CollisionFilter

Because colliders bake their Collision Filters from the Layer Collision Matrix, and queries also use a Collision Filter, unlike normal LayerMask-based physics queries, Unity Physics queries end up relying on the matrix.

See EditProject SettingsPhysicsSettings, and the Layer Collision Matrix at the bottom.

Both object's BelongsTo and CollidesWith must match to successfully interact.

Filtering logic

When determining if two colliders should collide or a query should be performed, Unity Physics checks the BelongsTo bits of one against the CollidesWith bits of the other. Both objects must want to collide with each other for the collision to happen.

public static bool IsCollisionEnabled(CollisionFilter filterA, CollisionFilter filterB)
{
    if (filterA.GroupIndex > 0 && filterA.GroupIndex == filterB.GroupIndex)
    {
        return true;
    }
    if (filterA.GroupIndex < 0 && filterA.GroupIndex == filterB.GroupIndex)
    {
        return false;
    }
    return
        (filterA.BelongsTo & filterB.CollidesWith) != 0 &&
        (filterB.BelongsTo & filterA.CollidesWith) != 0;
}

From the Filtering section of the Collision Queries documentation.

This means that even when BelongsTo is ~0u (all bits enabled), the target's CollidesWith must not be 0. There must be a layer enabled in the Layer Collision Matrix.
This also means you can set a collider's CollidesWith to 0 to deny queries from hitting it momentarily.

Looking at the data

You can check the Physics ColliderCollision Filter or Physics ColliderGeometryChildCollision Filter of the objects you expect to interact with, and validate that your collision filter matches. Note that the drawer may display 0-indexed layers, not the raw bitmask value.

Unity, huh, how?

CollisionFilter#

Filtering logic#

Looking at the data#

CollisionFilter

Filtering logic

Looking at the data