Problem with colliders in topdown 2d scene with many floors

Recently I ran into an complication in Unity that I don't know how to get out of. None of the solutions I came up with seems fully appropriate and requires some compromises. It looks like an engine weakness to me. But to the point.

I have 2D top down scene with has many levels (floors). Every floor is placed on different Z value (every 1 unit) and all floors are enabled all the time.

Focusing on specific floor is done by manipulating clipping plane of camera. This works great but the problem is with Physics2D.

I use Collider2D, but I do not want colliders on one floor (z = 0) to collide with colliders on another floor (z = 1, a higher ground). But 2D colliders ignore z-axis so player moving on one floor hits colliders on other floor.

I cannot use layers for this because amount of floors exceeds the number of layers in Unity. And also I use layers for something else.

Tried to use triggers, to manage manually collision, but it didn't work well.

I was also thinking of moving to Physis3D. But I use tilemaps with colliders, and I believe tilemaps do not work with Collider3D.

Is it possible to somehow make Collider2D to use the Z value?

if not, then I thing its a room for improvement by Unity. A setting like "Physics2D not ignore Z axis"