Three Attempts at Auth0's Rate Limits

TL;DR

• Auth0 caps queries at 4096 characters and roughly two requests per second.
• Direct user retrieval died at a few dozen users.
• Querying by role instead got us to 60-70 before data reassembly became the bottleneck.
• A rate-limiting wrapper that queues and deduplicates calls handles a couple of hundred users, slowly.
• It works for our current volume and will not survive growth.

Our admin portal needs user data from Auth0, and Auth0 would rather we didn’t have it that quickly. Three iterations, each buying a bit more headroom.

Attempt one: fetch users directly. Query our database for user IDs, then pull roles and details from Auth0 using its Lucene query syntax. Two limits killed it. Auth0 caps each query at 4096 characters, forcing the work into multiple requests, and those requests hit the rate cap of roughly two per second. We were stuck at a few dozen users.

Attempt two: query by role. Fetch roles first, then work out which users belong to them. Fewer requests means fewer rate limit hits, and we reached somewhere around 60-70 users. But reassembling the data on our side got complicated and backend performance suffered. Better, not good.

Attempt three: handle the rate limit directly. I wrote a rate-limiting class to wrap our API calls. It records and queues calls, deduplicates ones already queued, and times execution to respect Auth0’s replenishment rate. That gets us to a couple of hundred users, though it’s sluggish.

That’s where it stands. The wrapper is a holding position: it satisfies our current volume and it will not scale. The real question is whether we should be reaching into Auth0 for this data at all, or whether our authentication storage strategy needs rethinking so the portal reads from somewhere designed for it.