Skip to content

feat: apply API rate limits org-wide via an atomic fixed-window counter - #1019

Open
rohan-chaturvedi wants to merge 1 commit into
mainfrom
feat--unify-rate-limits
Open

feat: apply API rate limits org-wide via an atomic fixed-window counter#1019
rohan-chaturvedi wants to merge 1 commit into
mainfrom
feat--unify-rate-limits

Conversation

@rohan-chaturvedi

Copy link
Copy Markdown
Member

🔍 Overview

API rate limits are currently enforced per principal: each User (PAT), Service Account, and Service Token gets its own throttle bucket (user_<id> / sa_<id> / st_<id>). Since plan rate limits are conceptually org-scoped, this means an organisation's effective API capacity multiplies with every principal it creates, and the plan-based limit doesn't actually bound what an org can send.

This PR unifies rate limiting to a single shared bucket per organisation: all members, service accounts, and service tokens of an org draw from one limit, resolved from the org's plan.

💡 Proposed Changes

  • Org-wide throttle bucket. PlanBasedRateThrottle now keys the cache on org_<id>. A new get_organisation() resolves the org from request.auth (organisationenvironmentapp → principal fallbacks), covering every auth shape PhaseTokenAuthentication emits. Unauthenticated requests keep the per-IP anon_ bucket.
  • Atomic fixed-window counter. Enforcement replaces DRF SimpleRateThrottle's cached timestamp list (a non-atomic read-modify-write that loses updates under concurrency) with cache.add + cache.incr on a window-suffixed key — atomic SET NX / INCRBY on Redis. This matters now that one hot key is shared by all of an org's workers. It also drops the per-request cost from shipping an O(limit) pickled list to O(1) integer ops.
  • wait() / retry-after now reports the seconds until the current window resets.
  • Rate config hardening. An unset per-plan env var (e.g. RATE_LIMIT_PRO) now falls back to RATE_LIMIT_DEFAULT instead of silently disabling throttling for that plan tier.
  • Observability. An authenticated request whose org cannot be resolved logs a warning before falling back to the per-IP bucket.

🖼️ Screenshots or Demo

N/A — backend-only change. Throttled responses are unchanged in shape:

HTTP/2 429
retry-after: 52

📝 Release Notes

  • Phase Cloud API rate limits are now shared across your entire organisation: all Users, Service Accounts, and Service Tokens draw from one plan-based limit (previously enforced per account). If you run many parallel integrations, review your request volume against your plan's limit.
  • 429 responses continue to include a retry-after header.
  • Self-hosted: behavior is unchanged — RATE_LIMIT_DEFAULT sets the (now org-wide) limit; if unset, no rate limits are applied.
  • Companion docs update: phasehq/docs branch unified-rate-limits.

🧪 Testing

  • tests/api/test_throttling.py rewritten: 19 tests, all passing in the dev container.
  • Auth fixtures now mirror the exact request.auth shapes PhaseTokenAuthentication.authenticate() emits (env / app-only / org-only modes × User / Service Token / Service Account), instead of synthetic dicts.
  • New coverage: exact org cache-key assertions for all six auth shapes, shared-bucket enforcement across principals, cross-org isolation, DEFAULT fallback for unset plan rates, window reset, wait() remainder, fallback warning log, and rate-None disabling throttling.
  • The throttle clock is pinned in the fixture so fixed-window tests cannot straddle a real minute boundary.
  • Known unrelated flake: tests/api/views/identities/** can 429 when run back-to-back in a dev container with RATE_LIMIT_DEFAULT=5/min (they hit real throttled views against live Redis). Pre-existing on main, not introduced here.

🎯 Reviewer Focus

  • backend/api/throttling.pyallow_request(): the add/incr atomicity, window keying, and the plan-rate resolution path.
  • get_organisation() — confirm the resolution order against the auth dict shapes built in backend/api/auth.py (authenticate()), which always sets one of organisation / environment / app; the principal fallbacks are defensive only.

➕ Additional Context

  • auth.py already resolves the caller org (_resolve_caller_org) but only attaches it to request.auth in org-only/bootstrap modes. A follow-up could attach it on every request and collapse get_organisation() to a single dict read.
  • Docs changes (org-wide wording on the public API page, RATE_LIMIT_DEFAULT semantics on the self-hosting env vars page) ship separately via phasehq/docs#unified-rate-limits.

✨ How to Test the Changes Locally

  1. docker compose -f dev-docker-compose.yml up -d (set RATE_LIMIT_DEFAULT=5/min).
  2. In one organisation, create a PAT and a Service Token.
  3. Send 6 requests within a minute to https://localhost/service/public/v1/secrets/?app_id=<id>&env=development, alternating between the two tokens.
  4. The 6th request returns 429 with a retry-after header, regardless of which token sent it — both principals share the org bucket.
  5. Repeat with a token from a second organisation: it is unaffected.
  6. Run the suite: docker compose -f dev-docker-compose.yml exec backend pytest tests/api/test_throttling.py -v

💚 Did You...

  • Ensure linting passes (code style checks)? (no backend linter configured in CI; style matches surrounding code)
  • Update dependencies and lockfiles (if required) (not required)
  • Update migrations (if required) (not required)
  • Regenerate graphql schema and types (if required) (not required)
  • Verify the app builds locally?
  • Manually test the changes on different browsers/devices? (N/A — backend-only)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants