The free-tier problem
Picture a public API with a free tier. The best way to sell it is to let people try it before they sign up. The worst thing that can happen is a script hammering that same open door all night.
Auth doesn't help here. The visitor has no account, and you don't want to make them get one. Rate-limiting at the edge doesn't quite help either, because it can't tell a visitor on your landing page from a paying customer's backend.
What you need is a rule like: an anonymous visitor can make 20 requests a day to this route, and a logged-in user can make 1,000. That's a usage policy. It ties a caller to an allowance.
What a policy does here
Put UsageFlow in front of the routes. Every call to a metered route asks a question before the handler runs: is this caller allowed one more request?
- An anonymous visitor is counted by IP, under a
free-tierworkflow. - A logged-in user is counted by user ID, under a
memberworkflow. - If the answer is no, the caller gets a
429and the handler doesn't run.
Same code, three kinds of caller, three different rules. The rules aren't in the handler. They're in policy, so changing one doesn't mean a deploy.
The three parts of a policy
- Identity: who is asking? A user ID, an API key, an IP.
- Allowance: how much can they do, and how often does it reset?
- Decision: allow, block, or change how it runs.
That's it. It isn't specific to AI.
FAQ
What is a usage policy?
A rule that sets how much a given caller may do in your app and what happens when they hit the limit.
Why not use rate limiting?
Rate limits count requests per address. A policy counts per identity and per workflow, and can differ by plan.
Do policies need a deploy?
No. They're configured in the Console and applied at runtime.