Tier 2 serverless inference: Daily token bucket caps at 500K

Question

Trying to figure out the daily token budget for Tier 2 serverless inference (inference.do-ai.run) so to find out if it can be used for agentic coding work (bursts requests). I can’t find a consistent answer and would like to reach out in the community to see if anybody could help me figure this out.

Have done some requests and recorded the response (probing with “messages”: [] returns HTTP 400 and does not consume tokens, zero-cost way to pull headers):

The response header reports x-ratelimit-limit-tokens-per-day: 5,000,000, but x-ratelimit-remaining-tokens-per-day clamps at exactly 500,000. Observed the clamp value both after ~4h idle and after ~23h idle that sits at 500,000 and not 5,000,000.
The token bucket refills continuously at a measured value of 57.83 tok/s over a 601 s window. This matches the full refill rate of 5M/day (5,000,000 / 86,400 = 57.87 tok/s).

From this I think I can conclude that the bucket behaves like capacity: 500,000 is the tier 2 TPM range and max refill rate is 5,000,000/day and the tokens stop filling up the bucket after ~2.4h, when 500,000 tokens is reached.

Reading the docs I can’t find clarity:

Neither Tier table publishes a tokens-per-day value. I can’t find the 500,000 bucket I hit.
The quota-headers page defines x-ratelimit-limit-<type> as “total capacity configured for the evaluated quota bucket,” but response recordings give me day capacity at 500,000 while the header reports the refill rate (5,000,000).

With a 500,000 ceiling I can’t see how Tier 2 could be used for agentic coding. DO’s OpenCode tutorial (inference-router-agentic-coding) uses ~4.1M tokens over a few hours. At 57.87 tok/s, ~20h would be needed to run through it on my tier.

So, is 500,000 the intended Tier 2 daily capacity, and where is it documented?

KFSys · Answer

Heya, You’ve correctly spotted that this behaves like a token bucket, not a simple daily counter. The two numbers mean different things: x-ratelimit-limit-tokens-per-day: 5,000,000 is the refill total (the tokens added to your bucket per day), which lines up perfectly with the 57.87 tok/s you measured. The 500,000 clamp on remaining is the bucket capacity, i.e. the most tokens you can ever hold at once. That’s your burst ceiling. Here’s the key thing though: capacity is not your daily budget. Because the bucket refills continuously while you spend, the total you can consume over a full day is roughly capacity plus a full day of refill: 500,000 (burst) + 5,000,000 (daily refill) ≈ 5.5M tokens/day So you’re not limited to 500K a day. You’re limited to a 500K burst, and then a sustained ~57.87 tok/s after that. Over 24h that’s about 5M tokens of real throughput. That’s why the header is named per-day even though the visible ceiling is 500K. Now, for your actual question (is it usable for agentic coding?), the honest answer is: it depends on how bursty. The binding constraint isn’t the 500K, it’s that sustained refill rate. Running your math on the OpenCode ~4.1M token job: (4,100,000 - 500,000) / 57.87 tok/s ≈ 62,000s ≈ ~17h So a 4.1M run crammed into “a few hours” absolutely will throttle once you drain the 500K burst and start outrunning the refill. Spread across most of a day it fits fine, but heavy multi-hour agentic bursts on Tier 2 will hit the wall exactly like you saw. There’s also a separate per-minute TPM limit (Tier 2 is 500K to 750K per minute) layered on top, which governs your short spikes while the per-day bucket governs sustained use. On the docs: you’re not missing anything, this genuinely isn’t published clearly. The tier tables don’t list a per-day bucket capacity, and the quota-headers page describing limit as “total capacity” does conflict with what you (and I) observe, where limit is the refill rate and remaining reflects the actual capacity. That’s a real documentation gap worth flagging. So my suggestions: Treat Tier 2 as roughly 5M tokens/day sustained with a 500K burst, not a 500K/day cap. For genuinely bursty agentic coding, that burst ceiling is the pain point, so it’s worth requesting a limit increase or tier bump on the Resource Limits page in the control panel (or via support). Higher tiers lift both the burst and the refill. It’d be great to raise the header/doc inconsistency with DO support directly, since you’ve basically handed them a clean bug report already. Hope that reframes it a bit more usefully, nice work pulling all those measurements.

alexdo · Answer

One thing worth adding since this is for a recurring agentic workload rather than a one-off run: that “~5.5M/day” figure assumes you’re starting each session from a full 500K bucket. If you’re running this daily and the previous run drained the bucket close to empty, you don’t get a fresh 500K the next morning, you’re back to accumulating at 57.87 tok/s from wherever the bucket was left, which is roughly 2.4h of idle time to refill from empty to full. So the real planning number depends on your idle gaps between runs, not just a flat daily total, worth keeping an eye on the actual x-ratelimit-remaining-tokens-per-day header at the start of each run rather than assuming full capacity. Regards

Bobby · Answer

Hi there, Nice detective work on this. I cannot find a published tokens-per-day figure per tier anywhere in the docs, the tier tables only cover model access, not the actual bucket sizes. So the 500K ceiling you measured is not something I can confirm or contradict from public documentation. Given the discrepancy between the advertised 5M limit header and the observed 500K clamp, this looks like either a genuine bug in how the bucket is capped, or an undocumented burst ceiling separate from the daily refill rate. Either way it is not something you should have to reverse engineer to plan agentic workloads. Worth opening a support ticket with exactly the data you posted here, the header values, the refill rate measurement, and the OpenCode tutorial mismatch. That level of detail should get you a real answer fast, and if it turns out to be a bug, it is worth them fixing since agentic coding is clearly something they are positioning inference for given their own tutorial. If you do get an answer, posting it back here would help others who hit the same wall.

Become a contributor for community

DigitalOcean Documentation

Resources for startups and AI-native businesses

The developer cloud

Start building today

Report this

Tier 2 serverless inference: Daily token bucket caps at 500K

Related questions

Comparing model latency and cost on DigitalOcean Inference before you commit to one