Why x-ratelimit-remaining-tokens-per-day shows a tenth of your daily limit on DigitalOcean Inference

Question

I kept reading x-ratelimit-remaining-tokens-per-day as a balance. My account shows a limit of 60,000,000 tokens a day and a remaining figure of 6,000,000, and I had not spent 54 million tokens that morning. A tenth of the limit, sitting there on every response, with x-ratelimit-reset-tokens-per-day: 0 next to it. My first guess was that the counter was stuck, because six calls in a row with prompt sizes from 10 to 610 tokens did not move it by a single token. That guess was wrong, and working out why turned into the most useful hour I have spent with this endpoint. The headers are not reporting what I assumed they were reporting, and the thing they are actually describing is a better design than the one I had in my head. Below is what each header means, the burst that shows the counters are live after all, and the measurement that pins down what the daily number really is. Everything came off the live endpoint at inference.do-ai.run with an ordinary model access key.

KFSys · Accepted Answer

What you actually get back A chat completion carries three buckets, each with a limit, a remaining and a reset: x-ratelimit-limit-requests 240 x-ratelimit-remaining-requests 239 x-ratelimit-reset-requests 0 x-ratelimit-limit-tokens-per-minute 2000000 x-ratelimit-remaining-tokens-per-minute 2000000 x-ratelimit-limit-tokens-per-day 60000000 x-ratelimit-remaining-tokens-per-day 6000000 x-ratelimit-reset-tokens-per-day 0 That is already more than most inference providers will tell you. Plenty of endpoints give you a 429 and nothing else, so having three separate buckets exposed per response is the thing that made this investigation possible at all. Two of the values surprised me. Every reset-* field is 0, on every response I made. And the daily remaining is exactly a tenth of the daily limit. They look static, and that is a sampling problem Six calls, prompt sizes stepping from 10 tokens to 610: # prompt_t rem-req rem-tok-min rem-tok-day 0 10 239 2000000 6000000 1 130 239 2000000 6000000 2 250 239 2000000 6000000 3 370 239 2000000 6000000 4 490 239 2000000 6000000 5 610 239 2000000 6000000 Nothing moves. I concluded the counter was stuck, which was wrong and is an easy mistake to make, because serial requests from one laptop cannot outrun the refill. Twenty rapid serial calls also produced exactly one distinct value for remaining-requests: 239, every time. Thirty of the same calls in parallel produced seventeen distinct values, down to 223. The counter was live the whole time. I simply could not spend faster than it was topping itself back up, which in hindsight is a good sign rather than a bug. The burst that shows all three moving Twenty-four requests at about 4,000 prompt tokens each, twelve at a time, 96,675 tokens billed in 6.6 seconds: remaining-tokens-per-minute 2000000 -> 1966872 remaining-tokens-per-day 6000000 -> 5951988 remaining-requests 239 -> 233 The daily bucket dropped by roughly what the burst cost, so the accounting is honest and per-token. The daily limit is a refill rate Here is the part that reframed it for me. I drained the bucket with three waves, 973,818 tokens total, then sampled every 30 seconds while it recovered: t+ 0s 5047485 t+ 30s 5068927 t+ 60s 5090103 t+ 90s 5111532 t+120s 5132739 t+150s 5153894 t+180s 5175916 t+210s 5197451 t+240s 5218636 t+270s 5240282 A straight line. Over a 278-second window, subtracting the tokens my own probe calls cost: measured refill 694.7 tokens/sec limit / 86400 694.4 tokens/sec ratio 1.000 That is the whole answer. x-ratelimit-limit-tokens-per-day is not a balance you draw down until midnight. It is a daily allowance applied as a continuous trickle of limit / 86400 tokens every second. x-ratelimit-remaining-tokens-per-day is the size of the bucket that trickle fills, and that bucket holds a tenth of the daily figure. So on my tier: Burst capacity is 6,000,000 tokens. That is the most you can spend at once. Sustained rate is 694 tokens a second, which is the 60,000,000 a day the limit advertises. An empty bucket refills in 2.4 hours, which is a tenth of a day, exactly as the ratio implies. The reset field reads 0 because nothing resets, and once you see it as a token bucket that is the right answer rather than a missing one. I had been picturing a quota that empties and then makes you wait for a midnight rollover. This is better than that in two ways. You never sit blocked until tomorrow, because capacity comes back continuously and a short pause buys you real headroom. And there is no thundering herd at 00:00 UTC when every account on the platform refills at once, which is a failure mode I have been on the wrong end of elsewhere. The request bucket works the same way 120 requests at 40 concurrent pulled remaining-requests down to 191 of 240, all of them 200s. Recovery: t+ 0s 192 t+10s 234 (+42 in ten seconds) t+20s 239 That is 4.2 requests per second against a predicted limit / 60 of 4.0. Same mechanism, minute-scale instead of day-scale, and consistent enough across both buckets that you can rely on the formula. Note that the per-minute buckets hold their full limit, where the daily one holds a tenth. Two things worth knowing when you go looking The embedding headers exist, but only on embedding calls. access-control-expose-headers advertises fifteen headers on every response and nine of them are absent from a chat completion, including the whole embedding-tokens family. Call /embeddings and they appear: x-ratelimit-limit-embedding-tokens-per-day 180000000 x-ratelimit-remaining-embedding-tokens-per-day 18000000 Same one-tenth rule, different allowance, and a separate bucket from your chat tokens, which is the sensible arrangement: a big embedding job cannot starve your chat traffic. The router tells you which model it picked. x-model-router-selected-route is one of those nine absent headers on a normal call, but it does appear when the model is a router: target. Four routers, same prompt: router:writing Long-Form Article & Content Writing -> qwen3.8-max router:general fallback -> deepseek-v4-flash-0731 router:software-engineering fallback -> deepseek-v4-flash-0731 router:knowledge-base-document fallback -> deepseek-v4-flash-0731 Three of the four fell back rather than matching a named route on my prompt, and the model field in the response body names the model that actually ran. I like that this is visible at all. Routing you cannot inspect is routing you cannot debug, and between the header and the body field you can always tell exactly what served your request. What I would do with this Plan against the bucket, not the daily number. If you have a job that wants to push a few million tokens through in one go, 6,000,000 is your ceiling and then you are metered at 694 a second whatever the headline says. For most workloads that is a lot of headroom. Read remaining, not reset. The reset fields are zero and there is nothing to wait for. Backing off until the bucket has refilled enough is the useful behaviour, and you can compute the wait yourself: (needed - remaining) / (limit / 86400) seconds. Sample under concurrency before you believe a counter is stuck. That was my mistake and it cost me an hour of thinking the headers were broken when they were working correctly the whole time. Caveats The numbers are from one account on one tier. The 60,000,000 and the 240 will differ on yours; the ratio and the refill formula are the parts I would expect to carry over, and they held on both the chat and the embedding buckets. I never actually emptied the bucket. Draining 6,000,000 tokens costs real money, so I drained about a million and measured the recovery slope. What a genuinely exhausted bucket returns, and whether that 429 is distinguishable from the Platform overloaded 429 you get from concurrency, I did not test and would not guess at. The refill looked perfectly linear over 278 seconds. I did not test whether it stays linear over hours. Router routing was one prompt against four routers. That is enough to show the header works and that fallback happens, not enough to say anything about how often it happens.

KFSys · Answer

This comment has been deleted

Alex Taylor · Answer

The refill-rate explanation makes the behavior much clearer. One practical implication is that clients probably should not treat the daily “remaining” value as a traditional midnight-reset quota. For an SDK or application-level rate limiter, it seems useful to calculate the refill rate from the limit and window, then use the reported remaining value to decide whether to delay a request. I’d also be interested in whether DigitalOcean documents this token-bucket behavior officially, since relying on observed headers is useful but could become fragile if the implementation changes.

Become a contributor for community

DigitalOcean Documentation

Resources for startups and AI-native businesses

The developer cloud

Start building today

Report this

Why x-ratelimit-remaining-tokens-per-day shows a tenth of your daily limit on DigitalOcean Inference

Related questions

Comparing model latency and cost on DigitalOcean Inference before you commit to one

Tier 2 serverless inference: Daily token bucket caps at 500K

Tier 2 serverless inference: Daily token bucket caps at 500K

Why x-ratelimit-limit-tokens-per-day too little?

What are the best AI frameworks and tools for adding AI features to mobile apps?

Report this

Why x-ratelimit-remaining-tokens-per-day shows a tenth of your daily limit on DigitalOcean Inference

Related questions

Comparing model latency and cost on DigitalOcean Inference before you commit to one

Tier 2 serverless inference: Daily token bucket caps at 500K

Tier 2 serverless inference: Daily token bucket caps at 500K

Why x-ratelimit-limit-tokens-per-day too little?

What are the best AI frameworks and tools for adding AI features to mobile apps?