Fine-tuning the LLM Ornith 9b on a single H200 GPU Droplet: cost, latency, and serving overhead

Model providers building user-facing agentic products generally pick between two bad defaults: hide the reasoning entirely, which makes debugging and trust-building harder, or dump the raw chain-of-thought on the user, which is often long, repetitive, and occasionally contains information you don’t want surfaced verbatim. A model that natively produces a clean, structured summary of its own reasoning, without a separate summarization pass or a second model call, is a middle path worth having real numbers on.

This is not a novel idea in the abstract, but there’s a specific claim underneath it worth testing: does adding this behavior to an existing agentic model cost anything at serving time? Fine-tunes that change output format rather than output content are often assumed to be “free” from an inference standpoint. We wanted to confirm that rather than assume it; the full comparison against the unmodified base model is in the results below.

Ornith-1.0-9B is the smallest checkpoint in DeepReinforce AI’s Ornith-1.0 family, a set of open-source, MIT-licensed models built specifically for agentic coding (the family also includes 31B dense, 35B MoE, and 397B MoE checkpoints). It’s a dense 9B model, post-trained on top of Qwen 3.5, and small enough to run in bf16 on a single 80GB GPU (about 19GB of VRAM). What sets the family apart architecturally isn’t size, it’s the training approach: instead of pairing the model with a fixed, human-designed agent harness, Ornith-1.0 is trained via reinforcement learning to construct its own scaffold alongside its solutions, jointly optimizing both. DeepReinforce reports the 9B model reaching 43.1 on Terminal-Bench 2.1 and 69.4 on SWE-Bench Verified, matching or beating larger open models on comparable benchmarks despite its size. None of that changes with our fine-tune; we’re not touching the scaffold-generation behavior or coding performance, only how the model surfaces its reasoning to the end user.

The dataset, SupraLabs/reasoning-summaries-61k, is 61,000 examples built specifically for training models to produce clean, structured summaries of a reasoning chain rather than exposing the raw chain-of-thought. Each example pairs a raw reasoning trace with a summarized version, plus structured fields (title, subtitle, summary, current-task). It’s built from named, licensed upstream sources rather than scraped or proprietary data, which matters for anyone evaluating whether to reproduce this: the data lineage is traceable and the license terms are known going in.

We used LLaMA-Factory to run the fine-tune on a single H200 GPU Droplet.

At $3.44/hour for a single H200, 24.675 hours comes out to $84.88 to fine-tune the full 61K-example dataset for three epochs. That’s the full training cost, not an estimate (compute runtime only; note that DigitalOcean bills GPU Droplets per second and continues billing while a Droplet is powered off, so provisioning and idle time before you destroy the Droplet bill separately). Whatever a founder’s next question is (can we afford to try this, what’s the actual bill), this is the number that answers it.

We deployed the fine-tuned model on the same H200 Droplet using DigitalOcean’s Bring Your Own Model service, running vLLM behind an OpenAI-compatible endpoint.

To measure latency and throughput, we built a small async benchmarking harness: it fires a batch of concurrent requests using asyncio.gather, parses the streamed server-sent-events response to record time-to-first-token (TTFT) and tokens-per-second (TPS) per request, and reports both per-request and aggregate throughput. We ran five trials each at concurrency levels of 1, 5, and 20, with an explicit warmup request before any timed trial.

Two things came up during testing worth naming directly, since they shaped how we read the results. First, the initial run at a new concurrency level showed a much higher TTFT than every subsequent run at that same level (0.691s on the first hit at concurrency 5, versus roughly 0.035s on every run after). This lines up with how vLLM handles CUDA graph capture: it compiles an execution graph the first time it sees a given batch size, then caches it. A warmup pass at one batch size doesn’t warm every batch size you plan to serve at. If you’re benchmarking your own deployment, warm up at each concurrency level you intend to test, not just once at the start.

Second, one request out of 100 total (five trials of 20 concurrent requests each) on the fine-tuned model returned zero tokens, a full failure rather than a slowdown. We didn’t isolate whether that was a client-side timeout or a server-side rejection under load. We’re reporting it rather than smoothing it out of the averages, and we’d treat it as an open question rather than a settled finding given the sample size.

Fine-tuned model

(Concurrency-5 figures exclude the cold-start trial described above; concurrency-20 figures include the one failed request.)

TTFT stays close to flat, in the 20 to 47 millisecond range, from single-request all the way to 20 concurrent requests, once the server is warm. Per-request throughput drops by about 15% at 20x concurrency (187.63 down to 159.05 tokens/sec), while aggregate throughput scales roughly 17x (187.63 up to 3,181.02 tokens/sec). That’s the tradeoff that matters for anyone planning capacity: a modest slowdown per request in exchange for a large increase in total throughput.

Fine-tuned vs. base model

We ran the identical benchmark, same prompts, same hardware, same five trials per concurrency level, against the unmodified base Ornith-1.0-9B model.

Every difference here falls within the noise of the base model’s own trial-to-trial variation (its concurrency-20 TTFT alone ranged from 0.043s to 0.051s across its five trials). In plain terms: fine-tuning Ornith-1.0-9B to produce structured reasoning summaries added no measurable serving overhead, at any concurrency level we tested.

One data point worth flagging rather than overstating: the base model’s worst single-request TPS reading at concurrency 20 was 133.94 tokens/sec (a slowdown, not a failure), while the fine-tuned model had its one zero-token failure at that same concurrency level. With only 100 requests per model at that load level, this isn’t enough data to say the fine-tune caused the failure, or that it’s connected to the base model’s occasional slowdowns at all. It’s an open question, not a conclusion.

The numbers above show the fine-tune costs nothing at serving time. What it does change is the shape of the output itself. Here’s the same prompt (“how does carbon dioxide exhaust affect modern life?”) run against both models.

Base model:

The unmodified model exposes its full reasoning as a long, freeform “Thinking” block: numbered steps, nested bullet points, and running commentary on tone, goals, and structure before it ever gets to an answer. It’s legible, but it’s raw. A user-facing product would either have to hide this entirely or ship it as-is, unstructured text of unpredictable length.

Fine-tuned model:

The fine-tuned model, given the identical prompt, instead returns a single structured object:

{
  "title": "CO2 Impact Analysis",
  "sub_title": "Exploring how carbon dioxide emissions influence daily life and the environment.",
  "summary": "I'm considering how carbon dioxide exhaust influences modern life, focusing on environmental and health effects. I plan to examine air quality, climate impacts, and policy responses to provide a comprehensive overview.",
  "cur_task": "I'm thinking about how carbon dioxide emissions affect everyday life and the environment."
}

This is the exact schema the fine-tune was trained to produce, per the SupraLabs/reasoning-summaries-61k dataset card: a title and sub_title that work as a short, human-readable header for a reasoning step, a summary that condenses the actual reasoning content, and a cur_task field naming what the model is doing right now. Instead of a wall of nested reasoning, a UI can render a title and a one-line status, expand into the summary on demand, and never show the raw trace at all.

Same underlying task, same model family, and (per the benchmarks above) the same latency and throughput profile. The difference is entirely in whether the reasoning arrives as loose prose or as a small, predictable object a frontend can actually build against.

Use this approach if you want a model to produce user-facing reasoning transparency without exposing full chain-of-thought, and you want real cost and latency numbers before committing to a concurrency target in production.

Don’t use this as evidence of improved task accuracy or agentic reliability. This fine-tune changes output format, not task performance, and we haven’t measured the latter here.

• Ornith-1.0-9B model card (DeepReinforce AI)
• SupraLabs/reasoning-summaries-61k dataset card
• LLaMA-Factory, the fine-tuning framework used for training
• vLLM, the serving engine used for inference, and its CUDA graph capture design doc
• DigitalOcean GPU Droplets and GPU Droplet pricing
• DigitalOcean Bring Your Own Model, the deployment path used to serve the fine-tuned model
• Terminal-Bench 2.1 and SWE-Bench Verified, the benchmarks DeepReinforce reports for the base model