Qwen3.8-2.4T-A95B is now available on DigitalOcean Inference Engine

Served on NVIDIA HGX™ B300 GPUs with NVFP4-quantized weights, tuned in collaboration with Inferact.

Qwen3.8-2.4T-A95B is now available on DigitalOcean Inference Engine. It’s the open-weights, text-only release derived from Alibaba’s Qwen3.8-Max flagship: a 2.4 trillion-parameter mixture-of-experts model — roughly 95B parameters active per token — built for coding, tool use, and long-horizon agentic work. On Alibaba Cloud’s published benchmarks, the Qwen3.8-Max flagship leads PaperBench (93.0) and IFBench (82.8), and scores 86.6 on Terminal-Bench 2.1, ahead of Claude Opus 4.8 and Claude Fable 5 (both 84.6); no variant-specific public benchmarks exist yet (see Benchmarks below). List price is $2 per 1M input tokens and $6 per 1M output, against $10/$50 for Fable 5.

We’re serving it on NVIDIA HGX™ B300 GPUs with NVFP4-quantized weights, developed through a technical collaboration with Inferact. It’s available through DigitalOcean Serverless Inference with usage-based pricing and fully managed infrastructure, and through the DigitalOcean Inference Router, so you can add it to an existing routing mix and send requests to it based on cost, latency, or task fit. Sign up for DigitalOcean to start making calls.

At a glance

Note on the variant. Qwen3.8-2.4T-A95B is the open-weights, text-only release derived from the Qwen3.8-Max flagship — the version Qwen has made publicly available. It does not accept image or video input. Benchmark figures cited in this post are Qwen3.8-Max’s text-only benchmarks; we have deliberately excluded Alibaba’s multimodal results, which do not apply to this model.

Long-horizon agentic work. This is the capability Qwen built the model around, and it’s the clearest reason to reach for it. Alibaba’s own evaluations center on multi-day autonomous runs — sustained tool use, self-correction from execution feedback, and coherent strategy across hundreds of turns rather than one-shot generation.

Instruction-following in production workflows. The Qwen3.8-Max flagship’s IFBench 82.8 leads every model in Alibaba’s comparison set, including Opus 4.8, Fable 5, and GPT-5.6 Sol. If you’re building systems where the model has to respect format contracts and constraints reliably, this is the number that matters.

Large-document and large-codebase reasoning, up to the 262K context ceiling.

Cost-sensitive workloads at scale. At $2/$6, running a frontier-class model across high request volumes is materially cheaper than the alternatives.

The endpoint is OpenAI-compatible. Migrating an existing application is a base URL and model ID change. Note that the model ID on the platform is qwen3.8-max — the ID differs from the model’s full name.

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.do-ai.run/v1",
    api_key="<YOUR_DIGITALOCEAN_INFERENCE_KEY>",
)

response = client.chat.completions.create(
    model="qwen3.8-max",
    messages=[{"role": "user", "content": "Refactor this function for readability: ..."}],
    reasoning_effort="low",
    max_tokens=1024,
)

print(response.choices[0].message.content)

Reasoning effort. Qwen3.8-2.4T-A95B reasons before answering, and reasoning_effort accepts low, high, or xhigh. Reasoning tokens count toward both your output bill and the context window, so low is the right default for extraction, classification, formatting, and routing work — reserve high and xhigh for tasks where the chain of thought is doing real work. The latency figures later in this post were measured without setting the parameter, so they reflect the server default rather than low.

Streaming, which we recommend for anything user-facing:

stream = client.chat.completions.create(
    model="qwen3.8-max",
    messages=[{"role": "user", "content": "Walk me through the basics of stock trading"}],
    max_tokens=1024,
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        print(delta.content, end="", flush=True)

Calling a tool

Function calling uses the standard OpenAI shape: the model returns a tool_calls list, your code executes the function, and you pass the result back for the model to write a final answer.

Here’s the whole loop with a real implementation — Open-Meteo, no API key required:

import json
import urllib.parse
import urllib.request


def get_weather(city: str) -> str:
    """Look up current conditions for a city."""
    geo = json.load(urllib.request.urlopen(
        "https://geocoding-api.open-meteo.com/v1/search?"
        + urllib.parse.urlencode({"name": city, "count": 1})
    ))
    if not geo.get("results"):
        return "No location found for %r." % city

    loc = geo["results"][0]
    wx = json.load(urllib.request.urlopen(
        "https://api.open-meteo.com/v1/forecast?"
        + urllib.parse.urlencode({
            "latitude": loc["latitude"],
            "longitude": loc["longitude"],
            "current": "temperature_2m,wind_speed_10m",
        })
    ))

    now = wx["current"]
    return "%s, %s: %s°C, wind %s km/h" % (
        loc["name"], loc.get("country", ""),
        now["temperature_2m"], now["wind_speed_10m"],
    )


tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

messages = [{"role": "user", "content": "Is it jacket weather in Lisbon right now?"}]

resp = client.chat.completions.create(
    model="qwen3.8-max", messages=messages, tools=tools, max_tokens=512
)
msg = resp.choices[0].message

if msg.tool_calls:
    messages.append(msg)                       # keep the model's request in history
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_weather(**args)           # your function actually runs

        messages.append({
            "role": "tool",
            "tool_call_id": call.id,           # must match the call
            "content": result,
        })

    final = client.chat.completions.create(
        model="qwen3.8-max", messages=messages, max_tokens=512
    )
    print(final.choices[0].message.content)

The model decides get_weather is the right function, extracts {"city": "Lisbon"} from a question that never says “get weather,” reads the live temperature your function returned, and answers the question that was actually asked — whether to bring a jacket.

Two things to get right: append the assistant message itself, not just the tool result, and give every tool message the matching tool_call_id. Miss either and the follow-up request will fail or the model will lose track of what it asked for.

Structured outputs

Pass a JSON Schema and get back conforming JSON, so you can drop the parse-and-retry wrapper most pipelines carry:

resp = client.chat.completions.create(
    model="qwen3.8-max",
    messages=[{"role": "user", "content": "Extract the invoice fields from: ..."}],
    reasoning_effort="low",
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "invoice",
            "schema": {
                "type": "object",
                "properties": {
                    "vendor": {"type": "string"},
                    "total": {"type": "number"},
                    "due_date": {"type": "string"},
                },
                "required": ["vendor", "total", "due_date"],
            },
            "strict": True,
        },
    },
)

This is real constrained decoding, not a hint. We tested it with a prompt that explicitly instructed the model to use a value outside the enum, return a decimal where the schema requires an integer, add a forbidden field, and open with a paragraph of prose. With the schema attached, output conformed on every trial. With the same prompt and no schema, it violated the contract on every trial. You can drop the parse-and-retry wrapper.

One budget note: reasoning tokens draw on the same max_tokens pool as the answer. At default reasoning effort, a schema-constrained request for even a small object can exhaust a 512-token budget and return truncated JSON. Pair response_format with reasoning_effort="low" and a generous max_tokens, as above.

Qwen3.8-2.4T-A95B can use DigitalOcean’s server-side tools, which run on our infrastructure rather than requiring you to build and host the harness yourself.

MCP is the one to look at first if you’re building agents. A model tuned for long-horizon autonomous work, pointed at your existing MCP servers, with no harness to operate yourself — that’s the combination this release is built for.

These run on our side. Function calling, shown in the quickstart above, works differently: the model returns the call and your application executes it. Both mechanisms are available for Qwen3.8-2.4T-A95B and can be combined in a single request.

Some tools in the DigitalOcean catalog are provider-specific and are not available for Qwen3.8-2.4T-A95B: Tool Search, Computer Use, Bash/Local Shell, Text Editor, and Apply Patch.

The figures below are Alibaba Cloud’s published results for Qwen3.8-Max — the flagship from which this open-weights release is derived — restricted to text-only benchmarks. No variant-specific public benchmarks for Qwen3.8-2.4T-A95B exist yet, so treat these as indicative rather than exact; we’ll update this section when independent numbers for the open-weights release are published. We have not independently reproduced them.

Where it leads: PaperBench and IFBench, both by wide margins.

Where it trails: SWE-bench Pro, where Fable 5 is meaningfully ahead (80.0 vs 67.7), and text-only reasoning on HLE. If your workload is hard pure-SWE or frontier reasoning, benchmark before you commit.

A note on reading these: several of the strongest published results for this model come from Qwen’s own in-house benchmarks, scored by Qwen’s own judge models. We’ve excluded those here and cited only third-party benchmarks.

Why 4-bit. A 2.4T-parameter model is large enough that the FP8 variant would not load on a single node. NVFP4 quantization brings the weights within single-node reach on B300, which removes cross-node expert routing from the serving path entirely — simpler topology, no inter-node communication in the critical path, and better economics that we pass through in the price. This work was done in collaboration with Inferact, starting from the open weights Qwen released on Hugging Face.

Quality. An internal spot-check of the quantized build scored 88 on GPQA Diamond, against Alibaba’s published 92.6 for the Qwen3.8-Max flagship. The two figures come from different evaluation stacks and harness configurations, and the gap spans two differences at once — flagship versus open-weights variant, and unquantized versus NVFP4 — so this is an indicative data point rather than a controlled comparison, and it covers one benchmark in the model’s weakest category (text-only reasoning) rather than its strengths. We’re publishing it because a single honest number is more useful than none. Qwen3.8-2.4T-A95B is open-weights and other providers will serve it; when you compare, ask for the same disclosures we’ve made here: quantization format, hardware, quality data on the quantized build, and dated performance measurements.

The numbers below were measured on 2026-08-12 from a single client over the public internet against the production endpoint, using streaming requests with unique prefixes (so prompt caching is not in play) and without setting reasoning_effort, so they reflect the server default. They show what a developer would observe, which means they include network latency and represent a floor rather than a ceiling. We observed run-to-run variance in aggregate throughput across sessions; treat these as a snapshot, not a service guarantee.

Time to first token, by input length

Prefill scales linearly across the full context range, at roughly 16,000 tokens/sec, with no knee.

A usable rule of thumb: TTFT ≈ (input tokens ÷ 16,000) + 0.7 s.

Throughput under concurrency

Aggregate throughput scales close to linearly through 256 concurrent requests, while TTFT p50 stays near one second. We did not find a saturation point in this range — the ceiling is above 256.

~1,080-token inputs, ~128-token outputs. 1,536 requests at concurrency 256, zero errors.

Per-stream generation speed

Median inter-token latency is 108 ms at concurrency 1 and 115 ms at concurrency 8 — approximately 8–9 tokens/sec per stream, holding steady as concurrency rises.

This is the honest shape of the model’s performance: per-stream generation is modest, and capacity scales through concurrency rather than through single-stream speed. For agentic pipelines, batch processing, and background work, that tradeoff is the right one. For latency-sensitive interactive chat, benchmark against your own UX budget first.

The 262,144-token window is the model’s native context length as trained. The architecture is extensible to just over 1M tokens with context-extension techniques, but we serve the native window deliberately: extension runs the model outside its natively trained configuration, and it works against the single-node serving path that keeps latency and pricing where they are (see NVFP4 on B300, above). If your workload genuinely needs more than 262K in a single request, this build is the wrong fit — for most workloads, including agentic loops with large stable prefixes, native context plus prompt caching is the better trade.

In practice: a 254K-token input with 128 tokens of output succeeds; the same input with a large max_tokens will not. Prefill cost grows linearly with input length (see above), so a full-window request costs roughly 16 seconds before the first token arrives.

If you’re replaying a large stable prefix across turns — the common shape for agentic loops — see the prompt caching note under Pricing.

Qwen3.8-2.4T-A95B is available through the DigitalOcean Inference Router, so you can add it to an existing routing mix. Based on the published benchmarks, a reasonable starting policy:

For comparison, Claude Fable 5 lists at $10 input / $50 output. On output-heavy agentic workloads — where a single task may generate hundreds of thousands of tokens — that difference compounds quickly.

Prompt caching at $0.20 per 1M tokens is the biggest lever available on top of that: a 10x discount on input for any agentic loop that replays a large stable prefix across turns. Full rates for every model are in the inference pricing docs.

For workloads that don’t need an immediate answer, Batch Inference works with Qwen3.8-2.4T-A95B and processes large request volumes asynchronously: upload an input file, create a job, poll for completion, then download results. Jobs can be listed and cancelled through the same API.

This is the right shape for what the model does well. Per-stream generation is modest — around 8–9 tokens/sec — while aggregate throughput scales to roughly 1,900 tokens/sec across concurrent requests. So throughput-bound work like bulk classification, document extraction, dataset generation, and offline evaluation belongs in batch rather than a request-per-item loop.

If you’re weighing which to use, we’ve written a comparison of serverless, dedicated, and batch inference on DigitalOcean.

Qwen3.8-2.4T-A95B is live now on DigitalOcean Serverless Inference. Sign up for DigitalOcean, create an inference key, point your OpenAI client at https://inference.do-ai.run/v1, and pass qwen3.8-max as the model.

From there:

• Browse every model available on the platform in the supported models list
• Check rates in the inference pricing docs
• Decide between deployment shapes with our serverless vs. dedicated vs. batch inference comparison
• Read Qwen’s own Qwen3.8-Max release post for the model’s training and evaluation details
• Pull the open weights from Hugging Face if you’d rather host it yourself

What is Qwen3.8-2.4T-A95B? Qwen3.8-2.4T-A95B is Alibaba Cloud’s premiere open-source large language model, released in August 2026. It’s the open-weights, text-only release derived from the Qwen3.8-Max flagship: a mixture-of-experts model with 2.4 trillion total parameters and roughly 95 billion active per token, built for coding, tool use, and long-horizon agentic tasks. On DigitalOcean it runs as a text-in, text-out model.

What is the model ID for Qwen3.8-2.4T-A95B on DigitalOcean? The model ID on DigitalOcean Serverless Inference is qwen3.8-max. Pass that string as the model parameter — the platform ID differs from the model’s full Hugging Face name (Qwen/Qwen3.8-2.4T-A95B).

What is the context window for Qwen3.8-2.4T-A95B on DigitalOcean? 262,144 tokens total, shared between input, reasoning, and output. The endpoint treats this as a single budget — exceed it and the API returns an HTTP 400 reporting max_model_len=max_total_tokens=262144, along with your own token counts. Nothing is silently truncated, so budget output tokens against the same pool as input.

Does Qwen3.8-2.4T-A95B support images or video? No. Qwen3.8-2.4T-A95B is the text-only, open-weights release derived from the Qwen3.8-Max flagship, and it’s the variant DigitalOcean serves. Benchmarks cited in this post are text-only benchmarks; Alibaba’s published multimodal results for the flagship don’t apply to it.

How much does Qwen3.8-2.4T-A95B cost on DigitalOcean? $2 per 1M input tokens, $6 per 1M output tokens, and $0.20 per 1M cached input tokens. For comparison, Claude Fable 5 lists at $10 input and $50 output.

How fast is Qwen3.8-2.4T-A95B? In our measurements on 2026-08-12, time to first token was around 1.1 seconds at ~1,000 input tokens, and prefill ran at roughly 16,000 tokens/sec — so TTFT scales as about (input tokens ÷ 16,000) + 0.7 seconds. Per-stream generation is around 8–9 tokens/sec, and aggregate throughput scaled to roughly 1,900 tokens/sec at 256 concurrent requests without hitting saturation.

Does Qwen3.8-2.4T-A95B support function calling and tool use? Yes. It supports native function calling through the standard OpenAI tools parameter, plus DigitalOcean’s server-side tools: web search, web fetch, model synthesis, knowledge base retrieval, and MCP. The two mechanisms can be combined in a single request.

Does Qwen3.8-2.4T-A95B support structured outputs? Yes, with real schema enforcement. Pass a JSON Schema via response_format and output conforms — we verified this with a prompt that explicitly instructed the model to violate the schema, and the constrained output held every time. Note that reasoning tokens share the max_tokens budget, so pair schemas with reasoning_effort="low" and a generous token limit.

What is NVFP4 quantization? NVFP4 is a 4-bit floating-point format supported natively on NVIDIA Blackwell GPUs. DigitalOcean serves Qwen3.8-2.4T-A95B with NVFP4-quantized weights because the FP8 variant of a 2.4T-parameter model won’t load on a single node; 4-bit brings it within single-node reach on HGX B300, eliminating cross-node expert routing from the serving path.

Can I run Qwen3.8-2.4T-A95B myself? Yes. Qwen released the weights openly on Hugging Face. Self-hosting a 2.4T-parameter MoE requires substantial GPU capacity, which is what the managed serverless endpoint is for.

Is Qwen3.8-2.4T-A95B available for batch processing? Yes. It works with DigitalOcean Batch Inference for asynchronous, high-volume workloads — a better fit than a request-per-item loop for bulk classification, document extraction, and offline evaluation.