RadixArk serves DeepSeek V4 at 3.5K+ tokens/sec/GPU on DigitalOcean and AMD

RadixArk is an infrastructure-first company focused on increasing frontier AI adoption. Using open-source technologies, its founders are building the AI infrastructure layer that enables more teams to run frontier models at scale, so teams don’t rebuild inference engines, training frameworks, and orchestration layers from scratch.

To accomplish this, RadixArk staff builds on SGLang, an open-source inference engine launched in 2023. Designed for production-level LLM serving, it powers trillions of tokens daily for organizations including Google®, Microsoft®, NVIDIA®, Oracle®, AMD®, LinkedIn®, and xAI™.

One of SGLang’s main strengths is that it supports both NVIDIA GPUs (via CUDA) and AMD GPUs (via ROCm/HIP) from the same upstream project/framework, allowing developers to target multiple hardware platforms without maintaining separate serving frameworks. That broad hardware coverage is central to RadixArk’s goal of making frontier AI implementation and usage as widespread as possible.

Working in collaboration with DigitalOcean and AMD allowed RadixArk to effectively debug and troubleshoot DeepSeek V4 for production—at 3.5K tokens/sec/GPU on AMD Instinct MI350X/MI355X—and achieve ~10x throughput with HIP graph support thanks to AMD joint engineering efforts.

Gaining reliable hardware access

To “ship AI for all,” RadixArk needed dedicated GPU capacity for running SGLang on DeepSeek V4, both for development and production validation. DigitalOcean AMD Instinct™ MI350X GPU Droplets® gave the SGLang team always-on hardware on demand, ready in minutes whenever development or production validation was needed.

RadixArk’s priority was iterating fast on production AMD inference, which meant getting hardware the moment they needed it rather than queuing for capacity. “DigitalOcean had clear availability on the AMD hardware we needed,” says Harry He, a founding member of the RadixArk team. The team could spin up GPU Droplets in minutes across DigitalOcean’s global data center network and get straight to shipping.

Streamlining production and increasing performance

The GPU setup had to support DeepSeek V4 Pro. At 1.6 trillion parameters, it runs across multiple nodes and needs ample bandwidth for high throughput and low latency.

SGLang currently clears over 3.5K tokens/second/GPU running DeepSeek V4 Pro (FP4) on AMD Instinct MI350X/MI355X GPUs, according to the June 2026 public leaderboard on InferenceX, says Yihao Wang, who works on SGLang’s multi-hardware and multi-model teams at RadixArk.

“For multi-node setups, DigitalOcean Droplets work really well. The interconnect is set up automatically when you spin up the Droplet, which was a big benefit when we ran across multiple nodes,” Wang adds.

The deployment model is intentionally direct: spin up GPU Droplets, SSH in, pull Docker images, and run inference, all without a managed orchestration layer.

Beyond multi-node access, a substantial performance gain came from a joint engineering effort with AMD. “AMD joined our co-development effort and enabled HIP graph support, which improved throughput by approximately 10x,” says Yusheng Su, Member of Technical Staff at RadixArk.

Because SGLang runs on both NVIDIA and AMD from one codebase, this didn’t require a separate fork. The same engine that serves DeepSeek on NVIDIA now runs it on AMD. The joint work with AMD, including Hai Xiao’s team, also covered kernel optimization and multi-token prediction, which sped up decoding. RadixArk now runs multiple DeepSeek V4 variants, including FP8 and FP4.

Growing frontier AI adoption

RadixArk’s goal from the start has been frontier AI infrastructure that’s open, runs anywhere, and is cost-effective to operate. SGLang is how that shows up in practice: one inference engine that serves any open model on any hardware, with no vendor-specific forks.

The team keeps expanding what workloads can run on AMD Instinct MI350X Droplets, bringing up open frontier models like GLM-5.1, DeepSeek V4, and Kimi K2.6, all on the same SGLang engine. Each new model lands on infrastructure that’s already proven, so the roadmap grows without the engine fragmenting.

DigitalOcean is what makes that cycle fast. Beyond raw GPU capacity, RadixArk gets a broad model library, supported inference, and test environments to debug against before a rollout. DigitalOcean maintains the underlying AMD infrastructure and is a Slack message away when something breaks, which means the team spends its time on models, not machines.

“The whole point of building on open source is that you’re never locked in. One SGLang codebase serves DeepSeek on both NVIDIA and AMD. No fork, no rewrite. DigitalOcean gives us the capacity to prove that in production, so we can keep our roadmap open and deploy frontier models wherever they run best,” He says.

DISCLAIMER: Any references to third-party companies, trademarks, or logos in this document are for informational purposes only and do not imply any affiliation with, sponsorship by, or endorsement of those third parties.