It's Time to Break Up with Your Cloud: Why AI Teams are Switching

AI workflows are fundamentally different from traditional software workflows in their resource requirements, scaling patterns, billing models, cost management, and performance analysis. These factors make it difficult to rely on the traditional cloud platforms (Google Cloud, AWS, Azure, Oracle, etc.) that weren’t built with AI-first principles in mind. The conventional model training, hosting, and inference infrastructure can create friction that slows down AI development. Teams find themselves paying for always-on GPU instances that sit idle overnight, wrestling with permissions configurations, and navigating a range of services that weren’t purpose-built for AI workflows.

Purpose-built AI cloud platforms take a different approach. They often offer scale-to-zero functionality to eliminate idle costs, pre-configured environments optimized for model training and inference, and streamlined APIs that replace weeks of infrastructure setup with a few lines of code. These platforms compete on GPU costs, simplified developer experience, and AI-specific features like multi-LoRA serving.

To get a better idea of the landscape, I compare the cost, security, developer experience, and use cases of Baseten, Nebius, Fireworks AI, Modal, and Together AI. I also walk through a deployment scenario for each platform with a sample application to understand their pros and cons from a practical standpoint. Here’s what I found.

• Traditional cloud platforms create friction for AI workflows with always-on GPU billing, complex configurations, and services not purpose-built for AI workloads. Purpose-built AI platforms address these issues through scale-to-zero functionality, pre-configured environments, and streamlined APIs that reduce weeks of infrastructure setup to a few lines of code.

• The five platforms compared differentiate on cost structure, developer experience, and specialized features. Nebius offers the cheapest always-on GPUs ($2.95/hr H100) with the strongest compliance portfolio, Fireworks delivers the fastest inference with Multi-LoRA capabilities, Modal provides the best developer experience with per-second billing, while Baseten and Together AI balance features across regulated industries and model exploration respectively.

• None of these platforms provide complete infrastructure breadth. Most require external solutions for vector databases, document storage, and observability tooling. The right choice depends on matching platform strengths to your specific priorities.

Baseten: Cost

Baseten offers flexible pay-as-you-go pricing with scale-to-zero functionality, but they are relatively expensive for GPUs. They require per-minute GPU billing for dedicated deployments and per-token for the serverless API without reserved-instance or committed-use discounts. An H100 is $6.50/hour, and other GPU pricing runs from roughly $0.63/hr for a T4 up to $9.98/hr for a B200. The Basic tier is pay-as-you-go. Pro unlocks volume discounts, and enterprise starts around $5K/month via AWS Marketplace. Predictable enterprise pricing requires a sales call. Baseten’s scale-to-zero functionality automatically shuts down the GPU instance when the model is not receiving requests in order to stop charging entirely. This means we pay nothing overnight or during any idle period. When a new request arrives, the instance boots back up with a 5-10 second cold start.

Nebius: Cost

Nebius offers the cheapest GPU rates among the platforms. The raw GPU economics are a good differentiator for Nebius. The H100 runs at $2.95/hr. They have no egress fees, no ingress, and no charge for managed Kubernetes or public IPs. Batch inference runs at 50% of real-time prices. The tradeoff is that they lack the scale-to-zero functionality of Baseten, meaning the fine-tuned LLM service runs around the clock. There is no free tier, the minimum deposit is $25, and reserved discounts require large-scale, multi-month commitments.

Fireworks AI: Cost

Fireworks AI is cost-effective for serverless inference workloads. Serverless per-token pricing makes Fireworks cost-effective for bursty, low, or moderate volume workloads. At roughly $0.90 per million tokens for Llama-class models, it runs five to ten times cheaper than equivalent proprietary APIs. Cached tokens and batch inference each come in at 50% off. The downside is that an H100 is $6.00/hour.

Modal: Cost

Modal’s per-second billing minimizes costs for variable workloads. True per-second billing across GPU, CPU, and memory makes Modal pricing comparatively granular. Scale-to-zero is native, so you pay nothing during idle periods. An H100 runs at $3.95/hr, and the Starter plan includes $30/month in free credits. GPU instances are preemptible, meaning there is no guarantee the GPU instance will stay alive during the workload. This keeps costs low but requires your application to handle mid-request interruptions. Region multipliers are steep, up to 2.5x the base rate outside core regions.

Together AI: Cost

Together AI offers competitive serverless pricing with batch discounts. Together AI prices serverless inference at roughly $0.88 per million tokens for Llama-class models. The batch API halves that rate with up to 30 billion tokens of capacity. For the fine-tuned Llama, the cost path splits based on the base model. If it supports serverless LoRA, the adapter runs pay-per-token with no dedicated GPU to manage. If not, you are on a dedicated endpoint that bills continuously regardless of traffic volume. Fine-tuning also carries minimum charges of $6 to $100 depending on the model. An H100 is $3.99/hour.

Baseten: Security

Baseten covers the major regulatory bases, including SOC 2 Type II, HIPAA, and GDPR. They offer a self-hosted deployment option, allowing Baseten’s inference software to run inside the customer’s own cloud environment rather than on Baseten’s shared infrastructure. Model inputs, outputs, and weights never leave the customer’s network. This could be a hard procurement requirement for many regulated industries. As far as encryption goes, Baseten is notably opaque, with no publicly documented specifics on encryption algorithms, key management approach, or TLS version requirements.

Nebius: Security

Nebius’s SOC 2 Type II audit was conducted by Deloitte and includes a HIPAA section. Nebius holds ISO 27001, ISO 27701, ISO 27018, plus NIS2 and DORA alignment for EU financial institutions. EU data residency is available in Finland and France, and the US presence is Kansas City. Within the platform, you get VPC isolation, InfiniBand traffic segregation between tenants, zero-retention inference mode, and a public trust center. SOC 2 reports require an NDA to access, there is no FedRAMP, and there is no customer-managed encryption key (CMEK/BYOK) option documented.

Fireworks AI: Security

Fireworks has SOC 2 Type II, HIPAA, and zero data retention by default. They also have a BYOC storage option for integrating customer cloud buckets, and a Virtual Cloud Infrastructure product for running inference inside a customer’s own environment. Data residency guarantees span the US, Europe, and APAC, which is comparatively broad. They hold ISO 27001, ISO 27701, and ISO 42001 certifications, and customer-managed encryption keys are listed as “coming soon”. They also publicly document TLS 1.2+ and AES-256 at rest. Multi-tenancy risks on shared serverless infrastructure are not transparently addressed.

Modal: Security

Modal holds SOC 2 Type 2 and supports HIPAA, though the HIPAA BAA is Enterprise-only. Workload isolation runs via gVisor sandboxing. TLS 1.3 is in use, and function inputs and outputs carry a 7-day TTL before deletion. All function I/O routes through Modal’s US control plane in us-east-1, regardless of where compute runs, even for European customers. There is no VPC or private networking option, which makes this a blocker for GDPR-governed deployments and difficult for most enterprise security reviews.

Together AI: Security

Together AI holds SOC 2 Type 2 and HIPAA with BAAs, and is GDPR and CCPA compliant. A Zero Data Retention option removes prompt and output data immediately after processing, and EU data residency is available through European infrastructure. They do not hold ISO 27001 certification, and the multi-tenancy architecture for serverless inference is not publicly documented. A VPC deployment option is available via AWS VPC peering, keeping data within the customer’s environment.

Baseten: DE

The developer experience is smooth from the model deployment and inference perspective. They have an OpenAI-compatible Model API, and a “Chains SDK” for multi-model orchestration. Cold starts take 5-10 seconds, which would require some extra warming logic for reducing latency on first responses. Everything is centered around Python.

Nebius: DE

Nebius requires infrastructure skills. It provides managed Kubernetes and managed Slurm, a CLI, Python and Go SDKs over gRPC, and a Terraform provider. There is a marketplace of one-click stacks to help reduce bootstrap time, but wiring a multi-service AI pipeline with persistent storage still requires meaningful infrastructure work. They offer 24/7 support access with dedicated solution architects at no extra charge.

Fireworks AI: DE

Fireworks AI provides a good developer experience. Both the OpenAI and Anthropic SDKs work out of the box. You just need to swap the base URL and API key, and existing code runs on open-source models without any other change. Multi-LoRA is the standout feature for multi-customer deployments. Up to 100 fine-tuned adapters can run on a single base model deployment, enabling per-customer personalization without multiplying GPU costs. Inference is fast, especially with structured outputs. Their FireAttention engine, using FP8 quantization, serves models at roughly 4x the speed of vLLM.

Modal: DE

Depending on what your team is looking for, the developer experience can be good. There is no Docker or YAML. GPU selection, dependencies, and endpoint exposure are all defined in Python. Live reload makes iteration genuinely fast, and GPU memory snapshotting cuts model cold-start time from over a minute down to about 12 seconds. Modal’s decorator API is entirely platform-specific though and does not port anywhere else.

Together AI: DE

Together AI uses an OpenAI-compatible API. Just change the base URL and API key, and existing code runs against 200+ open-source models without modification. Latency is around 0.78 seconds time-to-first-token. This can be compared to faster platforms, like Fireworks AI’s 0.17 seconds. Anywhere from 4 to 13 models are retired per monthly cycle, so there are risks of losing the open-source model you build your app around, but this is common for other platforms as well.

Baseten: Use Case

Baseten is good for Python ML teams looking to deploy custom or open-source models for inference in regulated industries. It’s less of a fit for teams that need experiment tracking, cost predictability without sales negotiations, or sub-second response times (due to 5-10 second cold starts). It’s also not ideal for teams with non-Python stacks.

Nebius: Use Case

Nebius is the right fit for teams running both training and inference on one platform, regulated industries with ISO 27001 or EU data residency requirements, and large-scale always-on workloads where GPU unit economics matter most. It is not a fit for small teams without infrastructure expertise, startups trying to move fast, or polyglot stacks.

Fireworks AI: Use Case

Fireworks is the right fit for teams who want per-customer fine-tuning at scale, need fast structured output, or who want cheaper open-source inference. It is less suited for workloads that require guaranteed latency under sustained high throughput or a full ML platform with experiment tracking.

Modal: Use Case

Modal is the right fit for small teams who need to move fast without infrastructure overhead, and for bursty workloads where per-second billing and scale-to-zero translate into real savings. It is not a fit for teams with European data residency requirements, non-Python stacks, or enterprise buyers who will ask hard questions about network isolation.

Together AI: Use Case

Together AI is the right fit for teams in exploration mode who need to compare many models without committing to infrastructure. The batch API discount makes it particularly strong for offline processing workloads. It is not the right fit for latency-critical real-time applications or enterprise buyers with compliance requirements that go beyond SOC 2 and HIPAA.

To get a better understanding of what it would look like to use these platforms firsthand, I will discuss each one from the perspective of deploying an AI customer support agent using a fine-tuned Llama variant, a standard open-source model through an inference API, a set of markdown reference documentation, and a vector database on each platform. I chose this example application to understand broadly what each platform does well and could improve on.

Baseten

Deploying a fine-tuned Llama model is fully supported with Baseten’s open-source framework, Truss. With a bit of Python, loading custom weights from a checkpoint or Hugging Face path is not an issue, and if I had trained on Baseten, I could have deployed the latest checkpoint directly. Open-source model APIs are also available and can be run next to our fine-tuned model without issue. However, reference documents and vector storage will need to be on a different platform. They provide access to PostgreSQL databases, but Baseten is a supplemental tool rather than a complete infrastructure offering. I think this would be a better fit with faster cold starts (currently 5-10 seconds), transparent enterprise pricing without sales negotiations, and integrated vector storage.

Nebius

Nebius is a more infrastructure-heavy platform than Baseten. Deploying on Nebius is closer to building on AWS than using a managed inference service. If we had fine-tuned our Llama model on Nebius via their Token Factory API, deploying would be a single POST request, making a LoRA adapter queryable through an OpenAI-compatible endpoint. Bringing our own weights, on the other hand, requires us to spin up vLLM on a managed Kubernetes VM with adapter files pulled from Nebius Object Storage. Nebius doesn’t have an orchestration abstraction, and we need an external vector database.

Nebius is the “build it right the first time” choice. It has the strongest compliance portfolio of the group and the best raw GPU economics for always-on workloads, but it would be better for this project if it had managed abstractions to reduce the infrastructure complexity and scale-to-zero capabilities to avoid continuous billing.

Fireworks AI

Deploying models on Fireworks is relatively fast and easy. The serverless embedding API is simple and straightforward, and the fine-tuned Llama adapter uploads cleanly via their firectl command-line interface tool. If I still needed to fine tune my Llambda model, the platform handles SFT training directly. Fine-tuned LoRA models deploy to dedicated on-demand endpoints.

For this app, document storage and vector database deployment needs to be done on a different platform though. Their offering is simple fine tuning and custom model deployment with fast inference. Fireworks AI is the fastest path from OpenAI code to open-source models in production, and Multi-LoRA makes per-customer model personalization economically viable at scale, but like Baseten, it would be better for this application if it had more of an infrastructure offering.

Modal

Deploying on Modal requires building more of the inference stack myself and navigating a higher level of complexity. Deploying our fine-tuned Llama model is possible, but we have to run it inside a Python function with a decorator using the GPU we want. Document storage and vector databases need to come from another platform on Modal as well.

Modal offers a great developer experience with its Python-first approach, but it would be better for this app if document storage and vector databases didn’t need external platforms. I would also prefer if the Modal decorators didn’t cause vendor lock-in by making the code non-portable.

Together AI

Deploying on Together AI involves a moderate level of complexity. I can bring weights trained elsewhere with the CLI, but not as seamlessly as Fireworks. The open-source inference side is no problem at all. Document storage and vector databases need to come from external platforms here as well.

The 200+ models across every modality and the batch API discount make Together AI an attractive choice, but it would be ideal to have document storage and vector databases. It would also be good to have faster than 0.78-second time-to-first-token for our real-time customer support, and I would worry about the risk of model deprecation cycles disrupting the fine-tuned deployment.

The purpose-built AI cloud platform landscape offers compelling alternatives to traditional cloud infrastructure services for AI-specific workloads, but the right choice depends heavily on your team’s priorities.

The tradeoff is that none of these platforms offer the complete infrastructure breadth that some larger platforms have. Most require external solutions for vector databases, document storage, and observability tooling. But for teams focused specifically on model training, fine-tuning, and inference, the reduction in complexity and cost can outweigh the limitations. The key is matching the platform’s strengths to your use case rather than treating any of these single platforms as a universal replacement for your full infrastructure stack.