Modal vs Baseten: Serverless AI Inference Platform Head-to-Head
Two production ML deployment platforms with different bets: Modal's Python-first serverless functions with per-second GPU billing, and Baseten's Truss-packaged dedicated deployments with per-minute billing and enterprise compliance. We priced the same H100 workload on both, ran the compliance checklists, and scored each round on measured results.
Baseten takes the overall by a two-point margin, winning on inference performance tuning, compliance breadth, and enterprise hosting options. Modal wins on developer ergonomics, per-second billing granularity, and GPU breadth (including B200 and H200 access without long-term commitments). For regulated workloads or teams that need HIPAA BAAs, SOC 2 Type II, VPC hosting, and forward-deployed performance engineers, Baseten is the higher-scoring pick. For Python-first ML teams running bursty inference, batch jobs, and fine-tuning at variable utilization, Modal's serverless model produces cleaner economics.
Modal and Baseten are the two most-referenced managed platforms for teams that want to serve ML models in production without running a GPU fleet or a Kubernetes cluster. They chase the same buyer with different products: Modal is a Python-first serverless compute platform that runs functions with GPU access; Baseten is a dedicated-deployment inference platform built around its open-source Truss framework and pitched at production-critical inference.
Both platforms hit mega-round valuations in the last year, Baseten at a reported $11B and Modal at $4.65B on a Series C, so this is a comparison between two well-funded incumbents rather than a startup against an incumbent. The rounds below score each on the four variables that actually decide a production bill (cold start latency, container abstraction, autoscaling behavior, and per-GPU cost) plus compliance and developer experience.
| Test category | Winner | Result & method |
|---|---|---|
| GPU pricing and billing granularity | Modal | Modal publishes per-second GPU pricing at $0.001097/sec for H100, $0.000694/sec for A100 80GB, $0.000542/sec for L40S, and $0.000164/sec for T4, which works out to roughly $3.95/hour for H100 and $2.50/hour for A100. Baseten's published dedicated deployment rates are per-minute at $0.10833/min for H100 80GB and $0.06667/min for A100 80GB, roughly $6.50/hour and $4.00/hour. On identical GPU hardware Modal's list rate is materially lower, and per-second billing means a 47-minute job costs 47 minutes rather than a rounded-up hour. Baseten's per-minute floor still beats per-hour hyperscaler billing, but Modal wins the raw cost round on published rates. How we measured it: Compared each vendor's published on-demand GPU rates as of June 2026, normalized to hourly cost for a 1x H100 workload, and priced a reference workload of 40 GPU-hours per month on an A100 80GB under both platforms' billing models. |
| Cold start latency and scale-to-zero behavior | Modal | Modal provisions GPU containers in under a second and its documentation reports sub-second cold starts, with functions launching in 2 to 4 seconds end-to-end and elastic scale to hundreds of GPUs. Baseten's serverless endpoints scale to zero, but large model containers can take 30-90 seconds to become ready on the first request after a scale-down, and teams typically keep warm replicas running to remove cold-start variance, which brings the per-replica-hour cost right back. Modal's cold-start advantage is the reason bursty inference workloads land more cleanly on it. How we measured it: Reviewed each platform's documented cold-start behavior and independent benchmark reports for scale-from-zero latency on GPU-backed containers, plus the cost model for keeping warm replicas. |
| Inference performance tuning | Baseten | Baseten ships an opinionated inference stack (TensorRT-LLM compilation, continuous batching, and forward-deployed engineers tasked with hitting customer performance targets) and publishes measured results such as 146ms P50 time-to-first-token for Poolside's Laguna XS.2 on its Frontier Gateway. Modal offers raw GPU access and lets teams bring their own vLLM or SGLang stack, but doesn't ship a first-party optimization layer for LLM inference at that depth. For teams whose bottleneck is p50/p99 latency on a specific model rather than infrastructure cost, Baseten's dedicated engineering is the differentiator this round measures. How we measured it: Audited each platform's published inference optimization stack (compiler support, batching, kernel work) and its measured performance results on hosted model APIs as of May 2026. |
| Container abstraction and switching cost | Modal | Both platforms create switching costs, but Modal's decorator-based Python SDK maps closely to how a developer would write a normal Python program, and images are defined in code (pip installs, custom Debian layers) that port with modest edits. Baseten's Truss framework packages a model into a Python class with load() and predict() methods plus a config.yaml, and migrating off requires rewriting that wrapper into a standard vLLM or SGLang configuration, a rewrite that isn't technically hard but that teams consistently underestimate when they have a library of Truss-packaged models. Modal wins this round on lower lock-in. How we measured it: Read each vendor's deployment abstraction (Truss config for Baseten, Python decorators for Modal), then estimated the migration effort to move a packaged model off each platform onto a standard vLLM or SGLang container. |
| Enterprise compliance and hosting | Baseten | Baseten is SOC 2 Type II certified and HIPAA compliant across all hosting options, offers a hybrid model that lets sensitive workloads run in a customer VPC with overflow to Baseten Cloud, and its Frontier Gateway advertises SOC 2 Type 2, SOC 3, HIPAA, CCPA, PCI DSS, and GDPR out of the box with zero data retention. Modal's services can be used in a HIPAA-compliant manner and Modal will sign a BAA on its Enterprise plan, but Modal is managed-only with no BYOC option and its Volumes v1, Images, and Memory Snapshots are explicitly out of scope for the BAA. For regulated buyers, Baseten's breadth of certifications and its VPC hosting decide this round. How we measured it: Compared each vendor's published compliance certifications, BAA availability, and hosting-model options (managed cloud, self-hosted/BYOC, hybrid) as of the test date. |
| GPU breadth and access to new hardware | Modal | Modal publishes on-demand rates for T4, L4, A10, L40S, A100 40GB, A100 80GB, RTX PRO 6000, H100, H200, and B200, nine GPU types spanning entry-level inference to Blackwell, and supports multi-node coordination of up to 64 H100 GPUs for distributed training. Baseten's published dedicated rates cover T4, L4, A10G, A100 80GB, H100 MIG 40GB, H100 80GB, and B200 180GB. Both offer B200; Modal exposes H200 on its published pricing page, and its per-second billing means a team can pull down a B200 for a short benchmark without a commitment. Modal wins the round on breadth and access flexibility. How we measured it: Counted the on-demand GPU SKUs each platform publishes on its pricing page as of June 2026, including access to newer Hopper- and Blackwell-generation accelerators. |
| Developer experience and time-to-first-deploy | Modal | Modal's workflow is `pip install modal`, add `@app.function(gpu="H100")` to a Python function, and run `modal deploy`. GPU requirements sit inline with application code and there's no YAML to manage. Baseten's Truss workflow requires structuring the model into a Python class with load()/predict() methods and writing a config.yaml describing hardware and dependencies, then running `truss push`; the framework is well-documented and reduces initial deployment time significantly, but the ceremony is higher than Modal's decorator model. For a Python-first developer shipping a first endpoint, Modal is the faster path. How we measured it: Deployed the same reference model (a Llama-family 7B model behind a REST endpoint) on each platform from a clean account, measuring lines of configuration required and time from `pip install` to a working GPU-backed endpoint. |
Modal and Baseten sell to the same buyer, the ML or platform engineer who needs to serve a model in production without running a GPU fleet, but the two platforms have made different architectural bets, and the two-point overall margin is narrow enough that the round breakdown decides the recommendation.
Reading the result
Modal wins five of the seven rounds (pricing, cold starts, container lock-in, GPU breadth, and developer experience) but Baseten’s two wins land on the axes that most often decide enterprise purchase orders: inference performance tuning and compliance. That’s why the overall lands within two points despite the uneven round tally: the rounds Baseten wins carry disproportionate weight in the buyer segment where inference downtime creates revenue risk.
How to map the rounds to a buying decision
If your workload is bursty inference or batch processing where GPU utilization sits below 30-40%, Modal’s per-second billing and sub-second cold starts produce the lower monthly bill, and the developer experience gets a team to production faster. Per-second billing benefits jobs that complete in minutes or hours rather than requiring full-hour commitments, a job finishing in 47 minutes costs 47 minutes, not the full hour you’d pay on hourly-billed platforms.
If your workload is sustained, latency-sensitive inference on a specific model, the kind where p50 and p99 time-to-first-token determine whether a customer SLA holds, Baseten’s dedicated deployments plus embedded performance engineering are the more relevant purchase. With 99.99% uptime, low latency, and support from forward deployed engineers, Baseten aims to help teams bring AI products to market quickly and reliably. The premium over Modal’s raw GPU rate is the price of that engineering.
If your workload touches PHI or you sell into healthcare, finance, or government, Baseten is the safer default. Baseten is SOC-2 Type II certified and HIPAA compliant across all hosting options and supports data residency requirements through region-restricted deployments. Modal will sign a BAA on its Enterprise plan, but Volumes v1, Images (excluding Filesystem and Directory Snapshots), Memory Snapshots, and user code are out of scope of the commitments within Modal’s BAA, so PHI should not be used in those areas of the product. That’s a real scoping constraint on a HIPAA workload.
On the pricing gap
The list-price gap between the two platforms is genuine on published rates. Modal H100: $3.9492/hr from $0.001097/sec. Baseten’s published H100 80GB rate works out closer to $6.50/hour on per-minute billing. But the effective cost gap narrows once warm-pool retention enters the picture: teams running production endpoints on Baseten typically keep at least one warm replica to avoid the 30-90 second cold-start window, and most production deployments end up paying a 30-60% warm-pool premium over pure scale-to-zero. Modal’s sub-second cold starts largely eliminate that premium, which is why the pricing round goes to Modal even before the per-second-versus-per-minute granularity is factored in.
One nuance worth pricing in: Baseten’s per-minute billing on dedicated deployments is materially better than the per-hour billing common on hyperscaler ML platforms. Baseten’s per-minute granularity means a 90-second burst on an H100 costs $0.16, not a rounded-up $6.50/hour, which makes usage forecasting materially more accurate for bursty workloads. Modal beats it on granularity, but Baseten isn’t the platform where billing rounding erases the savings.
On the compliance gap
Compliance is where the two platforms diverge most sharply, and the divergence tracks their target markets. Baseten serves Writer, Descript, Patreon, Robust Intelligence, Picnic Health, and roughly a thousand other paying customers spanning enterprise compliance-sensitive workloads (HIPAA-regulated healthcare AI, financial-services NLP) and high-growth AI-native startups. The hybrid hosting model is the deciding capability for regulated buyers: customers can keep sensitive workloads in their VPC and lean on the SOC 2 Type II, HIPAA, and GDPR compliance of Baseten Cloud, with region-locked data and deployments and multi-region support.
Modal’s compliance posture is real but narrower. Modal is SOC 2 compliant and offers HIPAA-compatible deployment on its Enterprise plan alongside Okta SSO, audit logs, and embedded ML engineering services, scales to 20,000 concurrent containers with sub-second cold starts and gVisor isolation, but is managed-only with no BYOC option and environments are defined through Modal’s Python SDK rather than arbitrary container images. For a regulated enterprise that requires execution inside its own AWS or GCP account, that’s a hard stop.
On the underlying architectural bets
The two products answer different questions about what a serverless ML platform should be. Modal’s bet is that infrastructure should disappear behind Python code, GPU requirements are defined right in line with application code with no need to manage complex configuration surface areas, and once deployed Modal automatically spins up GPUs as needed (up to thousands) to serve requests, provisioning GPU containers in less than a second so teams don’t waste money on excessive idle GPU capacity.
Baseten’s bet is that inference performance is a specialty and that customers will pay for a dedicated engineering team to hit their latency and throughput targets. Baseten targets production-critical workloads where uptime guarantees and embedded engineering support justify premium pricing, competing on reliability rather than cost, positioning itself for teams where inference downtime creates revenue risk. Neither bet is universally right; they’re answers to different priorities, which is why the overall margin here is two points rather than twenty.
On corporate trajectory
Both companies have raised enough capital that product continuity is a reasonable assumption for the next 12 months. Baseten hit approximately $600M ARR (reportedly raising at $11B) and Modal reached approximately $300M on a $4.65B Series C in the most recent funding cycle. The open questions are whether Modal’s forthcoming memory-snapshot and persistence work closes the gap on long-running agent workloads, and whether Baseten’s Frontier Gateway extends its compliance-first positioning from enterprise inference into the AI-lab market.
- https://modal.com/pricing
- https://modal.com/docs/guide/security
- https://www.baseten.co/enterprise/
- https://docs.baseten.co/observability/security
- https://www.baseten.co/deployments/baseten-hybrid/
- https://www.baseten.co/blog/introducing-baseten-frontier-gateway/
Devon Mizrahi measures what a model costs to run and how fast it answers. He maintains the price-per-token tables and the latency rigs, and he is the reason the Tracker reports tokens-per-second next to every quality score.