Argonne National Laboratory announced on May 26, 2026 that it has launched what it describes as the first large-scale artificial intelligence inference service for open science. In plain terms, the U.S. Department of Energy lab is now operating a shared service that lets researchers run trained AI models on demand — the way commercial AI platforms serve their users — rather than reserving supercomputer time for each job.
The announcement, published by Argonne (anl.gov), positions the service as a resource for the open-science community, the network of publicly funded researchers whose methods and results are meant to be broadly shared.
Executive Summary
The significance here is less about any single piece of hardware and more about an operating model crossing an institutional boundary. Hyperscalers — the large cloud and AI companies — long ago mastered inference serving: keeping trained models resident and answering requests in real time, at scale, for many simultaneous users. National laboratories, by contrast, have historically run batch systems, where scientists queue jobs and wait their turn. Argonne is now claiming a first: bringing that always-on, request-driven serving model to open science at large scale.
If the service works as described, it changes the day-to-day texture of AI-assisted research. Scientists could embed model calls directly into instruments, workflows, and analysis pipelines instead of scheduling supercomputer allocations for every experiment. It also signals that DOE laboratories intend to be operators of AI infrastructure in their own right, not just consumers of commercial APIs — a stance with real implications for data governance, cost, and scientific reproducibility.
The public announcement is short on specifics, however. As of the release date, key details — the hardware behind the service, which models it serves, who qualifies for access, and how capacity is allocated — are not spelled out in the source available to us, and we flag those gaps below.
From Batch Queues to On-Demand Serving
Supercomputing centers were built around a simple economic logic: the machine is the scarce asset, so users line up for it. Jobs are submitted to a scheduler, wait in a queue, run to completion, and release the hardware. That model suits training runs and simulations that take hours or days. It suits inference badly. Inference — using an already-trained model to answer a question, label an image, or steer an experiment — is bursty, latency-sensitive, and interactive. A researcher who wants a model’s answer in two seconds cannot wait two hours in a queue.
Standing up a dedicated inference service means Argonne is carving out capacity that stays warm and answers requests continuously, which is a genuine architectural and operational departure for a national lab. It requires the disciplines hyperscalers developed over a decade: request routing, autoscaling, multi-tenancy, uptime engineering. The claim of being ‘first at large scale’ in the open-science context is Argonne’s framing, but the underlying shift it describes — labs adopting service-oriented AI operations — is real and consequential.
Why Labs Want Their Own Inference Layer
Commercial AI APIs already exist, so it is fair to ask why a national lab should run its own. Three answers are visible in the structure of the announcement. First, data governance: much scientific data is subject to policies that make shipping it to a commercial endpoint complicated or impossible, and an in-house service keeps sensitive or export-controlled data inside the fence. Second, cost and predictability: at the volumes scientific workflows can generate, metered commercial pricing becomes a research-budget problem, while a shared national resource spreads cost across the community. Third, reproducibility: open science depends on knowing exactly which model, at which version, produced a result — control that is easier to guarantee on infrastructure the community operates itself.
The counterweight is that operating inference infrastructure well is hard, and commercial providers iterate faster than public procurement cycles. Whether a lab-run service can keep pace with frontier commercial offerings — in model quality, tooling, and reliability — is the open competitive question, and the release, as available to us, does not yet provide the evidence to judge it.
The Infrastructure Signal: Inference Is Becoming a Baseload Workload
For the data-center industry, the notable thing is what this says about demand. Training gets the headlines, but inference is the workload that persists after the training run ends — continuous, growing with adoption, and increasingly treated as critical infrastructure. When a national laboratory stands up dedicated large-scale inference capacity, it confirms that inference is no longer an afterthought riding on spare cycles; it is a planned, provisioned workload with its own power, cooling, and availability requirements.
That has knock-on effects for everyone who builds and operates facilities. Inference favors sustained utilization and low-latency proximity to users and instruments, which shapes site selection and network design differently than training campuses do. Public-sector entrants also add a new class of buyer for accelerators and serving software — one whose requirements (openness, auditability, long service lifetimes) differ from the hyperscalers’. Vendors who can meet those requirements gain a market; those optimized purely for commercial serving economics may find the fit imperfect.
Background
Argonne National Laboratory, founded in 1946 and located outside Chicago, is one of the U.S. Department of Energy’s largest science and engineering research centers. Its Argonne Leadership Computing Facility provides supercomputing to researchers nationwide through peer-reviewed allocations, and in recent years the lab has been a focal point of DOE’s push into exascale computing and AI for science, including early testbeds for emerging AI accelerator hardware.
That history matters because national labs have traditionally delivered computing as scheduled batch time on flagship machines. The move to an always-on inference service represents the research-computing world adopting the service-oriented operating model that commercial AI platforms pioneered — a shift several labs have discussed, and which Argonne now claims to be first to deliver at large scale for open science.
Infrastructure investment firm I Squared Capital announced on May 26, 2026 the launch of a new United States data center platform focused on AI inference and edge colocation, backed by a $1 billion capital commitment. The announcement, distributed via Business Wire, positions the platform to serve the fast-growing market for running trained AI models close to users, rather than the massive centralized campuses where those models are built.
Executive Summary
I Squared Capital, a global infrastructure investor with a track record of building digital-infrastructure platforms from the ground up, is committing $1 billion to a US platform aimed at two intertwined markets: AI inference — the compute that answers queries after a model is trained — and edge colocation, meaning smaller data centers positioned in or near population centers where enterprises can rent space and power.
The bet matters because it stakes real capital on a specific view of where the AI buildout goes next. Most headline-grabbing investment to date has chased hyperscale training campuses measured in hundreds of megawatts, sited wherever cheap power exists. An inference-and-edge thesis argues the next wave of demand is distributed: many smaller facilities, closer to users, optimized for low latency and steady utilization rather than raw scale. If that view is right, data-center value will spread across many US metros instead of concentrating in a handful of power-rich regions.
Inference Is a Different Business Than Training
Training a large AI model is a batch job: it can run anywhere power is cheap, and users never interact with it directly. Inference is a service: every chatbot reply, search summary, and copilot suggestion is an inference call, and its economics are governed by latency (how fast a response travels to the user), utilization, and cost per query. That pushes inference capacity toward network-dense locations near people — the historic strength of colocation and edge facilities rather than remote gigawatt campuses.
By naming inference and edge together, I Squared is effectively arguing that the AI market is maturing from build-the-model to serve-the-model. Industry observers have long noted that if AI adoption follows the path of earlier computing waves, ongoing inference spending should eventually dwarf one-time training spending. A platform purpose-built for that phase is a bet on the durable, recurring part of the AI stack.
A Contrarian Read on Data-Center Geography
The prevailing US buildout has concentrated in a few power-abundant corridors — the kind of places where a utility can pledge hundreds of megawatts. Edge colocation inverts that logic: smaller footprints, more sites, and proximity to enterprises and consumers in secondary metros. The trade-off is that edge sites face urban land costs, tighter permitting, and constrained grid connections, but they can command premium pricing for low-latency capacity and are less exposed to the single-market risks of mega-campuses.
For enterprise buyers, a credible national inference-and-edge platform would offer an alternative to shipping every AI workload to a distant hyperscale region — relevant for latency-sensitive applications, data-residency requirements, and hybrid architectures that keep proprietary data close to home. For incumbent colocation providers, it signals a well-capitalized new competitor targeting exactly the niche where regional operators have historically differentiated.
What $1 Billion Buys — and What It Doesn’t
A $1 billion commitment is serious money and, at the same time, a measured entry. In today’s market, a single large hyperscale campus can absorb several billion dollars, so this commitment points toward a portfolio of smaller facilities rather than one flagship — consistent with the edge thesis. Infrastructure funds also routinely amplify equity commitments with project-level debt, so the platform’s ultimate buildout capacity could be a multiple of the headline figure, though the release itself does not say so.
I Squared has used the platform playbook before in digital infrastructure, assembling operating companies around a thesis and scaling them through acquisition and greenfield development. The open question is execution: inference-optimized facilities still need power, cooling for dense GPU racks, and — most importantly — tenants. The announcement describes a commitment and a strategy; converting that into leased, revenue-generating megawatts is a multi-year undertaking in a market where skilled operators, grid interconnection queues, and equipment lead times are all under strain.
Risks: The Edge-Inference Thesis Is Not Yet Settled
It is worth stating plainly that the distributed-inference future this platform anticipates is a forecast, not a fact. Today, a large share of inference still runs in the same hyperscale regions as training, because cloud providers concentrate their GPU fleets there and many applications tolerate tens of milliseconds of extra latency. If model efficiency improves faster than demand grows, or if hyperscalers simply extend their own regions closer to users, the addressable market for independent edge inference capacity could prove smaller than proponents expect.
None of that makes the bet unreasonable — infrastructure investing is precisely about positioning capital ahead of demand. But buyers and competitors evaluating this announcement should weigh that the release, as reported, substantiates a commitment and a strategy rather than contracted customers or operating assets.
Background
I Squared Capital is an independent infrastructure investment firm founded in 2012 and headquartered in Miami, managing capital across energy, utilities, transport, and digital infrastructure worldwide. In digital infrastructure specifically, the firm has favored a platform model — creating or acquiring an operating company around an investment thesis, then scaling it through greenfield development and bolt-on acquisitions, including prior edge data-center investments in Europe.
The announcement lands amid an unprecedented US data-center expansion driven by AI. Most capital to date has flowed to hyperscale training campuses in power-rich regions, but a growing school of thought holds that as AI applications reach mass adoption, the serving side — inference — will demand distributed, network-proximate capacity, reviving the strategic value of edge and metro colocation.
Infrastructure investor I Squared Capital has agreed to acquire data center assets from Cogent Communications for $225 million, according to a Reuters report dated May 25, 2026. The purchase anchors a new data center platform — reported at roughly $1 billion — that I Squared is positioning around artificial-intelligence inference, the day-to-day serving of AI models to users rather than the training of them.
Executive Summary
The transaction pairs a specific asset purchase with a bigger strategic wager. I Squared, a private-equity firm that specializes in infrastructure — roads, energy, and increasingly digital assets — is paying $225 million for facilities Cogent had been carrying on its books, and is using them as the foundation of a platform sized in press coverage at around $1 billion. The stated thesis is AI inference: the compute that answers queries, generates content, and runs AI features inside applications, which tends to sit closer to end users than the massive training campuses built by hyperscale cloud providers.
For Cogent, a company best known as a low-cost internet backbone and transit provider, the sale converts long-marketed real estate into cash. For the broader market, it is a data point that institutional capital now sees a distinct, investable asset class in smaller, distributed colocation sites — not just in the gigawatt-scale campuses that have dominated AI headlines. Whether inference demand materializes at these locations on the timeline investors hope is the open question the deal leaves unanswered.
Inference Is a Different Business Than Training
Most AI data center investment to date has chased training: enormous, power-hungry campuses where models are built, often in remote locations chosen for cheap land and available electricity. Inference — running the finished model every time a user asks a question — has a different profile. It is latency-sensitive, scales with user traffic rather than with model size, and in many architectures benefits from being distributed across metros closer to population centers. That is the logic behind putting inference capacity into smaller, geographically scattered facilities of the kind changing hands here.
The economics are also different. Training clusters are typically leased wholesale by a handful of very large tenants; inference capacity can, in principle, be sold in smaller increments to a broader customer base, which looks more like traditional retail colocation — renting secure, powered space to many customers. If that market develops, operators of distributed sites gain pricing power they have not had in years. If inference instead consolidates inside the hyperscalers’ own clouds, the thesis weakens. The release, as reported, does not settle which way demand is actually breaking.
A Payday for Cogent’s Conversion Thesis
Cogent acquired Sprint’s legacy wireline business from T-Mobile in 2023, a deal that brought with it a large portfolio of former telephone switching facilities across the United States. Management has spent the years since arguing that these buildings — hardened structures with existing power feeds and fiber connectivity — could be converted into sellable or leasable data centers. Skeptics noted that carrier hotels built for 1990s telecom gear are not automatically suited to modern high-density computing, and that monetization was slow to show up in reported results.
A $225 million sale to a sophisticated infrastructure buyer is the most concrete external validation of that thesis to date, though one transaction does not price the whole portfolio. It is worth being precise about what the deal does and does not prove: it shows a willing buyer at a real price for some assets, but the report does not disclose how many facilities are included, their capacity, or their condition — so extrapolating a value for Cogent’s remaining sites from this headline number would be premature.
Private Capital Moves Down-Market
I Squared’s entry continues a pattern of infrastructure funds treating digital assets — fiber, towers, and data centers — as core holdings alongside energy and transport. What is notable is the segment: rather than bidding on trophy hyperscale campuses, where competition from sovereign wealth funds and mega-funds has compressed returns, this platform targets the fragmented middle of the market. A reported $1 billion platform commitment suggests the firm intends to aggregate and upgrade additional sites, not simply hold what it bought.
The risks are equally clear. Retrofitting older facilities for AI-grade power density and cooling is capital-intensive, utility interconnection queues are long in many metros, and the platform will be competing for tenants against established colocation providers with existing sales channels and ecosystems. The strategy’s success likely depends less on the entry price than on execution: securing power upgrades, landing anchor customers, and timing capacity to a demand curve that remains genuinely uncertain.
Background
Cogent Communications built its business as an aggressive price competitor in internet transit, operating a global fiber backbone. Its 2023 acquisition of Sprint’s wireline business from T-Mobile brought hundreds of former telephone switching sites, and management has since pitched their conversion into data centers as a major source of untapped value — a claim the market has watched for proof in the form of actual sales or leases.
I Squared Capital is part of a wave of infrastructure private equity that has moved decisively into digital assets over the past decade, on the view that data centers, fiber, and towers offer the long-lived, contracted cash flows these funds seek. The AI boom has intensified that interest, first in massive training campuses and now, as this deal suggests, in the distributed facilities that may serve AI inference closer to end users.
Data Center Knowledge reported on May 23, 2026, that AI inference — the day-to-day serving of trained AI models to end users — is pulling infrastructure investment back toward metro data centers, reversing years of momentum toward remote hyperscale campuses. The driver, per the report’s framing, is latency: inference workloads live and die by response time, and response time is a function of physical distance to users.
Executive Summary
The trade publication’s thesis is straightforward: the AI buildout’s first act was dominated by training — the compute-intensive process of creating models — which rewarded remote sites with cheap land and abundant power, because training does not care where it runs. The second act is inference, the phase where those models actually answer queries for businesses and consumers, and inference is latency-sensitive in a way training never was.
If the thesis holds, it matters for nearly everyone in the infrastructure value chain. Metro colocation operators, carrier hotels, and interconnection-rich urban facilities — assets many analysts treated as yesterday’s story during the gigawatt-campus land rush — would regain strategic relevance. Site-selection criteria, capital allocation, and power procurement strategies would all tilt back toward proximity to population centers, precisely where power and real estate are scarcest.
Training Built the Campuses; Inference Pays the Bills
Training and inference are economically different animals. Training is a batch job: it runs for weeks or months, consumes enormous power, and produces a model. Because no end user is waiting on it in real time, operators could chase the cheapest available megawatt — which pushed campuses into rural and exurban regions with land, transmission access, and accommodating utilities. Inference is the opposite: it is the recurring, revenue-generating workload, triggered every time a user prompts a chatbot, a copilot drafts an email, or an application calls a model behind the scenes.
As AI products mature from demos into production services, the share of total AI compute devoted to inference grows structurally. That shifts the industry’s center of gravity from “where is power cheapest?” to “where are the users?” — a question metro data centers were built to answer. The report’s framing suggests the market is beginning to price this in.
Why Latency Is Redrawing the Map
Latency — the delay between a request and its response — is bounded by physics. Data cannot travel faster than light through fiber, and every additional kilometer between user and server adds round-trip time. For a monthly batch job, that is irrelevant. For an interactive AI assistant, a fraud-check API, or a voice agent, tens of milliseconds are perceptible and, at scale, commercially meaningful.
Newer AI application patterns compound the effect. Agentic and multi-step systems chain many model calls together to complete a single task, so per-call latency multiplies. Retrieval-augmented applications shuttle data between models and enterprise systems that already live in metro colocation facilities. Placing inference capacity near users and near enterprise data reduces both delay and data-transit cost — a pull toward the very urban markets the hyperscale era had de-emphasized.
Winners, Losers, and the Assets in Between
The clearest beneficiaries of a metro revival would be operators holding interconnection-dense urban facilities: carrier hotels, established colocation campuses in major metros, and providers with existing utility relationships in constrained markets. Those assets are hard to replicate — urban land, fiber density, and grid connections accumulate over decades. Enterprises also stand to gain optionality, since inference capacity near their existing colocation footprints simplifies hybrid architectures.
This is not, however, a zero-sum reversal. Remote hyperscale campuses remain essential for training and for latency-tolerant inference, and the report’s headline says infrastructure is being pulled “back into” metros, not out of the hinterlands. The more defensible reading is bifurcation: a two-tier geography where massive remote campuses handle training and batch work while a distributed metro layer serves real-time inference. The open question is how capital gets split between the tiers — and whether metro grids can absorb their share.
The Constraint That Follows the Workload: Power
The uncomfortable irony is that inference demand is heading toward the places least prepared to power it. Major metros already contend with constrained grids, long interconnection queues, and community resistance to new data center construction. AI inference hardware, while less power-dense per site than a training cluster, still pushes rack densities well beyond what many legacy urban facilities were engineered for, often requiring liquid cooling retrofits and electrical upgrades.
That constraint cuts both ways. It limits how fast the metro shift can happen, but it also makes existing permitted, powered metro capacity more valuable — scarcity is a landlord’s friend. Expect the competition for metro megawatts, substation capacity, and retrofittable urban shells to intensify if the trend the report describes continues.
Background
Data center geography has swung on a pendulum for two decades. The early internet clustered compute in urban carrier hotels where networks met; the cloud era then pushed capacity outward to remote regions where land and power were cheap, and the AI training boom of the mid-2020s accelerated that outward push into multi-hundred-megawatt and gigawatt-scale campuses.
Data Center Knowledge, the source of this report, is a long-running trade publication covering the data center industry. Its May 2026 piece captures a question the industry has been circling as AI products move from development into production: once models are built, the economics of serving them — inference — may favor a very different map than the one training drew.
Modal Labs, a startup that provides serverless infrastructure for artificial-intelligence workloads, has closed a $355 million funding round, as reported by SiliconANGLE on May 22, 2026. The round ranks among the larger financings to date for the emerging category of companies that let developers run GPU-powered AI code without managing the underlying servers.
Executive Summary
The announcement is straightforward: Modal Labs has secured $355 million in new funding. What makes it worth attention is the category it validates. “Serverless” computing means developers submit code and pay only for the seconds it actually runs, while the provider handles provisioning, scaling, and scheduling of the machines underneath. Applying that model to GPUs — the expensive, supply-constrained accelerator chips that power AI training and inference — is a harder engineering problem than classic serverless, and until recently most AI teams simply rented GPU servers by the month and absorbed the idle time.
A round of this size suggests investors believe the orchestration layer — the software that decides which workload runs on which GPU, and when — is becoming its own durable tier of the AI infrastructure stack, sitting between raw compute providers and the applications built on top. For data-center operators, GPU cloud providers, and enterprise buyers, that thesis has real implications for how AI capacity gets bought, priced, and utilized.
The Economics of Idle Silicon
The core problem serverless GPU platforms attack is utilization. High-end AI accelerators are among the most expensive line items in modern computing, and a GPU reserved around the clock but busy only a fraction of the time is capital burning quietly. Inference workloads — running a trained model to answer live requests — are especially bursty: traffic spikes and lulls make fixed reservations wasteful. A platform that pools GPUs across many customers and bills per second of actual execution converts that stranded capacity into revenue, and converts a customer’s fixed cost into a variable one.
That is the same economic argument that made serverless computing successful for ordinary CPU workloads a decade ago. The difference is difficulty: AI models can take tens of gigabytes of memory and long seconds to load, so starting them on demand — the “cold start” problem — requires genuine systems engineering. Solving it well is the moat companies in this category are selling, and a $355 million round indicates at least some investors believe the moat is real.
A New Layer Between the Chips and the Apps
The AI infrastructure stack has been visibly stratifying: chipmakers at the bottom; hyperscale clouds and specialist GPU cloud providers renting raw capacity; and application companies at the top. Orchestration platforms like Modal occupy the middle — they typically do not fabricate chips or, primarily, build data centers, but abstract other people’s hardware behind a developer-friendly interface. The bet embedded in this funding round is that the middle layer captures durable value, much as earlier developer-platform companies did atop the big clouds.
If the bet pays off, the winners include developers, who get cloud-like elasticity for AI; and, arguably, the upstream capacity providers, who gain a demand aggregator that keeps their fleets busy. The pressure lands on undifferentiated GPU rental businesses, because an orchestration layer that can shift workloads across suppliers commoditizes the raw compute beneath it.
The Risks the Category Still Carries
None of this is guaranteed. The largest cloud providers already offer their own serverless and managed inference products and can bundle them with existing enterprise agreements, so an independent orchestration layer must stay meaningfully better to justify its place. The category also depends on continued access to scarce accelerators at workable prices — a middle layer inherits the supply risk of its suppliers without controlling it. And the industry’s broader trajectory matters: if AI spending growth moderates, richly funded infrastructure startups will be judged on gross margins and retention rather than category narrative. The announcement, as reported, does not include the financial detail needed to assess Modal’s position on those measures, so the size of the round should be read as investor conviction, not as public evidence of unit economics.
Background
Modal Labs emerged in the early 2020s among a wave of startups rethinking developer infrastructure for the AI era, founded by engineers with backgrounds in large-scale data systems. Its platform focused on a specific technical wedge: making heavyweight AI workloads start in seconds inside a serverless model, so developers could treat GPUs the way earlier serverless products let them treat ordinary compute. The company raised conventional venture rounds before this financing and grew alongside the post-2022 boom in generative AI, which turned GPU capacity into one of the technology industry’s scarcest and most expensive resources.
That scarcity reshaped the infrastructure market it operates in. Hyperscale clouds, specialist GPU cloud providers, and a growing middle tier of orchestration and inference platforms now compete to serve AI developers, and utilization — how much of an expensive accelerator’s time is spent doing paid work — has become the economic metric the whole category is organized around.
CoreWeave, the GPU-focused AI cloud provider, announced support for Red Hat AI Inference Server on CoreWeave Kubernetes Service (CKS), its managed Kubernetes offering. The announcement, dated May 13, 2026, positions the pairing as an enabler of hybrid inference — running AI model-serving workloads consistently across CoreWeave’s cloud and other environments, such as enterprise data centers.
Executive Summary
The announcement joins two complementary layers of the AI stack. CoreWeave supplies large-scale GPU capacity delivered through CKS, its Kubernetes-based orchestration service; Red Hat supplies the inference-serving software layer — Red Hat AI Inference Server, an enterprise-supported model-serving platform built on the open-source vLLM project, a widely used engine for running large language models efficiently on GPUs. Together they aim at enterprises that want one consistent way to deploy and operate AI models wherever the workload runs.
It matters because the AI cloud market is shifting its center of gravity from training — the one-time, compute-intensive process of building models — to inference, the ongoing work of serving those models to users. Inference is where recurring revenue lives, and where enterprises face real portability questions: models trained in one place often need to run in another for latency, data-residency, or cost reasons. A hybrid inference story, if delivered, addresses exactly that friction — though the source release offers few specifics on how, when, or at what price.
Inference Is Where AI Clouds Will Be Judged Next
Training frontier models is a market with a handful of very large buyers. Inference is the opposite: every enterprise that deploys an AI application becomes an inference customer, and the spending recurs for as long as the application runs. For a specialized GPU cloud like CoreWeave — whose growth to date has leaned heavily on large training and capacity contracts with a concentrated set of customers — building a credible inference franchise is a route to broader, stickier, more diversified demand. Supporting an enterprise-standard serving layer on CKS is a logical step in that direction.
The competitive backdrop is that raw GPU access is commoditizing. Hyperscalers, neoclouds, and sovereign providers all sell similar silicon. Differentiation is migrating up the stack to orchestration, serving efficiency, and operational tooling — precisely the layer this announcement targets. An inference server matters economically because serving efficiency (how many tokens a GPU produces per dollar) directly sets gross margin for both the provider and the customer; vLLM, the engine underneath Red Hat’s product, exists specifically to raise that efficiency.
What Each Side Gets From the Pairing
For CoreWeave, Red Hat brings enterprise legitimacy. Red Hat — the open-source software company IBM acquired in 2019 — is already inside most large enterprises via Red Hat Enterprise Linux and OpenShift, and its support model is familiar to conservative IT buyers. Certifying Red Hat’s inference stack on CKS lowers the perceived risk of moving regulated or mission-critical inference workloads onto a young cloud provider, and lets CoreWeave sell to platform-engineering teams in language they already speak: Kubernetes, operators, supported software lifecycles.
For Red Hat, CoreWeave is distribution into the fastest-growing tier of GPU capacity. Red Hat’s AI strategy depends on its serving layer running everywhere customers have accelerators — on-premises, on hyperscalers, and on specialized AI clouds. Each certified venue strengthens its pitch that the inference layer, not the underlying cloud, is the portable standard. Notably, that pitch cuts both ways for CoreWeave: a genuinely portable serving layer makes it easier for customers to arrive, but also easier to leave.
Hybrid Inference: Real Need, Unproven Delivery
The hybrid framing responds to a genuine enterprise constraint. Latency-sensitive applications, data-residency rules, and existing data-center investments mean many organizations will run inference in several places at once. A consistent Kubernetes-plus-inference-server substrate across those venues would reduce duplicated engineering and make capacity fungible — burst to the cloud when demand spikes, serve locally when regulation requires it.
What the announcement does not yet substantiate is the hard part. Hybrid operation lives or dies on details the source leaves out: unified model registries and observability across sites, network paths between customer premises and CoreWeave regions, consistent GPU support matrices, and commercial terms that don’t penalize moving workloads. Until reference customers describe production hybrid deployments, this is a credible roadmap claim rather than a demonstrated capability — a caution that applies equally to every vendor currently marketing ‘hybrid AI.’
Background
CoreWeave began as a cryptocurrency-mining operation before pivoting into GPU cloud computing, and rose to prominence during the generative-AI boom as one of the largest independent providers of NVIDIA-based capacity, completing its Nasdaq IPO in March 2025. Its early revenue skewed toward very large training and capacity deals, making expansion into broader enterprise inference a recurring strategic theme. Red Hat, IBM’s open-source software arm since a $34 billion acquisition in 2019, has built its AI portfolio around portable, supported open-source layers — including inference serving based on the vLLM project — that run across on-premises and cloud infrastructure. The two companies’ stacks meet naturally at Kubernetes, the open-source container-orchestration standard both build upon.
On May 11, 2026, technology analyst Ben Thompson published an essay on his influential Stratechery newsletter titled “The Inference Shift,” arguing that the economic center of gravity in artificial intelligence is moving from training — the one-time, compute-intensive process of building a model — to inference, the ongoing work of running that model every time a user asks it a question.
Thompson’s framing matters because Stratechery is widely read by technology executives and investors, and because the training-versus-inference balance directly shapes where the next wave of infrastructure spending — chips, data centers, power, and networks — actually lands.
Executive Summary
The essay’s core contention, as its title signals, is that the AI buildout’s defining workload is changing. Training a frontier model is a bounded project: enormous, but finite, concentrated in a handful of massive facilities run by a handful of well-capitalized labs. Inference is different in kind. It scales with usage — every chatbot session, coding assistant, and AI-powered search query consumes compute — so as AI products find real adoption, serving them becomes a continuous, growing operating cost rather than a one-time capital project.
For infrastructure providers, that distinction is not academic. Training demand rewards maximum-density campuses wherever cheap power and land exist, with latency largely irrelevant. Inference demand rewards something closer to the traditional internet: capacity distributed nearer to users, resilient connectivity, and economics measured in cost per query rather than cost per training run.
Because the full essay sits behind Stratechery’s subscription, this analysis works from the thesis itself — the shift from training to inference economics — rather than from the piece’s specific figures or examples, and examines what that shift would re-rank across the infrastructure landscape.
Two Very Different Kinds of Compute Demand
Training and inference stress infrastructure in almost opposite ways. Training jobs run for weeks or months across thousands of tightly interconnected accelerators, which pushes builders toward gigantic single-site campuses where power is cheap and abundant — remoteness is a feature, not a bug. Inference workloads are short, bursty, and user-facing: a response has to come back in a second or two, which puts a premium on proximity to population centers, redundancy, and network quality.
The economics diverge just as sharply. Training is capital expenditure that a company chooses to make; it can be deferred, right-sized, or cancelled. Inference is tied to revenue-generating usage — if customers are querying your model, you must serve them, and your margins depend on how cheaply you can do it. A market organized around inference is one where efficiency per query, not raw peak capacity, becomes the competitive battleground.
What Gets Re-Ranked in Infrastructure Demand
If Thompson’s thesis holds, several categories of infrastructure move up the priority list. Metro and regional data centers — including colocation capacity near enterprise users — regain relevance after a period in which headlines were dominated by remote gigawatt-scale training campuses. Connectivity providers benefit, because distributed inference multiplies traffic between users, edge sites, and core facilities. Power demand becomes more geographically dispersed and steadier in profile, a different planning problem for utilities than a handful of enormous point loads.
The chip layer re-ranks too. Training has been dominated by the most powerful general-purpose GPUs, where flexibility justifies premium pricing. Inference, being a more predictable and repetitive workload, is friendlier to specialized silicon and to cost-optimized accelerators — which is precisely why cloud providers have invested in custom inference chips and why competition at this layer is more open than in training hardware.
Winners, Losers, and the Margin Question
The clearest beneficiaries of an inference-led market are operators with distributed footprints, strong interconnection, and the ability to sell capacity in smaller, latency-sensitive increments — along with any vendor that reduces cost per query, from silicon designers to cooling and power-efficiency specialists. The more exposed parties are those whose plans assume training demand grows indefinitely on its current trajectory: single-tenant mega-campuses purpose-built for one lab’s training runs carry concentration risk if that lab’s training appetite plateaus while its serving needs move elsewhere.
There is also a margin story embedded in the shift. When inference is the dominant cost, AI application companies face a squeeze between what users pay and what serving costs — which pressures them to negotiate hard with infrastructure suppliers, adopt cheaper hardware, and shrink models where quality allows. Infrastructure revenue may keep growing, but the pricing power within the stack could redistribute.
Reasons for Caution
The thesis has honest counterarguments, and they deserve equal scrutiny. Frontier labs continue to spend heavily on training, and newer techniques that make models “think longer” at answer time blur the line — they raise inference costs, supporting the thesis, but also keep demand for dense, training-class hardware high. It is also possible that both curves rise together, in which case “shift” overstates a rebalancing. And headline-level analysis of a subscription essay cannot verify which evidence Thompson marshals; readers should treat the thesis as a framework to test against disclosed capital-spending and usage data, not as settled fact.
Background
Stratechery, founded by Ben Thompson in 2013, is a subscription publication analyzing the strategy and economics of the technology industry, and it has been one of the more influential independent voices in debates over the AI buildout. The training-versus-inference question it takes up here has become central to that buildout: the industry’s first phase was defined by a race to train ever-larger foundation models, concentrating spending on top-end GPUs and massive single-site campuses.
As AI products have moved from demos to daily tools, attention has turned to the cost of actually serving them at scale. Cloud providers have developed custom inference chips, model developers have released smaller and cheaper model variants, and newer ‘reasoning’ models that consume extra compute per answer have pushed inference costs up further — all of which forms the backdrop against which Thompson’s May 2026 essay lands.
CoreWeave, the specialized AI cloud provider, announced on May 10, 2026 that it ranked first on Artificial Analysis’s public benchmark for serving the Kimi K2.6 large language model. The claim was published on the company’s own editorial blog, citing the independent third-party leaderboard as the source of the ranking.
Executive Summary
Artificial Analysis is a widely cited independent site that measures how AI cloud providers serve popular open-weight models, tracking metrics such as tokens produced per second, time-to-first-token latency, and price per million tokens. Topping one of its per-model leaderboards is a marketing and sales asset in the increasingly crowded market for GPU-backed inference, where dozens of providers now compete to host the same underlying model.
For CoreWeave, the ranking on Kimi K2.6 — a large model released by Chinese lab Moonshot AI — reinforces the company’s positioning as an inference-performance leader, not just a supplier of raw GPU capacity. The result matters because inference workloads, which run trained models in production, are becoming a larger share of AI cloud spending than the one-time training runs that first defined the market.
Why a Single Benchmark Win Actually Matters
Inference performance is not an abstract engineering metric. Every additional token per second a provider can squeeze out of the same GPU translates directly into lower cost per query and better user experience for downstream applications like chatbots, coding assistants, and agentic systems. A leaderboard-topping result on a widely followed public benchmark gives buyers a shorthand to compare providers without running their own tests, which shortens sales cycles for the winner.
That said, a benchmark victory is a snapshot on one model at one moment. Providers tune their deployments aggressively for popular tested configurations, and rankings shift as software stacks, batching strategies, and hardware allocations change. The commercial value of the win depends on whether CoreWeave can sustain the position across the models customers actually run in production.
The Inference Cloud Land Grab
The market for serving open-weight models has become a genuine competitive arena. CoreWeave sits alongside a growing roster that includes Together AI, Fireworks, Groq, SambaNova, Lambda, and the hyperscalers’ own inference endpoints. Each is chasing the same buyer: developers and enterprises who want to run models like Llama, DeepSeek, Qwen, and now Kimi without operating their own GPU fleet.
Differentiation in this market is thin. Everyone has access to broadly similar hardware, and the underlying model weights are identical across providers. That leaves the software layer — kernel optimizations, speculative decoding, KV-cache management, request routing — as the primary lever. Independent benchmarks like Artificial Analysis are one of the few places where those software investments become visible to buyers.
Kimi K2.6 and the Broadening Model Landscape
Kimi K2 is a family of large models from Moonshot AI, a Beijing-based lab. Its inclusion on Western inference benchmarks reflects the fact that competitive open-weight models increasingly originate from Chinese labs, alongside DeepSeek and Qwen. Providers that move quickly to host new releases can capture early demand from developers evaluating alternatives to closed models from OpenAI and Anthropic.
For infrastructure buyers, the practical read is that model provenance is decoupling from serving provider. A US-based enterprise can now run a Chinese-origin open-weight model on a US inference cloud, avoiding data-residency concerns tied to using the model developer’s own API. CoreWeave’s Kimi K2.6 result is one data point in that broader unbundling.
Background
CoreWeave started as a cryptocurrency mining operation before pivoting to become a GPU-focused cloud provider serving AI, visual effects, and other accelerated-compute workloads. Its rapid scale-up during the generative AI wave made it one of the most-discussed alternatives to the traditional hyperscalers for AI compute, with a customer roster that has included major model labs.
The inference segment where this benchmark result sits has emerged as a distinct competitive market, separate from long-running model training contracts. Independent benchmarking sites such as Artificial Analysis have grown in influence as buyers seek neutral comparisons across a growing roster of providers hosting the same open-weight models.
On May 7, 2026, CNBC reported that shares of Akamai Technologies surged roughly 20% after the company posted quarterly earnings and disclosed a $1.8 billion AI infrastructure deal. The headline pairing — an earnings beat narrative and a large AI-branded contract — was enough to produce one of the stock’s sharpest single-day moves in years.
Details of the deal itself, including the customer, the contract length, and how the $1.8 billion figure is measured, were not spelled out in the report summary, making the market reaction as notable as the disclosed facts.
Executive Summary
Akamai, best known as the company that pioneered the content delivery network (CDN) — the globally distributed layer of servers that speeds up websites and video by caching content close to users — is now being valued, at least for a day, as an AI infrastructure company. A $1.8 billion deal figure attached to AI infrastructure is large by Akamai’s historical contract standards, and the ~20% share-price response suggests investors see it as evidence of a genuine second act rather than a one-off.
The strategic significance is bigger than one contract. AI ‘inference’ — the work of running an already-trained model to answer queries, as opposed to the massive centralized job of training it — is widely expected to become the dominant, recurring cost of AI. Inference rewards low latency and proximity to users, which is precisely the asset CDN operators have spent decades building. This deal is an early, dollar-denominated data point for the thesis that edge networks can capture a meaningful slice of AI spending long dominated by hyperscale cloud providers and GPU ‘neocloud’ specialists.
That said, the public record here is thin: a headline number, a stock move, and an earnings print. What the deal actually obligates, over what period, and at what margin remains unstated — and those details determine whether this is a turning point or a well-timed press moment.
From Cache to Compute: A Second Act Decades in the Making
Akamai has reinvented itself before. Founded in 1998 out of MIT to solve web congestion, it built one of the world’s most distributed server networks, then layered a substantial security business on top of it, and in 2022 acquired cloud provider Linode to add general-purpose computing. The through-line is a single physical asset: thousands of points of presence wired close to end users. An AI inference business is the logical next tenant for that real estate — the servers change from caching video to running models, but the geographic advantage is the same.
The strategic question has always been whether that advantage is monetizable at scale, or whether AI spending would remain concentrated in a handful of giant centralized data centers. A $1.8 billion figure — if it represents committed customer revenue — would be the strongest public evidence yet that at least one large buyer believes distributed inference is worth paying for. The market’s 20% re-rating says investors are willing to extend that belief to the whole franchise.
Why Inference Economics Could Favor Distributed Networks
Training a frontier AI model is a centralized, power-hungry project measured in gigawatts and months. Inference is the opposite: billions of small, latency-sensitive requests arriving from everywhere, all day, forever. For chatbots, voice agents, translation, fraud scoring, and video analysis, shaving tens of milliseconds by serving the request near the user materially improves the product. That is the same physics that made CDNs valuable, and it is why edge operators argue the inference market will fragment geographically even as training consolidates.
There is also a cost argument. Inference does not always need the newest, scarcest GPUs; a distributed fleet of mid-range accelerators running close to demand can undercut centralized capacity that carries hyperscaler margins and long-haul network costs. If Akamai can fill its existing footprint with inference workloads, the incremental economics could be attractive — the network, facilities, and customer relationships are already paid for. The unproven part is utilization: an inference fleet only earns those economics if demand actually shows up across hundreds of locations rather than pooling in a few metros.
What $1.8 Billion Does — and Does Not — Tell Us
Headline contract values in infrastructure deserve scrutiny regardless of who announces them. A $1.8 billion deal could be a multi-year total contract value recognized over five or more years, a capacity reservation with usage-based true-ups, or something structured differently — each implies a very different annual revenue impact for a company of Akamai’s size. The reporting available at publication does not say which, nor does it identify the customer, and a deal this large is by definition concentrated: one counterparty’s fortunes and renewal decision matter enormously.
The same even-handedness applies to the skeptics’ case. A 20% single-day move on a deal without disclosed terms can look like AI-headline enthusiasm — but it coincided with an earnings report, so the market was plausibly repricing the whole business, not just one contract. The honest reading as of May 7, 2026: the deal is a substantiated, material fact; the interpretation that edge players are now structural winners in AI is a reasonable thesis this deal supports but does not yet prove.
Competitive Ripples: Hyperscalers, Neoclouds, and the Rest of the Edge
If distributed inference contracts of this size become repeatable, several markets shift. Hyperscale clouds (AWS, Microsoft Azure, Google Cloud) would face price and latency competition at the edge of the network they largely ceded to CDNs. GPU neoclouds — specialists that rent raw AI compute — would face a rival that bundles compute with a global delivery and security network. And Akamai’s CDN peers, along with data center operators with many small regional facilities, gain a template: the deal implicitly re-prices every well-distributed footprint as potential AI infrastructure.
For enterprise buyers, more credible suppliers is straightforwardly good news — inference pricing has been set in a sellers’ market. The caveat is execution risk: operating AI infrastructure at the edge means securing accelerator supply, power, and cooling across many sites, disciplines where hyperscalers have a decade of hard-won scar tissue. Winning the deal is the beginning of that test, not the end.
Background
Akamai Technologies was founded in 1998 by MIT researchers to solve early-web congestion and grew into the archetypal content delivery network, at one point carrying a substantial share of global web traffic across tens of thousands of distributed servers. As CDN pricing commoditized through the 2010s, Akamai diversified into web and API security, which became a major revenue pillar, and then into cloud computing with its 2022 acquisition of developer-favorite Linode.
The AI boom initially concentrated infrastructure spending in massive centralized training campuses built by hyperscalers and GPU specialists. By 2025–2026, attention was shifting toward inference — the ongoing cost of actually serving AI to users — reopening the question of whether distributed, latency-optimized networks would claim a structural role in AI economics. Akamai’s May 2026 deal disclosure landed squarely in that debate.
Cerebras Systems announced on 6 May 2026 that it is making inference on Kimi K2.6 — a trillion-parameter-class large language model from Moonshot AI — available to enterprise customers on its wafer-scale hardware. The announcement positions Cerebras as a route for companies that want to run a frontier-scale open-weight model without assembling their own GPU fleet.
The material available with the announcement is essentially the headline claim. Cerebras has not published, in the source reviewed here, the pricing, sustained throughput, context length, regional availability or capacity commitments that would let a buyer compare the offer directly against GPU-based inference providers.
Executive Summary
The substance of the news is straightforward: a specialist silicon vendor is putting a very large open-weight model in front of enterprise buyers on its own accelerators. The strategic question underneath it is larger. For most of the current AI build-out, the marginal dollar went into training — the one-time, capital-heavy process of creating a model. Spending is now shifting toward inference, the repeated act of running that model to answer requests, which behaves less like a construction project and more like a utility with a per-token meter attached.
That shift changes which hardware properties matter. Training rewards raw arithmetic throughput across enormous clusters. Generating text one token at a time rewards something different: how fast a machine can move model weights to its compute units. Cerebras builds a processor the size of an entire silicon wafer and keeps weights in fast on-chip memory rather than in the off-chip high-bandwidth memory GPUs rely on, an architecture aimed squarely at that bottleneck.
Whether that translates into better economics — not just faster demos — is unresolved by this announcement. Speed per token and cost per token are different metrics, and a trillion-parameter model stresses memory capacity in a way that cuts against wafer-scale’s main advantage. Enterprises evaluating the offer should treat it as a credible architectural bet that has not yet been priced in public.
Inference Is Becoming the Data Center’s Recurring Bill
Training a frontier model is a project: it has a start date, a budget and an end. Inference is an operating expense that scales with usage and never stops. As enterprises move AI features from pilots into products, the cost centre migrates from the training run to the serving fleet, and the buying criteria migrate with it — from peak cluster performance to cost per million tokens, tail latency and the ability to hold capacity when demand spikes.
This matters for the reasoning and agentic workloads enterprises are now deploying. A model that thinks step by step before answering emits a long chain of intermediate tokens the user never sees. If generation runs at a modest rate, a query that produces thousands of hidden tokens becomes a wait measured in tens of seconds — which rules out interactive use. Token generation speed stops being a benchmark curiosity and becomes the difference between a product and a demo.
That is the market Cerebras is aiming at, and it is a defensible one. It is also a narrower claim than it first appears: being fastest at generating tokens does not automatically mean being cheapest, because cost depends on how many concurrent requests a system can serve while staying fast. The announcement does not address that trade-off.
The Wafer-Scale Bet: Bandwidth Over Everything Else
Conventional accelerators are cut from a silicon wafer into many small chips, each paired with stacks of high-bandwidth memory (HBM) that hold the model’s weights. Every token generated requires reading those weights across that memory interface, so the interface, not the arithmetic units, usually sets the pace. Cerebras takes the opposite approach: it leaves the wafer whole, producing a single processor roughly the size of a dinner plate, and stores weights in memory distributed across the die itself. On-chip memory is dramatically faster to reach than off-chip memory, which is why the architecture has produced striking token-per-second figures on open models.
The catch is capacity. On-chip memory is fast but comparatively scarce per unit of silicon, while HBM is slower but plentiful. A trillion-parameter model is precisely the case where that asymmetry bites, because all of the model’s weights must be resident somewhere before a request can be served. Serving one at wafer scale implies spreading the model across multiple systems and moving activations between them — which reintroduces exactly the kind of interconnect cost the architecture was designed to avoid.
None of this makes the approach unworkable; Cerebras has run large models this way before, and mixture-of-experts designs help by activating only a fraction of parameters for any given token. But it means the headline claim — trillion-parameter inference — is where the engineering difficulty is concentrated, not where it is resolved. The disclosure that would settle the economics is how many systems constitute one serving instance, and the announcement does not provide it.
An Open-Weight Model Changes the Procurement Conversation
Kimi K2.6 comes from Moonshot AI, a Chinese lab whose K2 family has been released with open weights — the trained parameters are published, so anyone with sufficient hardware can run the model themselves. That property is what makes this announcement possible at all: a hardware vendor cannot offer a proprietary frontier model as a service, but it can offer an open one, and open weights have become the mechanism by which non-Nvidia silicon reaches enterprise buyers.
For buyers, open weights cut in two directions. They reduce lock-in, because the same model can in principle be moved between providers or brought in-house, which makes a specialist accelerator less of a one-way door. They also shift the governance question from the model’s origin to the serving arrangement: where inference physically runs, who retains prompts and outputs, and what the licence permits commercially. A model developed in one jurisdiction and served on infrastructure in another is a common and legitimate arrangement, but it is one enterprise compliance teams will want documented rather than assumed.
It is fair to note the competitive asymmetry this creates. Open releases from Chinese labs have given Western hardware challengers a supply of frontier-class models they would otherwise lack, while proprietary US models remain concentrated on GPU infrastructure. That is a genuine structural feature of the market, and it is worth stating without treating either the models or their provenance as inherently suspect.
Winners, Losers and the Benchmark Problem
If the offering performs as positioned, the clearest beneficiaries are enterprises with latency-sensitive AI products who currently face long queues for GPU capacity, and Cerebras itself, which has publicly disclosed heavy revenue concentration in a small number of customers and needs a broad enterprise base to diversify. Rival specialists pursuing similar high-speed inference strategies face more direct comparison. Incumbent GPU vendors are not meaningfully threatened by a single model launch, but they are affected by the general argument that inference and training may not want the same silicon.
The losers, if any, are harder to identify from an announcement this thin. A serving offer is only as good as its capacity, and capacity is a function of how much wafer-scale hardware exists and is deployed — a supply constraint that specialist vendors have historically found harder to solve than performance.
Buyers should also be alert to the benchmark problem. Tokens per second for a single request, cost per million tokens at realistic concurrency, and latency at the 99th percentile under load are three different numbers, and vendor materials across this entire market tend to lead with whichever is most flattering. That is not a criticism unique to Cerebras. It is the reason independent, workload-specific evaluation remains the only reliable basis for a purchasing decision here.
Background
Cerebras Systems, founded in 2016, took a contrarian approach to AI hardware: rather than dicing a silicon wafer into many chips, it manufactures a single processor spanning nearly the whole wafer, with memory and compute distributed across the surface. Successive generations of its Wafer Scale Engine have targeted first training and, more recently, high-speed inference sold as a cloud service. The company filed publicly to list its shares in 2024 and, in doing so, disclosed a heavy dependence on a small number of customers — a concentration that a broad enterprise inference business would help address.
Moonshot AI is a Chinese AI lab whose Kimi K2 family arrived as one of the largest openly released model lines available, built as a mixture of experts — a design in which only a subset of the model’s parameters is activated for any given token, making very large models cheaper to run than their headline parameter count suggests. Open-weight releases of this kind have become the principal way that alternative accelerator vendors gain access to frontier-scale models, since proprietary models are generally tied to their developers’ own infrastructure.
