jain.com — data centers, connectivity and security, analyzed daily

Tag: data center hardware

  • TSMC Reportedly to Double CoWoS Packaging Capacity by 2028

    TSMC Reportedly to Double CoWoS Packaging Capacity by 2028

    TL;DR · 30-second read

    The Short Version

    The chips that run artificial intelligence are not held up by the factories that print them. They are held up by the assembly step at the end, where a processor and its memory are bonded onto a single slab so they can talk to each other fast enough.

    Taiwan Semiconductor Manufacturing Company owns most of the world’s capacity for that step. Analysts in Taiwan say it plans to roughly double that capacity by 2028. Until it does, anyone building an artificial intelligence data center is standing in a queue. Intel is selling a rival method, and smaller assembly firms are absorbing the overflow.

    Wccftech, citing a Taiwanese media report that quotes analysts, said TSMC plans to expand its chip-on-wafer-on-substrate (CoWoS) advanced packaging capacity to 260,000 wafers per month by the end of 2028, roughly double the approximately 130,000 wafers per month the same forecast puts it at by the end of 2026. The expansion is said to center on TSMC’s Arizona campus and its AP7 packaging facility in Taiwan.

    The same forecast estimates that Intel’s competing EMIB-T packaging, converted into CoWoS-equivalent 12-inch wafer terms, could reach 15,000 to 20,000 wafers per month in 2027 and 40,000 to 45,000 in 2028, and says the CoWoS shortage has pushed work toward other packaging firms including Amkor, UMC and ASE. TSMC has not published a capacity target of its own. In a Form 6-K filed with the Securities and Exchange Commission on September 10, the company reported August 2026 consolidated revenue of NT$514.81 billion, up 53.3 percent from a year earlier.

    Executive Summary

    Advanced packaging is the step where a logic die and its stacks of high-bandwidth memory are mounted together onto a shared substrate so they behave as one component. CoWoS is TSMC’s version, and it is the packaging used for the accelerators that fill AI data centers. For roughly three years it, rather than leading-edge wafer manufacturing, has been the narrowest point in the supply chain for those parts.

    A doubling of that capacity by 2028 would be the single largest change to AI hardware availability that any one company can make. It would not, however, double accelerator output in a straight line. Package sizes are growing, the newest designs consume more interposer area and more memory stacks per unit, and a wafer of packaging substrate therefore yields fewer finished accelerators each generation than it did the year before. Capacity that doubles against demand that also rises per unit is not the same as supply that doubles.

    TSMC’s own disclosure does not address packaging capacity. What its September 10 filing does show is demand running hot: August revenue rose 53.3 percent year over year, and revenue for the first eight months of 2026 was up 39.3 percent. That is consistent with a company selling everything it can finish, which is the condition under which a capacity forecast of this kind becomes credible without being confirmed.

    The Bottleneck Moved Downstream

    For most of the history of the chip industry, the scarce resource was the front end: photolithography tools, leading-edge wafer starts, yield on a new process node. Packaging was the cheap, unglamorous back end. AI accelerators inverted that. A modern accelerator is not one chip but an assembly, a large logic die surrounded by stacks of memory, all sitting on a silicon interposer threaded with vertical connections called through-silicon vias. The density of those connections is what delivers the memory bandwidth the workload needs. That assembly step is CoWoS, and there is no abundant substitute for it.

    The practical consequence for anyone financing a data center is that wafer capacity is no longer the number to watch. A foundry can start more wafers than it can finish into packages, and unfinished silicon does not train models. The forecast of 130,000 wafers per month at the end of 2026 rising to 260,000 by the end of 2028 is, in effect, a two-year schedule for how quickly the industry can convert build-ready halls into loaded racks.

    Read it with care, though. Interposer area per accelerator has been climbing with every generation as designers add memory stacks and reticle-stitched logic. If the average package consumes meaningfully more of a wafer in 2028 than it does today, unit output grows by considerably less than the wafer count suggests. Doubling the input is a ceiling on supply growth, not a promise of it.

    Arizona Can Print the Chips but Cannot Finish Them

    The most operationally interesting detail in the forecast is geographic. Wafers produced at TSMC’s Arizona campus are flown to Taiwan to be packaged, because the packaging capacity sits on the island. Siting part of the expansion in Arizona alongside the AP7 facility in Taiwan would begin to close that loop, turning an American fab from a wafer source into something closer to a finished-component source.

    That matters for reasons beyond logistics. A round trip across the Pacific adds days of cycle time and a customs and freight cost to every high-value wafer, and it means that the resilience argument for building fabs outside Taiwan is currently incomplete. A domestic fab whose output must still transit Taiwan to become a usable product does not remove the concentration risk that justified it. TSMC’s own filing shows the Arizona entity is treated as a significant subsidiary, with parent guarantees extended to TSMC Arizona among other affiliates, but the filing says nothing about packaging there.

    For data center operators and the governments underwriting them, this is the question worth asking of any onshoring announcement: does the site produce a wafer or a shippable part? Until packaging follows fabrication, the answer for Arizona remains the former.

    Intel’s EMIB-T and the Value of a Second Source

    Intel’s EMIB-T takes a different architectural route. Instead of building the whole assembly on a large silicon interposer, it embeds small silicon bridges inside the organic substrate and places the logic and memory dies on top. Less silicon is used, which can mean lower cost and fewer of the handling and warpage problems that come with very large interposers, but the approach historically trades away some of the interconnect density that the largest, most bandwidth-hungry designs demand.

    The forecast puts EMIB-T at 40,000 to 45,000 CoWoS-equivalent wafers per month by 2028. Against 260,000, that is a meaningful second source rather than a competitive threat, on the order of a sixth of the leader’s volume. The claim that it suits custom accelerators of the kind Google and Amazon design is plausible on architecture, since in-house cloud silicon is generally tuned for a known workload and a known cost target rather than for maximum bandwidth per package. It is also unverified: neither Intel nor any cloud provider has announced such an arrangement.

    The spillover to Amkor, UMC and ASE follows the same logic. When the preferred supplier is sold out, work migrates to whoever can perform an adjacent version of it, and outsourced assembly and test firms have been the natural landing place. Buyers get optionality and a check on pricing. The risk is that second-source packaging is not a drop-in substitute. Requalifying a part on a different packaging architecture is an engineering program, not a purchase order, and the customers best placed to do it are the ones designing their own silicon.

    What Is Substantiated and What Is Not

    The demand side of this story rests on disclosed figures. TSMC’s September 10 filing with the Securities and Exchange Commission reports August consolidated revenue of NT$514.81 billion, a 10.1 percent increase on July and 53.3 percent on August 2025, with the first eight months of 2026 up 39.3 percent year over year. Growth at that rate at TSMC’s scale is difficult to explain without sustained AI accelerator demand behind it.

    The supply side is a forecast, not a commitment. The 260,000 figure, the 130,000 baseline, the Arizona and AP7 split and the EMIB-T range all come from analysts rather than from the companies. None of the three firms named has published a packaging capacity target, a capital budget attached to one, or a schedule. Analyst capacity models in this industry have a reasonable track record on direction and a poorer one on timing, because packaging expansions depend on tool delivery, cleanroom construction and qualification cycles that slip quietly.

    The sober reading is that the direction is well supported and the magnitude is an estimate. Treat 2028 as the earliest plausible date for materially looser accelerator supply, treat the doubling as a ceiling rather than a plan of record, and treat any procurement model that assumes it as carrying schedule risk that TSMC has not underwritten.

    Background

    TSMC is the world’s largest contract chip manufacturer, building processors designed by other companies. It commercialised CoWoS in the 2010s as a niche technique for high-performance parts, long before AI accelerators turned it into the scarcest resource in the industry. Today the packaging step determines how many finished accelerators reach the market, because a logic die without its memory bonded alongside it is not a usable product.

    The competitive response has come from two directions. Intel has pushed its own bridge-based packaging as part of its foundry ambitions, targeting customers who want a supplier outside Taiwan or a lower-cost assembly for custom silicon. Outsourced assembly and test firms such as Amkor, ASE and UMC have absorbed work that TSMC cannot schedule. None of this has yet changed the basic shape of the market, in which one company’s packaging line sets the pace at which AI data centers can be filled.

    Sources

    Source: Report: TSMC to Double CoWoS Capacity by 2028 as AI Chip Shortage Spills Over to Rivals, which relays a Taiwanese analyst forecast for TSMC’s advanced packaging capacity and for Intel’s competing EMIB-T volumes.

    Primary sources: Taiwan Semiconductor Manufacturing Company, Form 6-K filed September 10, 2026 (August 2026 revenue report), filed with the US Securities and Exchange Commission.

  • AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD announced on June 11, 2026 that its Instinct MI355X accelerator has set a new performance bar for inference on DeepSeek models — the open-weight large language models from the Chinese AI lab whose efficiency-focused releases reshaped expectations for serving costs. Inference is the work of running a trained model to answer real requests, as opposed to training it in the first place.

    The claim, published by AMD itself, positions the MI355X — the flagship of AMD’s MI350 series — as a leading choice for the inference-heavy workloads that increasingly dominate AI infrastructure spending.

    Executive Summary

    AMD’s announcement is a benchmark claim, not a product launch: the company says the MI355X, its current flagship data-center GPU, delivers record-setting throughput when serving DeepSeek models. Because DeepSeek’s open-weight models are among the most widely deployed for self-hosted inference, they have become a de facto proving ground for accelerator vendors — a benchmark customers can actually reproduce, unlike proprietary-model results.

    The timing matters. The AI hardware market is shifting from a training-dominated buildout, where Nvidia’s ecosystem advantage is strongest, toward an inference era where cost per token served — the price of generating each unit of model output — is the metric that decides purchase orders. AMD’s pitch has consistently been large memory capacity and better price-performance for exactly this phase.

    What the headline claim does not establish, at least in the material visible here, is the specific numbers, the comparison baseline, or independent verification. Vendor benchmarks are a legitimate signal, but buyers should treat them as the opening of a conversation rather than its conclusion.

    Why DeepSeek Became the Benchmark That Matters

    DeepSeek’s models occupy an unusual position in the AI market: they are open-weight, meaning anyone can download and run them on their own hardware, and they were engineered from the start for inference efficiency. That combination made them the workload of choice for enterprises and cloud providers that want frontier-class capability without paying per-token API fees to a model vendor. When a chipmaker claims leadership on DeepSeek inference, it is claiming leadership on one of the workloads real customers actually deploy — which gives the claim more commercial weight than a synthetic benchmark, and also makes it more checkable, since third parties can rerun it.

    There is a second, subtler point: DeepSeek’s mixture-of-experts architecture — where only a fraction of the model’s parameters activate per request — stresses memory capacity and memory bandwidth more than raw compute. That plays to the MI355X’s most widely cited hardware advantage, its large high-bandwidth memory pool (288 GB of HBM3E per GPU, per AMD’s published specifications for the MI350 series). Fitting a large model on fewer GPUs reduces the interconnect traffic and server count needed to serve it, which is where inference economics are won or lost.

    The Inference Era Rewrites the Competitive Math

    Training a frontier model is a rare, massive event; serving it to millions of users is a continuous, compounding cost. As deployed AI applications scale, industry spending is tilting toward inference, and that shift changes what buyers optimize for. In training, ecosystem maturity and cluster-scale networking — Nvidia’s strongholds — dominate the decision. In inference, the calculus is simpler and more mercenary: tokens per second, per dollar, per watt. Every point of throughput a rival accelerator gains translates directly into rack space, power, and capital that an operator does not have to buy.

    This is why AMD keeps aiming its benchmark artillery at inference rather than training. It is the segment where switching costs are lowest — an inference deployment of an open-weight model is far easier to port between hardware vendors than a training pipeline — and where AMD’s ROCm software stack, historically its weakest flank against Nvidia’s CUDA, faces the least demanding compatibility burden. For data-center operators, a credible second source of inference silicon is leverage in every negotiation, whichever vendor ultimately wins the deal.

    A Vendor Benchmark Is a Claim, Not a Verdict

    The announcement comes from AMD’s own newsroom, and the standard cautions apply — as they would to any vendor, including Nvidia, whose competitive benchmarks deserve identical scrutiny. Benchmark results are exquisitely sensitive to configuration: batch size, input and output sequence lengths, quantization (running the model at reduced numerical precision to go faster), and which competing hardware and software versions form the baseline. A ‘new bar’ can be genuine engineering progress, a favorable test setup, or both at once. The release headline, on its own, does not let a reader distinguish these cases.

    The constructive reading is that publishing reproducible claims on an open-weight model invites exactly the third-party validation that settles such questions. If independent labs and cloud customers can replicate the numbers on production-shaped workloads, the claim hardens into a real competitive fact. If the result holds only under narrow conditions, the market will find that out quickly too — one of the healthier dynamics the open-weight ecosystem has introduced to hardware marketing.

    Background

    AMD has spent a decade rebuilding itself into the principal challenger to Nvidia in data-center silicon, first in CPUs with EPYC and more recently in AI accelerators with the Instinct line. The MI300 series, launched in late 2023, gave AMD its first broadly adopted AI GPU; the MI350 series that followed in 2025, including the MI355X, extended its strategy of packing more high-bandwidth memory per chip than competing parts to win inference workloads.

    DeepSeek entered the global spotlight in early 2025 when its efficient open-weight models demonstrated that frontier-class AI could be trained and served at far lower cost than prevailing assumptions, briefly shaking AI-infrastructure markets. Since then its models have become a standard workload for measuring inference performance — turning each new hardware generation’s ‘DeepSeek numbers’ into a competitive scoreboard watched by chipmakers, cloud providers, and investors alike.

    Source: AMD Instinct MI355X GPU Sets a New Bar for DeepSeek Inference — AMD, the company’s announcement of record DeepSeek inference performance on its flagship accelerator.

  • d-Matrix Corsair Hits Full Production: A Challenger to the AI Inference Status Quo

    d-Matrix Corsair Hits Full Production: A Challenger to the AI Inference Status Quo

    Silicon Valley chip startup d-Matrix announced on June 10, 2026 that Corsair, its flagship AI inference accelerator, has entered full production, with the company attributing the ramp to customer demand. Corsair is a PCIe-card accelerator built on d-Matrix’s digital in-memory compute architecture, designed to run large language model inference — the work of generating answers from already-trained models — faster and more efficiently than general-purpose GPUs.

    Executive Summary

    d-Matrix says its Corsair inference platform has moved from early availability into full production. For a fabless semiconductor startup, that transition is one of the hardest milestones in the business: it signals that the design, manufacturing partners, packaging, and software stack are mature enough to ship at volume rather than in evaluation quantities. The company frames the ramp as demand-driven, though the release does not disclose shipment volumes, named customers, or revenue.

    The announcement matters because it lands in the middle of the industry’s most consequential architectural debate: whether AI inference — now widely expected to dwarf training as a share of total AI compute spending — will remain a GPU market, or fracture into specialized silicon. Corsair is a purpose-built bet that inference is fundamentally a memory problem, not a compute problem, and that an architecture which collapses the distance between memory and math can win on cost and energy per token. Full production is the point at which that thesis stops being a slide deck and starts being testable in customer data centers.

    The Memory-Bandwidth Wall, Explained

    When a large language model generates text, the dominant cost is not arithmetic — it is moving the model’s billions of parameters from memory to the processor over and over, once per generated token. Processors have gotten faster far more quickly than memory has gotten closer, a gap the industry calls the memory-bandwidth wall. GPUs attack it with expensive stacks of high-bandwidth memory (HBM) bolted alongside the compute die; d-Matrix attacks it by performing the math inside the memory arrays themselves, an approach called digital in-memory compute. Less data movement means, in principle, lower latency and less energy per token.

    The architectural logic is sound and the problem is real — memory bandwidth, not raw FLOPS, is the binding constraint on most production LLM serving today. The open question has never been whether in-memory compute is elegant, but whether it can be manufactured at scale, programmed easily, and priced competitively. A full-production milestone speaks directly to the first of those three tests.

    From Demo Silicon to Volume: Why This Milestone Is the Hard One

    The graveyard of AI chip startups is full of companies that produced impressive demonstration silicon but never crossed into volume manufacturing. Getting there requires acceptable yields from foundry partners, stable supply of advanced packaging, qualified server integrations, and a software stack that customers other than the vendor’s own engineers can actually use. By declaring full production, d-Matrix is asserting it has cleared those gates.

    What the release does not do is quantify the claim. “Full production to meet customer demand” is a statement about readiness, not about scale: no unit volumes, deployment sizes, or purchasers are disclosed. That is typical for a private company’s press release, but it means the milestone should be read as necessary rather than sufficient evidence of commercial traction. The verifiable signals — named customers, independent benchmarks, follow-on orders — come later, and observers should watch for them.

    The Economics of Challenging an Incumbent

    Every inference challenger faces the same asymmetry: Nvidia’s advantage is only partly the silicon. Its CUDA software ecosystem, developer familiarity, and guaranteed supply relationships make GPUs the default even where specialized chips post better numbers on paper. Challengers such as Groq, Cerebras, and SambaNova — and the hyperscalers’ in-house chips like Google’s TPUs and Amazon’s Inferentia — have each carved positions by competing on cost per token, latency, or energy rather than generality.

    d-Matrix’s opening is real, though. Inference is a workload buyers purchase continuously, priced per token, which makes operating cost — dominated by power and hardware amortization — brutally legible. Enterprises and cloud providers are also actively seeking second sources to gain pricing leverage over the GPU supply chain. A challenger does not need to displace the incumbent to build a substantial business; it needs to win the subset of workloads where its architecture’s advantages are largest and the switching costs are manageable.

    What It Means for the Data Center

    For data-center operators, the interesting property of accelerators like Corsair is the form factor: PCIe cards that slot into standard servers, rather than the dense, increasingly liquid-cooled rack-scale systems that frontier GPUs demand. If inference-optimized silicon delivers competitive throughput at meaningfully lower power per token — a claim d-Matrix has consistently made in its marketing, and one that independent benchmarking will need to validate — it extends the useful life of conventional air-cooled facilities that cannot economically retrofit for 100-kilowatt racks.

    That has second-order implications for the industry’s power crunch. Inference demand is growing at exactly the moment grid interconnection has become the limiting factor on data-center construction. Any architecture that serves more tokens per megawatt is, in effect, a capacity play — and that, more than any single benchmark, is why purpose-built inference silicon keeps attracting capital.

    Background

    Founded in 2019, d-Matrix spent its first years developing digital in-memory compute through successive test chips before unveiling Corsair in late 2024 as its first volume product, aimed squarely at low-latency large language model serving. The company has raised several hundred million dollars from investors including Microsoft’s M12, Temasek, SK hynix, and Playground Global — one of the better-capitalized entrants in a crowded field of AI chip startups formed on the thesis that inference workloads will eventually dwarf training.

    That thesis has moved from contrarian to consensus: as deployed AI applications scale, the recurring cost of serving models has become the industry’s central economic problem, and the market for inference-optimized alternatives to GPUs has drawn challengers ranging from venture-backed startups to the hyperscalers’ own silicon programs. Full production of Corsair marks d-Matrix’s transition from architectural argument to shipping product in that contest.

    Source: d-Matrix Corsair AI Inference Platform Enters Full Production to Meet Customer Demand — company press release via PR Newswire, June 10, 2026, announcing the production ramp of d-Matrix’s inference accelerator platform.

  • Anthropic Eyes Fractile’s DRAM-Less Inference Chips

    Anthropic Eyes Fractile’s DRAM-Less Inference Chips

    Anthropic is in early talks to buy AI inference chips from Fractile, a UK semiconductor startup whose architecture stores model weights in on-chip SRAM rather than external DRAM, according to a report published on 3 May 2026 by Tom’s Hardware. The stated appeal is that a DRAM-less design reduces dependence on high-bandwidth memory (HBM) at a moment of extreme memory pricing and constrained supply.

    The report describes talks at an early stage. No purchase volumes, prices, delivery dates, or contractual commitments were disclosed, and neither company is described as having confirmed a deal.

    Executive Summary

    The substance of the report is narrow but pointed: one of the largest buyers of AI inference capacity is looking at hardware that removes the single most expensive and supply-constrained component in a modern accelerator. HBM — the stacked DRAM that sits beside a GPU and feeds it data — has become both a cost centre and a scheduling risk. Fractile’s pitch, as characterised in the report, is an architecture that keeps model weights in static RAM on the compute die itself, eliminating the trip to external memory that dominates inference latency and power.

    Why this matters beyond one startup: inference at scale is not a compute-bound workload in the way training is. Generating tokens one at a time means repeatedly reading a model’s weights out of memory, so throughput tracks memory bandwidth far more closely than it tracks raw arithmetic. Anyone who can supply bandwidth without buying HBM is selling into a genuine bottleneck, not a marketing one.

    What the report does not establish is equally important. “Early talks” is the lowest rung of commercial engagement, the account appears to rest on a single publication, and the hardest engineering question for any SRAM-based design — whether on-die memory capacity can hold a frontier-scale model economically — is not addressed. The signal here is about buyer intent and market pressure, not about a validated product.

    Inference Is a Memory Problem Wearing a Compute Costume

    When a large language model answers a question, it produces one token at a time, and each token requires reading a large fraction of the model’s parameters. That makes the decode phase bandwidth-bound: the arithmetic units on a modern accelerator spend much of their time waiting for data to arrive. High-bandwidth memory exists to narrow that gap, stacking DRAM dies vertically and placing them next to the processor on the same package. It works, and it is expensive — HBM is one of the costliest components in an AI accelerator and among the hardest to secure, because it depends on advanced packaging capacity as well as DRAM fabrication.

    Static RAM changes the physics of that trade. SRAM sits on the logic die itself, delivers bandwidth measured in the hundreds of gigabytes to terabytes per second per chip, and consumes far less energy per bit moved than an off-package DRAM access. If a model’s weights fit in SRAM, the memory wall largely disappears for that model. This is not a novel insight — it is the same reasoning behind the wafer-scale and deterministic-dataflow approaches other inference specialists have pursued — but the memory market of 2026 has raised the value of the idea considerably.

    For infrastructure buyers, the second-order effect matters as much as the first. Moving data off-package is a meaningful share of accelerator power draw. An architecture that eliminates those transfers changes the energy-per-token calculation, and energy per token is the metric that ultimately determines how much inference a given megawatt of data centre capacity can serve.

    The Capacity Tax Nobody Escapes

    The counter-argument to SRAM is capacity, and it is a serious one. On-die SRAM is typically measured in tens to hundreds of megabytes per chip, while an HBM-equipped accelerator carries tens of gigabytes. Holding a large model entirely in SRAM therefore means distributing it across many chips and connecting them with an interconnect fast enough that the network does not become the new bottleneck. Silicon area is expensive, SRAM has scaled poorly relative to logic at recent process nodes, and a design that needs many dies to hold one model trades a memory bill for a wafer bill.

    Whether that trade is favourable is an empirical question about total cost of ownership, not a matter of architectural principle. It depends on how many chips a target model requires, what each chip costs to fabricate and package, how much power the resulting cluster draws, and how well utilised it stays across real request patterns. It also depends on the key-value cache — the growing scratchpad of intermediate state that long-context conversations generate at run time. KV cache scales with context length and concurrent users rather than with model size, and where it lives in a DRAM-less system is the question that separates a demonstration from a deployable product. The report does not address it.

    The honest framing is that SRAM-first designs are strongest where models are compact, batch behaviour is predictable, and latency is the product. They are weakest where a customer wants to run whatever model it likes at whatever context length users demand. Which of those descriptions fits Anthropic’s inference fleet is not something the report tells us.

    What a Frontier Lab Gains From Being Seen Shopping

    Anthropic already runs inference across multiple silicon platforms, including Google’s TPUs, Amazon’s Trainium, and Nvidia hardware. Adding an early-stage evaluation of a startup’s accelerator is consistent with that pattern rather than a departure from it. Frontier labs have strong incentives to hold options across suppliers: it hedges against shortage, it constrains pricing power, and it gives engineering teams early visibility into architectures that may matter in two or three years.

    That same logic should temper how much any single report is read to mean. Early-stage supplier talks are cheap for a buyer and valuable publicity for a young vendor, and the asymmetry in who benefits from disclosure is worth naming plainly. This is not a reason to doubt the reporting — it is a reason to treat “in talks” as evidence of interest in a category, which is well supported by the memory market, rather than evidence about a specific product’s readiness, which is not addressed. Neither party is described as confirming the discussions, and the account appears to originate from one publication.

    The category signal is nonetheless real. When the buyers with the deepest inference workloads start evaluating architectures whose main selling point is the absence of HBM, it tells you that the memory crunch has moved from a procurement irritation to an architectural forcing function.

    Winners, Losers, and the Data Centre Floor

    If DRAM-less inference gains commercial traction, the pressure lands first on HBM suppliers and on the packaging capacity that HBM consumes — though the near-term risk to them is modest, since training and the installed inference base remain firmly HBM-dependent. Nvidia’s position is likewise not threatened by an early-stage evaluation; the more plausible medium-term effect is on price discipline, as credible alternatives give large buyers a bargaining position they currently lack. The clearest beneficiaries of the trend, whether or not Fractile is the vehicle, are inference specialists of any architecture that can offer bandwidth without a DRAM bill of materials.

    For data centre operators, the interesting variable is density and power profile rather than chip count. SRAM-heavy, many-die inference systems concentrate compute differently from HBM-equipped GPU racks, and any shift in the mix changes assumptions about rack power, cooling approach, and interconnect topology. Operators planning capacity for 2027 and beyond should treat inference hardware as less settled than the current GPU-centric build-out implies.

    For enterprise buyers of inference capacity, the practical near-term takeaway is modest and worth stating without overclaiming: memory scarcity is now shaping the roadmaps of the companies you buy tokens from. That does not change procurement today. It does mean that assumptions about which silicon will serve your workload in three years deserve more scrutiny than they did a year ago.

    Background

    AI accelerators pair processing logic with memory, and for the current generation of large models that memory is usually HBM — DRAM stacked in vertical layers beside the processor. HBM solved a real problem, because model weights are far too large to fit on a processor die, but it introduced a cost and supply dependency that now shapes the entire AI hardware market. A parallel line of engineering has argued for the opposite trade: keep everything in fast on-chip SRAM and accept that a model must be spread across many chips. Wafer-scale and deterministic-dataflow inference startups have pursued versions of this idea for several years.

    Anthropic, the AI company behind the Claude models, is among the largest consumers of inference compute and has deliberately spread its workloads across multiple silicon platforms rather than standardising on one. Fractile is a UK semiconductor startup working on inference hardware that keeps weights in on-chip memory. The reported talks sit at the intersection of those two positions: a buyer with strong incentives to diversify supply, and an architecture whose central claim is that it does not need the component the market is short of.

    Source: Anthropic in early talks to buy DRAM-less AI inference chips from UK startup — Fractile’s SRAM architecture reduces need for pricey memory during extreme pricing and shortage crunch — Tom’s Hardware report, published 3 May 2026, describing early-stage discussions between Anthropic and UK chip startup Fractile.