jain.com — data centers, connectivity and security, analyzed daily

Tag: Nvidia competition

  • AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD announced on June 11, 2026 that its Instinct MI355X accelerator has set a new performance bar for inference on DeepSeek models — the open-weight large language models from the Chinese AI lab whose efficiency-focused releases reshaped expectations for serving costs. Inference is the work of running a trained model to answer real requests, as opposed to training it in the first place.

    The claim, published by AMD itself, positions the MI355X — the flagship of AMD’s MI350 series — as a leading choice for the inference-heavy workloads that increasingly dominate AI infrastructure spending.

    Executive Summary

    AMD’s announcement is a benchmark claim, not a product launch: the company says the MI355X, its current flagship data-center GPU, delivers record-setting throughput when serving DeepSeek models. Because DeepSeek’s open-weight models are among the most widely deployed for self-hosted inference, they have become a de facto proving ground for accelerator vendors — a benchmark customers can actually reproduce, unlike proprietary-model results.

    The timing matters. The AI hardware market is shifting from a training-dominated buildout, where Nvidia’s ecosystem advantage is strongest, toward an inference era where cost per token served — the price of generating each unit of model output — is the metric that decides purchase orders. AMD’s pitch has consistently been large memory capacity and better price-performance for exactly this phase.

    What the headline claim does not establish, at least in the material visible here, is the specific numbers, the comparison baseline, or independent verification. Vendor benchmarks are a legitimate signal, but buyers should treat them as the opening of a conversation rather than its conclusion.

    Why DeepSeek Became the Benchmark That Matters

    DeepSeek’s models occupy an unusual position in the AI market: they are open-weight, meaning anyone can download and run them on their own hardware, and they were engineered from the start for inference efficiency. That combination made them the workload of choice for enterprises and cloud providers that want frontier-class capability without paying per-token API fees to a model vendor. When a chipmaker claims leadership on DeepSeek inference, it is claiming leadership on one of the workloads real customers actually deploy — which gives the claim more commercial weight than a synthetic benchmark, and also makes it more checkable, since third parties can rerun it.

    There is a second, subtler point: DeepSeek’s mixture-of-experts architecture — where only a fraction of the model’s parameters activate per request — stresses memory capacity and memory bandwidth more than raw compute. That plays to the MI355X’s most widely cited hardware advantage, its large high-bandwidth memory pool (288 GB of HBM3E per GPU, per AMD’s published specifications for the MI350 series). Fitting a large model on fewer GPUs reduces the interconnect traffic and server count needed to serve it, which is where inference economics are won or lost.

    The Inference Era Rewrites the Competitive Math

    Training a frontier model is a rare, massive event; serving it to millions of users is a continuous, compounding cost. As deployed AI applications scale, industry spending is tilting toward inference, and that shift changes what buyers optimize for. In training, ecosystem maturity and cluster-scale networking — Nvidia’s strongholds — dominate the decision. In inference, the calculus is simpler and more mercenary: tokens per second, per dollar, per watt. Every point of throughput a rival accelerator gains translates directly into rack space, power, and capital that an operator does not have to buy.

    This is why AMD keeps aiming its benchmark artillery at inference rather than training. It is the segment where switching costs are lowest — an inference deployment of an open-weight model is far easier to port between hardware vendors than a training pipeline — and where AMD’s ROCm software stack, historically its weakest flank against Nvidia’s CUDA, faces the least demanding compatibility burden. For data-center operators, a credible second source of inference silicon is leverage in every negotiation, whichever vendor ultimately wins the deal.

    A Vendor Benchmark Is a Claim, Not a Verdict

    The announcement comes from AMD’s own newsroom, and the standard cautions apply — as they would to any vendor, including Nvidia, whose competitive benchmarks deserve identical scrutiny. Benchmark results are exquisitely sensitive to configuration: batch size, input and output sequence lengths, quantization (running the model at reduced numerical precision to go faster), and which competing hardware and software versions form the baseline. A ‘new bar’ can be genuine engineering progress, a favorable test setup, or both at once. The release headline, on its own, does not let a reader distinguish these cases.

    The constructive reading is that publishing reproducible claims on an open-weight model invites exactly the third-party validation that settles such questions. If independent labs and cloud customers can replicate the numbers on production-shaped workloads, the claim hardens into a real competitive fact. If the result holds only under narrow conditions, the market will find that out quickly too — one of the healthier dynamics the open-weight ecosystem has introduced to hardware marketing.

    Background

    AMD has spent a decade rebuilding itself into the principal challenger to Nvidia in data-center silicon, first in CPUs with EPYC and more recently in AI accelerators with the Instinct line. The MI300 series, launched in late 2023, gave AMD its first broadly adopted AI GPU; the MI350 series that followed in 2025, including the MI355X, extended its strategy of packing more high-bandwidth memory per chip than competing parts to win inference workloads.

    DeepSeek entered the global spotlight in early 2025 when its efficient open-weight models demonstrated that frontier-class AI could be trained and served at far lower cost than prevailing assumptions, briefly shaking AI-infrastructure markets. Since then its models have become a standard workload for measuring inference performance — turning each new hardware generation’s ‘DeepSeek numbers’ into a competitive scoreboard watched by chipmakers, cloud providers, and investors alike.

    Source: AMD Instinct MI355X GPU Sets a New Bar for DeepSeek Inference — AMD, the company’s announcement of record DeepSeek inference performance on its flagship accelerator.

  • d-Matrix Corsair Hits Full Production: A Challenger to the AI Inference Status Quo

    d-Matrix Corsair Hits Full Production: A Challenger to the AI Inference Status Quo

    Silicon Valley chip startup d-Matrix announced on June 10, 2026 that Corsair, its flagship AI inference accelerator, has entered full production, with the company attributing the ramp to customer demand. Corsair is a PCIe-card accelerator built on d-Matrix’s digital in-memory compute architecture, designed to run large language model inference — the work of generating answers from already-trained models — faster and more efficiently than general-purpose GPUs.

    Executive Summary

    d-Matrix says its Corsair inference platform has moved from early availability into full production. For a fabless semiconductor startup, that transition is one of the hardest milestones in the business: it signals that the design, manufacturing partners, packaging, and software stack are mature enough to ship at volume rather than in evaluation quantities. The company frames the ramp as demand-driven, though the release does not disclose shipment volumes, named customers, or revenue.

    The announcement matters because it lands in the middle of the industry’s most consequential architectural debate: whether AI inference — now widely expected to dwarf training as a share of total AI compute spending — will remain a GPU market, or fracture into specialized silicon. Corsair is a purpose-built bet that inference is fundamentally a memory problem, not a compute problem, and that an architecture which collapses the distance between memory and math can win on cost and energy per token. Full production is the point at which that thesis stops being a slide deck and starts being testable in customer data centers.

    The Memory-Bandwidth Wall, Explained

    When a large language model generates text, the dominant cost is not arithmetic — it is moving the model’s billions of parameters from memory to the processor over and over, once per generated token. Processors have gotten faster far more quickly than memory has gotten closer, a gap the industry calls the memory-bandwidth wall. GPUs attack it with expensive stacks of high-bandwidth memory (HBM) bolted alongside the compute die; d-Matrix attacks it by performing the math inside the memory arrays themselves, an approach called digital in-memory compute. Less data movement means, in principle, lower latency and less energy per token.

    The architectural logic is sound and the problem is real — memory bandwidth, not raw FLOPS, is the binding constraint on most production LLM serving today. The open question has never been whether in-memory compute is elegant, but whether it can be manufactured at scale, programmed easily, and priced competitively. A full-production milestone speaks directly to the first of those three tests.

    From Demo Silicon to Volume: Why This Milestone Is the Hard One

    The graveyard of AI chip startups is full of companies that produced impressive demonstration silicon but never crossed into volume manufacturing. Getting there requires acceptable yields from foundry partners, stable supply of advanced packaging, qualified server integrations, and a software stack that customers other than the vendor’s own engineers can actually use. By declaring full production, d-Matrix is asserting it has cleared those gates.

    What the release does not do is quantify the claim. “Full production to meet customer demand” is a statement about readiness, not about scale: no unit volumes, deployment sizes, or purchasers are disclosed. That is typical for a private company’s press release, but it means the milestone should be read as necessary rather than sufficient evidence of commercial traction. The verifiable signals — named customers, independent benchmarks, follow-on orders — come later, and observers should watch for them.

    The Economics of Challenging an Incumbent

    Every inference challenger faces the same asymmetry: Nvidia’s advantage is only partly the silicon. Its CUDA software ecosystem, developer familiarity, and guaranteed supply relationships make GPUs the default even where specialized chips post better numbers on paper. Challengers such as Groq, Cerebras, and SambaNova — and the hyperscalers’ in-house chips like Google’s TPUs and Amazon’s Inferentia — have each carved positions by competing on cost per token, latency, or energy rather than generality.

    d-Matrix’s opening is real, though. Inference is a workload buyers purchase continuously, priced per token, which makes operating cost — dominated by power and hardware amortization — brutally legible. Enterprises and cloud providers are also actively seeking second sources to gain pricing leverage over the GPU supply chain. A challenger does not need to displace the incumbent to build a substantial business; it needs to win the subset of workloads where its architecture’s advantages are largest and the switching costs are manageable.

    What It Means for the Data Center

    For data-center operators, the interesting property of accelerators like Corsair is the form factor: PCIe cards that slot into standard servers, rather than the dense, increasingly liquid-cooled rack-scale systems that frontier GPUs demand. If inference-optimized silicon delivers competitive throughput at meaningfully lower power per token — a claim d-Matrix has consistently made in its marketing, and one that independent benchmarking will need to validate — it extends the useful life of conventional air-cooled facilities that cannot economically retrofit for 100-kilowatt racks.

    That has second-order implications for the industry’s power crunch. Inference demand is growing at exactly the moment grid interconnection has become the limiting factor on data-center construction. Any architecture that serves more tokens per megawatt is, in effect, a capacity play — and that, more than any single benchmark, is why purpose-built inference silicon keeps attracting capital.

    Background

    Founded in 2019, d-Matrix spent its first years developing digital in-memory compute through successive test chips before unveiling Corsair in late 2024 as its first volume product, aimed squarely at low-latency large language model serving. The company has raised several hundred million dollars from investors including Microsoft’s M12, Temasek, SK hynix, and Playground Global — one of the better-capitalized entrants in a crowded field of AI chip startups formed on the thesis that inference workloads will eventually dwarf training.

    That thesis has moved from contrarian to consensus: as deployed AI applications scale, the recurring cost of serving models has become the industry’s central economic problem, and the market for inference-optimized alternatives to GPUs has drawn challengers ranging from venture-backed startups to the hyperscalers’ own silicon programs. Full production of Corsair marks d-Matrix’s transition from architectural argument to shipping product in that contest.

    Source: d-Matrix Corsair AI Inference Platform Enters Full Production to Meet Customer Demand — company press release via PR Newswire, June 10, 2026, announcing the production ramp of d-Matrix’s inference accelerator platform.