jain.com — data centers, connectivity and security, analyzed daily

Tag: DeepSeek

  • DeepSeek Open-Sources DSpark, Claiming Up to 85% Faster LLM Inference

    DeepSeek Open-Sources DSpark, Claiming Up to 85% Faster LLM Inference

    DeepSeek, the Hangzhou-based AI lab known for its unusually efficient open-weight models, has released DSpark, an open-source framework that it says can accelerate large language model (LLM) inference — the process of actually running a trained model to answer queries — by up to 85%, according to a VentureBeat report published June 28, 2026.

    The release continues DeepSeek’s pattern of publishing its internal efficiency tooling openly rather than keeping it proprietary, and lands at a moment when inference, not training, has become the dominant cost line for companies serving AI at scale.

    Executive Summary

    The announcement is straightforward on its face: DSpark is an inference framework, it is open source, and the headline claim is a speedup of “up to 85%.” What makes it noteworthy is who is making the claim. DeepSeek built its reputation on doing more with less — its earlier model releases were credited with achieving frontier-class results at a fraction of the compute budgets reported by Western rivals — so an efficiency claim from this lab gets taken more seriously than the average vendor benchmark.

    If the speedup holds up under independent testing, the implications run well beyond one company’s software stack. Inference speed translates almost directly into serving cost: a model that answers queries faster on the same hardware serves more users per GPU, which means fewer GPUs, less power, and less data center capacity per unit of AI demand. Because DSpark is open source, any operator — hyperscaler, neocloud, or enterprise running models in-house — can in principle adopt it without a licensing negotiation.

    The important caveat is that “up to 85%” is a ceiling, not an average, and the report available at publication does not detail the workloads, models, or hardware behind the number. That distinction should shape how buyers and investors read the news.

    Inference Is Where the Money Now Goes

    For the first years of the generative AI boom, the eye-watering costs were in training — the one-time process of teaching a model from massive datasets. That has flipped. Once hundreds of millions of people are querying models daily, the recurring cost of inference dwarfs the one-time cost of training, and it scales with every new user and every longer conversation. This is why the industry’s optimization energy has shifted to serving: techniques with names like speculative decoding, quantization, and KV-cache management all exist to squeeze more answers out of each GPU-hour.

    An 85% speedup, if achieved on realistic workloads, is not an incremental gain in this context. Serving capacity is the binding constraint for many AI providers, and GPUs remain supply-limited and expensive. Software that meaningfully raises throughput per chip is functionally equivalent to manufacturing more chips — without the fab, the lead time, or the export-control exposure that hardware carries.

    DeepSeek’s Open-Source Playbook, Continued

    DeepSeek has a track record here. The lab, spun out of the Chinese quantitative hedge fund High-Flyer, shook global markets in early 2025 when its R1 reasoning model demonstrated that frontier-adjacent capability did not require frontier-scale budgets. It followed up by open-sourcing chunks of its internal infrastructure code — low-level GPU kernels and communication libraries — rather than treating them as trade secrets. DSpark fits that pattern: release the tooling, let the ecosystem adopt it, and compete on the pace of research rather than on locked-down software.

    The strategic logic is worth spelling out. Open-sourcing inference tooling commoditizes the serving layer, which pressures companies whose business model depends on proprietary serving efficiency, while costing DeepSeek little — its own advantage lies upstream, in model quality and training efficiency. It also builds developer mindshare globally at a time when Chinese AI labs face restricted access to top-end accelerators, making software efficiency a competitive necessity as much as a virtue.

    What Cheaper Inference Means for Infrastructure Operators

    A natural first read is that faster inference is bearish for GPU and data center demand: if each chip does 85% more work, you need fewer chips and fewer megawatts. History suggests the opposite usually happens. Efficiency gains in computing have repeatedly triggered what economists call the Jevons paradox — when something gets cheaper, consumption expands enough to more than offset the savings. Cheaper inference makes previously uneconomic AI applications viable: always-on agents, AI in low-margin consumer products, long-context document processing at scale.

    For data center operators and connectivity providers, the more defensible conclusion is that efficiency software shifts demand rather than shrinking it. Lower serving costs favor deployment breadth — more applications, more regions, more inference happening closer to users — which tends to benefit distributed capacity and network infrastructure even if it moderates the growth rate of any single mega-campus. Operators planning around raw GPU scarcity should note that the scarcity premium softens every time the software stack gets meaningfully better.

    Reading an ‘Up To’ Claim Responsibly

    The 85% figure deserves the same scrutiny any vendor benchmark gets, and the fact that DSpark is open source cuts in its favor: the code can be tested independently, which is more than can be said for closed serving stacks making similar claims. Still, inference speedups are notoriously workload-dependent. Gains that appear on one batch size, sequence length, or model architecture can shrink dramatically on another, and the report available at publication does not specify the conditions behind the headline number.

    The practical test is adoption. The inference-serving field already has entrenched open-source incumbents — frameworks like vLLM and NVIDIA’s TensorRT-LLM ecosystem have large communities and production track records. DSpark’s real-world impact will be measured not by its launch benchmark but by whether major serving operations fold it, or its techniques, into production over the following quarters. DeepSeek’s prior open-source releases were rapidly picked apart and partially absorbed by the community; that is the most likely path here too, even if the framework itself does not displace incumbents wholesale.

    Background

    DeepSeek emerged from High-Flyer, a Chinese quantitative hedge fund, and stunned the AI industry in January 2025 when its R1 model matched much of the reasoning performance of leading Western systems at a reported fraction of the training cost — an announcement that briefly wiped hundreds of billions of dollars from AI-linked stocks as investors reassessed how much compute frontier AI truly requires. The lab has since maintained a strategy of releasing open-weight models and open-source infrastructure tooling, positioning efficiency as its core identity.

    The inference-serving market it is now entering more forcefully has its own history: open-source frameworks such as vLLM (from UC Berkeley researchers) and NVIDIA’s TensorRT-LLM became the workhorses of production LLM serving as the industry’s cost center shifted from training models to running them for hundreds of millions of users. Every meaningful gain in serving efficiency ripples outward into GPU procurement, data center planning, and the unit economics of AI products.

    Source: DeepSeek open sources DSpark, a new framework to speed up LLM inference by up to 85% — VentureBeat, reporting DeepSeek’s open-source release of its DSpark inference-acceleration framework, June 28, 2026.

  • AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD announced on June 11, 2026 that its Instinct MI355X accelerator has set a new performance bar for inference on DeepSeek models — the open-weight large language models from the Chinese AI lab whose efficiency-focused releases reshaped expectations for serving costs. Inference is the work of running a trained model to answer real requests, as opposed to training it in the first place.

    The claim, published by AMD itself, positions the MI355X — the flagship of AMD’s MI350 series — as a leading choice for the inference-heavy workloads that increasingly dominate AI infrastructure spending.

    Executive Summary

    AMD’s announcement is a benchmark claim, not a product launch: the company says the MI355X, its current flagship data-center GPU, delivers record-setting throughput when serving DeepSeek models. Because DeepSeek’s open-weight models are among the most widely deployed for self-hosted inference, they have become a de facto proving ground for accelerator vendors — a benchmark customers can actually reproduce, unlike proprietary-model results.

    The timing matters. The AI hardware market is shifting from a training-dominated buildout, where Nvidia’s ecosystem advantage is strongest, toward an inference era where cost per token served — the price of generating each unit of model output — is the metric that decides purchase orders. AMD’s pitch has consistently been large memory capacity and better price-performance for exactly this phase.

    What the headline claim does not establish, at least in the material visible here, is the specific numbers, the comparison baseline, or independent verification. Vendor benchmarks are a legitimate signal, but buyers should treat them as the opening of a conversation rather than its conclusion.

    Why DeepSeek Became the Benchmark That Matters

    DeepSeek’s models occupy an unusual position in the AI market: they are open-weight, meaning anyone can download and run them on their own hardware, and they were engineered from the start for inference efficiency. That combination made them the workload of choice for enterprises and cloud providers that want frontier-class capability without paying per-token API fees to a model vendor. When a chipmaker claims leadership on DeepSeek inference, it is claiming leadership on one of the workloads real customers actually deploy — which gives the claim more commercial weight than a synthetic benchmark, and also makes it more checkable, since third parties can rerun it.

    There is a second, subtler point: DeepSeek’s mixture-of-experts architecture — where only a fraction of the model’s parameters activate per request — stresses memory capacity and memory bandwidth more than raw compute. That plays to the MI355X’s most widely cited hardware advantage, its large high-bandwidth memory pool (288 GB of HBM3E per GPU, per AMD’s published specifications for the MI350 series). Fitting a large model on fewer GPUs reduces the interconnect traffic and server count needed to serve it, which is where inference economics are won or lost.

    The Inference Era Rewrites the Competitive Math

    Training a frontier model is a rare, massive event; serving it to millions of users is a continuous, compounding cost. As deployed AI applications scale, industry spending is tilting toward inference, and that shift changes what buyers optimize for. In training, ecosystem maturity and cluster-scale networking — Nvidia’s strongholds — dominate the decision. In inference, the calculus is simpler and more mercenary: tokens per second, per dollar, per watt. Every point of throughput a rival accelerator gains translates directly into rack space, power, and capital that an operator does not have to buy.

    This is why AMD keeps aiming its benchmark artillery at inference rather than training. It is the segment where switching costs are lowest — an inference deployment of an open-weight model is far easier to port between hardware vendors than a training pipeline — and where AMD’s ROCm software stack, historically its weakest flank against Nvidia’s CUDA, faces the least demanding compatibility burden. For data-center operators, a credible second source of inference silicon is leverage in every negotiation, whichever vendor ultimately wins the deal.

    A Vendor Benchmark Is a Claim, Not a Verdict

    The announcement comes from AMD’s own newsroom, and the standard cautions apply — as they would to any vendor, including Nvidia, whose competitive benchmarks deserve identical scrutiny. Benchmark results are exquisitely sensitive to configuration: batch size, input and output sequence lengths, quantization (running the model at reduced numerical precision to go faster), and which competing hardware and software versions form the baseline. A ‘new bar’ can be genuine engineering progress, a favorable test setup, or both at once. The release headline, on its own, does not let a reader distinguish these cases.

    The constructive reading is that publishing reproducible claims on an open-weight model invites exactly the third-party validation that settles such questions. If independent labs and cloud customers can replicate the numbers on production-shaped workloads, the claim hardens into a real competitive fact. If the result holds only under narrow conditions, the market will find that out quickly too — one of the healthier dynamics the open-weight ecosystem has introduced to hardware marketing.

    Background

    AMD has spent a decade rebuilding itself into the principal challenger to Nvidia in data-center silicon, first in CPUs with EPYC and more recently in AI accelerators with the Instinct line. The MI300 series, launched in late 2023, gave AMD its first broadly adopted AI GPU; the MI350 series that followed in 2025, including the MI355X, extended its strategy of packing more high-bandwidth memory per chip than competing parts to win inference workloads.

    DeepSeek entered the global spotlight in early 2025 when its efficient open-weight models demonstrated that frontier-class AI could be trained and served at far lower cost than prevailing assumptions, briefly shaking AI-infrastructure markets. Since then its models have become a standard workload for measuring inference performance — turning each new hardware generation’s ‘DeepSeek numbers’ into a competitive scoreboard watched by chipmakers, cloud providers, and investors alike.

    Source: AMD Instinct MI355X GPU Sets a New Bar for DeepSeek Inference — AMD, the company’s announcement of record DeepSeek inference performance on its flagship accelerator.