TL;DR · 30-second read

The Short Version

Chipmaker Marvell unveiled new parts aimed at a quiet problem in artificial intelligence: the expensive chips that run chatbots often sit idle, waiting for data.

When an assistant works through a long conversation or a multi-step task, it keeps a running memory of everything it has already read. That memory outgrows the chip. Marvell’s answer is a shared memory pool, reached by light over cables up to 50 meters long, that holds about as much as 32 laptops.

Marvell says this lets the same building produce two to three times more answers on the same electricity. That figure has not yet been tested by anyone else.

Marvell Technology announced on August 4, 2026, at the FMS 2026 memory and storage conference in Santa Clara, California, a set of products aimed at AI inference, the stage where trained models answer requests. The lineup spans three levels of the data center: the Bravera SC6 PCIe 6.0 SSD controller for individual servers, Structera X CXL memory expansion for racks, and Photonic Fabric memory modules, network cards and chiplets that form an optical shared-memory tier across multiple racks.

Marvell says Photonic Fabric reaches up to 50 meters, supports up to 32TB of warm KV cache offload and delivers up to 2-3x higher token throughput within existing footprints and power budgets. It says Bravera SC6 doubles the performance of its Bravera SC5 predecessor and is expected to begin sampling in Q4 2026. Marvell gave no availability dates for the Structera X or Photonic Fabric products.

Executive Summary

Marvell is presenting memory, not processors, as the next place where AI inference performance is won or lost. All three products in the announcement target the same object: the KV cache. This is the working memory a language model builds as it reads a prompt, and it grows with longer conversations and with more simultaneous users. The company argues that expanding, pooling and sharing memory separately from GPUs reduces the time those GPUs spend idle, which it calls stalls, and raises output per watt.

This matters to anyone who pays for AI infrastructure by the megawatt. If the claims hold, operators could get more usable work from GPU fleets they have already bought and powered, without adding floor space or grid capacity. The strongest number, the 2-3x throughput gain, is framed as an “up to” figure with no stated baseline or workload. Only one of the products, the SC6 controller, has a date, and that date is for sampling rather than volume shipment.

Three Tiers Aimed at One Problem

The announcement is best read as a memory hierarchy rather than three separate products. At the top sits high-bandwidth memory (HBM), the fast but scarce memory stacked beside each GPU. Below it, Structera X uses CXL (Compute Express Link, an open standard for attaching extra memory to processors over the same physical lanes as PCIe) to add and pool DRAM at the rack level. Photonic Fabric then extends a shared memory tier across racks over optical links of up to 50 meters. At the bottom, the Bravera SC6 controller makes SSDs fast enough that, in Marvell’s words, more KV cache can “move from high-bandwidth memory to SSD.”

Every tier has the same stated purpose: holding the KV cache somewhere cheaper and larger than HBM, while keeping it close enough that the GPU does not wait. Marvell’s framing, that the portfolio helps hyperscalers “scale memory more independently from compute,” is a statement about data center design. It describes memory as a resource provisioned at the rack and pod level, not a fixed amount soldered into each server.

Why KV Cache Placement Drives Throughput

The case for the headline rests on how inference works. When a model processes a prompt, it computes intermediate values for every token and stores them in the KV cache so it does not have to recompute them for each new word it generates. That cache grows with context length and with the number of concurrent sessions. Agentic AI, where a model runs multi-step tasks and carries long histories between steps, pushes both factors up at the same time. Once HBM fills, the operator has three options. It can evict the cache and recompute it later, which spends GPU cycles on repeated work. It can fetch the cache from storage, which is slow. Or it can fetch it from a faster intermediate tier.

Marvell’s throughput claim sits on that third option. The company says Photonic Fabric enables KV cache “to be loaded from a shared memory tier rather than storage,” offering up to 32TB of warm cache (recently used but not currently active) across multiple XPUs (Marvell’s umbrella term for GPUs and custom accelerators). From that it claims up to 2-3x higher token throughput within the same footprint and power envelope. The GPUs are unchanged. What changes is how often they sit idle waiting for context to be rebuilt or retrieved. Operators most affected are those running long-context, many-user inference on power-capped sites, where adding GPUs is not an option but raising their utilization is.

This supports a narrower claim than “memory has replaced GPUs as the bottleneck,” and the release itself does not go that far. It says memory is becoming “as critical as compute.” 650 Group analyst Alan Weckel, quoted in the release, calls memory capacity, bandwidth, latency and data movement “primary constraints.” The defensible reading is that for inference workloads with large KV caches, gains now come from memory placement as much as from raw compute.

What the Numbers Do and Do Not Establish

The concrete figures are the 32TB capacity, the 50-meter reach, the 2x performance step from SC5 to SC6, and the Q4 2026 sampling date for SC6. The 50-meter reach is significant for facility planners. It means the shared tier spans multiple racks, enough to cover a GPU pod, and it adds optical cabling inside the pod.

The 2-3x figure is the least specified. It carries an “up to” qualifier and no stated model size, context length, batch size or comparison system. Throughput gains from cache offload depend heavily on how often the cache is reused, so the figure is best read as an upper bound for favorable workloads, not a fleet-wide expectation. Likewise, “doubling the performance” of SC5 fits the generational jump from PCIe 5.0 to 6.0 but does not specify which metric doubles. “Extremely low latency” for the optical tier is not quantified at all.

Sourcing Flexibility and the Buyer’s Calculus

One of the more practical details concerns procurement. Bravera SC6 is NAND-agnostic, meaning it works with flash memory from multiple suppliers. For cloud providers, that lowers dependence on a single flash vendor, a real consideration in a market prone to supply swings. Structera X is pitched in similar terms, as a way to extend “the value of existing memory investments” rather than forcing wholesale replacement.

The trade-off is architectural complexity. Each tier adds devices, interconnects and software orchestration that must decide, in real time, where each session’s cache should live. The added power draw of CXL expanders, optical network cards and memory modules also counts against the power envelope the throughput gains are meant to protect. Buyers will need measured results on their own workloads before the net efficiency claim can be verified.

Background

Marvell Technology (NASDAQ: MRVL) is a Santa Clara-based semiconductor company focused on data infrastructure: chips that move, store, process and secure data for cloud, enterprise and carrier networks. It says it has served leading technology companies for over 30 years. Its Bravera line covers storage controllers, its Structera line covers CXL memory, and Photonic Fabric is its optical interconnect and memory technology.

As AI shifts from training models to serving them at scale, the memory attached to each GPU has become a practical limit. Models that handle long documents, long conversations or multi-step agent tasks build large KV caches that can exceed on-package high-bandwidth memory. Chipmakers and cloud providers are responding with tiered designs that spill that cache into pooled DRAM, shared memory and fast flash storage. FMS, the annual memory and storage conference, is a common venue for these announcements.

Sources

Source: Marvell Advances AI Memory Infrastructure Portfolio to Accelerate Agentic AI Inference: Marvell Technology’s August 4, 2026 announcement of Bravera SC6, Structera X and Photonic Fabric memory products at FMS 2026.