jain.com — data centers, connectivity and security, analyzed daily

Tag: AI Accelerators

  • TSMC Reportedly to Double CoWoS Packaging Capacity by 2028

    TSMC Reportedly to Double CoWoS Packaging Capacity by 2028

    TL;DR · 30-second read

    The Short Version

    The chips that run artificial intelligence are not held up by the factories that print them. They are held up by the assembly step at the end, where a processor and its memory are bonded onto a single slab so they can talk to each other fast enough.

    Taiwan Semiconductor Manufacturing Company owns most of the world’s capacity for that step. Analysts in Taiwan say it plans to roughly double that capacity by 2028. Until it does, anyone building an artificial intelligence data center is standing in a queue. Intel is selling a rival method, and smaller assembly firms are absorbing the overflow.

    Wccftech, citing a Taiwanese media report that quotes analysts, said TSMC plans to expand its chip-on-wafer-on-substrate (CoWoS) advanced packaging capacity to 260,000 wafers per month by the end of 2028, roughly double the approximately 130,000 wafers per month the same forecast puts it at by the end of 2026. The expansion is said to center on TSMC’s Arizona campus and its AP7 packaging facility in Taiwan.

    The same forecast estimates that Intel’s competing EMIB-T packaging, converted into CoWoS-equivalent 12-inch wafer terms, could reach 15,000 to 20,000 wafers per month in 2027 and 40,000 to 45,000 in 2028, and says the CoWoS shortage has pushed work toward other packaging firms including Amkor, UMC and ASE. TSMC has not published a capacity target of its own. In a Form 6-K filed with the Securities and Exchange Commission on September 10, the company reported August 2026 consolidated revenue of NT$514.81 billion, up 53.3 percent from a year earlier.

    Executive Summary

    Advanced packaging is the step where a logic die and its stacks of high-bandwidth memory are mounted together onto a shared substrate so they behave as one component. CoWoS is TSMC’s version, and it is the packaging used for the accelerators that fill AI data centers. For roughly three years it, rather than leading-edge wafer manufacturing, has been the narrowest point in the supply chain for those parts.

    A doubling of that capacity by 2028 would be the single largest change to AI hardware availability that any one company can make. It would not, however, double accelerator output in a straight line. Package sizes are growing, the newest designs consume more interposer area and more memory stacks per unit, and a wafer of packaging substrate therefore yields fewer finished accelerators each generation than it did the year before. Capacity that doubles against demand that also rises per unit is not the same as supply that doubles.

    TSMC’s own disclosure does not address packaging capacity. What its September 10 filing does show is demand running hot: August revenue rose 53.3 percent year over year, and revenue for the first eight months of 2026 was up 39.3 percent. That is consistent with a company selling everything it can finish, which is the condition under which a capacity forecast of this kind becomes credible without being confirmed.

    The Bottleneck Moved Downstream

    For most of the history of the chip industry, the scarce resource was the front end: photolithography tools, leading-edge wafer starts, yield on a new process node. Packaging was the cheap, unglamorous back end. AI accelerators inverted that. A modern accelerator is not one chip but an assembly, a large logic die surrounded by stacks of memory, all sitting on a silicon interposer threaded with vertical connections called through-silicon vias. The density of those connections is what delivers the memory bandwidth the workload needs. That assembly step is CoWoS, and there is no abundant substitute for it.

    The practical consequence for anyone financing a data center is that wafer capacity is no longer the number to watch. A foundry can start more wafers than it can finish into packages, and unfinished silicon does not train models. The forecast of 130,000 wafers per month at the end of 2026 rising to 260,000 by the end of 2028 is, in effect, a two-year schedule for how quickly the industry can convert build-ready halls into loaded racks.

    Read it with care, though. Interposer area per accelerator has been climbing with every generation as designers add memory stacks and reticle-stitched logic. If the average package consumes meaningfully more of a wafer in 2028 than it does today, unit output grows by considerably less than the wafer count suggests. Doubling the input is a ceiling on supply growth, not a promise of it.

    Arizona Can Print the Chips but Cannot Finish Them

    The most operationally interesting detail in the forecast is geographic. Wafers produced at TSMC’s Arizona campus are flown to Taiwan to be packaged, because the packaging capacity sits on the island. Siting part of the expansion in Arizona alongside the AP7 facility in Taiwan would begin to close that loop, turning an American fab from a wafer source into something closer to a finished-component source.

    That matters for reasons beyond logistics. A round trip across the Pacific adds days of cycle time and a customs and freight cost to every high-value wafer, and it means that the resilience argument for building fabs outside Taiwan is currently incomplete. A domestic fab whose output must still transit Taiwan to become a usable product does not remove the concentration risk that justified it. TSMC’s own filing shows the Arizona entity is treated as a significant subsidiary, with parent guarantees extended to TSMC Arizona among other affiliates, but the filing says nothing about packaging there.

    For data center operators and the governments underwriting them, this is the question worth asking of any onshoring announcement: does the site produce a wafer or a shippable part? Until packaging follows fabrication, the answer for Arizona remains the former.

    Intel’s EMIB-T and the Value of a Second Source

    Intel’s EMIB-T takes a different architectural route. Instead of building the whole assembly on a large silicon interposer, it embeds small silicon bridges inside the organic substrate and places the logic and memory dies on top. Less silicon is used, which can mean lower cost and fewer of the handling and warpage problems that come with very large interposers, but the approach historically trades away some of the interconnect density that the largest, most bandwidth-hungry designs demand.

    The forecast puts EMIB-T at 40,000 to 45,000 CoWoS-equivalent wafers per month by 2028. Against 260,000, that is a meaningful second source rather than a competitive threat, on the order of a sixth of the leader’s volume. The claim that it suits custom accelerators of the kind Google and Amazon design is plausible on architecture, since in-house cloud silicon is generally tuned for a known workload and a known cost target rather than for maximum bandwidth per package. It is also unverified: neither Intel nor any cloud provider has announced such an arrangement.

    The spillover to Amkor, UMC and ASE follows the same logic. When the preferred supplier is sold out, work migrates to whoever can perform an adjacent version of it, and outsourced assembly and test firms have been the natural landing place. Buyers get optionality and a check on pricing. The risk is that second-source packaging is not a drop-in substitute. Requalifying a part on a different packaging architecture is an engineering program, not a purchase order, and the customers best placed to do it are the ones designing their own silicon.

    What Is Substantiated and What Is Not

    The demand side of this story rests on disclosed figures. TSMC’s September 10 filing with the Securities and Exchange Commission reports August consolidated revenue of NT$514.81 billion, a 10.1 percent increase on July and 53.3 percent on August 2025, with the first eight months of 2026 up 39.3 percent year over year. Growth at that rate at TSMC’s scale is difficult to explain without sustained AI accelerator demand behind it.

    The supply side is a forecast, not a commitment. The 260,000 figure, the 130,000 baseline, the Arizona and AP7 split and the EMIB-T range all come from analysts rather than from the companies. None of the three firms named has published a packaging capacity target, a capital budget attached to one, or a schedule. Analyst capacity models in this industry have a reasonable track record on direction and a poorer one on timing, because packaging expansions depend on tool delivery, cleanroom construction and qualification cycles that slip quietly.

    The sober reading is that the direction is well supported and the magnitude is an estimate. Treat 2028 as the earliest plausible date for materially looser accelerator supply, treat the doubling as a ceiling rather than a plan of record, and treat any procurement model that assumes it as carrying schedule risk that TSMC has not underwritten.

    Background

    TSMC is the world’s largest contract chip manufacturer, building processors designed by other companies. It commercialised CoWoS in the 2010s as a niche technique for high-performance parts, long before AI accelerators turned it into the scarcest resource in the industry. Today the packaging step determines how many finished accelerators reach the market, because a logic die without its memory bonded alongside it is not a usable product.

    The competitive response has come from two directions. Intel has pushed its own bridge-based packaging as part of its foundry ambitions, targeting customers who want a supplier outside Taiwan or a lower-cost assembly for custom silicon. Outsourced assembly and test firms such as Amkor, ASE and UMC have absorbed work that TSMC cannot schedule. None of this has yet changed the basic shape of the market, in which one company’s packaging line sets the pace at which AI data centers can be filled.

    Sources

    Source: Report: TSMC to Double CoWoS Capacity by 2028 as AI Chip Shortage Spills Over to Rivals, which relays a Taiwanese analyst forecast for TSMC’s advanced packaging capacity and for Intel’s competing EMIB-T volumes.

    Primary sources: Taiwan Semiconductor Manufacturing Company, Form 6-K filed September 10, 2026 (August 2026 revenue report), filed with the US Securities and Exchange Commission.

  • Gimlet Labs’ $300M Raise Puts Power at the Center of AI Compute

    Gimlet Labs’ $300M Raise Puts Power at the Center of AI Compute

    TL;DR · 30-second read

    The Short Version

    A company called Gimlet Labs has raised $300 million from investors, who now put the value of the whole business at $3 billion. It builds the computing systems that run artificial intelligence.

    Two things stand out. It plans to use six different kinds of computer chip instead of relying on a single supplier. And it wants several hundred megawatts of electricity, roughly what a small city consumes.

    That second part is the hard part. Getting power, not getting chips, is now the main thing holding new artificial intelligence sites back.

    Ascendants.in reported that Gimlet Labs has raised $300 million at a $3 billion valuation, and that the company is scaling a six-chip artificial intelligence strategy toward several hundred megawatts of capacity. A megawatt is a unit of electrical demand; several hundred of them is the scale of a mid-sized data center campus rather than a single leased hall.

    The report does not name the investors in the round, the six silicon platforms the strategy spans, or the sites where the power would be delivered. What it does establish is the shape of the company’s pitch: multi-vendor compute, sold at a scale measured in electricity rather than in racks.

    Executive Summary

    The headline number is the $300 million. The more interesting number is the megawatts. A startup describing its roadmap in units of power rather than units of silicon is telling investors that its scarcest input is grid capacity, and that it believes it has secured, or can secure, a claim on some.

    The six-chip element is a second signal. Most AI compute providers have built their businesses on a single accelerator family, which makes them fast to deploy and hostage to one vendor’s allocation, pricing and product cadence. Spanning six platforms is a deliberate trade: more engineering overhead and a harder software story, in exchange for supply optionality and the ability to match different workloads to different silicon economics.

    Together, the two claims describe a company positioning itself between the merchant AI cloud operators and the traditional colocation landlords, and asking to be valued at ten times the capital it just raised on that positioning.

    Power Procurement Is Now the Product Roadmap

    For most of the last decade, an AI compute company’s competitive position was a function of chip allocation. Whoever could get accelerators first could sell capacity first. That constraint has not disappeared, but a second one has moved ahead of it in many markets: interconnection, the process by which a new large load is granted a connection to the electricity grid. In congested regions, the queue for that connection is measured in years, and it is not something a funding round can shorten.

    This is why a several-hundred-megawatt figure functions as a credential. It implies the company has done, or is doing, the unglamorous work that separates announced capacity from energized capacity: site control, substation and transformer procurement, utility load studies, and in many cases some form of on-site or contracted generation. None of that is visible in a valuation, but all of it determines whether the valuation is underwritten by anything.

    The corollary is a genuine risk. Megawatt figures in this sector are quoted at very different stages of certainty. There is capacity under a signed power agreement, capacity in an interconnection queue, capacity in a letter of intent with a landlord, and capacity that is simply a planning target. Until a company distinguishes among them, a several-hundred-megawatt claim describes an ambition rather than an asset.

    Why Six Chips Instead of One

    Running six accelerator platforms is harder than it sounds. Each has its own software stack, its own compiler and libraries, its own memory architecture and its own thermal and power profile. Supporting all of them means either forcing customers to port their models repeatedly or building an abstraction layer that hides the differences, which is one of the more demanding pieces of systems engineering in the field. It also fragments spare-parts inventory, firmware management and the staff expertise needed to operate the fleet.

    The payoff is negotiating position and workload fit. A buyer with six qualified suppliers is not exposed to any one vendor’s allocation decisions or lead times, and can place inference work, which is often cost- and latency-sensitive, on cheaper or more power-efficient parts while reserving the largest chips for training. In a market where accelerator supply and pricing have both moved sharply, that flexibility has real option value.

    Whether it is worth the overhead depends on something the company has not put a number to: how much of its fleet is actually diversified versus how much is a single dominant platform with five qualified alternatives. Those are very different businesses with the same description.

    What a $3 Billion Valuation Has to Underwrite

    AI compute at this scale is a capital-intensive, thin-margin business dressed in software multiples. Servers depreciate on a schedule shorter than the buildings that house them; power contracts and leases run longer than most customer commitments; and the residual value of a given accelerator generation is difficult to forecast when the next generation is announced annually. The companies that have made this work have generally done so by matching long-dated contracted revenue against long-dated obligations.

    A $3 billion mark on a $300 million round therefore raises a specific question rather than a rhetorical one: what fraction of the several hundred megawatts is contracted, to whom, and for how long? Take-or-pay commitments from creditworthy customers turn a power position into an annuity. Uncontracted capacity turns it into inventory with a carrying cost.

    The strategic logic is nonetheless coherent. If power is the binding constraint and silicon is the fungible input, then the durable asset in AI infrastructure is the energized site, and the chips inside it are a refresh cycle. A company that internalizes that ordering early, and can genuinely operate heterogeneous fleets, is building something less exposed to any single vendor’s roadmap. The valuation is a bet that it can execute on both halves at once.

    Background

    AI compute providers — sometimes called neoclouds — emerged as a distinct category as demand for accelerated computing outran what established cloud platforms could deliver. They buy or lease power and space, fill it with AI accelerators, and sell that capacity to model developers and enterprises. The model is capital-intensive: hardware depreciates faster than the buildings and power contracts that support it, so operators typically try to match long-term customer commitments against long-term obligations.

    Over the past two years the sector’s binding constraint has shifted. Early on, the scarce resource was accelerator allocation. Increasingly it is electricity — specifically, the ability to secure a grid connection for a large new load in a reasonable timeframe. That shift has changed how these businesses describe themselves, with megawatts replacing rack counts as the headline metric, and has pushed some operators toward multi-vendor silicon strategies to reduce dependence on any single chip supplier’s roadmap and allocation decisions.

    Sources

    Source: Gimlet Labs Raises $300 Million at $3 Billion Valuation as Six-Chip AI Strategy Scales to Several Hundred Megawatts — ascendants.in reports the funding round, valuation and multi-platform compute strategy.

  • Broadcom’s AI Silicon Now Outsells Everything Else It Makes

    Broadcom’s AI Silicon Now Outsells Everything Else It Makes

    TL;DR · 30-second read

    The Short Version

    • Broadcom designs the specialist chips that big internet companies use to train and run artificial intelligence. It just said sales of those chips hit $16.7 billion in three months — triple the same period last year.
    • That single product line is now bigger than everything else Broadcom sells combined.
    • The company expects the next quarter to be bigger still: $21.7 billion.
    • Why it matters: the biggest technology firms are no longer just buying chips off the shelf. They are paying to have their own designed, and they are committing years ahead.

    Broadcom reported results for its third fiscal quarter, ended August 2, 2026, on September 2, disclosing them in an 8-K filed with the Securities and Exchange Commission that day. Revenue reached $29.6 billion, up 86 percent year over year. GAAP operating income was $16.0 billion and non-GAAP operating income $20.1 billion. The company generated $14.2 billion in cash from operations against $0.5 billion of capital expenditure, leaving $13.7 billion in free cash flow — 46 percent of revenue. The board declared a quarterly dividend of $0.65 per share, payable September 30 to holders of record on September 21.

    The number that moved the story sits in the release’s first quote. “Demand for our custom AI accelerators and networking continues to be very strong,” said president and CEO Hock Tan, putting third-quarter AI semiconductor revenue at $16.7 billion — up 221 percent year over year and 54 percent from the prior quarter — and guiding the fourth quarter to $21.7 billion, up 236 percent. Total fourth-quarter revenue guidance is approximately $34.8 billion at a non-GAAP operating margin of about 66 percent. Bloomberg reported that Broadcom predicted a surge in AI chip sales over the next two years.

    Executive Summary

    Broadcom’s AI semiconductor line crossed a threshold this quarter that is easy to miss in the percentages: at $16.7 billion out of $29.6 billion in total revenue, custom AI accelerators and the networking silicon that connects them are now the majority of what the company sells. The remainder of the business — the legacy semiconductor franchises and the infrastructure software estate built around VMware — accounts for roughly $12.9 billion, implied by subtraction from the reported totals. On the company’s own fourth-quarter guidance, AI would rise to about 62 percent of a $34.8 billion quarter while everything else stays close to flat.

    The sequential figure matters more than the annual one. A 54 percent quarter-over-quarter jump is not the shape of a business being compared against an easy year-ago base; it is the shape of a business shipping against orders placed some time ago and ramping into volume production. Custom accelerators — application-specific integrated circuits, or ASICs, designed for one customer’s workloads rather than sold to the market at large — carry long design cycles. Revenue arriving now reflects commitments made well before this quarter.

    For infrastructure buyers and investors, the strategic reading is that hyperscaler silicon programs have moved past the pilot stage into committed multi-year production, and that they are running alongside merchant GPU purchases rather than replacing them. Broadcom’s results substantiate the demand and the margins. They do not, on their own, substantiate the durability, and the company has disclosed nothing about which customers, at what contracted volumes, for how long.

    The Crossover Quarter

    Until recently, Broadcom was a diversified semiconductor and software company that also happened to have a promising AI franchise. The third quarter inverts that description. AI semiconductor revenue of $16.7 billion against $29.6 billion in total revenue means the AI line is the company, and the rest is the ballast. Guidance extends the trend: $21.7 billion of AI revenue inside $34.8 billion of total revenue would put the non-AI business at roughly $13.1 billion, essentially unchanged from this quarter’s implied $12.9 billion.

    That mix shift has consequences beyond the headline. A company whose growth is concentrated in one product line and, by the nature of custom silicon, a small number of buyers is valued and stressed differently from a diversified one. Broadcom’s non-GAAP operating margin held at about 68 percent in the quarter, and management guided to approximately 66 percent for the fourth — flat year over year, as CFO Amie Thuener framed it in the release. Holding margin while nearly doubling revenue is the genuinely difficult part of this result, and it suggests custom accelerator work is not being won on price.

    Custom Silicon Is Additive, Not Substitutive

    The convenient narrative is that hyperscalers build their own chips to escape their GPU supplier. The economics point somewhere more mundane. Custom accelerators are built for workloads a cloud operator runs at enormous, predictable scale — a specific model architecture, a specific inference pattern — where a chip stripped of general-purpose flexibility wins on performance per watt and per dollar. Merchant GPUs remain the tool for everything else: new model architectures, third-party customers renting capacity, and any workload whose shape is not yet settled.

    The evidence in Broadcom’s own numbers supports the additive reading. AI revenue tripling year over year during a period when merchant accelerator demand has also been strong is not the profile of a substitution; a zero-sum shift would show up as one line rising while another falls. Broadcom does not disclose the split between custom accelerators and networking within that $16.7 billion, which matters because the networking half — the Ethernet switching and connectivity silicon that stitches thousands of accelerators into one training fabric — grows with every deployed cluster regardless of whose compute sits inside it. That is a structurally more defensible position than accelerator design alone.

    Fabless Economics in a Capital-Intensive Boom

    The single most striking line in the release is not a revenue figure. Broadcom spent $0.5 billion on capital expenditure in a quarter that generated $14.2 billion of operating cash flow — capex under two percent of revenue. Its customers, meanwhile, are pouring tens of billions into buildings, substations, transformers and cooling plant to house what Broadcom designs. Broadcom carries design risk and supply-chain allocation risk; the balance-sheet risk of the buildout sits with the operators and with the foundries that fabricate the wafers.

    That asymmetry cuts both ways. It explains 46 percent free-cash-flow conversion and a dividend that cost roughly $3.1 billion last quarter against $13.7 billion of free cash flow — a payout the company can sustain through a downturn. It also means Broadcom has limited ability to absorb a demand pause: with no large fixed asset base to depreciate against, revenue that does not arrive simply does not arrive, and the cost structure that produces 66 percent operating margins is largely engineering headcount that cannot be flexed quickly without damaging the design pipeline that generates the next cycle’s orders.

    What the Guidance Actually Commits To

    Broadcom guided one quarter. The $21.7 billion AI figure for the fourth quarter is a forecast the company has put its name to in an SEC filing, with the standard caveat that actual results will vary and may vary materially. The longer-horizon framing that dominated coverage of the results — a multi-year surge — came from management commentary around the report, not from the guidance in the release itself. The distinction is worth holding onto: one is a number a company can be held to next quarter, the other is a direction of travel.

    For buyers of infrastructure, the practical signal is about lead times and allocation rather than about Broadcom’s stock. If custom accelerator programs are ramping at this rate, they are competing for the same advanced packaging capacity, high-bandwidth memory supply and foundry slots as merchant GPUs, and for the same grid interconnections and long-lead electrical equipment once the silicon lands in a building. Tightness in any of those upstream inputs would show up in Broadcom’s shipment schedule before it showed up in its order book — which is precisely why the absence of disclosed backlog makes the trajectory harder to underwrite than the quarter itself.

    Background

    Broadcom is a Palo Alto-based designer of semiconductors and infrastructure software, assembled over more than a decade of large acquisitions — networking, storage connectivity, mainframe software, security, and most recently the virtualisation company VMware. It does not own the factories that make its chips; it designs them and contracts manufacturing to foundries, a model that keeps capital spending low and cash conversion high.

    Its position in AI infrastructure rests on two adjacent franchises. The first is co-designing custom accelerators with large cloud operators who run workloads at sufficient scale to justify a bespoke chip. The second is Ethernet switching and connectivity silicon, which links thousands of accelerators into the single fabric a large training cluster needs. The first is high-value but concentrated among a handful of buyers; the second grows with total industry deployment. Both are now reported together as AI semiconductor revenue, the line that crossed half of company sales this quarter.

    Sources

    Source: Broadcom Q3 Results: CEO Tan Forecasts Surging AI Infrastructure Buildout — coverage of Broadcom’s third-quarter fiscal 2026 results and management’s outlook for AI infrastructure demand.

    Primary sources: Broadcom Inc., Form 8-K filed September 2, 2026 (results of operations and declaration of a $0.65 quarterly dividend); Exhibit 99.1 — Broadcom Inc. Announces Third Quarter Fiscal Year 2026 Financial Results and Quarterly Dividend (revenue, margin, cash flow and fourth-quarter guidance, including AI semiconductor revenue figures quoted by Hock Tan and Amie Thuener).

  • TSMC Moves the Cooling Fight Inside the Chip

    TSMC Moves the Cooling Fight Inside the Chip

    TL;DR · 30-second read

    The Short Version

    The chips that run artificial intelligence are getting so hot that cooling them from the outside is starting to run out of room. TSMC, the Taiwanese company that manufactures most of the world’s advanced processors, is looking at cutting channels finer than a human hair into the chip itself and pumping liquid through them, right next to where the heat is made.

    Forecasts put the power a single artificial intelligence system draws at roughly six times today’s level within five years. If cooling moves inside the chip, the company that builds the chip starts selling the cooling too.

    TrendForce reported that TSMC is evaluating microchannel cooling integrated at the chip level — etching or bonding fluid channels directly into the silicon package so coolant flows microns from the transistors — as projected power draw for AI systems rises roughly sixfold over the next five years. The report frames the technology as a response to the point at which conventional external cooling, including metal cold plates strapped to the top of a processor, stops keeping pace with heat generated inside the package.

    TSMC has not addressed the technology in its recent filings with United States regulators. Its 6-K filed August 25, 2026 covering July activity reported no board-approved capital appropriations for the month, and its 6-K filed September 1, 2026 dealt with a routine dividend adjustment.

    Executive Summary

    For a decade, cooling improvements in computing happened outside the chip: better fans, then cold plates, then facility water loops and immersion tanks. Chip-level microchannel cooling breaks that pattern. It puts the coolant inside the package itself, shortening the path heat must travel from transistor to fluid — the single largest source of thermal resistance in a modern accelerator stack. If TSMC productizes it, cooling stops being purely a data center purchase and becomes partly a foundry and advanced-packaging purchase.

    The sixfold power projection is the forcing function. Each generation of AI accelerator has raised per-package power while shrinking the area over which that heat can be spread. External cooling can move more fluid faster, but it cannot shorten the distance between the hot spot and the fluid. Microchannels can, which is why the approach has moved from research curiosity toward roadmap discussion.

    The commercial implications run in several directions at once: advanced packaging gains value, cold-plate and thermal-interface suppliers face a long-run substitution question, and data center operators inherit a component that is sealed, unserviceable and warranted by the chipmaker rather than the integrator. None of that is settled, and TSMC has published no timeline, node, capacity or customer detail.

    Why the Thermal Bottleneck Has Moved Below the Heatsink

    Heat leaving a processor passes through a chain of obstacles: the silicon itself, a thermal interface material, a metal lid, another interface layer, and finally a cold plate or heatsink carrying the heat away. Each junction adds resistance. Data center engineering over the past few years has attacked the last link in that chain — swapping air for water, moving from rear-door heat exchangers to direct-to-chip cold plates, adding coolant distribution units. Those changes worked, but they optimise the part of the path furthest from the problem.

    Microchannel cooling attacks the first link. By routing fluid through channels formed in or immediately against the silicon, it removes several interface layers and places coolant adjacent to the hot spots rather than millimetres above them. That matters most for the tightly packed, high-power dies used in AI training accelerators, where heat is not evenly distributed and a small area of the die can limit the performance of the whole package.

    The projected sixfold rise in AI system power over five years is what makes this urgent rather than academic. Cooling capacity outside the package can be scaled by spending more — bigger pumps, colder water, more plumbing. Thermal resistance inside the package cannot be spent away. Once it dominates, the only remaining lever is to redesign the package, which is precisely where a foundry with leading advanced-packaging capability has an advantage.

    Cooling Becomes a Foundry Product

    The business consequence is a shift in where cooling value is captured. Today a hyperscaler — an operator of very large cloud data centers — buys accelerators from a chip designer and cooling from a separate thermal supply chain of cold-plate makers, manifold vendors, pump and coolant distribution unit manufacturers. If the highest-value part of the thermal path is fabricated at the packaging stage, part of that spend migrates upstream into the price of the package.

    That is a favourable structural position for TSMC, which already assembles AI accelerators using advanced packaging that stacks logic dies and high-bandwidth memory on a shared substrate. Adding fluid channels to that flow extends an existing chokepoint rather than opening a new competitive front. For established thermal suppliers, the risk is gradual rather than immediate: facility-side cooling, plumbing and heat rejection remain necessary regardless, because heat still has to leave the building. What could shrink is the premium attached to the interface between chip and coolant.

    There are real costs on the other side of the ledger. A sealed fluid path inside a package is not field-serviceable. Leaks, corrosion, particulate clogging and mechanical stress become manufacturing yield and warranty problems for the chipmaker rather than maintenance problems for the operator. Testing a package that only performs correctly when fluid is flowing complicates production test. Those are engineering problems with known research literature behind them, but solving them at high volume and acceptable yield is a different exercise from demonstrating them.

    What TSMC’s Filings Do and Do Not Show

    TSMC’s recent disclosures to United States regulators contain nothing about the technology. The 6-K filed on August 25, 2026, which reports the company’s July activity, listed no capital appropriations approved by the board that month, recorded a NT$0.7 billion equity investment disposition, and reported no share pledges or cancellations. The 6-K filed on September 1, 2026 was narrower still: an adjustment of the first-quarter 2026 cash dividend from NT$7.0 to NT$7.00000137 per common share, payable October 8, 2026, after shares were reclaimed from 2024 restricted stock awards.

    This absence is not evidence against the programme. Research and early development work does not normally surface in monthly shareholding and appropriation reports, and companies routinely discuss roadmap directions publicly long before they appear as line items. But it does bound what can currently be said: there is no disclosed capital commitment, no announced capacity, and no financial scale attached to chip-level cooling at TSMC.

    For investors, that suggests where to look next rather than what to conclude now. Board-approved capital appropriations in the monthly filings, advanced-packaging capacity announcements, and any customer disclosure from accelerator designers would be the first hard signals that microchannel cooling has moved from evaluation to committed investment.

    What It Changes for Data Center Buyers — and What It Doesn’t

    Operators planning liquid-cooled halls should not read this as a reason to wait. Chip-level cooling reduces resistance inside the package; it does not eliminate the need for a facility water loop, coolant distribution units, heat rejection equipment or the electrical capacity to feed the racks. If anything, a package designed to shed more heat more efficiently makes the facility-side loop more important, because more watts per rack must ultimately be rejected to air, water or the outside environment.

    The more consequential change is contractual. Cooling that ships inside the processor moves specification authority to the chipmaker: coolant chemistry, filtration standards, inlet temperature and flow requirements would become conditions of the chip’s warranty rather than choices made by the integrator. Operators used to negotiating thermal design with a systems vendor may find those parameters arriving pre-set.

    None of this is imminent. What the direction of travel does justify is designing new capacity for liquid distribution and water quality management as a baseline rather than an upgrade, since every plausible version of the next five years — cold plates, microchannels or both — ends with liquid at the rack.

    Background

    TSMC is the world’s largest contract chip manufacturer, producing leading-edge processors for companies that design but do not fabricate their own silicon. Its role in AI hardware extends beyond making transistors: its advanced packaging capacity, which bonds logic dies and high-bandwidth memory onto a shared substrate, has been one of the industry’s tightest bottlenecks through the current build-out. As a foreign private issuer listed in New York, TSMC files monthly 6-K reports with United States regulators covering board capital appropriations, asset transactions, bond issuance and insider shareholdings.

    Cooling has escalated in parallel. Air cooling gave way to rear-door heat exchangers, then to direct-to-chip cold plates and facility water loops as accelerator power outgrew what fans could handle. Each step moved the working fluid closer to the processor while leaving the package itself untouched. Chip-level microchannel cooling is the logical next step in that sequence — and the first one that would be manufactured by the chipmaker rather than installed by the data center.

    Source: [News] TSMC Eyes Chip-Level Microchannel Cooling as AI System Power Could Rise 6× in Five Years — TrendForce report on TSMC’s evaluation of microchannel cooling integrated at the chip level against projected growth in AI system power.

    Primary sources: Taiwan Semiconductor Manufacturing Company, Form 6-K filed September 1, 2026 (adjustment to first-quarter 2026 cash dividend per share); Taiwan Semiconductor Manufacturing Company, Form 6-K filed August 25, 2026 (monthly report covering July 2026 shareholdings, asset transactions and capital appropriations).

  • Google’s $12.2B Marvell Deal Reshapes the Custom AI Chip Race

    Google’s $12.2B Marvell Deal Reshapes the Custom AI Chip Race

    Google has expanded its custom AI chip partnership with Marvell Technology in a deal reported at $12.2 billion, according to multiple Yahoo Finance reports published this week. Broadcom — long regarded as Google’s incumbent partner for custom AI accelerators — saw its shares fall 6.2% on the news, while analyst fair-value estimates for Marvell edged higher.

    Executive Summary

    The reported agreement deepens Google’s relationship with Marvell for custom silicon — chips designed to a single customer’s specification rather than sold off the shelf. In AI infrastructure, these custom accelerators (often called XPUs or ASICs) are the hyperscalers’ primary lever for reducing dependence on Nvidia’s general-purpose GPUs, and the design partner that wins the engagement captures years of high-visibility revenue.

    The market reaction tells the story in one frame: Broadcom, which has been widely credited as the co-design partner behind Google’s Tensor Processing Units (TPUs), dropped 6.2%, while Marvell’s bull case strengthened. A $12.2 billion figure, if it represents committed or expected purchases, would be one of the larger custom-silicon engagements publicly reported — though the source articles leave the deal’s structure, duration, and scope largely undefined.

    For the broader AI infrastructure market, the significance is less about one stock move and more about confirmation of a trend: hyperscalers are dual-sourcing their chip design partners the same way they dual-source power, fiber, and data center capacity — to control cost, schedule risk, and negotiating leverage.

    Why Hyperscalers Refuse to Depend on One Chip Partner

    Custom AI accelerators are multi-year commitments. A hyperscaler like Google picks a design partner, co-develops a chip over 18–36 months, then ramps production across successive generations. That timeline creates lock-in — and lock-in creates pricing power for the partner. Broadcom’s custom-silicon business has been a major beneficiary of exactly that dynamic. By expanding work with Marvell, Google gains a credible second source, which pressures pricing on every future generation and insulates its TPU roadmap from any single vendor’s execution stumbles.

    This mirrors how large infrastructure buyers behave everywhere in the stack. No serious operator single-sources grid power, network transit, or construction contractors for a multi-gigawatt buildout. As custom silicon becomes as strategically important as the data centers that house it, the same procurement discipline is arriving in chip design.

    Broadcom’s 6.2% Drop: Signal Versus Substance

    A one-day 6.2% decline reflects what investors fear, not necessarily what Google has decided. The reports do not state that Google is reducing its Broadcom engagement — only that it is expanding Marvell’s. Those are different things: Google’s total accelerator demand is growing fast enough that two partners could both see rising volumes. The bearish reading is about share and leverage, not necessarily absolute revenue.

    That said, the concern is not irrational. In custom silicon, the design win for generation N strongly influences who builds generation N+1. If Marvell’s expanded role includes compute (the accelerator itself) rather than adjacent components such as networking or interconnect silicon, the competitive implications for the incumbent are materially larger. The source reporting does not settle that question — and it is the single most important unknown in this story.

    What $12.2 Billion Does — and Doesn’t — Tell Us

    Headline deal values in semiconductors deserve careful reading. A $12.2 billion figure could represent firm purchase commitments, a cumulative multi-year revenue expectation, or an analyst’s sizing of the opportunity — each with very different levels of certainty. The reports cited here frame it as changing Marvell’s bull case, which suggests investors are treating it as durable pipeline, but the articles do not disclose contract structure, timeline, or margin profile.

    Custom silicon also carries structurally lower gross margins than merchant chips, because the customer funds the design and captures much of the value. Marvell’s win is real in revenue-visibility terms; whether it is equally attractive in profitability terms depends on details not yet public.

    Downstream Effects on AI Infrastructure Buyers

    For enterprises and operators who buy cloud AI capacity rather than chips, this competition is quietly good news. Every credible alternative to Nvidia GPUs — and every second source within the custom-silicon supply chain — adds capacity to a market that has been supply-constrained for years. More TPU supply at better economics ultimately shows up as more available accelerated compute, and potentially better pricing, for Google Cloud customers. It also intensifies demand on the physical layer: more accelerator volume means more high-density data center space, more power procurement, and more advanced cooling — the parts of the stack where constraints now bind hardest.

    Background

    Google has designed its own AI accelerators — the TPU line — for roughly a decade, working with external semiconductor partners on design and production. Broadcom has long been identified in industry reporting as the principal partner behind that program, and custom accelerators for hyperscalers have become one of the fastest-growing segments in semiconductors as cloud providers seek alternatives to merchant GPUs. Marvell, meanwhile, has built its own custom-compute franchise serving hyperscale customers, making it the most frequently cited challenger to Broadcom in this market.

    The reported $12.2 billion expansion lands in that context: a two-horse race for hyperscaler design partnerships, where each win shapes multiple future chip generations and, downstream, the data center, power, and cooling infrastructure required to deploy them.

    Source: Broadcom (AVGO) Is Down 6.2% After Google Expands AI Chip Ties With Marvell — Yahoo Finance, with related Yahoo Finance coverage of Marvell’s reported $12.2 billion Google partnership expansion and its impact on analyst fair-value estimates.

  • AMD’s Buy of 2023-Founded Taalas Shows AI Inference Is Getting Its Own Silicon

    AMD’s Buy of 2023-Founded Taalas Shows AI Inference Is Getting Its Own Silicon

    TL;DR · 30-second read

    The Short Version

    Chipmaker AMD is buying Taalas, a Toronto startup founded only in 2023. Taalas designs chips shaped around the artificial intelligence model they will run, instead of chips built to run anything.

    Every time someone asks a chatbot a question, a computer in a data center does the work, and that happens constantly. Flexible, do-everything chips waste time and electricity on that job. AMD is betting that purpose-built chips can answer faster and use less power.

    The price and the first products have not been announced.

    AMD announced on August 6, 2026 that it has reached a definitive agreement to acquire Taalas, a Toronto-based company founded in 2023 that develops specialized silicon for AI inference, the work of running a trained model to answer requests. AMD said it plans to integrate Taalas’ technology into its accelerator roadmap and build system-level products that pair it with AMD Instinct GPUs.

    Financial terms were not disclosed. The deal is subject to customary closing conditions and regulatory approvals, and AMD gave no expected closing date.

    Executive Summary

    AMD, one of two major suppliers of general-purpose GPUs for AI data centers, is buying a company whose stated purpose is to get around the limits of general-purpose hardware. AMD’s own announcement describes Taalas’ technology as reducing “compute and memory bottlenecks associated with general-purpose architectures.” Taalas CEO and co-founder Ljubisa Bajic described the company’s approach as “building the hardware around the model.”

    The significance is directional. AMD is not abandoning GPUs. It frames Taalas as a complement to Instinct accelerators, EPYC CPUs, ROCm software and Helios rack-scale systems. But a leading GPU vendor is treating inference as a workload that justifies dedicated silicon on its own roadmap. That matters for anyone planning data center capacity, power budgets or AI hardware purchases over the next several years.

    What the announcement does not yet provide is proof. There is no price, no performance figure behind the “breakthrough” claim, no named customer and no product timeline. The deal is best read as a strategic commitment whose payoff remains to be demonstrated.

    A GPU Maker Buying Non-GPU Silicon Is the Signal

    AMD’s flagship AI product, the Instinct GPU, is a general-purpose accelerator. It can train models, run them, and handle scientific computing. So it is notable that AMD’s release justifies this acquisition by pointing to the bottlenecks of general-purpose architectures. AMD says it will integrate Taalas’ technology into its accelerator roadmap and pair it with Instinct GPUs in system-level solutions. In other words, AMD is adding inference-specific hardware alongside the GPU rather than asking the GPU to do everything.

    The mechanism behind that bet comes from how the two main AI workloads differ. Training teaches a model and happens periodically, at enormous scale, on hardware that benefits from flexibility. Inference runs the finished model every time a user sends a request. It is repetitive: the same model performs the same operations, over and over, at high volume. Inference is often limited less by raw arithmetic than by moving the model’s data between memory and the compute units, the “memory bottleneck” AMD’s release refers to. Taalas says it optimizes inference dataflows, the path data takes through the chip, for the model being served. That is what Bajic means by building the hardware around the model, an approach sometimes described as hard-wiring a model into silicon.

    If the approach delivers, the people most affected are those who pay for inference at scale: cloud providers, AI service companies and the data center operators who host them. Inference runs continuously, so efficiency per answer compounds into power, cooling and floor-space requirements. More answers per unit of electricity would let operators serve more demand within a fixed grid connection, and grid connections are increasingly the constraint on new AI capacity.

    The Price of Specialization Is Flexibility

    Hardware tuned to a model is efficient for as long as that model is in use. AI models are revised and replaced quickly, while chips take years to design, validate and manufacture. A chip fitted too tightly to today’s model risks being stranded when customers move on. This likely explains why AMD positions Taalas as a complement rather than a replacement. Boppana’s framing, the “right compute solutions for every AI workload,” implies a division of labor. GPUs would handle training and fast-changing models, and specialized silicon would take stable, high-volume inference.

    For infrastructure buyers, that points toward mixed fleets: racks combining different accelerator types, each doing the job it is best at. Mixed fleets complicate capacity planning, procurement and software. Workloads have to be routed to the right hardware, and utilization has to stay high across all of it. How Taalas’ technology is exposed through ROCm, AMD’s software platform for programming its accelerators, will largely decide whether customers can adopt it without rewriting their deployment stacks.

    What AMD Is Actually Buying

    Taalas was founded in 2023, making it roughly three years old at signing. The release names no Taalas product, customer or performance figure. Instead, it leans heavily on the “world-class engineering team” and on retaining and growing Canadian talent. That suggests AMD is acquiring a design approach and the people behind it as much as any finished product line. Acquiring engineering teams to accelerate a chip roadmap is a well-established move in semiconductors, and it is a reasonable way to enter a design space quickly.

    Read even-handedly, the claims in the release fall into two groups. Some are substantiated by the fact of the deal itself: AMD has committed to acquiring the company and has stated where the technology will go. Others remain assertions until products ship, including “breakthrough inference performance and efficiency” and “significantly reducing” bottlenecks. The competitive logic is clear enough. AMD is building a full-stack AI platform of CPUs, GPUs, networking, software and rack-scale systems, and inference-specific silicon would fill a gap in that stack. Whether it closes that gap depends on execution the market cannot yet evaluate.

    Background

    AMD, based in Santa Clara, California, sells EPYC server CPUs and Instinct GPU accelerators for AI and high-performance computing. It also sells ROCm, its software platform for programming those accelerators, and Helios, a rack-scale system that packages them for data centers. The company has used acquisitions to broaden its data center portfolio before, most notably its purchase of programmable-chip maker Xilinx. It positions itself as a full-stack alternative in AI infrastructure.

    Taalas was founded in 2023 in Toronto to design silicon specifically for AI inference. Inference is the stage where trained models serve real users, and AMD calls it one of the fastest-growing segments of the AI market. As inference volumes rise, the cost and power consumed per answer have become central concerns for cloud providers and data center operators. That has drawn interest in specialized chips designed to do that one job more efficiently than general-purpose hardware.

    Sources

    Source: AMD Acquires Taalas to Advance Compute Solutions for Rapidly Growing AI Inference Market. AMD’s announcement of its definitive agreement to acquire Toronto-based AI inference silicon developer Taalas.

  • AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD Says Instinct MI355X Sets a New Bar for DeepSeek Inference

    AMD announced on June 11, 2026 that its Instinct MI355X accelerator has set a new performance bar for inference on DeepSeek models — the open-weight large language models from the Chinese AI lab whose efficiency-focused releases reshaped expectations for serving costs. Inference is the work of running a trained model to answer real requests, as opposed to training it in the first place.

    The claim, published by AMD itself, positions the MI355X — the flagship of AMD’s MI350 series — as a leading choice for the inference-heavy workloads that increasingly dominate AI infrastructure spending.

    Executive Summary

    AMD’s announcement is a benchmark claim, not a product launch: the company says the MI355X, its current flagship data-center GPU, delivers record-setting throughput when serving DeepSeek models. Because DeepSeek’s open-weight models are among the most widely deployed for self-hosted inference, they have become a de facto proving ground for accelerator vendors — a benchmark customers can actually reproduce, unlike proprietary-model results.

    The timing matters. The AI hardware market is shifting from a training-dominated buildout, where Nvidia’s ecosystem advantage is strongest, toward an inference era where cost per token served — the price of generating each unit of model output — is the metric that decides purchase orders. AMD’s pitch has consistently been large memory capacity and better price-performance for exactly this phase.

    What the headline claim does not establish, at least in the material visible here, is the specific numbers, the comparison baseline, or independent verification. Vendor benchmarks are a legitimate signal, but buyers should treat them as the opening of a conversation rather than its conclusion.

    Why DeepSeek Became the Benchmark That Matters

    DeepSeek’s models occupy an unusual position in the AI market: they are open-weight, meaning anyone can download and run them on their own hardware, and they were engineered from the start for inference efficiency. That combination made them the workload of choice for enterprises and cloud providers that want frontier-class capability without paying per-token API fees to a model vendor. When a chipmaker claims leadership on DeepSeek inference, it is claiming leadership on one of the workloads real customers actually deploy — which gives the claim more commercial weight than a synthetic benchmark, and also makes it more checkable, since third parties can rerun it.

    There is a second, subtler point: DeepSeek’s mixture-of-experts architecture — where only a fraction of the model’s parameters activate per request — stresses memory capacity and memory bandwidth more than raw compute. That plays to the MI355X’s most widely cited hardware advantage, its large high-bandwidth memory pool (288 GB of HBM3E per GPU, per AMD’s published specifications for the MI350 series). Fitting a large model on fewer GPUs reduces the interconnect traffic and server count needed to serve it, which is where inference economics are won or lost.

    The Inference Era Rewrites the Competitive Math

    Training a frontier model is a rare, massive event; serving it to millions of users is a continuous, compounding cost. As deployed AI applications scale, industry spending is tilting toward inference, and that shift changes what buyers optimize for. In training, ecosystem maturity and cluster-scale networking — Nvidia’s strongholds — dominate the decision. In inference, the calculus is simpler and more mercenary: tokens per second, per dollar, per watt. Every point of throughput a rival accelerator gains translates directly into rack space, power, and capital that an operator does not have to buy.

    This is why AMD keeps aiming its benchmark artillery at inference rather than training. It is the segment where switching costs are lowest — an inference deployment of an open-weight model is far easier to port between hardware vendors than a training pipeline — and where AMD’s ROCm software stack, historically its weakest flank against Nvidia’s CUDA, faces the least demanding compatibility burden. For data-center operators, a credible second source of inference silicon is leverage in every negotiation, whichever vendor ultimately wins the deal.

    A Vendor Benchmark Is a Claim, Not a Verdict

    The announcement comes from AMD’s own newsroom, and the standard cautions apply — as they would to any vendor, including Nvidia, whose competitive benchmarks deserve identical scrutiny. Benchmark results are exquisitely sensitive to configuration: batch size, input and output sequence lengths, quantization (running the model at reduced numerical precision to go faster), and which competing hardware and software versions form the baseline. A ‘new bar’ can be genuine engineering progress, a favorable test setup, or both at once. The release headline, on its own, does not let a reader distinguish these cases.

    The constructive reading is that publishing reproducible claims on an open-weight model invites exactly the third-party validation that settles such questions. If independent labs and cloud customers can replicate the numbers on production-shaped workloads, the claim hardens into a real competitive fact. If the result holds only under narrow conditions, the market will find that out quickly too — one of the healthier dynamics the open-weight ecosystem has introduced to hardware marketing.

    Background

    AMD has spent a decade rebuilding itself into the principal challenger to Nvidia in data-center silicon, first in CPUs with EPYC and more recently in AI accelerators with the Instinct line. The MI300 series, launched in late 2023, gave AMD its first broadly adopted AI GPU; the MI350 series that followed in 2025, including the MI355X, extended its strategy of packing more high-bandwidth memory per chip than competing parts to win inference workloads.

    DeepSeek entered the global spotlight in early 2025 when its efficient open-weight models demonstrated that frontier-class AI could be trained and served at far lower cost than prevailing assumptions, briefly shaking AI-infrastructure markets. Since then its models have become a standard workload for measuring inference performance — turning each new hardware generation’s ‘DeepSeek numbers’ into a competitive scoreboard watched by chipmakers, cloud providers, and investors alike.

    Source: AMD Instinct MI355X GPU Sets a New Bar for DeepSeek Inference — AMD, the company’s announcement of record DeepSeek inference performance on its flagship accelerator.

  • d-Matrix Corsair Hits Full Production: A Challenger to the AI Inference Status Quo

    d-Matrix Corsair Hits Full Production: A Challenger to the AI Inference Status Quo

    Silicon Valley chip startup d-Matrix announced on June 10, 2026 that Corsair, its flagship AI inference accelerator, has entered full production, with the company attributing the ramp to customer demand. Corsair is a PCIe-card accelerator built on d-Matrix’s digital in-memory compute architecture, designed to run large language model inference — the work of generating answers from already-trained models — faster and more efficiently than general-purpose GPUs.

    Executive Summary

    d-Matrix says its Corsair inference platform has moved from early availability into full production. For a fabless semiconductor startup, that transition is one of the hardest milestones in the business: it signals that the design, manufacturing partners, packaging, and software stack are mature enough to ship at volume rather than in evaluation quantities. The company frames the ramp as demand-driven, though the release does not disclose shipment volumes, named customers, or revenue.

    The announcement matters because it lands in the middle of the industry’s most consequential architectural debate: whether AI inference — now widely expected to dwarf training as a share of total AI compute spending — will remain a GPU market, or fracture into specialized silicon. Corsair is a purpose-built bet that inference is fundamentally a memory problem, not a compute problem, and that an architecture which collapses the distance between memory and math can win on cost and energy per token. Full production is the point at which that thesis stops being a slide deck and starts being testable in customer data centers.

    The Memory-Bandwidth Wall, Explained

    When a large language model generates text, the dominant cost is not arithmetic — it is moving the model’s billions of parameters from memory to the processor over and over, once per generated token. Processors have gotten faster far more quickly than memory has gotten closer, a gap the industry calls the memory-bandwidth wall. GPUs attack it with expensive stacks of high-bandwidth memory (HBM) bolted alongside the compute die; d-Matrix attacks it by performing the math inside the memory arrays themselves, an approach called digital in-memory compute. Less data movement means, in principle, lower latency and less energy per token.

    The architectural logic is sound and the problem is real — memory bandwidth, not raw FLOPS, is the binding constraint on most production LLM serving today. The open question has never been whether in-memory compute is elegant, but whether it can be manufactured at scale, programmed easily, and priced competitively. A full-production milestone speaks directly to the first of those three tests.

    From Demo Silicon to Volume: Why This Milestone Is the Hard One

    The graveyard of AI chip startups is full of companies that produced impressive demonstration silicon but never crossed into volume manufacturing. Getting there requires acceptable yields from foundry partners, stable supply of advanced packaging, qualified server integrations, and a software stack that customers other than the vendor’s own engineers can actually use. By declaring full production, d-Matrix is asserting it has cleared those gates.

    What the release does not do is quantify the claim. “Full production to meet customer demand” is a statement about readiness, not about scale: no unit volumes, deployment sizes, or purchasers are disclosed. That is typical for a private company’s press release, but it means the milestone should be read as necessary rather than sufficient evidence of commercial traction. The verifiable signals — named customers, independent benchmarks, follow-on orders — come later, and observers should watch for them.

    The Economics of Challenging an Incumbent

    Every inference challenger faces the same asymmetry: Nvidia’s advantage is only partly the silicon. Its CUDA software ecosystem, developer familiarity, and guaranteed supply relationships make GPUs the default even where specialized chips post better numbers on paper. Challengers such as Groq, Cerebras, and SambaNova — and the hyperscalers’ in-house chips like Google’s TPUs and Amazon’s Inferentia — have each carved positions by competing on cost per token, latency, or energy rather than generality.

    d-Matrix’s opening is real, though. Inference is a workload buyers purchase continuously, priced per token, which makes operating cost — dominated by power and hardware amortization — brutally legible. Enterprises and cloud providers are also actively seeking second sources to gain pricing leverage over the GPU supply chain. A challenger does not need to displace the incumbent to build a substantial business; it needs to win the subset of workloads where its architecture’s advantages are largest and the switching costs are manageable.

    What It Means for the Data Center

    For data-center operators, the interesting property of accelerators like Corsair is the form factor: PCIe cards that slot into standard servers, rather than the dense, increasingly liquid-cooled rack-scale systems that frontier GPUs demand. If inference-optimized silicon delivers competitive throughput at meaningfully lower power per token — a claim d-Matrix has consistently made in its marketing, and one that independent benchmarking will need to validate — it extends the useful life of conventional air-cooled facilities that cannot economically retrofit for 100-kilowatt racks.

    That has second-order implications for the industry’s power crunch. Inference demand is growing at exactly the moment grid interconnection has become the limiting factor on data-center construction. Any architecture that serves more tokens per megawatt is, in effect, a capacity play — and that, more than any single benchmark, is why purpose-built inference silicon keeps attracting capital.

    Background

    Founded in 2019, d-Matrix spent its first years developing digital in-memory compute through successive test chips before unveiling Corsair in late 2024 as its first volume product, aimed squarely at low-latency large language model serving. The company has raised several hundred million dollars from investors including Microsoft’s M12, Temasek, SK hynix, and Playground Global — one of the better-capitalized entrants in a crowded field of AI chip startups formed on the thesis that inference workloads will eventually dwarf training.

    That thesis has moved from contrarian to consensus: as deployed AI applications scale, the recurring cost of serving models has become the industry’s central economic problem, and the market for inference-optimized alternatives to GPUs has drawn challengers ranging from venture-backed startups to the hyperscalers’ own silicon programs. Full production of Corsair marks d-Matrix’s transition from architectural argument to shipping product in that contest.

    Source: d-Matrix Corsair AI Inference Platform Enters Full Production to Meet Customer Demand — company press release via PR Newswire, June 10, 2026, announcing the production ramp of d-Matrix’s inference accelerator platform.

  • Google TPU v8 vs Nvidia: Inference Is Redrawing the AI Compute Map

    Google TPU v8 vs Nvidia: Inference Is Redrawing the AI Compute Map

    On May 29, 2026, investment research firm IO Fund published an analysis arguing that Google’s eighth-generation Tensor Processing Unit (TPU v8) represents a meaningful challenge to Nvidia’s dominance of AI computing — and that the industry’s shift from training AI models to running them, known as inference, is rewriting who captures value in the AI market.

    The piece is analyst commentary rather than a company announcement: neither Google nor Nvidia issued the claims, and the material available does not include chip specifications, benchmarks, pricing, or customer commitments.

    Executive Summary

    The thesis at the center of the analysis is straightforward: the AI compute market that Nvidia came to dominate was built on training — the enormously expensive, one-time process of teaching a model. As AI products mature, spending shifts toward inference — the everyday work of answering queries, generating text and images, and serving applications to users. Inference runs continuously, at massive scale, and its economics reward cost-per-query and energy efficiency over raw peak performance.

    Google is the one hyperscaler that has designed its own AI accelerator across eight generations, and it both consumes TPUs internally and rents them to customers through Google Cloud. If inference becomes the dominant workload, the argument goes, a vertically integrated chip tuned for serving costs could take share that merchant GPUs currently hold by default.

    Why it matters: even a partial shift of inference workloads to non-Nvidia silicon would ripple through chip suppliers, cloud pricing, and the design of the data centers that house all of it. But readers should note what is being claimed versus what is being shown — the source material asserts the competitive framing without publishing head-to-head performance or cost data.

    From Training Arms Race to Inference Economics

    Training a frontier AI model is a capital project: a huge cluster runs for weeks or months, and buyers pay almost any price for the fastest available hardware. Inference is an operating expense: every chatbot reply, search summary, and generated image is a small compute job repeated billions of times. That changes the buying criteria. For training, time-to-result dominates; for inference, what matters is cost per token served, latency, and performance per watt — how much useful output a chip produces for each unit of electricity.

    This is why analysts increasingly frame inference as the market’s center of gravity. A workload that runs 24/7 in production is exquisitely sensitive to efficiency, and a chip that is modestly slower but meaningfully cheaper to operate can win business that a peak-performance chip cannot. The IO Fund headline captures that logic; what the available material does not provide is data quantifying how TPU v8 actually performs on those metrics against Nvidia’s current parts.

    Custom Silicon and the Limits of the CUDA Moat

    Nvidia’s advantage has never been hardware alone. CUDA, its programming platform, is the software layer nearly all AI development targets, and switching away from it carries real engineering cost. That moat is strongest where code is bespoke and experimental — which describes training research well. Inference is different: production models are increasingly served through standardized frameworks and compilers that can target multiple chip types, lowering the switching cost that protects the incumbent.

    Google’s structural position is also unusual. Unlike merchant chipmakers, Google does not need to win sockets in other companies’ data centers to justify TPU development — its own search, ads, and Gemini workloads provide guaranteed internal demand, and Google Cloud monetizes the surplus. Amazon and Microsoft have followed the same playbook with their own accelerators. The open question, which the source material does not answer, is whether any hyperscaler chip has yet attracted large third-party inference workloads at scale, or whether custom silicon remains mostly an internal cost-reduction tool.

    What Inference-First Compute Means for Physical Infrastructure

    The training-to-inference shift is not just a chip story; it reshapes data centers. Training concentrates compute in a few gigawatt-scale campuses. Inference pulls in the opposite direction: serving users at low latency favors capacity distributed closer to population centers, with high-bandwidth connectivity to move requests and responses rather than model weights. For data center operators and network providers, an inference-heavy market means demand for more sites, in more markets, with different power and cooling profiles than monolithic training clusters.

    Efficiency claims matter here too. Power availability is the binding constraint on data center growth in most major markets, so performance-per-watt improvements in accelerators translate directly into how much AI capacity a given substation can support. Any credible challenger to Nvidia will be judged as much on watts as on FLOPS — a reminder that the AI market’s referee is increasingly the electric grid.

    Reading the Claim Like a Buyer

    For enterprises and cloud customers, the practical takeaway is not to pick a winner but to price the competition. A credible TPU alternative — even one adopted mainly inside Google — pressures accelerator pricing and cloud inference rates across the board, because Nvidia’s largest customers gain negotiating leverage. Buyers evaluating platforms should ask vendors for workload-specific benchmarks (their models, their traffic patterns) rather than headline chip comparisons, and should weigh portability: an inference stack built on open frameworks preserves the option to chase better economics as this rivalry plays out.

    It is equally fair to stress-test the bear case on Nvidia. The company has repeatedly absorbed inference-era challenges by iterating its own inference-optimized products and software, and market-share shifts in semiconductors tend to be slower than analyst narratives suggest. A headline announcing that the market is being ‘rewritten’ is a thesis, not a measurement — and the same skepticism should apply to Google-favorable and Nvidia-favorable framings alike.

    Background

    Google disclosed its first Tensor Processing Unit in 2016, making it the earliest hyperscaler to design custom AI silicon rather than rely solely on merchant chips. Successive TPU generations scaled from internal inference workloads to full training clusters offered through Google Cloud, and the seventh generation, Ironwood, announced in April 2025, was explicitly positioned as an inference-first chip — a signal of where Google believed the market was heading.

    Nvidia, meanwhile, converted its graphics-processor franchise into overwhelming leadership of AI training hardware, propelled by the generative-AI buildout that began in late 2022 and reinforced by its CUDA software ecosystem. The tension between merchant GPUs and hyperscaler custom silicon — Amazon’s Trainium, Microsoft’s Maia, Google’s TPUs — has become one of the defining structural questions of the AI infrastructure market, and the training-versus-inference spending mix is the variable most likely to decide it.

    Source: Google TPU v8 vs Nvidia: How Inference Is Rewriting the AI Market — IO Fund analysis, published May 29, 2026, arguing that the shift from AI training to inference is reshaping competition between Google’s custom TPU silicon and Nvidia’s GPUs.

  • Multi-Kilowatt AI Chips Push Direct-to-Chip Liquid Cooling From Option to Mandate

    Multi-Kilowatt AI Chips Push Direct-to-Chip Liquid Cooling From Option to Mandate

    Engineering trade publication Electronics360 published an analysis on May 24, 2026 arguing that direct-to-chip (D2C) liquid cooling — circulating coolant through cold plates mounted directly on processors — has crossed from a design option to a practical requirement, driven by AI accelerator chips whose power draw has reached the multi-kilowatt range per device.

    The piece frames this as the end of an era: air cooling, the default thermal strategy for data centers since the industry’s beginning, can no longer keep pace with the heat that flagship AI silicon produces in the small area of a single chip package.

    Executive Summary

    The core claim is thermodynamic rather than commercial: individual AI processors now dissipate thousands of watts each, and moving that much heat out of a dense rack with air alone requires airflow volumes and temperature differentials that become impractical or impossible at the densities AI clusters demand. Direct-to-chip liquid cooling, which places a liquid-filled cold plate against the chip itself, removes heat far more efficiently because liquids carry heat orders of magnitude better than air.

    Why it matters: if D2C is genuinely mandatory rather than optional, every layer of the data-center stack changes — facility design, plumbing, power distribution, rack architecture, maintenance skills, and capital budgets. Operators of existing air-cooled facilities face retrofit decisions, and new builds are being designed liquid-first. For an industry that standardized on air handling for decades, this is a foundational transition, not an incremental upgrade.

    Physics Ended the Debate Before the Market Did

    Air cooling persisted as the default not because it was elegant but because it was cheap, simple, and universally understood. Its limitation is fundamental: air is a poor heat conductor, so cooling a hotter chip means moving more air, faster, across larger heatsinks. As AI accelerators pushed past one kilowatt per device — with roadmaps pointing well beyond — the heat concentrated in a few square centimeters of silicon began to exceed what any realistic airflow can absorb. Water and engineered coolants transfer heat dramatically more effectively, which is why cold plates bolted directly onto the chip package have become the pragmatic answer.

    The word ‘mandatory’ in the source’s framing is worth taking seriously but precisely. Air cooling is not disappearing from data centers generally — the vast installed base of conventional enterprise and cloud workloads runs at rack densities air handles fine. The mandate applies to the frontier: dense AI training and inference clusters built around multi-kilowatt accelerators. That distinction matters for anyone budgeting a transition.

    The Retrofit Question Splits the Market

    Liquid-first design is straightforward in a new build: coolant distribution units, manifolds, leak detection, and higher floor loading are engineered in from day one. Retrofitting an existing air-cooled facility is harder. Piping must be routed through spaces never designed for it, water supply and heat-rejection capacity must be added, and operations teams must learn to manage a system where a leak — rare but nonzero — sits inches from expensive silicon.

    This creates a divergence in asset value across the industry. Facilities that can economically accept liquid cooling — because of their power capacity, structure, and location — become more valuable as AI demand grows. Older facilities that cannot may be relegated to lower-density workloads. Colocation providers, hyperscalers, and enterprise operators are all making that assessment now, and the answers will shape which real estate wins the AI buildout.

    A New Supply Chain Rises Around the Cold Plate

    A shift of this scale redraws the vendor landscape. Demand moves toward cold plates, coolant distribution units, quick-disconnect fittings, dielectric and water-based coolants, leak-detection systems, and rear-door or facility-level heat exchangers — categories that were niche a few years ago. Established thermal-management and precision-cooling vendors are competing with newer specialists, and chip and server makers increasingly ship liquid-ready designs, effectively deciding the question for their customers.

    There is also an efficiency dividend. Because liquid captures heat at the source, less energy is spent on fans and air handling, and the warm coolant leaves at temperatures useful for heat reuse in some settings. For operators facing scrutiny over data-center energy consumption, D2C offers a genuine efficiency story — though it introduces its own considerations around water use and coolant handling that deserve equally honest accounting.

    Background

    For most of computing history, data centers were cooled the same way: chilled air pushed through raised floors or ducts, across finned metal heatsinks, and back to air-handling units. That model worked because individual chips drew tens or hundreds of watts. The AI era broke the assumption — training and running large models rewards packing the most powerful accelerators as densely as possible, and each generation of AI silicon has raised per-chip power substantially, crossing the kilowatt mark and continuing upward.

    Liquid cooling itself is not new; mainframes and supercomputers used water cooling decades ago before commodity air-cooled servers displaced them on cost. What has changed is that the physics that once made liquid cooling a supercomputing niche now applies to mainstream AI infrastructure, pulling a once-specialist discipline back to the center of data-center design.

    Source: Multi-kilowatt chips make D2C cooling mandatory — Electronics360 analysis (May 24, 2026) on why multi-kilowatt AI processors are forcing data centers from air cooling to direct-to-chip liquid cooling.