Trends in the AI ​​semiconductor industry: from chip to system

For decades, the semiconductor industry competed over one thing: the individual chip. One of the biggest trends in the AI ​​semiconductor industry right now is the shift toward treating chips as components of much larger systems. Performance, cost and reliability are now determined across the entire package, rack and supply chain, not just in a single chip.

This shift moves the most difficult problems upward. Packaging, connectivity, yield, sourcing and verification used to follow design, but now they shape it from day one. AI shortens development cycles while increasing the complexity that engineers must manage. The next phase will depend on how well companies coordinate across the system.

“I think the AI ​​system is our new chip,” said Letizia Giuliano, vice president of product management and marketing at Qualcomm Technologies Inc. “If before we optimized everything for the silicon, today we optimize everything for the system.”

Giuliano and other industry experts spoke with John Furrier of theCUBE Research and co-host Alison Kosik at the GSA Women’s Leadership Initiative event in Palo Alto, California. Interviews covered AI semiconductor industry trends, optical interconnects, advanced packaging, AI-driven yield learning, physical AI, strategic sourcing, and AI-powered chip design.

Here are six standout takeaways from the event:

1. Connectivity is overtaking computing power as a bottleneck in AI infrastructure.

According to Giuliano, moving data between chips, storage and racks now limits AI performance more than pure computing power. At 200 gigabits per second per track, half of the signal budget goes into exiting the package alone. This involves moving optical connections from rack-to-rack connections into the rack, the blade and the chassis itself, using copper and optics alongside each other. Qualcomm just launched High Bandwidth Compute, a low-power method of connecting computing power to storage tuned to tokens per watt. Building such products in large volumes means that product development must be linked early on to design-for-test, manufacturing and reliability planning, said Sophie Wang, senior director of product engineering at Marvell Technology Inc.

Check out theCUBE’s full interview.

2. Advanced packaging moves from an afterthought to system architecture.

The boundaries of an AI system are moving from the chip to the package, emphasized Nisha Ananthakrishnan, senior director of advanced packaging at Intel Corp. Intel uses its EMIB bridge technology to connect chiplets and scale packages from 3x to more than 10x form factors. Outsourced assembly and test providers are becoming system integrators, noted Patricia MacLeod, senior director of corporate communications and industry partnerships at Advanced Semiconductor Engineering Inc. She pointed to ASE’s VIPack platform, which supports technologies ranging from fan-out chip-on-substrate to co-packaged optics. “Many of the chips that AI relies on, such as 12- to 28-nanometer power management chips and voltage regulators, utilize sophisticated nodes,” added Tazrien Kamal, senior director of technology and innovation at GlobalFoundries Inc.

Watch theCUBE’s full interview.

3. AI-driven yield learning turns speed into a competitive advantage.

About 70 to 80 percent of the problems found when scaling up mass production could have been identified before tapeout, emphasized Preeti Prasher, chief operating officer of Stathera Inc. By using AI to analyze characterization, simulation and in-line manufacturing data, teams can start new wafers before a defective batch reaches test. Adaptive testing also reduces testing time. According to Iliana Chen, senior director of program management at Lattice, Lattice Semiconductor Corp. Released Lattice Prompt, its first AI design assistant that converts a plain text query into a field-programmable gate array design and recommends the best part for form factor and cost. Demand from robotics and other markets has made volume the biggest challenge for Lattice, Chen noted.

Check out theCUBE’s full coverage.

4. Physical AI becomes a system-level challenge rather than a chip war.

“Physical AI is much broader than humanoid robots and covers any autonomous system that senses its environment, reasons and acts in the physical world,” explained Gina Longoria, vice president of external communications at Arm Ltd. The market is in its early stages and no single company can build the architecture, security and protection standards it needs on its own, she emphasized. “Firmware is becoming the layer that connects GPUs, CPUs, NICs and Compute Express Link devices in a fleet-sized system,” said Lakshmi Ramagiri, firmware engineering partner at Microsoft Corp. Security must go beyond secure booting and achieve ongoing trustworthiness while a system is running, and the Open Compute Project and DMTF are the standards bodies to watch, Ramagiri added.

Don’t miss the full interview on theCUBE.

5. Strategic sourcing becomes a product decision as AI strains the supply chain.

“Sourcing has now gone far beyond negotiating prices to shape product development, risk and customer experience,” said Alisa Wang, AVP of global planning and procurement at Credo Technology Group Holding Ltd. Because AI-driven demand can change within a year, suppliers need to be treated as strategic partners who receive product roadmaps early, she emphasized. Leading-edge logic capacity is at Taiwan Semiconductor Manufacturing Co. Ltd. the biggest limitation, followed by packaging, specialized materials and critical equipment. In response, Credo is qualifying secondary suppliers and expanding its production footprint to additional regions. Stathera’s Prasher emphasized dual sourcing for key segments and leveraging supplier feedback when introducing new products to co-develop lower cost products.

Hear the full story on theCUBE.

6. AI agents are compressing chip design as development cycles shrink to six to nine months.

AI is evolving from a productivity assistant to an integrated part of electronic design automation workflows, said Rimi Dasgupta, chief of staff for product management and markets at Synopsys Inc., and Vidya Neerkundar, head of product management at Siemens Digital Industries Software. According to Dasgupta, Synopsys has seen verification completion speed up by 50x for register transfer level designs and debug time reduced by 40%. The demand for workload-optimized custom silicon has also resulted in development cycles being shortened from 18 months to six to nine months. AI is probabilistic, but chips must be deterministic. Therefore, Synopsys combines AI with multiphysics solvers from the Ansys acquisition and is working with the OpenAI Group PBC on a special GPT-Synopsys model. Multi-die design is the next big shift as chips push the limits of reticle, Neerkundar noted.

Follow the full conversation on theCUBE.

Photo: SiliconANGLE