Key technology of AI chip and its packaging: SiP technology

What is AI Chip?

AI chips, that is, artificial intelligence chips. In a broad sense, chips that can run AI algorithms are called AI chips, but they usually refer to chips that have been specially designed for AI algorithms to accelerate. At the beginning of the 20th century, due to the constraints of algorithms and data volume, there was not much market demand for AI chips, and general-purpose CPUs could meet the needs of AI algorithms and applications. Later, with the development of cloud computing and mobile devices, data With the accumulation of a large amount of data, the requirements for computing power have increased, and AI chips have experienced GPUs with parallel computing capabilities, and gradually developed TPUs, NPUs, FPGAs, and ASICs with the development of the AI industry. For AI chips, the core is powerful computing power. Since 2012, the computing power required for artificial intelligence training tasks has doubled every 3.43 months, greatly surpassing Moore's Law that the chip industry has followed for a long time.

AI Chip Background and Significance

In recent years, with the rapid development of integrated circuits, a series of great inventions such as mobile phones, computers, and the Internet have gradually and completely changed people's communication methods, changed people's work, study, and lifestyle, and the production, work, and study of today's human society And life has been inseparable from various electronic products. The development of technology has brought us a rich experience, and at the same time stimulated people's wider demand for integrated circuit applications. People have gradually seen the power of combining integrated circuit electronic products with artificial intelligence, smart phones, smart speakers, and smart sweepers. Robots, smart food delivery robots, smart driving, smart clinics, etc. With the development of technologies such as 5G and the Internet of Things (IoT), people have a beautiful vision of the interconnection of all things in the future, and artificial intelligence will bloom everywhere. Change. The AI chip is the physical basis of all these intelligent applications, and it is also a powerful driving force for the sustainable development of artificial intelligence technology.

Improving the Computing Power of AI Chips

One is to increase the computing unit, that is, to increase the number of transistors in the chip and increase the chip area;

The second is to shorten the physical distance between the memory and the computing unit, increase the bandwidth between the memory and the computing unit, and solve the memory bottleneck, thereby reducing the waiting time of the computing unit and reducing data transfer.

For the first method, since the development of Moore's Law, the development of integrated circuits has followed the technological trajectory of Moore's Law for more than half a century. The number of transistors accommodated on an integrated circuit chip doubles every 18 to 24 months. At the same time, the function and processing speed of the processor are doubled, and the price is doubled. It is precisely because of this that various electronic products with more and more functions, better performance and lower prices can be obtained and their wide applications. However, after 2010, the growth rate of transistor density slowed down and gradually deviated from Moore's Law. In recent years, the transistor technology has developed to 5nm, 3nm, and 2nm, and has gradually reached the physical limit. To increase the number of transistors, the chip area must also increase, but the increase in chip area will lead to a decrease in yield and an increase in cost. At this stage, Moore's Law is approaching its limit in terms of physics, power consumption, and cost. It has entered the so-called post-Moore era. Relying solely on transistor technology to improve chip performance can no longer meet the needs of the era of 5G, AI, and the Internet of Everything. If we want to continue Moore's Law, the concept of integrated circuits needs to be richer, and the concept of integration has begun to shift more to the packaging level. Through heterogeneous and heterogeneous packaging technology, integrated circuits can develop in multiple dimensions, and the extension of the equivalent meaning of Moore's Law can be realized.

For the second method, reducing the physical distance between the computing unit and the memory is the most effective way to solve the memory bottleneck and the main goal of the development of high-performance chip packaging technology. It can not only improve computing power, but also reduce power consumption and heat generation. . The realization of these two methods can actually be realized through packaging. One generation of chips, one generation of packaging, and packaging technology appear with the generation of chips. The role and status of packaging in integrated circuits from the early simple protection of chips to the present Greatly improved, rich integration concepts, product performance, power consumption, volume, etc. are increasingly implemented by packaging. AI chips require low latency, low power consumption, high performance, and small size. SiP system-in-package has the advantages of high flexibility, small size, and high performance. It is a good choice for AI chip packaging. Using SiP system-in-package technology to integrate AI chips and memory chips into one package, from the perspective of package integration, the number of transistors per unit volume can be increased, and AI chips and memory chips of different process nodes can be integrated heterogeneously. It shortens the physical distance between computing and storage, breaks the limitation of the traditional chip memory wall, and is conducive to improving computing power, reducing power consumption, reducing volume, and enhancing performance.

SiP system-in-package, according to the definition of ITRS, the international semiconductor routing organization, SiP is to preferentially assemble multiple active electronic components with different functions and optional passive devices, as well as other devices such as MEMS or optical devices, to achieve certain A single standardized package of functions that forms a system or subsystem. Since Apple first publicly announced the use of SiP packaging technology in iwatch smart watches, SiP has attracted much attention and is considered to be one of the key technologies to save Moore's Law. With the development of packaging technology and process, the development of SiP is more of a system-level concept, not a specific process, but a solution to package a variety of chips with different functions in parallel or stacked. There are many specific implementation methods. Since system-level subpackaging can significantly reduce package volume and realize more complex functions, it is regarded as an important path to achieve beyond Moore's Law.

The realization of SiP system-in-package involves various technologies such as substrate technology, chip stacking technology, WB and FC chip interconnection technology, BGA technology, and packaging technology. A key to the realization, the substrate is used as a carrier for carrying multiple chips and a bridge for chip interconnection and chip and PCB interconnection. In the design process, the above-mentioned various technologies must be considered comprehensively to ensure the manufacturability of the substrate and the reliability of the package. Feasibility, the quality of the substrate has a crucial impact on the system performance. The miniaturization of the SiP package and the high integration of the chip require the substrate to have a small size. At the same time, the AI chip and the memory chip have a high I/O pin density, which makes the substrate wiring density high, and the various high-speed interface technologies of the system have high-speed signal transmission. Requirements, higher requirements are placed on the signal integrity of the package substrate interconnection. The success or failure of substrate design, quality and cycle are crucial to product quality and cycle, so the research on substrate design technology is of great significance for the packaging of AI chips based on SiP technology.

SiPResearch status and development trend

The demand for AI chips as the underlying infrastructure of intelligent applications is strong, prompting many chip and Internet companies to enter the AI field. There are many manufacturers in the field of AI chips. Representative companies include Tesla, Intel, Intel, Google, Huawei HiSilicon, Apple and Samsung and others. Among them, Huawei launched the Kirin 970 chip equipped with Cambrian NPU (Neural-network Processing Unit, embedded neural network processor) in September 2017. It is the world's first smartphone terminal AI chip and has been successfully applied in Apple released the 980 and 990 series of AI chips on the Mate10 smart phone; Apple also released the A11 Bionic chip in September 2017 and applied it to the iPhone mobile phone terminal, which improved the user's experience in taking pictures and other aspects, and in 2020 From 2022 to 2022, M1 series chips will be released successively. In addition to smart phones, there are also many AI chips used in smart voice, smart vision, wearable devices and other fields, such as Rockchip's RK3588 AIoT chip, Qingwei Smart's TX510 smart vision processor chip, Coolchip's AR9341 chip, And Zhicun Technology's storage and calculation integrated Soc chip WTM2101 and so on.

The realization of the high performance of AI chips also requires appropriate packaging technology to ensure that the development of packaging technology is also driven by 5G and AI. Various manufacturers have developed many packaging technology routes to achieve high-bandwidth and high-density interconnection. Including Intel's EMIB2.5D and Foveros 3D packaging technologies, TSMC's CoWoS, FOWLP-based InFO and 3D SoIC packaging technologies, and Huatian Technology's self-developed eSIFO technology; Packaging manufacturers such as Tongfu have taken it as an important layout technology. Driven by the development and application requirements of AI intelligence and 5G, there have been many researches on high-performance packaging of AI chips in recent years. In 2017, Dyi-Chung Hu studied and analyzed the packaging structure classification of high-performance computing, from 7 main interconnection From the perspective of elements, the 2.5D advanced packaging technology is classified according to the interconnection application, which is helpful to understand the characteristics of the advanced packaging structure, and compares the line width and aperture of the 2.5D interposer, substrate, and PCB. At the parameter level, the substrate is in the middle, and the line width and aperture of the interposer are at the level of 1-10 μm, which can realize the integration of smaller pitches; Tadahiro Kuroda studied 3D IC stacked memory chips in 2019 to solve the storage wall problem; Luke England, Eric Tremble, etc. studied 3D packaging technology and Chiplet packaging technology supporting AI applications in 2019; TSMC's F.C. Chen, M.F. Chen, W.C, etc. studied 3D heterogeneous integration technology for SoIC on-chip integration in 2019, and developed The first use of SoIC technology based on front-end 3D technology for logic-logic and logic-memory stacking provides higher I/O bonding density and achieves strong reliability and stacking yield; Guan L T et al. 2020 Research has proved that FOWLP technology can support various high-performance applications such as 5G and AI; Rao V S, Chong C T et al. studied the application of high-density fan-out wafer-level packaging HD-FOWLP in AI packaging in 2021, and the design integrates four A system-in-package of DNN chiplets has been verified by simulation experiments. Through the above research on the packaging technology of AI chip applications, it can be seen that the characteristics of AI chips mainly lie in power consumption and bandwidth, because AI chips require more calculations and a large amount of access memory than traditional chips, and require high bandwidth and low bandwidth. Delay to improve response speed, energy saving and low cost at the same time, this requires the distance between the logic chip and the memory to be as close as possible, reduce the communication distance between chips, and break through the memory wall problem. To deal with these problems, logic and memory can be integrated together through chip integration, but as Moore's Law goes to the limit, the chip area, yield and cost of on-chip integration will be challenged, so packaging technology is more used to achieve heterogeneous Integrating chips of different process nodes, especially 3D packaging can directly stack memory on logic chips. The above research mainly focuses on the application of advanced technologies such as FO, interposer, and TSV in AI chip packaging. Heterogeneously integrated AI logic chips and memory chips also belong to the SiP system-in-package based on front-end advanced technology.

SiP(System In Package) system-in-package technology has been widely used in a variety of portable smart electronic products, and there are many types depending on the specific implementation form and technology. Figure 1 below shows several main types of SiP packages. SiP packaging can be based on the substrate, frame or wafer level, and the interconnection methods also include WB, FC, TSV, and a mixture of various interconnection methods. This article focuses on substrate-based SiP packaging. The SiP package based on the packaging substrate is usually BGA type; the way of integrating different chips on the substrate is 2D horizontal tiling and 3D vertical stacking. Compared with tiling, vertical stacking can reduce The small packaging area is conducive to the realization of a smaller packaging volume; the interconnection technology between the chip and the substrate mainly includes WB wire bonding technology and FC flip-chip welding technology, among which WB technology is the most widely used, most mature technology, and low price and cost. Interconnection technology, but the longer interconnection lead leads to larger parasitic parameters, high impedance and high radiation at high frequencies, which has certain limitations on chip performance, and FC technology greatly shortens the interconnection time compared with WB technology path, the parasitic parameters are small, which has great advantages in improving chip performance, and has been widely used in high-performance chip interconnection; the interconnection between chips can be realized through the internal interconnection of the substrate, compared with traditional The interconnection of the upper single-chip package through the PCB reduces the communication transmission path between chips, which is conducive to improving system performance and bandwidth; the communication interconnection between the SiP system and the system PCB is realized through BGA solder balls.

Figure 1 Several major SiP package types

Due to the increasing clock frequency of modern electronic systems and the application of various high-speed interface technologies in electronic design, signal integrity issues must be considered in package design, and whether the package design meets the system performance requirements must be confirmed before product production. The core of SiP packaging design is the SiP packaging substrate. Due to the characteristics of multi-chip integration, high chip pin IO density, and high signal transmission rate in high-speed design, the design requirements for the substrate are higher. In order to ensure the signal integrity of the substrate, the current substrate design process mainly includes pre-simulation, substrate design and post-simulation. The pre-simulation is based on the design requirements of the package obtained from the front-end system simulation to simulate the key signal routing parameters, mainly Impedance matching is performed according to the trace impedance requirements and stacked structure design line width and line spacing; the substrate layout design is mainly designed using Cadence APD/SiP package design software. The software provides a rule- and constraint-driven design environment. Designers can base on Substrate manufacturers and packaging manufacturers’ process capability specifications and packaging design key network differential pairs, equal length lines, etc. require that relevant physical constraints and electrical performance constraints be set before design, and then the design is carried out; post-simulation is the completed substrate design layout Import the layout simulation software for signal integrity simulation, extract signal S parameters, or RLGC parameters, and time-domain eye diagrams, etc., to verify whether the substrate design meets the system requirements. If the requirements are not met, return to the substrate design for optimization and re-simulation, iterative optimization until the requirements are met.

The substrate is separated from the PCB with the development of packaging technology. The size and process parameters are smaller than that of the PCB. It can be understood as a reduced version of the PCB, which belongs to the printed circuit board. For high-speed design, scholars have done a lot of research on the signal integrity of interconnection design. In 2014, Ai-Chie W and Hui C C studied the signal integrity that needs to be considered in the physical design of wiring in the package design of DRAM and NAND memory chips. In 2020, Rossi V et al. studied and compared the 4-layer and 6-layer DDR3 interface BGA packages. The electrical performance of the substrate design, the insertion loss and return loss S parameters under different designs, and the PDN impedance of the power distribution system have been simulated. The research shows that the 6-layer substrate design is better for signal integrity than the 4-layer design. However, the 4-layer substrate is sufficient to meet the performance and the price is lower; Wang J, Xu C, etc. studied the influence of differential via design on crosstalk in high-speed PCBs in 2020 and the optimization direction; Chen Zhangjin et al. The equivalent circuit model studies the influence of line length and via structure size on high-speed signal reflection; Yan Dong et al. studied the analysis and optimization of differential vias in high-speed PCBs in 2020, and established the equivalent circuit model and HFSS 3D structure model is used to predict and simulate the influence of differential via structure parameters on transmission characteristics, and the influence of adding ground holes on the signal integrity of differential vias is studied; The transmission characteristics of vias are studied.

The current research on signal integrity is based on the system simulation analysis and optimization based on the designed circuit board on the one hand, and on the other hand is the modeling and simulation analysis of the interconnection structure. The multiple optimization iterations of design and simulation take a long time, which increases the design cycle. In substrate design, stack design, interconnect wiring design and power distribution system design are important steps in SiP design, and are also the main factors affecting signal integrity. Before the signal integrity research on the substrate interconnection structure, the study of the influence of the interconnection design on the transmission characteristics is helpful to guide the substrate design in the design process based on the signal integrity of the interconnection design, and after the design is enlarged, the simulation passes the signal integrity verification , reduce the number of iterations and shorten the design cycle. At present, the research on the interconnection structure is mainly based on the PCB level, and the size of the substrate and the PCB structure and the size of the interconnection structure differ greatly. Large line width, with the development of substrate technology, the minimum line width of the substrate is usually about 25 μm, or even 10 μm; and some structural features are not the same, for example, for via holes, PCBs are usually mechanical holes with copper-plated hole walls, and multi-layer substrates It is generally a laser hole filled with copper in the hole, so the PCB interconnection and signal integrity characteristics may be different from those of the substrate. It is necessary to conduct signal integrity research on the interconnection structure of the substrate size and substrate characteristics to better guide the substrate. design.

With the development of the smart application industry and smart applications becoming more and more popular, people will have more and more demands for intelligence in the fields of smart voice, smart vision, smart home and other terminal devices and edge devices in the future. Compared with packaging based on front-end advanced technologies such as interposer and TSV, SiP technology realized by traditional WB, FC and other technologies will become a more widely used AI chip packaging type. The advantages of SiP technology will be further enhanced with the development of technology in a more refined direction, the improvement of packaging process technology capabilities, and the advancement of materials.