Neural-network Processing Unit Hardware-software Co-design IP

Large language models (LLMs) have become an increasingly prominent feature in various computing platforms, such as data centers, smartphones, and microcontrollers. As such, their potential applications in both human-to-machine interfaces (HMIs) and machine-to-machine interfaces (M2MI) have been extensively studied and discussed. However, implementing hyper-scale LLMs on heterogeneous multicore System-on-Chips (SoCs) poses significant challenges. In this talk, we will discuss the key challenges, including issues related to hardware/software interfaces, verification, and SoC architecture exploration. At the end of this talk, we will propose a practical workflow and corresponding solutions we’ve applied in many cases. And see how our standard tool - ONNC - is used in the workflow.

大型語言模型（LLM）已成為各種計算平台（例如資料中心、智慧型手機和微控制器）中日益突出的功能。因此，它們在人機介面（HMI）和機器對機器介面（M2MI）中的潛在應用已被廣泛研究和討論。然而，在異質多核晶片上系統（SoC）上實現大型語言模型（LLM）面臨著重大挑戰。在本次演講中，我們將討論關鍵挑戰，包括與硬體 / 軟體介面、驗證和 SoC 架構探索相關的問題。在本次演講的最後，我們將提出一個實用的架構以及我們在許多案例中應用的解決方案。並了解我們的標準工具 - ONNC - 如何在架構中使用。