Foxconn Announces Large Language Model (LLM) Refined from Metas Llama 3.1

Foxconn, the company best known for manufacturing iPhones and other Apple hardware products, has just surprised everyone by announcing its first large language model (LLM), called FoxBrain, which is intended to be used to improve manufacturing and supply chain management.

The Taiwanese manufacturer said that FoxBrain was trained with just 120 Nvidia H100 GPUs. This LLM is essentially built on Meta's Llama 3.1 architecture, with 70 billion parameters through a distillation process. The concept of LLM distillation involves taking a "parent" model and training a "child" model based on its feedback. Foxconn also admitted that its LLM is not as good as the refined model of China's DeepSeek, but the overall performance is very close to world-class standards.

Speaking about this achievement, Dr. Yung-Hui Li, Director of the Artificial Intelligence Research Center at Hon Hai Research Institute (Foxconn), said:

In recent months, the increasing inference capabilities and efficient use of GPUs have gradually become the main trends in the field of AI. Our FoxBrain model has adopted a very effective training strategy, focusing on optimizing the training process rather than blindly accumulating computing power.

Through carefully designed training methods and resource optimization, we have succeeded in building a local AI model with strong reasoning capabilities."

Foxconn not only assembles Apple products, but also manufactures Nvidia’s AI servers. With 120 H100 GPUs, FoxBrain is scaled up with Nvidia’s Quantum-2 InfiniBand network, and the training process took only about 4 weeks (with a total computational cost of 2,688 GPU days). Foxconn generated 98 billion high-quality pre-training data tokens in traditional Chinese with a context window length of up to 128,000 tokens.

The partnership between Foxconn and Nvidia isn't new, and the two companies are also working on other projects, including building the world's largest Blackwell GPU manufacturing facility.

Nvidia also provided Foxconn with the Taipei-1 supercomputer to complete the model pre-training process. Foxconn said FoxBrain will become a “key engine” to upgrade the company’s three main platforms: Smart Manufacturing, Smart Electric Vehicles, and Smart Cities.