Kempner Institute adds ~400 H100 GPUs to its computing cluster

The Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard has purchased 384 H100-80 GB Nvidia graphics processing units (GPUs). These GPUs will be added to the Kempner’s state-of-the-art computational cluster to support research in natural and artificial intelligence.

With this addition, the Kempner’s cluster is positioned to be one of the largest academic ML computing clusters in the world. The expanded cluster, which is specifically designed to train generative AI models, will allow researchers to process massive amounts of data quickly, and to develop and test new computational models that explain how brains and machines learn.

“Academia is uniquely equipped to push the frontiers of scientific knowledge in an open and curiosity-driven fashion, and this cluster is a uniquely powerful resource with which to do so,” said Kempner affiliate Jonathan Frankle, who played a key role in shaping the direction of the Institute’s computing cluster. “Our students will be able to work with state-of-the-art equipment and industry-scale budgets as they develop the next generation of machine learning technology.”

Nvidia’s A100 and H100 GPUs have emerged as popular choices in the competitive landscape of high-performance computing, with the latter model offering nearly three times more training and inference speed as the already powerful A100 model. Networked using Infiniband, the Kempner’s expanded cluster will be capable of 1600 Gigabyte/second transfer speeds, allowing for scientists to use the entire cluster for training language models quickly and efficiently.

The Kempner’s current rack of 144 A100-40GB GPUs is already powerful, capable of training LLaMA2-7B, a popular 7 billion parameter open-source large language model, in about six weeks. However, the expanded cluster will be capable of training the same model in less than a week, or will be able to train LLaMA-70B, the 70 billion parameter version, in two and a half months, a time savings that allows the Kempner to pursue truly innovative research.

The Kempner’s computing nodes reside at the Massachusetts Green High Performance Computing Center (MGHPCC), a green, nonprofit modern datacenter in western Massachusetts. The MGHPCC, which is shared by a number of other Boston-area universities, was the first university research data center to achieve LEED Platinum Certification, the highest level awarded by the Green Building Council’s Leadership in Energy and Environmental Design Program.

About the Kempner

The Kempner Institute seeks to understand the basis of intelligence in natural and artificial systems by recruiting and training future generations of researchers to study intelligence from biological, cognitive, engineering, and computational perspectives. Its bold premise is that the fields of natural and artificial intelligence are intimately interconnected; the next generation of artificial intelligence (AI) will require the same principles that our brains use for fast, flexible natural reasoning, and understanding how our brains compute and reason can be elucidated by theories developed for AI.

Join the Kempner mailing list to learn more, and to receive updates and news.