Apple releases open-source AI models

Apple has launched eight open-source artificial intelligence large language models (LLMs), called OpenELM (Open-source Efficient Language Models).

This was announced in a research article by Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, Maxwell Horton, Yanzi Jin, Chenfan Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal, and Mohammad Rastegari.

The paper is titled “OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework.”

These LLMs run on-device rather than through cloud servers and have been made available by Apple on the Hugging Face Hub and GitHub communities.

The tech giant said transparency and reproducibility are essential for advancing open research of LLMs.

This will ensure the trustworthiness of results and enable investigations into data and model biases, Apple stated.

Four of the eight models have been pre-trained using the CoreNet library, and the other four are instruction-tuned models.

“OpenELM uses a layer-wise scaling strategy” that will lead to enhanced efficiency and accuracy, the researchers point out.

“For example, with a parameter budget of approximately one billion parameters, OpenELM exhibits a 2.36% improvement in accuracy compared to OLMo while requiring 2× fewer pre-training tokens.”

Apple has made the code, training logs, and multiple versions of the LLM available to encourage faster innovation and progress within the open research community.

“This extensive release aims to empower and strengthen the open research community, facilitating future research efforts.”

Developers and companies can modify the models or use them as-is.

Apple has recently made significant developments in AI by releasing ReALM, an LLM, to challenge OpenAI’s GPT-4 and Google Gemini.

The tech giant will host its Worldwide Developers Conference on 10–14 June and has hinted that AI will feature heavily.