Explained: What are foundational models, examples of common ones?

Foundational models are a form of artificial intelligence models that can perform a wide range of tasks

The debate on whether India should have its own foundational models or build large language models (LLMs) is intensifying, with IT Minister Ashwini Vaishnaw stating that the government is exploring indigenous AI models. Adding to this, Perplexity CEO Aravind Srinivas emphasised that India should create its own foundational model. In a recent post on X, Srinivas said he disagrees with Nandan Nilekani's view that India should not develop its own LLMs.

 

What are foundational models? 

Foundational models are a type of artificial intelligence model capable of performing a wide range of tasks. These models are created by training on vast and diverse datasets, enabling their use across various applications. Foundational models have existed for some time, but earlier versions were specialised tools trained for specific applications.

 

According to Amazon Web Services (AWS), the term "foundational model" was coined by researchers to describe machine learning (ML) models trained on broad, generalised, and unlabelled data. These models can perform a variety of general tasks, such as understanding language, generating text and images, and engaging in natural language conversations.

 

Which are some of the common foundational models? 

Indian entrepreneurs and businesses are using AI engines or foundational models developed by OpenAI, Microsoft, Google, and Meta, among others. In India, Ola is creating Krutrim, an LLM from scratch.

 

What are some of the challenges in building foundational models? 

There are two critical elements for building foundational models: first, the compute power or GPUs needed to create powerful servers, and second, the investments required. As Satya Nadella, chairman and CEO of Microsoft, said during his recent visit to India: “India must get into frontier work in artificial intelligence and build foundational models, but investment is a real entry barrier, and just one mathematical breakthrough can change the entire dynamics.”

 

The Indian government’s stand 

Ashwini Vaishnaw, Minister of Electronics and Information Technology, told Business Standard in an earlier interview that the country is focused on building its own GPUs. The target is to have a GPU built in India within the next three to five years. In a recent interview with CNBC TV18 at Davos, he stated that India is working on preparing datasets for training AI models, leveraging large pools of non-personal data, such as transport, agriculture, and weather datasets.

 

Why the debate now? 

One reason for the current debate is the recent advancement by DeepSeek, a Chinese startup that unveiled DeepSeek V3, an LLM with 671 billion parameters. Srinivas of Perplexity has argued that India should focus on creating foundational models for Indic languages while remaining competitive on global benchmarks.