Nvidia and IIT Madras research lab AI4Bharat have introduced Nemotron-pretraining-Indic-v1, an open synthetic dataset containing approximately 2.8 trillion tokens across 22 Indian languages, according to an announcement published on Hugging Face. The corpus covers the 22 languages recognized in India's Constitution, representing first languages for 97 percent of Indians, or nearly 1.2 billion citizens at the 2011 Census.
Mitesh Khapra, who heads AI4Bharat, noted that Indian language technology trails English because "we do not have enough data for Indian languages." The project pairs the dataset with an open GPU-accelerated pipeline built on Nvidia's NeMo Data Designer, allowing developers to adapt prompts and quality checks to their own text sources.
Token counts using the Gemma-4 tokenizer show 2,601.0 billion native-script tokens and 224.6 billion romanized tokens across 34 language-script configurations. Twelve languages include both native-script and Latin-script romanized text: Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu, and Urdu. Ten additional languages appear solely in native scripts: Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali, and Sindhi.
Two generation pipelines
Engineers drew source documents from 15 subsets of Nemotron-CC-v2, covering web text, mathematics, programming code, and question-answer material. NeMo Data Designer handled translation, quality scoring, and text repairs while retaining original source identifiers for auditing.
For the twelve dual-script languages, the pipeline translated complete documents using Gemma-4-31B-it to maintain paragraph context and formatting. The same model evaluated outputs against an error rubric inspired by Multidimensional Quality Metrics, reviewing meaning, fluency, terminology, style, and formatting. The team retained translations scoring at least 75 out of 100, repairing lower-scoring candidates to recover 438.4 billion tokens. Romanized text was generated by converting accepted native-script translations into Latin script rather than translating directly from English.
The remaining ten languages used Indic Translate, a document translation model developed by Bodhan AI at IIT Madras, with Gemma-4-31B-it handling evaluation. Santali documents were translated across overlapping text windows before reconstruction, while Manipuri text was translated into Bengali script prior to conversion into Meitei Mayek.
Testing on Nemotron models
Nvidia tested the data during continued pretraining for Nemotron 3.5 Super VL, allocating 10 billion Indic tokens from a twelve-language synthetic subset and natural text out of a 100-billion-token training budget. Model scores gained between 4.98 and 9.82 percentage points across five benchmarks: MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy, and Indic GSM8K. Odia was excluded from GSM8K testing because evaluation exceeded the serving context limit.
Nemotron 3.5 Super VL outperformed the earlier Nemotron 3 Super on five of six task evaluations, dipping 2.04 points on IFEval 1.5 instruction following. It also showed improvements across all four dimensions of Sarvam AI's IndiVibe benchmark: usefulness, fluency, script fidelity, and verbosity.
Developers can inspect splits and stream samples through Hugging Face Datasets or customize translation runs using the open-source Nemotron Bharat plugin. AI4Bharat and Nvidia are also developing Indic-LLM-Eval-Suite, an open collection of evaluation benchmarks designed to measure language quality and task performance across Indian languages.
