Carbon: Decoding the Language of Life

External resource·Submitted May 19, 2026·47 pages·0 citations

Loubna Ben Allal, Qiuyi Li, Maurizio Fiusco, Lewis Tunstall, Kashif Rasul, Ed Beeching, Dana Aubakirova, Carlos Patiño, Thibaud Frere, Anton Lozhkov, Georgia Channing, Thomas Wolf

Source page Code Project page

TL;DR · AI-generated

Introduces Carbon, a family of 3B and 8B generative DNA language models trained on 1T DNA tokens with efficient 6-mer tokenization and long genomic contexts. Carbon is competitive with larger Evo2 models on training-free DNA benchmarks while running much faster, and the release includes models, data, training code, and evaluation tools.

Abstract

Genomic foundation models have emerged alongside the rapid progress of large language models, offering a promising framework for learning general-purpose sequence priors for DNA understanding, generation, and design. This connection to LLMs creates a major opportunity: modern architectures, scaling infrastructure, autoregressive training, and token-based mod…

+ read full abstract

Tasks3 tagged

BiologyLanguage ModelingRetrieval

Methods0 used

Results8 benchmarks

Biology8 results

Benchmark	Model	Metric	Value
Perturbation Bench	Carbon-8B (synonymous codon)	Accuracy	91.5	Compare →
Perturbation Bench	Carbon-8B (triplet expansion)	Accuracy	89.0	Compare →
Perturbation Bench	Carbon-3B (synonymous codon)	Accuracy	88.9	Compare →
Perturbation Bench	Carbon-3B (triplet expansion)	Accuracy	85.2	Compare →
Genomic-NIAH	Carbon-8B	Accuracy	86.0	Compare →
Genomic-NIAH	Carbon-3B	Accuracy	79.0	Compare →
Sequence Recovery	Carbon-8B	Accuracy	64.0	Compare →
Sequence Recovery	Carbon-3B	Accuracy	61.5	Compare →