Interactive-Active-AI / NextTransformer

:sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.

Why this job: matched by what the repo talks about: benchmark, llm, platform, open

Open on GitHub Website

Many architectures, one framework: attention, sparse, linear, looped, residual and hybrid models such as GQA, NSA, MoBA, GDN, KDA, Ouro, AttnRes and GDN+SWA plug into one shared trainer, data pipeline and evaluation.

A fair, explicit protocol: same data, tokens, optimizer, schedule and evaluation for every model; training compute matched to within 1% and audited over the whole run.

49B tokens of pretraining for a model with about 1.4B parameters, in 2–4 days on one 8-GPU node; 400B tokens and larger models are planned for November.

Agent-native research: hand the arena task to your agent; the full version, Arena v2, ships by November 14, 2026.

The NextTransformer observatory: learning curves, leaderboard, model atlas and arena

1 of 5

Get started

git clone https://github.com/Interactive-Active-AI/NextTransformer.git && cd NextTransformer
uv sync --frozen --extra gpu --extra upstream && source .venv/bin/activate
uv pip install --no-deps --require-hashes -r requirements/linear.txt
# linear-attention kernels

Maintenance: Active; Last commit yesterday
Stars: 407; +78 this week
Licence: MIT; Allows business use
Downloads for: None listed; No release files
Latest version: None yet; No releases
Built with: Python; 77% of the code

Is it looked after?

Contributors: 10; Last 90 days
Issues: 50% closed; 4 still open
Pull requests: 30 merged; 3 waiting
Releases: None; Never published

Similar repos

Repos similar to this one
Repo	Stars	Last commit	Licence	Downloads for
InternScience / ResearchClawBench	277	15 days ago	MIT	No downloads

More: languages, topics and project record

Languages

Python 77%
TypeScript 13%
CSS 7%
JavaScript 2%

Topics

auto-research
benchmark
deep-learning
language-models
linear-attention
llm
looped-transformer
pytorch
sparse-attention
transformer
nexttransformer

Project record

JobEvals and observability
StatusNew to our list
Commits95
Forks33
Watching16
CreatedOct 31, 2026
First seen by usNov 7, 2026
CheckedNov 12, 2026