MinishLab / tokenlearn
Pre-train Static Word Embeddings
Why this job: works in the areas we cover, but does not say which job it does
What it does
Tokenlearn is a method to pre-train Model2Vec static embedding models. The original version, used to train the first Potion models (potion-base-32M, potion-base-8M, potion-base-4M, potion-base-2M), is described in our original blogpost. The current version, used to train potion-multilingual-128M, is covered in our Tokenlearn 2.0 post.
Get started
pip install tokenlearn- Maintenance
- Active
- Last commit 43 days ago
- Stars
- 112
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.2.1
- Released 17 months ago
- Built with
- Python
- 99% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- 88% closed
- 1 still open
- Pull requests
- 28 merged
- 0 waiting
- Releases
- 5
- 0 in the last year
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| huggingface / text-embeddings-inference | 5,120 | 6 days ago | Apache-2.0 | No downloads |
| AnswerDotAI / ModernBERT | 1,781 | 8 months ago | Apache-2.0 | No downloads |
| curiosity-ai / catalyst | 863 | 35 days ago | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 99%
- Makefile 1%
Topics
- ai
- embeddings
- machine-learning
- model2vec
- nlp
- python
- torch
Project record
- JobOther AI tools
- StatusListed
- Commits38
- Forks11
- Watching5
- Stars in 28 days+1
- CreatedOct 13, 2024
- First seen by usNov 7, 2026
- CheckedNov 7, 2026