Yolande1994 / CUDA_GPT2_Inference
纯 CUDA 实现 GPT2 推理，逐步优化算子与架构，最终在单请求下推理速度领先 vLLM 约 1.5 倍。含详细 NCU 分析与 README 优化报告。
Why this job: topics: llm-inference, quantization
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 11
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Cuda
- 81% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- None
- No issues filed
- Pull requests
- 0 merged
- 0 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| pegainfer-project / pegainfer | 740 | yesterday | Apache-2.0 | Linux |
| syv-ai / HyperQwen | 2,687 | 2 days ago | Apache-2.0 | No downloads |
| zengxiao-he / tessera | 557 | 5 months ago | Custom | No downloads |
More: languages, topics and project record
Languages
- Cuda 81%
- C++ 8%
- Python 7%
- C 2%
Topics
- cpp
- cuda
- cuda-graphs
- cuda-kernels
- gemv
- gpt-2
- kv-cache
- llm-inference
- quantization
- transformer
- flash-attention
- flash-decoding
Project record
- JobLocal LLMs
- StatusNew to our list
- Commits7
- Forks0
- Watching0
- CreatedNov 10, 2026
- First seen by usNov 12, 2026
- CheckedNov 12, 2026