uber / petastorm
Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Why this job: matched by what the repo talks about: evaluation, dataset, learn, model
What it does
Petastorm is an open source data access library developed at Uber ATG. This library enables single machine or distributed training and evaluation of deep learning models directly from datasets in Apache Parquet format. Petastorm supports popular Python-based machine learning (ML) frameworks such as Tensorflow _, PyTorch _, and PySpark _. It can also be used from pure Python code.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Inactive
- Last commit 10 months ago
- Stars
- 1,895
- On GitHub
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.13.1
- Released 10 months ago
- Built with
- Python
- 100% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 49% closed
- 158 still open
- Pull requests
- 408 merged
- 26 waiting
- Releases
- 50
- 1 in the last year
- No contributing guide
- No security policy
- Code of conduct (opens in new tab)
More: languages, topics and project record
Languages
- Python 100%
- Dockerfile <1%
- Makefile <1%
- Shell <1%
Topics
- tensorflow
- pytorch
- deep-learning
- machine-learning
- sysml
- pyspark
- pyarrow
- parquet
- parquet-files
Project record
- JobEvals and observability
- StatusListed
- Commits701
- Forks285
- Watching36
- Stars in 28 days0
- CreatedJun 15, 2018
- First seen by usNov 7, 2026
- CheckedNov 8, 2026