Data-centric Libraries

6 Components & Libraries

Sortby

Data-centric Libraries

Low-code framework for building custom LLMs, neural networks, and other AI models

Key features: Install from PyPi. Be aware that Ludwig requires Python 3.8+. Or install with all optional dependencies: Let's fine-tune a pretrained LLaMA-2-7b large language model to follow instructi…

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

cleanlab is: cleanlab supports Linux, macOS, and Windows and runs on Python 3.7+. cleanlab can find label issues from any model's predicted class probabilities if you can produce them yourself. This …

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

Lance is a modern columnar data format that is optimized for ML workflows and datasets. Lance is perfect for: The key features of Lance include: Download the sift1m subset Convert it to Lance Build …

Modern columnar data format for ML implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

The data scientist's open-source choice to scale, assess and maintain natural language data. Treat training data like a software artifact.

Does one of these scenarios sounds familiar to you? If so, you are one of the people we've built refinery for. refinery helps you to build better NLP models in a data-centric approach. Semi-automate …

The open source active learning toolkit to find failure modes in your computer vision models, prioritize data to label next, and drive data curation to improve model performance.

Use Encord Active to visualize your data, evaluate your models, surface model failure modes, find labeling mistakes, prioritize high-value data for re-labeling and more! Encord Active helps you under…

Related tags