Dataset Libraries

59 Components & Libraries

Sortby

Dataset Libraries

Qlib is an AI-oriented quantitative investment platform that aims to realize the potential, empower research, and create value using AI technologies in quantitative investment, from exploring ideas to implementing productions. Qlib supports diverse machine learning modeling paradigms. including supervised learning, market dynamics modeling, and RL.

Recent released features Features released before 2021 are not listed here. Qlib is an open-source, AI-oriented quantitative investment platform that aims to realize the potential, empower research, …

An open source multi-tool for exploring and publishing data

Datasette is a tool for exploring and publishing data. It helps people take data of any shape or size and publish that as an interactive, explorable website and accompanying API. Datasette is aimed a…

Open source annotation tool for machine learning practitioners.

doccano is an open-source text annotation tool for humans. It provides annotation features for text classification, sequence labeling, and sequence to sequence tasks. You can create labeled data for …

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

cleanlab is: cleanlab supports Linux, macOS, and Windows and runs on Python 3.7+. cleanlab can find label issues from any model's predicted class probabilities if you can produce them yourself. This …

pix2tex: Using a ViT to convert images of equations into LaTeX code.

The goal of this project is to create a learning based system that takes an image of a math formula and returns corresponding LaTeX code. To run the model you need Python 3.7+ Model checkpoints will …

Deep learning with satellite & aerial imagery

Extracting roads is challenging due to the occlusions caused by other objects and the complex traffic environment In instance segmentation, each individual 'instance' of a segmented area is given a u…

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

These three functionalities enable a variety of use cases for data scientists, machine learning engineers, and data engineers: From here, you can quickly log a dataset: And there you have it, you now…

A procedural Blender pipeline for photorealistic training image generation

A procedural Blender pipeline for photorealistic rendering. The simplest way to install blenderproc is via pip: Alternatively, if you need to make changes to blenderproc or you want to make use of th…

Medical imaging toolkit for deep learning

If you like this repository, please click on Star! If you use this package for your research, please cite our paper: BibTeX entry: This project is supported by the following institutions:

fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale.

🚀 Introducing VL Profiler! 🚀 We're excited to announce our new cloud product, VL Profiler. It's designed to help you gain deeper insights and enhance your productivity while using fastdup. With VL P…

Open source audio annotation tool for humans

audino is an open source audio annotation tool. It provides features such as transcription and labeling which enables annotation for Voice Activity Detection (VAD), Diarization, Speaker Identificatio…

Up to 10x faster string search, split, sort, and shuffle for long strings and multi-gigabyte files in Python and C, leveraging SIMD with just a few lines of Arm Neon and x86 AVX2 & AVX-512 intrinsics 🦖

StringZilla is the Godzilla of string libraries, splitting, sorting, and shuffling large textual datasets faster than you can say "Tokyo Tower" 😅 StringZilla is built on a very simple heuristic: If t…

An open-source framework for real-time anomaly detection using Python, ElasticSearch and Kibana

An open-source framework for real-time anomaly detection using Python, Elasticsearch and Kibana. The recommended installation method is to use pip within a Python 3.x virtalenv. You can use dsio thro…

Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision

This is an official repository of the paper Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision. The project was formerly referred by RingNet. The codebase consists …

An open source framework for building data analytic applications.

CDAP is an integrated, open source application development platform for the Hadoop ecosystem that provides developers with data and application abstractions to simplify and accelerate application dev…

A review of change detection methods, including codes and open data sets for deep learning. From paper: change detection based on artificial intelligence: state-of-the-art and challenges.

Change detection based on remote sensing (RS) data is an important method of detecting changes on the Earth’s surface and has a wide range of applications in urban planning, environmental monitoring,…

Dataset Management Framework, a Python library and a CLI tool to build, analyze and manage Computer Vision datasets.

A framework and CLI tool to build, transform, and analyze datasets. Dataset reading, writing, conversion in any direction. Dataset building Dataset quality checking Dataset comparison Dataset statist…

Keep code, data, containers under control with git and git-annex

will install the most recently released version, and release candidates are available via MIT/Expat The DataLad project received support through the following grants: US-German collaboration in compu…

Data Science Hacks consists of tips, tricks to help you become a better data scientist. Data science hacks are for all - beginner to advanced. Data science hacks consist of python, jupyter notebook, pandas hacks and so on.

Data Science Hacks is created and maintained by Analytics Vidhya for the data science community. It includes a variety of tips, tricks and hacks related to data science, machine learning These Hacks …

FastDup is a tool for gaining insights from a large image collection. It can find anomalies, duplicate and near duplicate images, clusters of similaritity, learn the normal behavior and temporal interactions between images. It can be used for smart subsampling of a higher quality dataset, outlier removal, novelty detection of new information to be sent for tagging. FastDup scales to millions of images running on CPU only.

FastDup is a tool for gaining insights from a large image collection. It can find anomalies, duplicate and near duplicate images, clusters of similarity, learn the normal behavior and temporal intera…

🗂 Split folders with files (i.e. images) into training, validation and test (dataset) folders

The input folder should have the following format: In order to give you this: This package is Python only and there are no external dependencies. MIT

Understanding emotions from audio files using neural networks and multiple datasets.

This project presents a deep learning classifier able to predict the emotions of a human speaker encoded in an audio file. The classifier is trained using 2 different datasets, RAVDESS and TESS, and …

Python library for working with Music Information Retrieval datasets

This library provides tools for working with common MIR datasets, including tools for: To install, simply run: There are two ways of citing mirdata: If you are using the library for your work, please…

:mag_right: Data analysis with real world data sets using Python :mag:

White balance camera-rendered sRGB images (CVPR 2019) [Matlab & Python]

The original source code of our paper was written in Matlab. We also provide a Python version of our code. We tried to make both versions identical. However, there is no guarantee that the Python ver…

Python library for computer vision labeling tasks. The core functionality is to translate bounding box annotations between different formats-for example, from coco to yolo.

PyLabel is a Python package to help you prepare image datasets for computer vision models including PyTorch and YOLOv5. It can translate bounding box annotations between different formats. (For exa…

Python CLI tool and library for diffing CSV and JSON files

Consider two CSV files: You can also import the Python library into your own code like so: If the columns in the CSV have changed, those added or removed columns will be ignored when calculating chan…