Pipeline Libraries

47 Components & Libraries

Sortby

Pipeline Libraries

Another curated list of Python frameworks

A HTML page with curated list of Python frameworks! That's it! Add a Bootstrap card element, like this: If you need a new category for Python frameworks, try this: And, update the navbar links,

Another curated list of Python frameworks

A HTML page with curated list of Python frameworks! Fork and clone this repository, and run Add a Bootstrap card element, like this: If you need a new category for a Python framework, try this:

A reactive notebook for Python — run reproducible experiments, execute as a script, deploy as an app, and version with git.

Click this link to open the repo in a Gitpod Workspace: Create or edit notebooks with We're building a community. Come hang out with us! We believe that the tools we use shape the way we think — bett…

An open-source computer vision framework to build and deploy apps in minutes

Pipeless is inspired by modern serverless technologies. You provide some functions and Pipeless takes care of executing them for new video frames and everything involved. With Pipeless you create se…

Your single tool to express data, ML, and LLM pipelines with simple python functions. Runs anywhere that python runs, E.G. spark, airflow, jupyter, fastapi, etc. Incrementally adoptable. Use Hamilton to build testable, reusable, and self-documenting dataflows with lineage and metadata out of the box.

Hamilton is a novel paradigm for specifying a flow of delayed execution in python. It works on python objects of any type and dataflows of any complexity. Core to the design of Hamilton is a clear ma…

🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

If that sounds like a lot, don't worry. After you cover this course, you will understand everything we said before. Most importantly, you will know WHY we used all these tools and how they work toget…

BuildFlow, is an open source framework for building large scale systems using Python. All you need to do is describe where your input is coming from and where your output should be written, and BuildFlow handles the rest. No configuration outside of the code is required.

Turn any function into a component of your backend system. Inject any dependency with full control over its setup and lifecycle Installing Pulumi unlocks:

Complete log pipeline solution in Python. Runs fast in Go.

Vaero's Python syntax is modern and easy to use. Log pipelines are executed with high-performance Go code.

SORDI dataset has per frame annotation file in json format. Following tools create a COCO style annotation out of it. Thus the SORDI data can be easily fed into COCO style training pipelines.

SORDI dataset has per frame annotation file in json format. Following tools create a COCO style annotation out of it. Thus the SORDI data can be easily fed into COCO style training pipelines. Sample …

Project demonstrating how to automate Prefect 2.0 deployments to AWS ECS Fargate

The goal of this repository template is to make it easy for you to get started with Prefect. Ideally, you should be able to:

New generation opensource data stack

This repository contains Docker compose script that creates opensource data analytics stack on your local machine. Currently, the stack consists of multiple components: I plan to add more components …

Build and deploy simple flask application using Jenkins and Kubernetes

Build and deploy code a simple flask application using Jenkins and Kubernetes Login to your github account and create a new repository. Do make sure that you have given a unique name for the reposito…

Turns Data and AI algorithms into production-ready web applications in no time.

Taipy is designed for data scientists and machine learning engineers to build full-stack apps. To install Taipy stable release run: Below is our filter function. This is a typical Python …

The RAP community of practice includes all analysts and data scientists who are interested in adopting the working practices included in reproducible analytical pipelines (RAP) at NHS Digital.

Pythonic tool for running machine-learning/high performance/quantum-computing workflows in heterogeneous environments.

Covalent is a Python library for AI/ML engineers, developers, and researchers. It provides a straightforward approach to running compute jobs, like LLMs, generative AI, and scientific research, on v…

One framework to develop, deploy and operate data workflows with Python and SQL.

This repository provides a working, deployable, open source-based, serverless service template with an AWS Lambda function and AWS CDK Python code with all the best practices and a complete CI/CD pipeline.

This project provides a working, open source based, AWS Lambda handler skeleton Python code including DEPLOYMENT code with CDK and a pipeline. This project can serve as a template for new Serverless …

Distributed Machine Learning Patterns from Manning Publications by Yuan Tang https://bit.ly/2RKv8Zo

This book teaches you how to take machine learning models from your personal laptop to large distributed clusters. You’ll explore key concepts and patterns behind successful distributed machine learn…

:zap: Data quality management for SQL, Spark, and Pandas-accessible data

Soda Core is a free, open-source, command-line tool that enables you to use the Soda Checks Language to turn user-defined input into aggregated SQL queries. When it runs a scan on a dataset, Soda Cor…

ETL Project with the Spotify API. Automating extracting data from the Spotify API, transforming it, and loading it into a Postgres SQL database using Python. Scheduling of script using Apache Airflow. Additionally, automating weekly summary statistics sent to my email.

ML pipeline orchestration and model deployments on Kubernetes.

Any stage in a Bodywork pipeline can do one of two things: You can use these to compose pipelines for many common ML use-cases, from serving pre-trained models to running continuous training on a sch…

Regular practice on Data Science, Machien Learning, Deep Learning, Solving ML Project problem, Analytical Issue. Regular boost up my knowledge. The goal is to help learner with learning resource on Data Science filed.

Machine Learning Algorithm Data Cleaning Process Data Mining Tehnique Data Analysis Database Connection--- MySql, PostgresSql, Elasticsearch Dashboard Development Feature Engineering & Feature Se…

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

These three functionalities enable a variety of use cases for data scientists, machine learning engineers, and data engineers: From here, you can quickly log a dataset: And there you have it, you now…

A scalable general purpose micro-framework for defining dataflows. You can use it to build dataframes, numpy matrices, python objects, ML models, etc. Embed Hamilton anywhere python runs, e.g. spark, airflow, jupyter, fastapi, python scripts, etc.

The general purpose micro-framework for creating dataflows from python functions! Specifically, Hamilton defines a novel paradigm, that allows you to specify a flow of (delayed) execution, that forms…

🏭 PDF text extraction pipeline: self-hosted, local-first, Docker-based

PDFs are hard to process and it's hard to extract information. So the results of this tool may not satisfy you. There will be more work to improve this software but altogether, it's unlikely that it …

The repository contains beginner friendly DevOps projects for automating stuffs using DevOps concepts.

SaaS Boilerplate - Open Source and free SaaS stack that lets you build SaaS products faster in React, Django and AWS. Focus on essential business logic instead of coding repeatable features!

Supercharge your SaaS development with our comprehensive starter kit, designed to accelerate your project and save you valuable time and resources. Our battle-tested boilerplate eliminates the need f…

Beneath is a serverless real-time data platform ⚡️

Beneath is a serverless real-time data platform. Our goal is to create one end-to-end platform for data workers that combines data storage, processing, and visualization with data quality management …

Related tags