Big-data Libraries

22 Components & Libraries

Sortby

Big-data Libraries

Apache Spark - A unified analytics engine for large-scale data processing

Spark is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for da…

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

GitHub GitHub issues -- file bug reports / track issues here Stack Overflow -- ask all code/software questions here Cross Validated (Stack Exchange) -- ask algorithm/theory questions here h2ostream G…

Source code for the CERN Open Data portal

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

The reference book for these and other Spark related topics is: The following notebooks can be examined individually, although there is a more or less linear 'story' when followed in sequence. By usi…

An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset

Feel free to contact me to discuss any issues, questions, or comments. This repository contains a variety of content; some developed by Jose A. Dianes, and some from third-parties. The third-party c…

Efficient video analysis at scale

Scanner is a system for developing applications that efficiently process large video datasets. Please submit a pull-request rebased against the most recent version of the master branch and we will re…

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Install CatBoost by following the guide for the Next you may want to investigate: If you cannot open documentation in your browser try adding yastatic.net and yastat.net to the list of allowed domain…

Open Source Indonesian Python Programming Tutorial Site

Belajarpython adalah situs kolaboratif tutorial pemrograman Python bahasa Indonesia

A multi-cloud framework for big data analytics and embarrassingly parallel jobs, that provides an universal API for building parallel applications in the cloud ☁️🚀

Lithops is a Python multi-cloud distributed computing framework. It allows you to run unmodified local python code at massive scale in the main serverless computing platforms. Lithops delivers the u…

VerticaPy is a Python library that exposes sci-kit like functionality to conduct data science projects on data stored in Vertica, thus taking advantage Vertica’s speed and built-in analytics and machine learning capabilities.

⭐ 2023-12-01: VerticaPy secures 200 stars. 📢 2020-06-27: Vertica-ML-Python has been renamed to VerticaPy. The easiest and most accurate way to find documentation for a particular function is to use t…

Intel(R) Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application

With Intel(R) Extension for Scikit-learn, you can: To install Intel(R) Extension for Scikit-learn, run: The software acceleration is achieved through patching. It means, replacing the stock scikit-le…

Fundamentals of Spark with Python (using PySpark), code examples

But, PySpark+Jupyter combo needs a little bit more love :-) Resilient Distributed Datasets (RDD) is a fundamental data structure of Spark. It is an immutable distributed collection of objects. Each d…

:snowflake: :whale: Awesome tools and libs for AI, Deep Learning, Machine Learning, Computer Vision, Data Science, Data Analytics and Cognitive Computing that are baked in the oven to be Native on Kubernetes and Docker with Python, R, Scala, Java, C#, Go, Julia, C++ etc

The entire computing industry has adopted Kubernetes as the way forward to running distibuted computing loads from on-premise servers to large multi-cloud deployments. For Machine Learning and Data S…

Feature Store for Machine Learning

Feast allows ML platform teams to: The list below contains the functionality that contributors are planning to develop for Feast. We welcome contribution to all items in the roadmap! Feast is a commu…

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs for Scala, Java, Rust, Ruby, and Python.

There are two types of APIs provided by the Delta Lake project. Delta Lake guarantees backward compatibility for all Delta Lake tables (i.e., newer versions of Delta Lake will always be able to read …

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

Eland is a Python Elasticsearch client for exploring and analyzing data in Elasticsearch with a familiar Pandas-compatible API. Where possible the package uses existing Python APIs and data structur…

My journey to learn and grow in the domain of Machine Learning and Artificial Intelligence by performing the #100DaysofMLCode Challenge. Now supported by bright developers adding their learnings :+1:

LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.

LakeSoul is a cloud-native Lakehouse framework that supports scalable metadata management, ACID transactions, efficient and flexible upsert operation, schema evolution, and unified streaming & ba…

Data Science Roadmap from A to Z

Pandas

Multi-Modal Database replacing MongoDB, Neo4J, and Elastic with 1 faster ACID solution, with NetworkX and Pandas interfaces, and bindings for C 99, C++ 17, Python 3, Java, GoLang 🗄️

Would you prefer to connect to a remote UStore server? UStore comes with an Apache Arrow Flight RPC interface! Function calls may look identical, but the underlying implementation can be addressing h…

Replacing MongoDB, Neo4J, and Elastic with 1 transactional database. Features: zero-copy semantics, swappable backends, bindings for C, C++, Python, Java, GoLang

UKV is more than a database. It is a "build your database" toolkit and an open standard for NoSQL potentially-transactional databases, defining zero-copy binary interfaces for "Create, Read, Update, …

Artificial-Intelligence-Important-Documents-Collections

AI technology is significant because it allows software to do human functions—understanding, reasoning, planning, communication, and perception—increasingly effectively, efficiently, and affordably.

AI technology is significants because it allows softwares to do human functions—understanding, reasoning, planning, communication, and perception—increasingly effectively, efficiently, and affordably…

Related tags