Evaluation Libraries

13 Components & Libraries

Sortby

Evaluation Libraries

🪢 Open source LLM engineering platform. Observability, metrics, evals, prompt management, testing -- 🍊YC W23 🤖 integrate via Typescript, Python, OpenAI, Langchain, LlamaIndex, Litellm

Managed deployment by the Langfuse team, generous free-tier (hobby plan), no credit card required. You may want to use another integration to get started quickly or implement a use case that we do no…

:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI

EvalAI is an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence (AI) algorithms at scale. In recent years, it has become increasingly difficult to com…

Multi-class confusion matrix library in Python

Fig1. ConfusionMatrix Block Diagram After that, two scores are calculated for each confusion matrices, overall and class-based. The overall score is the average of the score of seven overall benchmar…

Open source Python SDK for agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

AgentOps helps developers build, evaluate, and monitor AI agents. Tools to build agents from prototype to production. Initialize the AgentOps client and automatically get analytics on every LLM call.…

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks s…

Machine Learning notebooks for refreshing concepts.

Helpful jupyter noteboks that I compiled while learning Machine Learning and Deep Learning from various sources on the Internet. Linear & Multiple Regression Logistic Regression Regularization

A petrophysics python package for geoscience python computing of conventional and unconventional formation evaluation. Reads las files and creates a pandas dataframe of the log data. Includes a basic petrophysical workflow and a simple log viewer based on XML templates.

Install PetroPy through pip via the command line To read in an las file, pass the file reference:

Clustering methods in Machine Learning includes both theory and python code of each algorithm. Algorithms include K Mean, K Mode, Hierarchical, DB Scan and Gaussian Mixture Model GMM. Interview questions on clustering are also added in the end.

Measure and visualize machine learning model performance without the usual boilerplate.

Measure, visualize, and compare machine learning model performance without the usual boilerplate. Breaking API improvements to be expected. Or, for the latest unreleased version:

An open, minimalist Gymnasium environment for autonomous coordination in wireless mobile networks.

mobile-env supports multi-agent and centralized reinforcement learning policies. It provides various choices for rewards and observations. mobile-env is also easily extendable, so that anyone may add…

Python client library for improving your LLM app accuracy

⚡ Unified LLM data management to drive accuracy at scale ⚡ A one-line Python integration to manage your LLM data. Use Log10 to log both closed and open-source LLM calls. It helps you: Log10 offers va…

Artificial intelligence (AI, ML, DL) performance metrics implemented in Python

Please include these citations if you plan to use this library: LaTeX: APA: After installation, you can import Permetrics as any other Python module: Below is the most efficient and effective way to …

Audio degradation toolbox in python, with a command-line tool. It is useful to apply controlled degradations to audio: e.g. data augmentation, evaluation in noisy conditions, etc.

Audio degradation toolbox in python, with a command-line tool. It is useful to apply controlled degradations to audio. A small set of sounds and impulse responses are installed along with the script,…

Related tags