The LLM Evaluation Framework
evaluation-metrics evaluation-framework llm-evaluation llm-evaluation-framework llm-evaluation-metrics
-
Updated
Dec 28, 2024 - Python
#
evaluation-metrics
Here are 463 public repositories matching this topic...
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen
agent ai openai evaluation-metrics mistral cost-estimation autogen groq agentops llm langchain anthropic evals ollama crewai
-
Updated
Dec 28, 2024 - Python
《大模型白盒子构建指南》：一个全手搓的Tiny-Universe
-
Updated
Dec 28, 2024 - Python
(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"
tracking machine-learning real-time computer-vision robotics evaluation evaluation-metrics multi-object-tracking kitti 3d-tracking 3d-multi-object-tracking 2d-mot-evaluation 3d-mot 3d-multi kitti-3d
-
Updated
Apr 3, 2024 - Python
Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!
-
Updated
Dec 16, 2024 - Jupyter Notebook
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
-
Updated
Dec 26, 2024 - Python
[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.
-
Updated
Aug 12, 2024 - Jupyter Notebook
OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)
nlp natural-language-processing hyperparameter-optimization topic-modeling nlp-library bayesian-optimization hyperparameter-tuning latent-dirichlet-allocation evaluation-metrics neural-topic-models latent-semantic-analysis topic-models hyperparameter-search non-negative-matrix-factorization nlproc
-
Updated
Jul 25, 2024 - Python
Evaluate your speech-to-text system with similarity measures such as word error rate (WER)
-
Updated
Nov 1, 2024 - Python
📈 Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.
-
Updated
Aug 31, 2024 - Python
A Neural Framework for MT Evaluation
nlp machine-learning natural-language-processing machine-translation artificial-intelligence evaluation-metrics
-
Updated
Dec 5, 2024 - Python
⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍
python information-retrieval evaluation comparison numba recommender-systems evaluation-metrics metasearch data-fusion score-fusion ranking-metrics information-retrieval-evaluation information-retrieval-metrics rank-fusion
-
Updated
Jul 1, 2024 - Python
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Mor…
python nlp machine-learning natural-language-processing library linguistics computational-linguistics text-processing nlp-library search-algorithms evaluation-metrics folia language-modelling
-
Updated
Sep 14, 2023 - Python
Data-Driven Evaluation for LLM-Powered Applications
information-retrieval evaluation-metrics evaluation-framework rag llmops retrieval-augmented-generation llm-evaluation
-
Updated
Dec 27, 2024 - Python
Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)
audio video pytorch transformer gan multi-modal evaluation-metrics video-understanding vas video-features vqvae bmvc melgan audio-generation vggsound
-
Updated
Jul 12, 2024 - Jupyter Notebook
Resources for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper
-
Updated
Jul 22, 2023 - Python
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
evaluation-metrics evaluation-framework rag large-language-models llm llms llmops retrieval-augmented-generation
-
Updated
Nov 14, 2024 - Python
[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
evaluation vision vqa llama object-detection gpt evaluation-metrics iclr multimodal vision-and-language hallucination vicuna gpt-4 foundation-models prompt-engineering chatgpt llava iclr2024
-
Updated
Mar 13, 2024 - Python
MapEval: Towards Unified, Robust and Efficient SLAM Map Evaluation Framework.
point-cloud slam evaluation-metrics lidar-point-cloud open3d wasserstein-distance map-evaluation pointcloud-registration
-
Updated
Dec 27, 2024 - C++
A Python wrapper for the ROUGE summarization evaluation package
-
Updated
Feb 10, 2021 - Python
Improve this page
Add a description, image, and links to the evaluation-metrics topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the evaluation-metrics topic, visit your repo's landing page and select "manage topics."