Skip to content

About

This repository serves as a comprehensive collection of resources focused on testing AI/ML systems, which present unique challenges beyond traditional software testing. The curated list provides resources for ensuring the quality, reliability, and fairness of AI/ML systems throughout their lifecycle.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

1 watching

Forks

Latest commit

ย 

History

14 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Awesome AI/ML Testing Awesome GitHub stars Link Check License: CC0-1.0 Visitors Contributors Last Commit

The Ultimate Resource for Testing AI/ML Systems - Curated Collection of Frameworks, Tools, Methodologies, and Best Practices

Awesome AI/ML Testing

๐Ÿค” Why This Repository?

Testing AI/ML systems is crucial yet challenging due to their unique characteristics:

  • Non-deterministic behavior: Models may produce different outputs for same inputs
  • Data dependencies: Performance heavily relies on training data quality
  • Evolving nature: Models degrade over time requiring continuous monitoring
  • Complex evaluation: Traditional metrics may not capture real-world performance

This repository provides comprehensive resources to address these challenges and ensure your AI/ML systems are reliable, fair, and production-ready.

๐ŸŒŸ Share This Repository

Help spread the word about this resource by sharing it with your network:

Twitter LinkedIn Reddit

๐Ÿ“ˆ Star History

Star History Chart

๐Ÿš€ Getting Started

Testing AI/ML systems presents unique challenges beyond traditional software testing. This curated list provides resources for ensuring the quality, reliability, and fairness of your AI/ML systems throughout their lifecycle.

Whether you're new to ML testing or an experienced practitioner, this list offers valuable resources for implementing effective testing strategies for your AI/ML applications.

๐Ÿ“‹ Contents

๐Ÿ› ๏ธ General ML Testing Frameworks

  • Deepchecks - Test suite for data and models, validating a wide range of potential issues.
  • MLtest - Testing infrastructure for machine learning with focus on TensorFlow.
  • Giskard - Testing framework focused on vulnerability scanning and detection.
  • Pytest-ml - Extension of pytest for validating ML code and models.
  • Manifold - Visual debugging tool for machine learning models.
  • ModelDB - System to version ML models and track experiments.
  • MLFlow - Platform for the ML lifecycle, including model testing.
  • HyperDrive - Hyperparameter tuning and neural architecture search.
  • Kedro - Framework for creating reproducible ML pipelines.
  • Seldon Alibi - Algorithms for monitoring and explaining machine learning models.
  • Determined AI - Deep learning training platform with built-in validation.
  • Ray Tune - Distributed hyperparameter tuning framework.

๐Ÿค– LLM & Chatbot Testing

  • LangChain Evaluation Framework - Testing framework for LLM applications.
  • HELM - Holistic Evaluation of Language Models.
  • LLM-Eval - Framework for evaluating language models.
  • Botium - Testing automation for conversational AI.
  • BERTScore - Evaluating text generation with BERT.
  • ChatEval - Scientific framework for evaluating chatbots.
  • ROUGE - Evaluation metric for summarization tasks.
  • Rasa Testing Framework - Built-in testing for Rasa conversational agents.
  • promptfoo - Testing and evaluation framework for LLM prompts.
  • TruLens - LLM evaluation focused on faithfulness, quality and safety.
  • Guardrails - Framework to validate and clean language model outputs.
  • DeepEval - Evaluation framework for LLMs with human-like metrics.
  • Ragas - Framework for evaluation of RAG (Retrieval Augmented Generation) systems.
  • Promptimize - A/B testing and optimization for LLM prompts.
  • LlamaIndex Evaluation - Evaluation modules for RAG applications.
  • TruLens-Eval - Advanced evaluation metrics for LLMs beyond traditional metrics.
  • MT-Bench - Multi-turn benchmark for LLM conversation quality.
  • Anthropic's Claude Evaluations - Specialized testing framework for Constitutional AI models.
  • FairEval - Framework for evaluating fairness in language models.
  • AgentBench - Framework for evaluating LLM agents on complex tasks.
  • ai-evaluation - Open-source LLM evaluation framework with 50+ metrics, LLM-as-Judge, and guardrail scanners (jailbreak, PII, injection).

๐Ÿ“Š Data Validation & Quality

  • Great Expectations - Framework for data validation and quality.
  • TFDV - TensorFlow Data Validation for analyzing and validating ML data.
  • Deequ - Data quality validation for large datasets (by AWS).
  • Pandera - Statistical data validation for pandas.
  • Cerberus - Lightweight and extensible data validation library.
  • Data-Linter - Tool to identify common data quality issues.
  • DataProfiler - Profiling and detecting sensitive information in data.
  • Soda Core - Data quality testing and monitoring.
  • PyDeequ - Python wrapper for Deequ library.
  • Deepchecks Data Integrity - Specialized tests for data integrity.
  • Validio - Data validation pipeline tool.
  • Frictionless Data - Data validation framework for tabular data.

โš–๏ธ Fairness, Bias & Ethics Testing

  • AI Fairness 360 - Toolkit for detecting and mitigating bias in ML models.
  • What-If Tool - Visual tool for investigating model behavior.
  • FairML - Auditing black-box predictive models for bias.
  • Aequitas - Bias and fairness audit toolkit.
  • Fairlearn - Tools for assessing and improving fairness of ML systems.
  • ML Fairness Gym - Simulation platform for fairness research.
  • LiFT - LinkedIn Fairness Toolkit for measuring bias in ML workflows.
  • Themis ML - Library for fairness-aware machine learning.
  • FairKit - Tools for fair and explainable ML models.
  • Bias Buccaneers - Detecting and mitigating bias in NLP models.
  • Gender Bias Test - Testing and mitigating gender bias in word embeddings.
  • AI Ethics Toolbox - Practical tools for ethical AI development.

๐Ÿ›ก๏ธ Adversarial Testing

  • TextAttack - Framework for adversarial attacks on NLP models.
  • Adversarial Robustness Toolkit - Testing model robustness against attacks.
  • CleverHans - Library for benchmarking vulnerability to adversarial examples.
  • FoolBox - Python toolbox to create adversarial examples.
  • DeepFool - Simple algorithm to find adversarial perturbations.
  • EvalAI - Open source platform for evaluating AI algorithms.
  • Counterfit - Command-line tool for security testing of ML systems.
  • RAID - Robust AI framework for detecting adversarial inputs.
  • Alibi Detect - Tools for outlier, adversarial, and drift detection.
  • PGD Attack - Implementation of projected gradient descent attack.
  • FGM Attack - Fast gradient sign method for adversarial attacks.
  • Threat of Neural Fake News - Tools for detecting neural fake news.

๐Ÿ“ก Monitoring & Observability

  • Evidently AI - Tools for ML model monitoring and evaluation.
  • WhyLogs - Profiling library for ML/AI data and model monitoring.
  • Seldon Core - Platform for deploying ML models with monitoring.
  • Prometheus - Monitoring system adaptable for ML metrics.
  • Grafana - Analytics & monitoring solution for metrics visualization.
  • Arize AI - ML observability platform.
  • NannyML - Post-deployment model performance monitoring.
  • BentoML - Model serving with monitoring capabilities.
  • Datadog - ML monitoring integrated with infrastructure.
  • Fiddler AI - Model monitoring focused on explainability.
  • traceAI - Open-source OpenTelemetry-native tracing for LLM and agent apps with 50+ framework integrations.
  • Gantry - Production ML monitoring and debugging.
  • Aporia - Model observability platform.
  • Helicone - Open-source LLM observability platform.
  • LangSmith - Observability and evaluation platform for LLM applications.
  • Superwise - ML monitoring focused on sensitive data applications.
  • Prometheus ML Metrics - Specialized ML metrics for Prometheus.
  • MLflow Monitoring Extensions - Advanced monitoring plugins for MLflow.

๐Ÿ”„ CI/CD for ML

  • Kubeflow - ML toolkit for Kubernetes with CI/CD capabilities.
  • GitHub Actions for ML - GitHub Actions workflows for ML.
  • DVC - Git for data science, enables versioning of data and models.
  • CML - Continuous Machine Learning - CI/CD for ML projects.
  • Jenkins ML Plugin - Plugin connecting Jenkins with ML libraries.
  • MLReef - GitLab-based collaboration platform for ML.
  • Pachyderm - Data lineage and version control for data science.
  • ZenML - MLOps framework focusing on reproducibility.
  • MLflow CI/CD - Continuous integration features of MLflow.
  • Metaflow - Netflix's framework for building data science pipelines.
  • TFX - TensorFlow Extended for ML pipelines.
  • Flyte - Kubernetes-native workflow orchestration platform.

๐ŸŽญ Multi-modal Testing

  • MMMU - Massive Multi-discipline Multimodal Understanding benchmark for testing multimodal models across 30+ disciplines.
  • MME - Multimodal Evaluation benchmark for testing vision-language models on fine-grained capabilities.
  • MultiModal-Eval - Framework for testing multimodal models across diverse tasks and data types.
  • LLaVA-Bench - Benchmarking suite for vision-language models on instruction following and reasoning.
  • Modular Testing Framework for Multimodal Models - Testing framework that separates modality-specific testing from integration testing.

๐Ÿ“œ Regulatory Compliance Testing

๐ŸŽจ Generative AI Evaluation

๐Ÿ’ป Hardware-aware ML Testing

  • DeepPerf - Framework for testing ML models across various hardware configurations.
  • TVM-Benchmark - Testing suite for model performance on diverse hardware targets.
  • MLPerf for Edge Devices - Benchmark suite for ML inference on edge devices.
  • PowerInfer - Testing framework for power consumption of ML inference.
  • TensorRT Test Automation - Tools for testing hardware acceleration of neural networks.

๐Ÿ›ก๏ธ AI Safety Testing

๐Ÿ“‚ Benchmarks & Datasets

๐Ÿ“š Books & Academic Papers

๐ŸŽ“ Courses & Tutorials

๐Ÿ“ฐ Blogs & Newsletters

๐Ÿ‘ฅ Communities & Conferences

  • ML-Ops Community - Community focused on ML operations including testing.
  • Papers with Code - Platform for ML research including benchmarking.
  • SciKit-Learn Community - Community for ML in Python with testing practices.
  • ICLR - Conference on representation learning with testing tracks.
  • NeurIPS - Machine learning conference with ML testing topics.
  • ACM FAccT - Conference on Fairness, Accountability, and Transparency.
  • CIKM - Conference on Information and Knowledge Management with ML focus.
  • MLSys - Conference focused on ML systems including testing.
  • ML Safety Workshop - Workshop on safety aspects of ML systems.
  • MLOps World - Conference dedicated to ML operations.
  • Responsible AI Forum - Forum on ethical AI development.
  • AIES Conference - AI, Ethics, and Society conference.

๐Ÿ“‹ Case Studies

๐Ÿš€ Emerging Trends

๐Ÿ“ˆ Weekly Updates

API Updates

Automated updates from GitHub and arXiv APIs

Community Contributions

Resources submitted by our community

Editor's Picks

Curated selection of top resources

Partner Highlights

Featured content from our partners

๐ŸŽฎ Interactive Resources

๐Ÿงฉ Testing Challenges

๐ŸŒŸ Community Showcase

๐ŸŒŸ Community Contributors

We gratefully acknowledge the contributions from our community members:

Your name could be here!
Learn how to contribute

Contributing

Your contributions are always welcome! Please read the contribution guidelines first.

License

About

This repository serves as a comprehensive collection of resources focused on testing AI/ML systems, which present unique challenges beyond traditional software testing. The curated list provides resources for ensuring the quality, reliability, and fairness of AI/ML systems throughout their lifecycle.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages