<h1 align="center">
<a href="https://prompts.chat">
This comprehensive guide covers development workflows for the Ragas monorepo, designed for both human developers and AI agents.
Sign in to like and favorite skills
This comprehensive guide covers development workflows for the Ragas monorepo, designed for both human developers and AI agents.
# 1. Clone and enter the repository git clone https://github.com/vibrantlabsai/ragas.git # 2. Install uv (if not already installed) curl -LsSf https://astral.sh/uv/install.sh | sh # 3. Choose your installation type: # RECOMMENDED: Minimal dev setup (fast) make install-minimal # FULL: Complete dev environment (comprehensive) make install # 4. Verify everything works make check # 5. Start developing! make help # See all available commands
AI agents working with this codebase should use these standardized commands:
# Essential commands for AI development make help # See all available targets make install-minimal # Minimal dev setup (fast) make install # Full environment (modern uv sync) make check # Quick health check (format + type) make test # Run all tests make run-ci # Full CI pipeline locally # Individual development tasks make format # Format and lint all code make type # Type check all code make clean # Clean generated files
Key Points for AI Agents:
make commands rather than direct tool invocationuv run prefix for any direct Python tool usagemake help for the complete command referenceThis repository is organized as a single project with integrated experimental features:
/ # Main ragas project ├── src/ragas/ # Main source code │ └── experimental/ # Experimental features ├── tests/ # Tests (unit, e2e, benchmarks) │ └── experimental/ # Experimental tests ├── examples/ # Example code ├── pyproject.toml # Dependencies and configuration ├── docs/ # Documentation ├── .github/workflows/ # CI/CD pipeline ├── Makefile # Build commands └── CLAUDE.md # AI assistant instructions
src/ragas/)src/ragas/experimental/examples/ as an installable package ragas-examplespublish-examples.ymlexamples-v (e.g., examples-v0.1.0)uv pip install -e . -e ./examplespython -m ragas_examples.benchmark_llm.prompt# Recommended: Minimal dev setup make install-minimal # Full: Complete environment make install
# Install uv if not available curl -LsSf https://astral.sh/uv/install.sh | sh # Minimal dev: Core + essential dev tools uv pip install -e ".[dev-minimal]" # Full dev: Everything (uses modern uv sync) uv sync --group dev
Use
if you're:make install-minimal
Use
if you're:make install
install-minimal: Uses uv pip install -e ".[dev-minimal]" for selective minimal dev dependenciesinstall: Uses uv sync --group dev for complete modern dependency managementmake check # Runs format + type checking make test # Runs all tests
Run
make help to see all targets. Here are the essential commands:
make install-minimal - Install minimal dev setup (recommended)make install - Install full environment with uv sync (complete)make format - Format and lint all code (includes unused import cleanup)make type - Type check all codemake check - Quick health check (format + type, no tests)make test - Run all unit testsmake test-e2e - Run end-to-end testsmake benchmarks - Run performance benchmarksmake benchmarks-docker - Run benchmarks in Dockermake run-ci - Run complete CI pipeline locallymake clean - Clean all generated filesmake build-docs - Build all documentationmake build-docs-pdf - Build documentation with PDF export (requires WeasyPrint)make serve-docs - Serve documentation locallydocs/community/pdf_export.md for PDF export details and limitations# 1. Start your work git checkout -b feature/your-feature # 2. Make changes to code # 3. Check your work make check # Format and type check make test # Run tests # 4. Commit and push git add . git commit -m "feat: your feature description" git push origin feature/your-feature
make run-ci # Run full CI pipeline # Ensure all checks pass before creating PR
# Use the Makefile for all development make help # See available commands make format # Format all code (core + experimental) make type # Type check all code make test # Run all tests (core + experimental) make check # Quick format + type check make run-ci # Run full CI pipeline # Or use direct commands for specific tasks uv run pytest tests/unit # Run core unit tests uv run pytest tests/unit # Run unit tests uv run pyright src # Type check source code
# All tests make test # Specific test categories uv run pytest tests/unit uv run pytest tests/e2e # With coverage or specific options uv run pytest tests/unit -k "test_name"
tests/unit/tests/e2e/tests/benchmarks/The
make format command runs:
make type # Type check all code with pyright
Our GitHub Actions CI runs:
make run-ci # Runs: format + type + test
pyproject.tomlpyproject.tomlmake install and make test# Reinstall in development mode make install
# Run specific failing test uv run pytest tests/unit/test_specific.py -v # Check experimental test dependencies uv run pytest tests/unit --collect-only
# Fix formatting make format # Check specific files uv run ruff check path/to/file.py --fix
# Run the same checks locally make run-ci # Individual checks make format # Must pass make type # Must pass make test # Must pass
# Install uv curl -LsSf https://astral.sh/uv/install.sh | sh # or use pip: pip install uv
# Clean install make clean make install
CLAUDE.md for AI assistant guidancemake help for all available targetsgit checkout -b feature/amazing-featuremake run-cifeat: add new evaluation metric fix: resolve import error in experimental docs: update development guide test: add unit tests for metric base
make test)make format)make type)make targetsmake check frequently during developmentmake run-ci before completing# Always start with understanding the current state make help ls -la # Check current directory structure # For code changes make format # After making changes make test # Verify functionality # For project-specific work make help # See available commands # For investigation uv run pytest --collect-only # See available tests uv run ruff check --no-fix # Check issues without fixing
make format)make install attempts to build numpy==2.0.x from source on Python 3.13 (no prebuilt wheel), failing with C/C++ errors.Workarounds:
uv python install 3.12 rm -rf .venv uv venv -p 3.12 make install
rm -rf .venv uv venv -p 3.13 make install-minimal uv pip install "ragas[tracing,gdrive,ai-frameworks]"
uv pip install "numpy>=2.1" --only-binary=:all:
If conflicts pin NumPy to 2.0.x, temporarily set
numpy>=2.1 in pyproject.toml and run uv sync --group dev.
Happy coding! 🚀
For additional context and instructions specific to AI assistants, see CLAUDE.md.