JevalKit

A common ground for evaluating decision models.

JevalKit is an open-source evaluation toolkit for Jev-like decision models. It provides unified model interfaces and prepared evaluation datasets, so you can evaluate models without preparing each benchmark in a separate repository.

What's included

• A common evaluation command across eight benchmark families.
• Prepared evaluation data in ten TSV views.
• Adapters for twenty model families.
• Saved predictions, manifests, probability metrics, and leaderboard exports.

Registration does not imply a completed evaluation. The leaderboard distinguishes full evaluations, smoke tests, and imported predictions. See its manifests for coverage and provenance.

Start evaluating

1. Install and run your first evaluation.
2. Set up a model's native dependencies.
3. Read the benchmark's scoring protocol.
4. Explore the leaderboard.

Add an integration

Open an issue with the model or benchmark ID. Include the author release and a minimal usage example where available.

GitHub · Datasets · Leaderboard