Judge with structured evaluations and add a classification judge

[pushpak1300]

Classification models such as TypeSafe's JEV are faster and cheaper than text models for grading. This lets them be the judge. Scorers now describe their evaluation as data, so each judge gets a request in its own format.

Usage

Add the provider key to .env:

TYPESAFE_API_KEY=...

Turn on the classifier judge in tests/Pest.php:

pest()->evals()->classify();                                          // ai.default_for_classification
pest()->evals()->classify(provider: 'typesafe', model: 'jev-latest'); // or pick one

Write evals as before. toSatisfy(), toBeCorrect(), toBeRelevant() and toBeSafe() use the classifier:

it('answers refund questions', function () {
    expect(RefundPolicyAgent::class)
        ->prompt('Can I get a refund on my purchase from two weeks ago?')
        ->toBeRelevant()
        ->toSatisfy('States the 30-day refund window.');
});

Run with vendor/bin/pest --evals. PEST_EVALS_LARAVEL_CLASSIFICATION_PROVIDER and PEST_EVALS_LARAVEL_CLASSIFICATION_MODEL override the provider and model per environment.

Note

Only one judge is active. classify() is shorthand for judgeUsing(new LaravelAiClassifier(...)), so a later judgeUsing() replaces it.

Warning

Upgrade note: JudgeDriver::generate() becomes judge(Evaluation $evaluation): Verdict. judgeUsing() closures receive an Evaluation and return a float or a Verdict. Custom judged scorers use Judge::evaluate() because JudgeResponse is removed.