Skip to content

Ground truth: decide on gradient validation (currently fully unimplemented) #986

Open

Description

@michalharakal

Sub-issue of #984.

Despite the "gradienttracer" name, no gradient is ever actually checked today.
ToleranceConfig.GRADIENT, GroundTruthTestCase.expectedGradients, and grad_*-tensor
parsing all exist on the Kotlin side (GroundTruthLoader.kt,
GroundTruthValidator.validateGradientsInternal) — but store_experiment_as_gguf
(gradienttracer's gt/pytorch/io/writer.py) never calls .backward() or writes any
grad_* tensor. validateGradientsInternal is explicitly a placeholder (// TODO: Implement actual gradient computation via tape) that always reports failure if ever
invoked with real data.

Decision needed

  • Implement it: add .backward() + grad_* tensor serialization to
    store_experiment_as_gguf (every @Executable input already has requires_grad=True
    set, so the gradient is available, just not captured), and implement SKaiNET-side
    backward-pass execution in GroundTruthValidator.validateGradientsInternal to actually
    compare against it. This is real, nontrivial work on both sides — SKaiNET's own
    autodiff/backward execution path needs to be driveable from
    skainet-test-groundtruth, which it currently isn't.
  • Explicitly de-scope it: if gradient validation isn't a near-term priority, say so
    in CONTRACT.md and TESTING.md (which currently reads as if this already works),
    and consider whether the dead expectedGradients/ToleranceConfig.GRADIENT/
    validateGradients parameter surface should be removed rather than left as
    looks-implemented-but-isn't.

Either answer is fine — the point of this issue is picking one, since right now the code
reads as "supported" when it silently isn't.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    assessmentAssessment task (DARC: A)questionFurther information is requestedsub-issueSub-issue of a tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions