Add unsafe_untracked_call for impure lazy directories

[mbouaziz]

Summary

• Adds the @allow_tracked_call annotation: an untracked function or method carrying it appears tracked to callers — the annotation is part of the declaration's type. Its body is still typechecked in an untracked context. Because the semantics lives in the type, everything follows from the ordinary tracking rules with no special call-site logic:
  • direct calls, calls through function values, and pipe applications from tracked code all work;
  • an unannotated untracked override of an annotated method is rejected by the existing override lattice (untracked (() -> T) is not a subtype of (() -> T)), so dynamic dispatch can never reach an implementation that didn't opt in;
  • an annotated method can implement/override a tracked signature (impure implementation behind a tracked interface, with explicit opt-in);
  • the optimizer is deliberately not fooled: annotated declarations are still side-effecting, so they remain ineligible for CSE (same treatment as @debug).
• The annotation is rejected on declarations that are not untracked (it would be meaningless).
• Fixes lambda tracked_context in the type checker: a lambda body's tracking context is derived from the lambda's own modifiers and snapshot at its definition site (TUtils.Lambda.lam_tracked_context, mirroring lam_frozen_level), so untracked lambdas get an untracked body context and the verdict does not depend on where or when the lambda body happens to be solved.
• Pipe applications (x |> f) now run the same tracking check as ordinary calls (previously they bypassed it entirely).
• Adds Unsafe.unsafe_untracked_call<T>, a generic escape hatch that allows tracked code to run an untracked computation.

Fixes #1268.

Motivation

SKStore lazy directory compute functions run in a tracked context, which means they cannot call untracked functions. This is correct for pure reactive computations, but becomes a blocker for use cases like lazy file system access in mappers, where:

1. We don't know all needed files upfront — they're discovered during computation
2. File reads are inherently side-effectful and should be untracked
3. The caller takes responsibility for correctness and invalidation

Currently, volatile functions like Time.time_ms(), print_string, and FileSystem.readTextFile are not marked untracked even though they should be — precisely because doing so would make them uncallable from mapper contexts. This escape hatch allows us to eventually mark these functions as properly untracked without breaking existing code.

The opt-in is transitive by design: closures created inside the annotated body have tracked types, so an annotated declaration can hand untracked computations to tracked callers. The annotation's author takes responsibility for the full closure of behavior they expose.

Commits

1. Set tracked_context for untracked lambda bodies — lambda body contexts derive from the lambda's own modifiers, snapshot at the definition site (solve-order independent; pinned by tracked_lambda_solve_order).
2. Add @allow_tracked_call annotation — make_tfun_tracking consults the annotation at the two declaration-type construction sites (functions and methods) and in the backend lowering; naming rejects the annotation on non-untracked declarations; invalid tests pin the placement rules and the override direction.
3. Add AllowTrackedCall compiler test — tracked callers exercising the annotated function directly, through an untracked lambda, through a function value, and via a declared tracked function-typed parameter.
4. Add AllowTrackedCallMethod compiler test — annotated methods: dispatch through a base-typed receiver, method values, this calls from tracked methods, statics, implementing a tracked signature, and the covariant override direction.
5. Add Unsafe.unsafe_untracked_call to stdlib — the blessed generic escape hatch, exercised cross-module by the test.
6. Check tracking for pipe applications — |> previously bypassed check_tracking entirely; it now resolves the callee's tracking (including through type-parameter upper bounds) and applies the same rule.

Usage

fun myLazyCompute(context: mutable Context, path: String): String {
  Unsafe.unsafe_untracked_call(untracked () -> FileSystem.readTextFile(path))
}

Test plan

• AllowTrackedCall: direct, lambda-wrapped, through-a-value, declared-param, stdlib hatch, and pipe callers
• AllowTrackedCallMethod: base-receiver dispatch, method values, this, statics, tracked-signature implementation, covariant override
• Invalid: annotation on tracked fun/method; unannotated untracked override of annotated method; pipes into untracked callees (fun/lambda/tparam); existing untracked_fun2/untracked_lambda2 still fail
• Full compiler test suite passes locally
• CI passes

🤖 Generated with Claude Code

[mbouaziz]

Code review findings (multi-angle adversarial review; every finding below was probe-verified against a compiler built from this branch)

Correctness

1. [major] Virtual-dispatch CSE deletes side effects of annotated overrides — canCSECall (src/peephole.sk:383-409) allows CSE when any implementation at a callsite is CSE-able. An @allow_tracked_call untracked override sharing a virtual callsite with a tracked CSE-able sibling impl gets its second invocation deleted (reproduced at -O0 and -O2). Pre-PR this impl mix was inexpressible; the same hole exists on main for @debug impls. Filed as Dynamic dispatch calls are CSE'd even when a possible callee is @debug, deleting its side effects #1270 — the all-implementations fix there covers this PR's case with no extra logic (annotated impls keep funType.isTracked = false).
2. [minor] Pipe check regression on inferred callees — the pipe's synthesized expectation type carries {Fmutable, Funtracked} modifiers; once a free type variable is bound to it by a first pipe, a second pipe through the same value spuriously errors with "Cannot call an untracked function from a tracked context". v = Vector[]; f = v[0]; _ = "a" |> f; _ = "b" |> f compiles on main, rejected on this branch (src/skipTyping.sk:2848).
3. [minor] Pipe check bypass for forward-constrained callees — a still-free type variable defaults to Ftracked (src/skipTyping.sk:2836), so f = v[0]; _ = "a" |> f; v.push(side) performs an untracked call from a tracked context with no diagnostic, while the direct call or the reordered program is rejected. Both 2 and 3 share one fix: derive the synthesized expectation's tracking from env.tracked_context and run the check before the join.
4. [minor] Parameterized annotation silently ignored — @allow_tracked_call("reason") is stored with its argument suffix, but all three checks use exact-string annotationsContain: the opt-in is dropped and the misplacement rejection is bypassed (src/skipNaming.sk:1697, :1780, :2203). Fix: annotationsContainParam, or reject the parameterized form.
5. [minor] Chained tparam bounds skip the pipe tracking scan — fun callIt<G: untracked (String) -> String, F: G>(f: F, x: String): String { x |> f } passes typing and dies in the pre-existing lowering ICE (ICE: piping into a tparam bounded by a tracked function type — "Expected closure type but got F" #1269) instead of getting the clean tracking error the single-level bound gets (src/skipTyping.sk:2821).
6. [minor] Annotation flips isFrozen — annotated declarations are now Fpure + Ftracked, so their values are frozen: capturable by pure ~> lambdas and storable in frozen consts, both previously hard errors (pinned by frozen_lambda_captures_untracked). Arguably implied by "appears tracked", but undocumented, untested, and it is exactly the isDeepFrozen precondition CSE requires. Needs a decision: document + test, or key purity off untracked_.
7. [minor] Misplacement rejection only covers funs/methods — @allow_tracked_call on a class, const, or type alias is a silent no-op; class-level placement (expecting it to distribute over methods) is the natural misuse.
8. [nit, pre-existing] Malformed annotation arguments ICE — @allow_tracked_call() (empty parens) crashes with Invariant violation: Expected single argument to annotation (src/convertTree.sk:2157); shared by all annotations, newly exposed by a user-facing one.

Cleanup / consistency

9. get_instance_method_of_type (src/skipOuterIstUtils.sk:1577) re-derives method tracking from untracked_ only — consistent with the deliberate backend split, but deserves the same explanatory comment as convertFunDef/convertMethodDef; its consumers are currently dead code, leaving a latent two-types-for-one-declaration hazard.
10. The override diagnostic for an unannotated untracked override suggests an impossible fix ("annotate as this type: untracked (() -> String)") pointing at a parent that is already syntactically untracked — the annotation is invisible in printed types, and no tracking diagnostic ever mentions it (invalid_tests/typechecking/allow_tracked_call_override.exp_err).
11. Unsafe.unsafe_untracked_call's comment warns about closure reachability but not that invocation count through tracked-typed values is not preserved by the optimizer (resolved for the dispatch case by Dynamic dispatch calls are CSE'd even when a possible callee is @debug, deleting its side effects #1270; worth one sentence).
12. The pipe's Tparam upper-bound scan duplicates call_or_constructor_'s (src/skipTyping.sk:2821-2835 vs :1671-1682) — extract a shared helper; a future fix to one fold otherwise silently misses the other.
13. The pipe's tracking extraction + check_tracking re-implements what tfun_call does for direct calls; a shared callee_tracking_of_type helper would keep f(x) and x |> f in lockstep.
14. The placement check in fun_def is a verbatim copy of method_def's, with the annotation string spelled at three sites — one shared helper next to make_tfun_tracking would carry both.
15. Altitude: the pipe case inlines a check-only copy of callee resolution instead of sharing the resolve-callee path (including the Tparam → TAst.Annotated promotion) — which is precisely why pipes ICE (ICE: piping into a tparam bounded by a tracked function type — "Expected closure type but got F" #1269) where direct calls succeed. A shared resolution helper would fix 5, 12, 13, and the ICE divergence together.

The review also surfaced a handful of lower-severity notes not listed here. Items 2-5 and 12-15 are concentrated in the pipe path and largely collapse into one refactor; item 1 is tracked in #1270; item 6 is the one open semantics decision.

🤖 Generated with Claude Code