Skip to content

[core] Editor and type-checker ergonomics for the public API #161

Description

@Sam123ben

Situation

A library whose autocompletion does not work is a library people copy examples from and guess at. Protocol-heavy interfaces and decorator-based tools are exactly the shapes that lose type information, so a @tool-decorated function or an agent.run() result can arrive as Any in an editor — and the typed-boundary rule stops being enforced at the one moment enforcing it is free.


Task

User or System Story

As a engineer writing agent code in an editor
I want the public API to carry full type information end to end, verified by a conformance suite
So that autocompletion, inline signatures and static errors work, and structured-output guarantees are enforced before the code ever runs

Scope

In scope

  • py.typed shipped in the wheel and re-export hygiene via __all__, so imports resolve to stable documented paths
  • @tool typed with ParamSpec and TypeVar so the decorated callable keeps its original signature and its generated schema type stays inferable
  • Agent[InputT, OutputT] generic so run() is inferred as the declared output model rather than a union or Any
  • Literal and TypedDict for task classes, config keys and provider names, so a typo is a static error not a runtime routing failure
  • Overloads distinguishing streaming from non-streaming run, so each returns a precise type
  • A typing conformance suite of assert-type cases run in CI under both mypy --strict and pyright strict, across every supported Python version
  • Deprecation shims annotated so editors surface the replacement, consistent with the documented deprecation policy

Out of scope

  • Documentation content and tutorials, owned by the docs workstream
  • Runtime validation behaviour, owned by the core and runtime epics
  • Editor plugins or language-server extensions

Acceptance Criteria

Primary Scenario

Given a consumer project on mypy --strict defining a @tool that returns a Pydantic model and an Agent[TripRequest, TripPlan]
When the project is type-checked
Then agent.run(...) is inferred as TripPlan, the decorated tool retains its original parameter types, and passing a wrong argument type is reported at the call site rather than at runtime

Failure or Validation Scenario

Given a consumer passes an unknown task class or a mistyped config key
When the project is type-checked
Then type checking fails with a Literal mismatch listing the allowed values, instead of the mistake surfacing later inside provider routing

Edge Cases

  • from __future__ import annotations and postponed evaluation of annotations
  • Any leaking in through a third-party adapter without stubs, which must be contained at the adapter boundary
  • Generic behaviour differences between supported Python versions
  • Cases where pyright strict and mypy --strict legitimately disagree and one must be declared authoritative
  • Decorated tools with default, keyword-only or variadic parameters

Engineering Guardrails

Complete the applicable items before the story is closed.

Code Quality

  • The change is limited to the agreed scope.
  • Existing behaviour remains backward-compatible unless explicitly approved.
  • Code follows the team's agreed standards and architecture.
  • No temporary code, debug output or unnecessary complexity remains.
  • Error handling is explicit and does not silently ignore failures.

Test-Driven Development

  • Acceptance criteria were converted into test scenarios.
  • New or changed business logic was developed using Red–Green–Refactor where practical.
  • Bug fixes include a regression test that fails before the fix and passes afterwards.
  • Tests validate behaviour rather than private implementation details.

Automated Testing

  • Unit tests cover new and changed logic.
  • Positive, negative and important boundary scenarios are covered.
  • Integration or contract tests are updated where service boundaries change.
  • End-to-end coverage exists for any path that crosses a process, provider or service boundary.
  • Existing tests pass.
  • No new flaky or skipped tests are introduced without justification.

Security and Access

  • Authentication and authorisation are enforced at the appropriate layer.
  • Users and services can access only the data and actions they require.
  • Input is validated and sensitive information is not exposed in logs or errors.
  • Secrets and credentials are not stored in source code.
  • Security and dependency scans pass.
  • Tenant, account or customer isolation is tested where applicable.

Public API Obligations

  • Every public name is fully typed and passes mypy --strict.
  • Every public name has a docstring stating its behaviour and its failure modes.
  • A runnable example exists for the new surface and is executed in CI.
  • A test covering the public behaviour runs with no network access.
  • Any change to an existing public surface follows the documented deprecation policy.
  • The import-linter layering contract still passes.

Performance and Reliability

  • The change does not introduce unnecessary API calls, database queries or processing.
  • Timeouts, retries and failure behaviour are defined where applicable.
  • Duplicate requests do not create inconsistent data.
  • Performance-sensitive changes are tested against an agreed threshold.
  • Partial failures do not leave the system in an invalid state.

Observability

  • Important failures are logged with useful diagnostic context.
  • Logs do not contain sensitive information.
  • Metrics or traces are added where the change affects a critical workflow.
  • The team can determine whether the feature is working correctly after deployment.

Documentation

  • Relevant README, API or technical documentation is updated.
  • Configuration or environment changes are documented.
  • Deployment, migration or rollback steps are documented where required.
  • Known limitations are recorded.

Result

  • Expected user or system outcome: A kit consumer gets working autocompletion, accurate inline signatures and static errors for wrong tool arguments or config keys.
  • Expected business or operational outcome: The typed-boundary non-negotiable is enforced by the type checker in every consuming repository, catching a class of defects before any test runs.
  • Success metric or validation method: Typing conformance suite green under both strict type checkers on all supported Python versions; no Any in the public API surface per the typing coverage report
  • Monitoring period, where applicable: Reviewed at the M5 milestone against typing coverage of the public API and consumer type-check adoption

Pull Request Evidence

The pull request should include:

  • Link to this story.
  • Summary of the change.
  • Test evidence.
  • Screenshots or recordings for UI changes.
  • Security, accessibility or performance evidence where relevant.
  • Deployment and rollback notes for higher-risk changes.
  • Known risks or follow-up work.

Definition of Done

The story is complete when:

  • All acceptance criteria pass.
  • Required tests pass locally and in CI.
  • No critical or high-severity security or quality issues remain.
  • The critical user or system journey has been validated.
  • Documentation is updated.
  • Code review is complete.
  • The change has been deployed or is ready for deployment through the standard pipeline.
  • Post-deployment validation is defined or completed.
  • Any exception is documented with an owner and reason.

Exceptions

For any guardrail marked not applicable, record:

Guardrail:
Reason:
Alternative validation:
Residual risk:
Approved by:

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions