feat(stock-watchlist-agent-js): accept image inputs and annotate them on spans

[cdfox]

What

Adds an image input path to the JS stock watchlist demo, and attaches the input image to the LLM Observability span that consumes it.

Inputs can now be ticker symbols, images, or a mix of both:

# a) all three inputs are ticker symbols
npm start -- AAPL GOOGL NVDA

# b) first two inputs are images, third is a ticker symbol
npm start -- logos/apple.png logos/google.png NVDA

When images are given, a vision step runs first: one LLM call per image identifies the public company shown and returns its ticker symbol. Resolved tickers are merged with any tickers passed directly, deduped, and handed to the existing orchestrator. Images that match no public company come back as UNKNOWN and are skipped.

Changes

• src/agents/vision.js (new) — resolveTickersFromImages() fans out one identify_ticker span per image. That span is kind: 'llm' and is annotated with a user message carrying imageParts: [{ mimeType, content }], so the image renders on the span. Images are always materialized as base64 (http(s) URLs are downloaded first) so the same bytes are sent to OpenAI and attached to the trace.
• src/agents/orchestrator.js — image resolution happens inside the root analyze_portfolio agent span, so image translation and research share a single trace rather than producing two.
• src/main.js — argument parsing accepts images by extension / http(s):// / data:image/..., plus an explicit --image <path|url> override. Prints what each image resolved to. Evaluations run against the resolved ticker list.
• src/models.js — tickerFromImageSchema + validateTickerFromImage.
• logos/ — three small wordmark PNGs (Apple, Google, NVIDIA; ~900 bytes each, 12K total) so the image path is runnable without supplying your own files.
• Model defaults — all four call sites (agent loop, vision, web search, eval judges) now default to gpt-5.4-nano, matching the direction of Unpin openai, trim requirements, and update notebooks to gpt-5.4-nano #78. All remain overridable via OPENAI_MODEL, OPENAI_VISION_MODEL, OPENAI_SEARCH_MODEL, OPENAI_EVAL_MODEL.

Trace shape

analyze_portfolio (agent)                    ← evals attach here
├── resolve_tickers_from_images (workflow)   ← only when image inputs are given
│   └── identify_ticker (llm, one per image) ← image_parts annotated here
└── orchestrator (agent)
    └── delegate_research (tool) → stock_researcher ...

Testing

Tested against @joizddog's dd-trace-js branch for image support, dd-trace-js#9684 (jose/mlob-7916-llmobs-image-parts), installed as a local file dependency:

npm install ../../../dd-trace-js
DD_LLMOBS_ML_APP=stock-watchlist-agent-js-chris dd-auth -- npm start -- logos/apple.png

The Apple wordmark renders on the identify_ticker span in the LLM Observability trace view, alongside the user message and the assistant's structured {ticker: "AAPL", ...} output.

Two findings from that testing shaped this implementation:

1. Provider auto-instrumentation does not capture images. The dd-trace-js PR is manual-annotation only (auto-capture is a stated follow-up), so the auto-instrumented OpenAI.createResponse child span carries no image. The image has to be annotated by hand.
2. imageParts is only honored on llm-kind spans. In sdk.js, only spanKind === 'llm' routes to tagLLMIO → tagMessages, which is the sole reader of imageParts. On task / workflow / agent spans the input goes through tagTextIO and the whole message array is stringified into an opaque value — imageParts passes through as inert text with no warning. identify_ticker was originally a task span, which is why no image appeared at first.

Also verified end to end:

• npm start -- AAPL GOOGL NVDA — unchanged text-only path
• npm start -- logos/apple.png logos/google.png NVDA — mixed path; resolves AAPL + GOOGL, analyzes three tickers, evals submitted
• All three bundled logos resolve correctly at high confidence (AAPL, GOOGL, NVDA)
• Vision, research, and eval spans all share one trace_id

Note on dd-trace version

package.json still points at dd-trace: latest; the local-checkout dependency swap used for testing is deliberately not committed. Until image-part support ships in a release, the app runs normally on the published dd-trace — the imageParts field is simply ignored and no image appears on the span. Everything else in this PR is version-independent.

🤖 Generated with Claude Code