Skip to content

Add OpenCode + NVIDIA Nemotron custom app - #473

Open
samhornstein wants to merge 7 commits into
masterfrom
BENCH-9850-opencode-nemotron
Open

Add OpenCode + NVIDIA Nemotron custom app#473
samhornstein wants to merge 7 commits into
masterfrom
BENCH-9850-opencode-nemotron

Conversation

@samhornstein

@samhornstein samhornstein commented Aug 24, 2026

Copy link
Copy Markdown

Adds src/opencode-nemotron: code-server with the opencode agent driving a local NVIDIA Nemotron model on Ollama. Nothing leaves the VM. For the NVIDIA dev day demo. Derived from feature/applegath/ollama_gemma-4-26B-A4B.

This PR:

  • Serves code-server on 8443 and Ollama on 11434 (container-local). Pins opencode 1.18.22.
  • Defaults to nemotron-3.5-lightning:30b — 30B MoE, tool calling, 25 GB of weights. Needs an A100 40 GB.
  • Resolves the model tag in resolve-model.sh: /config/.opencode-model first, then OLLAMA_MODEL from docker-compose.yaml. Both start-ollama.sh and configure-opencode.sh call it, so the server and the agent config cannot drift.
  • Writes ~/.config/opencode/opencode.json on create and restart, with share: "disabled" so sessions never reach opencode's hosted sharing service.
  • Sets OLLAMA_CONTEXT_LENGTH=32768. Tool calls are unreliable below ~16k.

Two deviations worth review:

  • The model tag is not a template option. The VM substitutes only login, cloud, containerImage, containerPort, shmSize, and memoryLimit (startupscript/butane/050-parse-devcontainer.sh). A custom option stays literal and breaks docker-compose config. So the default lives in OLLAMA_MODEL in docker-compose.yaml. To change the model on an existing app, write the tag to /config/.opencode-model and restart. /config is a volume, so the override survives a restart and a machine-type change. An A100 to L4 move needs a smaller model, and this is how you make that change.
  • OpenJDK 17 comes from apt, not ghcr.io/devcontainers/features/java. That feature is currently broken for all 13 apps that use it, including vscode-with-llm and nemo_jupyter: Microsoft's Java 17 SDKMAN identifier is now 17.0.20+1-ms, and the feature's regex rejects the +1 segment. Upstream: java: major-only version fails to resolve once a vendor publishes a four-component release (21.0.12.1 → 21.0.12+1-ms) devcontainers/features#1712, fix in #1713. Pinning the digest does not help — the lookup is against SDKMAN's live catalog. The other apps need a separate ticket. This app stays unaffected by the upstream outcome either way; install-java.sh only needs /usr/bin/java.

Testing

Deployed to samh-sandbox on a2-highgpu-1g (1x A100 40 GB, us-central1-c):

  • ollama psnemotron-3.5-lightning:30b, 25 GB, 100% GPU, context 32768.
  • opencode run "Read hello.txt and tell me exactly how many lines it has" → agent called its read tool and answered correctly.
  • java 17.0.19 and wb on the PATH, so post-startup.sh succeeds. code-server answers on 8443.
  • Model override → resolve-model.sh returns nemotron-3.5-lightning:30b by default and nemotron-3-nano:4b with /config/.opencode-model present. configure-opencode.sh then writes ollama/nemotron-3-nano:4b. Deleting the file reverts to the default. The abc user can write /config directly, so the documented command needs no sudo.
  • Direct API example → with max_tokens=100 the reply stops at finish_reason=length and content is empty. With no cap, finish_reason=stop and content is 'Hello! How can I help you today?'. Nemotron always returns its thinking in Ollama's non-standard reasoning field, so the README prints both.

Not tested: ollama pull of a changed tag on restart. That path is a single line and reads the verified resolver.

Not registered in test-pr.yaml: CI runs devcontainer up, which would pull 25 GB on a GPU-less runner.

BENCH-9850

@samhornstein
samhornstein requested review from a team as code owners August 24, 2026 21:53
@samhornstein
samhornstein marked this pull request as draft August 24, 2026 22:05
@samhornstein
samhornstein requested a review from PeterSu92 August 24, 2026 22:18
@samhornstein
samhornstein marked this pull request as ready for review August 24, 2026 22:25
@pantherman594

Copy link
Copy Markdown
Contributor

If this is for a NVIDIA demo, would it be fine staying in/running from a branch? Does it need to be merged into master?

@samhornstein

Copy link
Copy Markdown
Author

If this is for a NVIDIA demo, would it be fine staying in/running from a branch? Does it need to be merged into master?

@pantherman594 Technically speaking, we can pull in a custom app from a feature branch, so we don't need to merge into master. The question then is if we want this in master or not. Thoughts?

@samhornstein

Copy link
Copy Markdown
Author

Decision: Wait on merge until after demo

Comment thread src/opencode-nemotron/README.md Outdated
Comment thread src/opencode-nemotron/README.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants