Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
2faa08d
Attempting to run vllm locally instide of a vwb app.
May 14, 2026
a7b6589
Updated to multi stage docker
May 14, 2026
da032fc
updated
May 14, 2026
804f035
updated
May 15, 2026
fd45c7f
update
May 15, 2026
e744e4e
Added Readme
May 15, 2026
6e55adb
using vscode base
May 15, 2026
7225cf4
This installs PyTorch built for CUDA 12.8 first, then vLLM will use t…
May 15, 2026
dd8bff1
Update CUDA driver missmatch. This uses --extra-index-url so both vLL…
May 15, 2026
fae2827
Switch from vLLM to Ollama to resolve CUDA driver compatibility issues
May 15, 2026
840d1e6
replaced curl with sh
May 15, 2026
20674b3
Fix Ollama install: use tar.zst download and correct gemma4:e4b mod…
May 15, 2026
1774245
removed the keep warm issue
May 15, 2026
bffdbff
added dark theme to vscode
May 15, 2026
4d7e892
Add ollama_gemma-4-26B-A4B app template for Gemma 4 26B MoE on A100
May 18, 2026
f1f8b7e
Remove main.py from ollama_gemma-4-26B-A4B template
May 18, 2026
4b3239c
Start Ollama on every VM boot, not just first creation
May 18, 2026
75549d4
Enable flash attention for more efficient inference
May 18, 2026
39e072f
Add tesseract-ocr to Dockerfile
May 18, 2026
6c481a5
Add uv Python package manager to Dockerfile
May 19, 2026
cea74d1
Add pep-ppp-prod app template with dark mode default
Jun 10, 2026
45c544e
Align pep-ppp-prod with pep-cli-acc config (root user, code-server on…
Jun 10, 2026
af63abf
Add Claude Code setup with Vertex config, GCP auth, and dark mode
Jun 10, 2026
5598e0e
Remove interactive gcloud auth from startup script to prevent hang
Jun 10, 2026
be34f47
Revert to original working config (abc user, port 8443, linuxserver f…
Jun 11, 2026
a2ec2aa
Restore pep-ppp-prod to last known working state (5598e0e)
Jun 11, 2026
1f8a04e
Fix pep-ppp-prod Java feature: switch JDK distro from ms to tem
Aug 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions src/ollama_gemma-4-26B-A4B/.devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
{
"name": "Ollama Gemma 4 26B-A4B",
"dockerComposeFile": "docker-compose.yaml",
"service": "app",
"shutdownAction": "none",
"workspaceFolder": "/workspace",
"postCreateCommand":
"./startupscript/post-startup.sh abc /config \"${templateOption:cloud}\" \"${templateOption:login}\"; ./sudo-passwordless.sh abc; ./start-ollama.sh",
"postStartCommand":
"./startupscript/remount-on-restart.sh abc /config \"${templateOption:cloud}\" \"${templateOption:login}\"; ./start-ollama.sh",
"features": {
"ghcr.io/devcontainers/features/java:1": {
"version": "17"
},
"ghcr.io/devcontainers/features/aws-cli:1": {},
"ghcr.io/dhoeric/features/google-cloud-cli:1": {},
"ghcr.io/anthropics/devcontainer-features/claude-code@sha256:cfc2e7d3e9fd3b9b01f8d5cb158508a884c8c0ede2e23ed10f32dea5d4ffe69a": {},
"./.devcontainer/features/workbench-tools": {
"cloud": "${templateOption:cloud}",
"username": "abc",
"userHomeDir": "/config"
}
},
"remoteUser": "root",
"customizations": {
"workbench": {
"opens": {
"extensions": [
".py",
".ipynb",
".sh",
".md",
".html",
".csv",
".json",
".jsonc",
".yaml",
".yml"
],
"fileUrlSuffix": "?payload=[[\"openFile\",\"vscode-remote:///config/{path}\"]]"
}
}
}
}
24 changes: 24 additions & 0 deletions src/ollama_gemma-4-26B-A4B/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
FROM lscr.io/linuxserver/code-server@sha256:7bd334657f13505abc1e20afeeee5670ad8f818e68853c810889184e597f3051

RUN apt-get update \
&& apt-get install -y --no-install-recommends jq python3 python3-pip python3-venv tesseract-ocr \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /config/extensions /config/.cache \
&& chown -R abc:abc /config

RUN apt-get update && apt-get install -y --no-install-recommends zstd \
&& curl -fsSL https://ollama.com/download/ollama-linux-amd64.tar.zst | tar --zstd -x -C /usr \
&& rm -rf /var/lib/apt/lists/* \
&& ollama --version

RUN pip install --break-system-packages openai uv

USER abc
ENV HOME=/config

RUN /app/code-server/bin/code-server --extensions-dir /config/extensions --install-extension ms-python.python \
&& mkdir -p /config/data/User \
&& echo '{"workbench.colorTheme":"Default Dark Modern"}' > /config/data/User/settings.json

USER root
WORKDIR /config
78 changes: 78 additions & 0 deletions src/ollama_gemma-4-26B-A4B/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Ollama + Gemma 4 26B-A4B

Code-server IDE with an Ollama inference server running Gemma 4 26B MoE (4B active parameters). Opens a browser-based VS Code editor on port 8443 with Ollama serving an OpenAI-compatible API on port 11434 inside the container.

## Recommended VM Configuration

| Model | GPU | Machine type | CPUs | RAM | Notes |
|---|---|---|---|---|---|
| **Gemma 4 26B MoE** | **A100 (40 GB)** | **n1-standard-8** | **8** | **30 GB** | **Default config for this app** |
| Gemma 4 26B MoE | A100 (80 GB) | n1-standard-8 | 8 | 30 GB | More VRAM headroom for longer contexts |

## Changing the Model

Update the `ollama pull` command in `start-ollama.sh` and the `model=` parameter in your Python scripts. See available models at https://ollama.com/library.

## Usage

Once the app is ready, open the code-server IDE and run:

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="unused")

response = client.chat.completions.create(
model="gemma4:26b",
messages=[{"role": "user", "content": "Hello!"}],
max_tokens=100,
)
print(response.choices[0].message.content)
```

### Using `uv`

1. Install uv

```sh
pip install uv
```

2. Initialize the project

```sh
uv init local-ollama-gemma4-26b
```

3. Add openai package

```sh
cd local-ollama-gemma4-26b/
uv add openai
```

4. Run the application

```sh
uv run main.py
```

## Debugging

Check Ollama server logs:

```bash
tail -f ~/ollama-server.log
```

Verify the server is running:

```bash
curl http://localhost:11434/v1/models
```

List downloaded models:

```bash
ollama list
```
30 changes: 30 additions & 0 deletions src/ollama_gemma-4-26B-A4B/devcontainer-template.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"id": "ollama_gemma-4-26B-A4B",
"version": "0.0.1",
"name": "Ollama Gemma 4 26B-A4B",
"description": "Code-server IDE with Ollama inference server (Gemma 4 26B MoE). Requires A100 GPU.",
"documentationURL": "https://github.com/verily-src/workbench-app-devcontainers/tree/master/src/ollama_gemma-4-26B-A4B",
"licenseURL": "https://github.com/verily-src/workbench-app-devcontainers/blob/master/LICENSE",
"options": {
"cloud": {
"type": "string",
"description": "VM cloud environment",
"proposals": [
"gcp"
],
"default": "gcp"
},
"login": {
"type": "string",
"description": "Whether to log in to workbench CLI",
"proposals": [
"true",
"false"
],
"default": "false"
}
},
"platforms": [
"Any"
]
}
32 changes: 32 additions & 0 deletions src/ollama_gemma-4-26B-A4B/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
services:
app:
container_name: "application-server"
build:
context: .
dockerfile: Dockerfile
restart: always
volumes:
- .:/workspace:cached
- work:/config:cached
ports:
- "8443:8443"
environment:
USER: "abc"
DEFAULT_WORKSPACE: "/config"
SUDO_PASSWORD: "pwd"
OLLAMA_KEEP_ALIVE: "-1"
OLLAMA_FLASH_ATTENTION: "true"
networks:
- app-network
cap_add:
- SYS_ADMIN
devices:
- /dev/fuse
security_opt:
- apparmor:unconfined

networks:
app-network:
external: true
volumes:
work:
18 changes: 18 additions & 0 deletions src/ollama_gemma-4-26B-A4B/start-ollama.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash

set -o errexit
set -o nounset
set -o pipefail

OLLAMA_LOG="/config/ollama-server.log"

nohup ollama serve > "${OLLAMA_LOG}" 2>&1 &
sleep 2

echo "Pulling gemma4:26b model (this may take a few minutes)..."
ollama pull gemma4:26b >> "${OLLAMA_LOG}" 2>&1

echo "Preloading model into GPU memory..."
curl -s http://localhost:11434/api/generate -d '{"model":"gemma4:26b","prompt":"warmup","options":{"num_predict":1}}' > /dev/null 2>&1

echo "Ollama ready — model loaded — logs at ${OLLAMA_LOG}"
34 changes: 34 additions & 0 deletions src/ollama_gemma-4-26B-A4B/sudo-passwordless.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/bin/bash

# This script is used to set up passwordless sudo for the core user on the VM.
# It requires to be run with root priviledges and USER_NAME to be set in the environment.
# It is typically called from post-startup.sh.

USER_NAME="${1}"

if [[ -z "${USER_NAME}" ]]; then
echo "Usage: $0 <username>"
exit 1
fi

sudoers_file="/etc/sudoers"
sudoers_d_file="/etc/sudoers.d/${USER_NAME}"

# Make sure user exists
if ! id "${USER_NAME}" &>/dev/null; then
echo "User ${USER_NAME} does not exist."
exit 1
fi

# Check if there's an old rule in the main sudoers file that requires a password
if grep -q "^${USER_NAME} ALL=(ALL:ALL) ALL" "${sudoers_file}"; then
echo "Found password-requiring rule for ${USER_NAME} in /etc/sudoers. Commenting it out."

# Comment out the old rule in /etc/sudoers
sed -i "s/^${USER_NAME} ALL=(ALL:ALL) ALL/# ${USER_NAME} ALL=(ALL:ALL) ALL/" "${sudoers_file}"
fi

echo "${USER_NAME} ALL=(ALL) NOPASSWD:ALL" > "${sudoers_d_file}"
chmod 440 "${sudoers_d_file}"

echo "User ${USER_NAME} has been given passwordless sudo access."
69 changes: 69 additions & 0 deletions src/pep-ppp-prod/.devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
{
"name": "pepcli-prod app",
"dockerComposeFile": "docker-compose.yaml",
"service": "app",
"shutdownAction": "none",
"workspaceFolder": "/workspace",
"postCreateCommand": "./startupscript/post-startup.sh root /config \"${templateOption:cloud}\" \"${templateOption:login}\"; ./sudo-passwordless.sh root; ./setup-claude-code.sh",
"postStartCommand": [
"./startupscript/remount-on-restart.sh",
"root",
"/config",
"${templateOption:cloud}",
"${templateOption:login}"
],
"features": {
"ghcr.io/devcontainers/features/java:1.6.3": {
"version": "17",
"jdkDistro": "tem"
},
"ghcr.io/devcontainers/features/aws-cli:1.1.1": {},
"ghcr.io/dhoeric/features/google-cloud-cli:1.0.1": {},
"ghcr.io/coder/devcontainer-features/code-server:1": {
"port": 8080,
"host": "0.0.0.0",
"auth": "none"
},
"ghcr.io/devcontainers/features/go:1": {
"version": "1.22"
},
"ghcr.io/devcontainers/features/python:1": {
"version": "3.12"
},
"ghcr.io/anthropics/devcontainer-features/claude-code:1": {}
},
"remoteUser": "root",
"customizations": {
"vscode": {
"settings": {
"workbench.colorTheme": "Default Dark Modern"
}
},
"workbench": {
"opens": {
"extensions": [
".c",
".cjs",
".cpp",
".go",
".java",
".js",
".mjs",
".php",
".scala",
".sh",
".ts",
".md",
".html",
".csv",
".json",
".jsonc",
".tsv",
".xml",
".yml"
],
"fileUrlSuffix": "?payload=[[\"openFile\",\"vscode-remote:///config/{path}\"]]"
}
}
}
}
17 changes: 17 additions & 0 deletions src/pep-ppp-prod/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@

# PEP CLI Prod (pep-ppp-prod)

A Template to run PEP CLI prod on workbench.

## Options

| Options Id | Description | Type | Default Value |
|-----|-----|-----|-----|
| cloud | VM cloud environment | string | gcp |
| login | Whether to log in to workbench CLI | string | false |



---

_Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/pep-ppp-prod/devcontainer-template.json). Add additional notes to a `NOTES.md`._
23 changes: 23 additions & 0 deletions src/pep-ppp-prod/devcontainer-template.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"id": "pepcli-prod",
"version": "0.0.1",
"name": "PEP CLI Prod",
"description": "A Template to run PEP CLI prod on workbench",
"documentationURL": "https://github.com/verily-src/workbench-app-devcontainers/tree/master/src/pepcli-prod",
"licenseURL": "https://github.com/verily-src/workbench-app-devcontainers/blob/master/LICENSE",
"options": {
"cloud": {
"type": "string",
"description": "VM cloud environment",
"proposals": ["gcp", "aws"],
"default": "gcp"
},
"login": {
"type": "string",
"description": "Whether to log in to workbench CLI",
"proposals": ["true", "false"],
"default": "true"
}
},
"platforms": ["Any"]
}
Loading
Loading