sureshot-lite

A single-file, offline evidence extractor for video and audio. It turns a media file into a timestamped timeline of observable facts (speech, visible text, scene cuts, visual motion, silences, audio energy events) and then gets out of the way. The LLM reading the output is the one that interprets it.

sureshot-lite never tells you a clip is "an interview" or "a product demo". It tells you there is speech from 00:00:21 to 00:00:51, a scene cut at 00:00:07, and 1.4 s of silence over a static frame at 00:01:02. Interpretation is your job, or your model's.

It was built for a hostile runtime: the ChatGPT code sandbox (Linux x86-64, CPython 3.13, CPU-only, ~4 GiB RAM without swap, no general internet, roughly 30-40 s per interpreter call). That is why it is a single file with no package to install, why every unit of work checkpoints to disk, and why analyze accepts a wall-clock budget and resumes where it stopped. It also runs fine on a normal machine with Python and ffmpeg.

Requirements

• Python 3 (tested on 3.12 and 3.13), standard library only.
• ffmpeg and ffprobe on PATH (or next to the script). Required.
• Optional STT: faster-whisper plus a local CTranslate2 model directory passed with --model-dir.
• Optional OCR: the tesseract binary with the language packs you want.

Check what is available:

python sureshot_lite.py doctor

doctor exits non-zero only when ffmpeg/ffprobe is missing. STT, the model, and OCR are reported as unavailable and degrade gracefully.

Quick start

# Factual pass: ffprobe metadata, scene cuts, black/freeze, motion, audio events.
python sureshot_lite.py analyze video.mp4 --json

# Where is the activity?
python sureshot_lite.py view video.mp4 --map

# Read one layer of flat evidence.
python sureshot_lite.py view video.mp4 --layer audio

# Open a range: frames plus the events inside it.
python sureshot_lite.py inspect video.mp4 00:00:21-00:00:51 --frames 6

# Export everything stored as a versioned YAML artifact.
python sureshot_lite.py report video.mp4

Add --stt --model-dir /path/to/faster-whisper-small for transcription and --ocr for visible text. Both are opt-in.

Audio-only input works: pass a .wav/.mp3 and the video-dependent detectors are skipped instead of failing.

Commands

Command	What it does
analyze	Extract evidence into the timeline. Resumable, budget-aware.
view	Read the stored timeline: --map, --layer, --words, --phrases.
inspect	Open a time range: extracted frames plus the events inside it.
report	Export stored evidence as sureshot/video-reference/v1 YAML.
files	List stored files and persisted artifacts.
doctor	Check required and optional dependencies.
spec	Describe the CLI (for a calling agent to read).

view, inspect, report, and files never re-analyze. They read what analyze already stored.

State

State lives under .sureshot/ next to the script, or under /mnt/data/.sureshot/ when /mnt/data exists (the sandbox case). Override it with --state-dir DIR or the SURESHOT_STATE_DIR environment variable.

.sureshot/sureshot.sqlite             # media + timeline_items
.sureshot/temp/<media_id>/            # transient frames and wav chunks
.sureshot/artifacts/<media_id>/       # frames kept with --save-artifacts
.sureshot/jobs/<media_id>/job.json    # progress, the source of truth for --resume

Resumable runs

Long jobs are split into small units (STT chunks, OCR frames). Each unit commits atomically, so an interrupted run loses at most one unit.

python sureshot_lite.py analyze video.mp4 --stt --model-dir ./faster-whisper-small \
  --max-seconds 30 --resume --json

Exit codes:

0   work complete
10  progress saved, units remain (call again with --resume)
20  recoverable error
30  permanent error (missing model or binary)

The intended loop is: call with --max-seconds and --resume until you get exit code 0. Without --max-seconds the job runs to completion with no time limit. Without --resume it starts over.

Evidence types

speech                  text                    frame
boundary.scene_cut      interval.black          interval.freeze
visual.motion_high      visual.static
audio.silence           audio.activity_high     audio.activity_low
audio.energy_change     audio.onset_cluster
av.silence_over_motion  av.silence_over_static

When an item carries a value, it is the observed relative magnitude of the normalized signal (0..1) inside that same file. It is not a confidence, a probability, an importance score, or a label, and it is not comparable across files. There is no tempo, BPM, or semantic audio tagging here.

Using it from an LLM

SKILL.md is the operating guide written for the calling model: command order, how to pick a speech detail level without drowning in words, and the failure modes to avoid. If you are wiring this into an agent, give it that file.

ChatGPT sandbox

The sandbox reaches an internal PyPI mirror but not Hugging Face, so faster-whisper installs normally while model files must be uploaded by hand.

cd /mnt/data
unzip -q model.zip -d faster-whisper-small
MODEL_DIR=/mnt/data/faster-whisper-small bash bootstrap.sh

bootstrap.sh installs from the mirror, falls back to an attached offline wheelhouse if that fails, verifies the runtime imports, and validates the four CT2 model files. It exits 30 rather than letting a transcription start on a broken install. Package state does not survive a sandbox restart, so rerun it each session.

To rebuild the uploadable attachments on a machine with internet:

powershell -File scripts\stage_attachments.ps1 -Zip

See attachments/MANIFEST.md for the upload checklist, requirements.lock for the pinned offline dependency closure, and docs/limitations.md for the measured constraints of the target environment.

Tests

python test/smoke_test.py            # compare against the committed golden
python test/smoke_test.py --update   # regenerate it

A golden regression net over the deterministic ffmpeg pipeline, plus audio-only analyze and cover-art probe gating. It does not touch OCR, STT, or any model.

The fixture is a 16 s clip synthesized from ffmpeg lavfi sources (test/media/make_synthetic.sh), so no third-party media ships with this repo. It is shaped to trigger every deterministic detector at least once: two scene cuts, motion and static stretches, black and freeze intervals, silence, audio activity, and one audio/video overlap. Detector output can legitimately shift across ffmpeg versions; if the golden drifts after an upgrade, review the diff before regenerating it.

Relation to sureshot

sureshot-lite is the offline extraction core of a larger private tool. The semantic layer of that tool (ask, agent, digest, and its VLM/LLM workers) is deliberately absent here, along with every network path. What remains is the factual pipeline: ffprobe metadata, scene and black/freeze detection, portable visual motion, audio silence and energy events, the timeline_items model, and the read commands.

License

MIT. See LICENSE.