ReCall turns hours long CCTV or camera footage into searchable events.
Upload a video, or point ReCall at a local folder, and search for moments using natural language:
"person in a white shirt near the counter"
"someone entered the restricted area"
"what happened before the machine stopped?"
"show me every vehicle after 10 PM"
ReCall analyzes the footage, detects and tracks objects, builds a time-aware understanding of what is happening, and lets you jump directly to the relevant moment in the original video. Best part? it does everything locally.
 |
 |
 |
 |
final_compressed.mp4
ReCall watches your footage for you and makes it searchable.
Instead of:
Open 8-hour recording
↓
Scrub through the timeline
↓
Guess where the event happened
↓
Watch the surrounding footage
you can ask:
"Show me when someone entered the restricted area."
"Find every time a person fell near the entrance."
"Show me every vehicle that entered after 10 PM."
"What happened right before the warning light?"
ReCall returns relevant moments from the footage with:
- the timestamp
- a description of what happened
- the matching video
- relevant frames
- detected objects and tracks
- bounding boxes where available
You can click a result and jump directly to that point in the original video.
Raw video
↓
Understand what is in each frame
↓
Track objects across time
↓
Understand what happened
↓
Store the resulting information
↓
Search it later
↓
Jump back to the original footage
ReCall is designed to run locally by default, so sensitive footage does not need to leave your machine. A cloud vision model can optionally be used when desired.
Cameras are good at recording everything.
Finding one specific thing in that recording is a different problem.
A camera may record eight hours of footage containing thousands of frames. If something happened at 3:17 PM, finding it manually means searching through that recording yourself.
Object detection helps, but individual detections are not the same as understanding an event.
For example, a detector can tell you:
14:31:04 Person
14:31:08 Person
14:31:12 Person
14:31:16 Person
That is useful, but it does not directly tell you:
14:31:04
A person approaches the machine.
14:31:18
The person opens the machine panel.
14:31:32
A warning light appears.
14:31:41
The machine stops.
The useful information is not just what appeared in a frame.
It is what happened over time.
That is what ReCall is built around.
There are already good systems for analyzing and searching video. ReCall is not trying to replace all of them.
The difference is in what ReCall treats as the fundamental unit of information.
Traditional video analytics often starts with labels:
Person
Car
Truck
Face
License plate
ReCall is being built around the idea that the useful unit is an event over time:
Person
↓
Track
↓
Movement
↓
Context
↓
Event
↓
Description
↓
Searchable evidence
| Approach | Typical strength | ReCall's focus |
|---|---|---|
| Traditional NVR / VMS | Recording, playback, timelines, motion search | Search by what happened, not just when it happened |
| Cloud video APIs | Large set of managed detection and media-analysis capabilities | Local-first processing and an application-specific event representation |
| Video understanding APIs | Natural-language search and high-level video understanding | Open, inspectable pipeline from detection to track to event to evidence |
| Enterprise video analytics | Multi-camera investigation, appearance search, alerts and security workflows | A developer-oriented system that can be run and modified on your own infrastructure |
| Local NVR / CV systems | Local detection, tracking, recording and increasingly semantic search | Move from tracked objects toward temporal descriptions and searchable events |
| DIY CV pipelines | Maximum flexibility | One pipeline that connects video processing, tracking, event generation, embeddings, search and evidence |
Local
The core pipeline can run on your own hardware. Sensitive footage does not have to be sent to a cloud AI provider.
Temporal
ReCall does not treat every frame as an independent observation. Objects are tracked across time and used to build events and descriptions.
Searchable
The output of processing is not just a collection of detection labels. Video segments receive semantic descriptions and embeddings that can be searched using natural language.
Inspectable
Search results remain connected to their source data:
Query
↓
Segment
↓
Description
↓
Events
↓
Tracks
↓
Frames
↓
Original video
Composable
The pipeline is built from replaceable components rather than being tied to a single proprietary video analytics stack.
Today that means:
YOLO
+
ByteTrack / IoU
+
SmolVLM2 or Gemini
+
BGE
+
PostgreSQL / pgvector
The individual components can evolve independently as better models and processing techniques become available.
ReCall is best thought of as the layer between computer vision and video search.
Video
│
▼
Computer Vision
detection + tracking
│
▼
ReCall
events + descriptions
+ semantic search
│
▼
Evidence
frames + video
The goal is not simply to detect more objects.
The goal is to make recorded video queryable.
The current pipeline is intentionally straightforward.
Video
│
▼
Video Ingestion
│
▼
Segment + Sample
│
│
frames every 2s
│
▼
Object Detection
│
│
YOLO
│
▼
Tracking
│
ByteTrack / IoU
│
▼
Event Generation
│
▼
Temporal Description
│
SmolVLM2 / Gemini
│
▼
Embedding
│
BGE
│
▼
PostgreSQL
+ pgvector
│
▼
Search
│
▼
Relevant Segments
│
▼
Evidence
│
▼
Video + Frames
The current implementation processes videos sequentially through this pipeline.
There is no Kafka, Redis, or distributed processing queue in the current system.
Processing is driven by a Go polling worker that claims uploaded videos from PostgreSQL and records progress using processing checkpoints.
This keeps the current system simple while the core pipeline is being developed.
A 30 FPS camera produces:
30 × 60 × 60 = 108,000 frames per hour
Running every expensive model on all 108,000 frames would be wasteful.
ReCall currently samples footage at a lower rate for the initial processing stage:
30 FPS video
↓
1 frame every 2 seconds
↓
30 frames / minute
This makes the pipeline substantially cheaper.
However, sampling introduces a fundamental tradeoff.
An event can happen between two sampled frames:
Frame A
│
│
│ Person falls
│
│
Frame B
If the event is never captured, it cannot be recovered later.
This is why ReCall is designed around high recall.
The current implementation uses fixed sampling. An adaptive multi-stage pipeline that increases processing around potentially interesting activity is planned for a future stage.
Once the video has been processed, the descriptions of its segments are embedded using:
BAAI/bge-small-en-v1.5
and stored in PostgreSQL using pgvector.
A search works roughly like this:
User query
↓
Generate query embedding
↓
pgvector similarity search
↓
Find relevant segments
↓
Enrich with detections, tracks and events
↓
Return matching moments
↓
Jump to timestamp in video
For example:
Query:
"person in white shirt near the counter"
can retrieve a segment whose description contains:
A person wearing a white shirt approaches
and stands near the counter.
The result still points back to the actual footage.
This is important because ReCall is not intended to produce an answer and leave the user to trust it.
The result should be verifiable against the video.
Every search result is connected to the underlying video data.
Search result
│
▼
Segment
│
├── Description
├── Timestamp
├── Detections
├── Tracks
└── Events
│
▼
Frames
│
▼
Original video
From the result, the user can:
- jump to the relevant timestamp
- play the surrounding video
- inspect matching frames
- inspect detected objects
- inspect tracks and events
- view bounding boxes
The goal is simple:
If ReCall tells you something happened, you should be able to see the footage that supports it.
ReCall currently consists of a Go application, a web application, Python-based ML workers, and PostgreSQL.
┌───────────────┐
│ React Web │
│ Vite + TS │
└───────┬───────┘
│
▼
┌───────────────┐
│ Go API │
│ Fiber │
└───────┬───────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
Processing Search Video
Worker API Storage
│ │
│ ▼
│ PostgreSQL
│ + pgvector
│
▼
Python ML Workers
│
┌─────────┼─────────┐
▼ ▼ ▼
YOLO SmolVLM2 BGE
The current processing architecture is deliberately simple.
A Go worker polls PostgreSQL for uploaded videos, claims work using PostgreSQL row locking, and runs the processing pipeline.
Progress is stored in:
video_processing_checkpoints
This allows processing to resume from completed segments rather than starting an entire video from scratch.
The ML components are currently invoked as Python processes from the Go processing pipeline rather than operating as independently scaled services.
| Layer | Technology |
|---|---|
| API | Go + Fiber |
| Web | React + Vite + TypeScript |
| Database | PostgreSQL 16 |
| Vector search | pgvector |
| Video processing | FFmpeg + ffprobe |
| Object detection | Ultralytics YOLO |
| Tracking | ByteTrack / IoU |
| Vision-language model | SmolVLM2-500M-Video-Instruct |
| Optional cloud vision | Gemini |
| Embeddings | BAAI/bge-small-en-v1.5 |
| ML workers | Python |
| Video storage | Local filesystem |
| Deployment | Docker + Docker Compose |
Detection
YOLOv8n / YOLO11n
Tracking
ByteTrack / IoU
Temporal description
SmolVLM2-500M-Video-Instruct
or Gemini
Embeddings
BAAI/bge-small-en-v1.5
Vector storage
PostgreSQL + pgvector
The local pipeline does not require a cloud vision API.
Gemini is an optional provider for users who want to use a cloud VLM instead of the local vision model.
recall/
├── migrations/
│ └── PostgreSQL + pgvector migrations
│
├── pkg/
│ └── database/
│ └── PostgreSQL connection and queries
│
├── services/
│ ├── api/
│ │ ├── API
│ │ ├── processing
│ │ ├── search
│ │ ├── tracker
│ │ └── vision
│ │
│ └── web/
│ └── React + Vite application
│
├── workers/
│ ├── detector/
│ │ └── YOLO inference
│ │
│ ├── vision/
│ │ └── temporal descriptions
│ │
│ └── embedding/
│ └── BGE embeddings
│
├── storage/
│ └── local video and frame storage
│
├── docker-compose.yml
├── Makefile
├── go.mod
└── go.sum
- Go
- Docker
- Docker Compose
- Node.js
- Python
- FFmpeg
Clone the repository:
git clone https://github.com/berzz26/recall.git
cd recallConfigure the environment:
cp .env.example .envFor fully local vision processing:
VISION_PROVIDER=local
For Gemini:
VISION_PROVIDER=gemini
GEMINI_API_KEY=...
Start the backend:
docker compose up -dThe API runs on:
http://localhost:8081
Start the web application:
cd services/web
npm install
npm run devThe web application runs on:
http://localhost:5173
Videos can be added in two ways:
Upload
Videos → + Add Video
Local folder
Settings → Add Local Source
Provide an absolute path to the directory containing your videos.
ReCall will ingest the video and begin processing it through the pipeline.
ReCall is under active development.
The current end-to-end pipeline is working for local video files:
Video
↓
Detection
↓
Tracking
↓
Events
↓
Temporal description
↓
Embeddings
↓
pgvector
↓
Semantic search
↓
Video evidence
The current system uses fixed frame sampling and sequential processing.
- Local video ingestion
- Video segmentation
- Frame sampling
- YOLO object detection
- Batched detection
- ByteTrack tracking
- IoU tracking
- Track persistence
- Event generation
- Temporal video descriptions
- Local SmolVLM2 inference
- Optional Gemini vision
- BGE embeddings
- PostgreSQL
- pgvector semantic search
- Search result enrichment
- Timestamp-based video playback
- Frame and bounding-box inspection
- Processing checkpoints
The following are planned rather than current capabilities:
- Adaptive frame sampling
- Automatic cheap-to-expensive processing
- Audio and speech understanding
- OCR
- Cross-camera re-identification
- Distributed processing queues
- Complex temporal queries such as
before,after, andstayed > 30s - Large-scale distributed video processing
The immediate focus is reliability and better temporal understanding.
- Improve processing checkpoints
- Improve track persistence
- Improve temporal context
- Reduce unnecessary model inference
- Adaptive frame sampling
- Better semantic retrieval
- Temporal queries
- Event-aware ranking
- Combined object and event queries
- Search across multiple videos
- Better temporal event detection
- Audio and speech
- OCR
- More object and action types
- Cross-camera identity tracking
- Distributed processing
- Queue-based workers
- GPU worker pools
- Large video collections
- Multi-camera deployments
There is an enormous amount of information trapped inside recorded video. The cameras already captured it.
The problem is finding it.
ReCall is an attempt to build a system where video can be treated less like a recording that you manually inspect and more like a dataset that you can query.
The project combines:
- Computer vision
- Object detection
- Object tracking
- Vision-language models
- Temporal reasoning
- Vector search
- Video processing
- Backend systems
- Distributed systems
The long-term goal is simple:
Ask what happened. Find the moment. See the evidence.
ReCall is actively being developed.
Issues, discussions, experiments, and pull requests are welcome.