Skip to content
View stevenybuilder's full-sized avatar

Block or report stevenybuilder

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
stevenybuilder/README.md

LinkedIn Email GitHub


I build ML systems, and I'm pointed at physical AI and world models — self-supervised representation learning (JEPA-family predictive embeddings), computer vision, and robotics that closes the loop with real hardware (ESP32 firmware, live sensors, a physical actuator). Before that: probability-of-default models for lending, agent orchestration, and data products on Databricks and GCP.

LanguagesPython, C/C++ (embedded/ESP32 firmware), Kotlin, JavaScript, TypeScript, SQL, HTML/CSS
ML/DataPyTorch, TensorFlow, CatBoost, scikit-learn, pandas, NumPy, calibration, conformal intervals, Bayesian scoring, Markov chains, survival/censoring, causal caution
Physical AI / Representation learningJEPA-family predictive embeddings (self-supervised, world-model-style architectures), frozen-encoder inference, Vision Transformers, linear/MLP probing, 3D volumetric perception, confound/bias ablation
Robotics / EmbeddedESP32/ESP-IDF firmware (C/C++) for real-time arm control and sensor tracking, computer-vision perception closing the loop with a physical actuator
GPU/CPU EngineeringPyTorch inference and training on GPU (Colab) driven from a CPU-only dev machine, per-unit checkpointed resumability, pinned CUDA/PyTorch dependency stacks
AI SystemsLLM tool loops, MCP servers, RAG/retrieval, golden-dataset evals, deterministic gates, generated-code validation
Cloud/InfraGoogle Cloud Run, BigQuery, Gemini, Vertex AI, Databricks Asset Bundles, Delta Lake, MLflow, Docker, FastAPI, React

Research Background: Cognitive Neuroscience & Data Science

Before enterprise AI product work, I did cognitive neuroscience research at UC Irvine's Center for the Neurobiology of Learning and Memory / Stark Lab, using Python for fMRI image processing and analysis in controlled studies of spatial navigation, hippocampal pattern separation — how the hippocampus keeps similar experiences from collapsing into the same memory — and human memory. I proposed and secured funding for a study outside the lab's traditional hippocampus focus; the approach was later adopted by postdoctoral researchers and presented to 2,000+ symposium attendees.

Currently extending that grounding at UC San Diego with coursework in Deep Learning for Computer Vision, Linear Algebra for Machine Learning, and a Deep Neural Networks practicum — building the same representation-learning toolkit behind the physical-AI work above.


Things I've Built

NeuroAD Discovery Engine

Evaluation rigor for a frozen JEPA-style world-model encoder — proving what it actually learned before trusting it.

Repo Hackathon

Watch the NeuroAD demo

Watch the demo video

Built for the Claude Life Sciences hackathon with Gladstone Institutes. JEPA (Joint-Embedding Predictive Architecture) is the self-supervised family behind current world-model research — it learns by predicting representations, not pixels. NeuroAD stress-tests that promise: a foundation-model embedding separates disease from control, but it separates scanners and sites just as well. I ran frozen inference on Neuro-JEPA (a third-party NYU model extending V-JEPA 2 to 3D volumetric imaging) and built the extraction, confound-ablation, probing, and evaluation layers that tell you which signal the model is actually reading before you trust it downstream.

Representation learningFrozen ViT-MoE forward pass over preprocessed 3D volumetric scans, token mean-pool to 768-d, across 1,787 subjects spanning four public research cohorts
ProbingOne reused logistic head pointed at either the outcome or the confound, so a result and its own falsifier share an estimator; site-disjoint StratifiedGroupKFold, bootstrap CIs, label-permutation nulls, PCA fit inside each fold
Headline findingOn 96 healthy brains with no disease signal, the frozen embedding predicts scanner field strength at AUC 0.931 at PCA-10 — acquisition physics is linearly decodable from a self-supervised representation and survives dimensionality reduction
HarmonizationEmpirical-Bayes site-effect correction, deliberately label-blind: protected covariates in, outcome excluded so the correction never sees the label it will later be evaluated against. Cohort separability drops 0.9996 → 0.5634 while the disease-vs-control signal survives at 0.8313 cross-cohort
Why it matters for world modelsAny frozen predictive encoder — JEPA-family or otherwise — can look like it learned the physics of a domain while it's actually keyed on an acquisition artifact. This is the confound-ablation methodology for telling those apart before you build on top of the representation
Agentic layerClaude Sonnet router with enum-constrained routing plus an Opus agent that reads the evidence and drafts the next experiment; Claude never produces a number — every figure comes from the probe
StackPython, PyTorch, deepbet, scikit-learn, NumPy, Anthropic API, FastAPI, Docker, Google Cloud Run, Colab GPU runtimes (CPU-only dev machine)

TrueDose AI

Robotic medication-safety cabinet — physical AI end to end: embedded control, computer vision, and a real actuator.

Hackathon

A robotic cabinet that identifies medication, detects dispensing errors, and physically delivers a dose — perception and control closing the loop on real hardware, not a simulation. I owned the embedded firmware and the vision stack end to end.

Embedded controlESP32/ESP-IDF firmware in C/C++ driving robotic-arm motion, camera capture, and sensor tracking for physical dispensing
PerceptionPython computer-vision modules for medication identification, dispensing-error detection, and prescription reconciliation against a scanned label
Systems integrationOMI-captured prescriptions synced through Convex; real-time family alerts and a monitoring dashboard on the same event stream
StackC/C++ (ESP32/ESP-IDF), Python, computer vision, Convex, real-time dashboards
ResultWon Best Use of Convex, AI Valley Robotics Hackathon

Intuit Data Science Competition: Model Performance

Explainable ML system for small-business lending decisions.

Team Repo Hackathon

Team build for the Intuit TechWeek NYC 2026 Explainable ML hackathon. I owned modeling and calibration work for underwriting decisions: estimating probability of default, producing 90% intervals, and iterating on profit-aware approval logic under selection bias, leakage risk, censoring, and missing-not-at-random bank-feed data.

Decision problemApprove or decline SMB loan applicants to maximize realized portfolio profit under APR, origination-fee, LGD, and default-definition constraints
ML depthCalibrated PD modeling, uncertainty intervals, profit break-even thresholds, reject-inference awareness, leakage controls, MNAR missingness indicators
Challenge scopeLoan decisions, default trajectory forecasting, causal counterfactual PDs, monotonicity checks, and explainable methodology defense
StackPython, pandas, NumPy, scikit-learn, SciPy, statsmodels, HGB/value ensembles, calibration pipelines, CSV validators
ValidationSubmission schema/ID/range/monotonicity gate with PASS status before upload

CareGap

Healthcare access intelligence for finding and explaining medical deserts.

Repo Cloud Run Devpost

CareGap ranks likely medical deserts, estimates confidence, surfaces provider evidence, and recommends concrete deployment actions for healthcare planners. It combines lakehouse-style data workflows with statistical uncertainty layers so users can inspect not just where gaps exist, but why the system believes they exist.

Data pipelineBronze/silver/gold flow for facility records, geography, provider claims, district health indicators, and review queues; bundled as Databricks serverless jobs
ModelingCatBoost capacity and doctor-count imputers, native categorical handling, log1p targets, 5-fold OOF validation, clipped predictions, MLflow logging
UncertaintyBayesian validity posterior, empirical-Bayes trust smoothing, Wilson intervals, split-conformal facility trust sets
StackPython, Databricks Apps, Databricks SQL connector, Delta Lake, Unity Catalog, MLflow, Streamlit, CatBoost, PyDeck/H3, Google Cloud Run, Docker

Guardia

Deployment-risk copilot inside JetBrains IDEs.

Repo

Guardia brings production-risk analysis into the developer workflow before code ships. It reads touched services, Git context, incident patterns, and observability signals, then explains likely deployment risks with cited evidence and targeted remediation paths inside IntelliJ.

IDE engineeringNative IntelliJ plugin with PSI, Git4Idea diff extraction, PasswordSafe credentials, tool windows, editor highlights, and undoable WriteCommandAction patches
Risk engineSub-5ms deterministic Kotlin baseline plus bounded OpenAI/Codex Responses API override grounded in incident citations
RetrievalDatadog incident context, BM25/structural matching, reciprocal-rank fusion, and offending-code snippet scans
StackKotlin, IntelliJ Platform SDK, Gradle, JDK 21, OkHttp, Moshi, Python fixture generation, Datadog APIs, Supabase, OpenAI
Verification28 Kotlin test files plus remote-robot UI smoke-test source set

Sentinel

Runtime security layer for autonomous AI agents.

Repo Live Demo

Sentinel treats agent behavior as a security surface. Independent investigators verify payment-agent claims against ground truth, a deterministic safety gate blocks risky actions, and confirmed attacks generate new Python scoring rules that are AST-validated, regression-tested, and hot-deployed.

Core loopSupervisor agent, payment agent, risk/compliance/forensics investigators, verdict board, deterministic policy gate
ConcurrencyPython 3.11 asyncio.TaskGroup dispatch for parallel sub-agent investigations with structured cancellation
Learning systemGenerated Python scoring functions validated by AST parse, attack/clean regression checks, forbidden-token scan, and RestrictedPython execution
StackPython, FastAPI, Pydantic, Anthropic async client, React, Vite, JavaScript, XYFlow, Zustand, Aerospike, RestrictedPython, Auth0, Docker
Verification22 pytest files plus GitHub Actions backend deployment workflow to EC2

AI Meeting Autopilot

Multimodal meeting agent with real-time audio, vision, memory, and action dispatch.

Repo Cloud Run

This system transcribes live browser audio, extracts commitments, checks sentiment, and dispatches actions across Slack, Calendar, Gmail, documents, and BigQuery reports. Sessions run through WebSockets and async background execution so follow-up work can happen while the meeting continues.

Realtime pipelineBrowser PCM at 16kHz, WebSocket transport, Cloud STT streaming, transcript buffering, Gemini extraction, deterministic sentiment gate
ReliabilityPer-session dataclass registry, background task retention, proactive STT reconnect before streaming limits, async action fanout
Actions/dataSlack updates, Calendar events, Gmail summaries, document revision, BigQuery NL-to-SQL reports, DigitalOcean meeting memory
StackPython, FastAPI, JavaScript, HTML/Tailwind, WebSockets, Gemini, Google Cloud STT/Vision, BigQuery, Slack SDK, DigitalOcean inference/KB, Cloud Run, Terraform
Verification8 regression/smoke tests with GitHub Actions install, smoke, and test workflow

Together

Live translation and memory generation for cross-language presence.

Repo

Together is a real-time conversation system for people separated by distance and language. It translates speech bidirectionally, narrates the other person's environment on request, then turns call moments into a generated storybook and memory video.

Realtime modeFastAPI/WebSocket rooms, per-participant streaming sessions, Gemini Live bidirectional speech translation, translated captions, session affinity on Cloud Run
Vision/memoryVision scene analysis with semaphores, markdown transcript logging, storybook generation through Gemini interleaved output, memory video pipeline through Veo
ReliabilityDeepgram STT sessions, ElevenLabs/Gradium TTS fallback path, billing monitor controls for demo safety
StackPython, FastAPI, WebSockets, Gemini Live, Google Cloud Vision, Deepgram, Veo, ElevenLabs, HTML/JavaScript, Docker, Cloud Run

Tech Stack

Languages

Python Kotlin JavaScript TypeScript SQL

Data / ML

Databricks MLflow BigQuery pandas NumPy scikit--learn PyTorch

Cloud / Backend / Infra

Google Cloud Cloud Run FastAPI React Docker Supabase

AI Systems

OpenAI Gemini Anthropic Datadog


Recent Wins

Intuit AI/ML Hackathon2nd Place, June 2026 - calibrated underwriting PDs and profit-aware decision logic
Google DeepMind HackathonBest Use of DigitalOcean, March 2026 - multimodal meeting memory and agent workflow
Experian Global AI Hackathon1st of 91 teams, November 2025
AI Valley Robotics HackathonBest Use of Convex, November 2025

Operating Range

  • Pointed at physical AI and world models: self-supervised/predictive representation learning, computer vision, and robotics perception that closes the loop with real hardware
  • Build production-shaped AI systems across backend, frontend, data, ML, and cloud infrastructure
  • Design decision systems with calibration, uncertainty, evals, explainability, and data-quality guardrails
  • Ship agentic workflows with MCP servers, tool use, RAG/retrieval, deterministic gates, and human review
  • Translate ambiguous business problems into measurable systems while staying close to the code

Popular repositories Loading

  1. together-presence-agent together-presence-agent Public

    Together Presence Agent: live translation and memory-making companion for shared moments, built with Gemini Live, Veo, FastAPI, and Cloud Run.

    Python

  2. sentinel-agent-security sentinel-agent-security Public

    Sentinel: runtime security layer for autonomous AI agents using independent investigators, deterministic safety gates, generated rules, and episodic memory.

    Python

  3. ai-meeting-autopilot ai-meeting-autopilot Public

    AI Meeting Autopilot: multimodal meeting agent that captures audio/video context and turns conversations into Slack, Calendar, Gmail, and BigQuery follow-through.

    Python

  4. guardia guardia Public

    Guardia: JetBrains copilot that predicts deployment risk from Datadog incident history, Kotlin heuristics, and OpenAI/Codex remediation loops.

    Kotlin

  5. caregap-medical-desert-map caregap-medical-desert-map Public

    CareGap: maps medical-desert risk and provider trust using Databricks, CatBoost imputation, and conformal uncertainty.

    Python

  6. stevenybuilder stevenybuilder Public

    Profile README