Skip to content

[Suggestion] Electron desktop voice agent with function calling and system audio capture (TypeScript) #318

Description

@deepgram-robot

What to build

An Electron desktop application implementing a voice agent with Deepgram's Voice Agent API, featuring both microphone and system audio capture with function calling for desktop automation tasks.

Why this matters

Desktop voice agents are an emerging category — developers building AI assistants, accessibility tools, and productivity apps need a reference for integrating Deepgram into Electron. Unlike browser-based voice agents, Electron apps can capture system audio (for meeting transcription), access the filesystem (for saving transcripts), and run local tools (for desktop automation). An Electron live transcription example exists, but a full voice agent with function calling demonstrates a significantly deeper integration pattern that developers building desktop AI assistants need.

Suggested scope

  • Framework: Electron with electron-forge or electron-builder
  • Language: TypeScript
  • Deepgram APIs: Voice Agent API (STT + LLM + TTS), function calling
  • Architecture: Main process manages Deepgram WebSocket connection and audio routing; renderer process shows conversation UI with waveform visualization; IPC bridge for function calling results
  • Features: Microphone input, system audio capture (optional — for transcribing other apps), voice agent conversation with function calling (e.g., open URL, read clipboard, search files), transcript history with local persistence
  • Complexity: Medium-high — Electron main/renderer architecture + Deepgram Voice Agent API + audio routing

Acceptance criteria

  • Runnable with minimal setup (clone, add API key, run)
  • README explains the pattern clearly
  • Uses current SDK version
  • Demonstrates at least two function calling tools (e.g., open URL, clipboard read)
  • Works on macOS and Windows (Linux optional)
  • Includes audio input device selection UI
  • Conversation history persists across sessions

Raised by the DX intelligence system.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions