SoundVisualizer is a desktop application that captures real-time system audio and translates its direction, intensity, and meaning into a graphical overlay. Going beyond simple waveform rendering, it integrates an AI sound classifier powered by YAMNet and ONNX Runtime to instantly identify the type of sound playing (ambient, speech, danger) and provide adaptive visual feedback.
📱 There is also an Android version: SoundVisualizer for Android.
- Background & Objectives
- Key Features
- System & AI Architecture
- Installation & Quick Start
- Build Guide (For Developers)
- License
Modern media content, including YouTube, OTT, and gaming, relies heavily on spatial audio to maximize immersion. However, this creates a severe barrier for the deaf and hard of hearing. Traditional subtitle systems only deliver dialogue, completely omitting crucial acoustic information such as sound direction, footsteps, or urgent sound effects (e.g., gunshots). SoundVisualizer was created to break down these barriers by visualizing the intensity, classification, and spatial direction of invisible sounds. Our primary goal is to resolve this information asymmetry and guarantee an equal, fully accessible media consumption environment for everyone.
This overlay technology is also highly beneficial for gamers, providing a tactical visual indicator (situational awareness) in competitive environments. Additionally, it serves as a perfect alternative for users consuming media in public or silent environments where audio output is restricted.
We provide 4 unique rendering modes that intuitively map sound intensity and direction (supporting 2.0, 5.1, and 7.1 channels):
- Wave Mode: Renders dynamic audio waves along the screen edges that fluctuate based on intensity.
- Circle Mode: Radiates circular ripples outward from the center of the screen.
- Pad Mode: Displays glowing pads anchored to specific spatial grid directions.
- Outline Mode: A minimalist variation of Wave mode, showing only the thin borders of the wave to minimize screen occlusion.
- Animations move at the same speed at any frame rate, from 30 FPS up to your monitor's refresh rate.
Modify settings instantly while in a full-screen application or game without minimizing the window.
- F2 / F3 Hotkeys: Switch between sound modes and visualization modes on the fly. Hotkeys can be customized.
- Editor Mode (Default F4): Drag the guideline boundaries on your screen to physically resize the rendering limits in real-time. Adjust colors, opacity, and all other settings directly from the pop-up control panel.
- Automatically detects the system's audio configuration (Stereo, 5.1, 7.1 Surround) and accurately maps the sound's origin (Front/Back/Left/Right) to produce a 3D visual effect on a 2D screen.
- The sound mode switches to match your output device whenever the visualizer starts or the device changes, and you can still change it yourself.
- With stereo output, sound common to both channels is drawn at the top and only the difference between left and right goes to the sides, so a centred sound no longer looks like sound coming from both sides.
- 3-Class Detection: Analyzes all incoming audio into
Ambient,Speech, andDangercategories. Each category triggers independent, customizable UI colors and dynamic opacity changes. - Warning cues: Sirens, alarms, explosions, fireworks and similar warning sounds count as
Dangereven when they are only among the model's top five candidates (at 5% or more), so music or game audio can't drown them out. Gunshot labels are not promoted this way, to avoid false alarms. - Stable colors:
Dangerappears on the first qualifying result;SpeechandAmbientneed two in a row, so the color doesn't flicker. The AI pauses while it's quiet and resumes as soon as sound returns. - Every channel counts: On 5.1/7.1 systems, the AI hears the rear and side channels too, not just the front.
- Late danger: If a
Dangerresult arrives a moment after the sound, the overlay still draws that sound at the size it had, even when its earlier type was hidden.
- Browse the 521 sounds the AI knows, grouped like the Android version, and choose whether each group (or, in advanced mode, each sound) counts as
Ambient,Speech, orDanger. - Search by name, filter by type or by the sounds you changed, and reset everything in one step. The overlay's AI follows your changes immediately.
- Help tab: Getting started, visual modes, sound types, hotkeys, FAQ, a report button that opens a prefilled GitHub issue or email, and the open source licenses.
- Notices: The Home tab tells you when there is no playback device to capture, when sound type detection is unavailable, or why the visualizer stopped.
- Tray icon: Start or stop the visualizer, return to the launcher, or exit from the notification area.
- Tutorial: The first time you open the app, a short tutorial shows how sound, direction, and sound types are drawn, using the real renderer. You can open it again from the Home tab.
- Developer mode: Shows what the AI heard (its top five sounds, how sure it was, and how long it took) at the top left of the overlay, and can record every result to a CSV file in the
ai-logfolder next to the app. - Keyboard and screen readers: Every control in the launcher can be reached with the Tab key. On the tab row, the arrow keys switch tabs, and
Ctrl + Tabmoves to the next tab from anywhere. Dialogs and the overlay editor close withEsc. The language list opens with the arrow keys so you can browse it, and the language only changes when you choose one withEnter. Screen readers such as Narrator and NVDA read icon buttons, hotkey buttons, color buttons, and the sound list by name, and announce the running status, notices, and the number of changed sounds as they change. Everything you can drag with the mouse in the overlay editor can also be changed with the sliders in its panel.
- Supports 18 languages: Korean, English, Japanese, Chinese (Simplified and Traditional), Vietnamese, Thai, Indonesian, Spanish, Portuguese (Brazil), French, Italian, German, Russian, Polish, Turkish, Hindi, and Arabic (right-to-left). By default it follows the Windows display language.
- The interface uses the Pretendard font, with Windows fonts for scripts Pretendard doesn't cover.
The project is rigorously engineered for high-performance, real-time background processing with minimal system overhead.
- Core Audio API (WASAPI): Loopback captures system-wide audio with zero latency.
- Per-Channel Levels: The capture thread measures each channel's peak level for every buffer, and the render loop takes the loudest value since the previous frame.
- Zero-Allocation Rendering: The WPF/C# rendering loop is designed to minimize Garbage Collector (GC) allocation, entirely preventing frame drops.
- Frame-Rate Independent and Idle-Aware: Smoothing is normalized to a 60 FPS reference. When nothing has been visible for a second, the overlay stops redrawing every frame and wakes as soon as sound arrives.
- ONNX Inference Pipeline: Every 250 ms, the last 0.975 seconds of audio is downmixed to mono, resampled to 16 kHz through an anti-aliasing filter, turned into the log-mel spectrogram the model was exported with, and classified by YAMNet via
Microsoft.ML.OnnxRuntime. The model files are checked against pinned SHA-256 hashes at load. Each step is unit-tested against reference outputs.
SoundVisualizer is provided as a lightweight, portable (no-install) open-source package for Windows 10/11 (64-bit). You don't need to install .NET.
- Go to the GitHub Releases page.
- Download the latest
SoundVisualizer_vX.Y.zipfile and extract it to any location on your PC. - Run
SoundVisualizer.exe. - Configure your initial settings and language in the launcher, then click Start to activate the overlay.
Your settings are saved in settings.json next to SoundVisualizer.exe, so the folder can be moved or copied as is.
To build from source to modify the code or contribute to the project:
- Prerequisites: Windows 10/11 and the .NET 10 SDK. Visual Studio 2026 (with the .NET Desktop Development workload) is optional.
- Clone the repository:
git clone https://github.com/amophi/SoundVisualizer.git
- Build and test from the repository folder:
Or open
dotnet build SoundVisualizer.slnx -c Release dotnet test SoundVisualizer.Tests/SoundVisualizer.Tests.csprojSoundVisualizer.slnxin Visual Studio, build (Ctrl + Shift + B), and pressF5to run.
The solution has three projects: SoundVisualizer (the WPF app), SoundVisualizer.Core (audio, AI, rendering logic, settings, and the 18 language tables, with no UI dependency), and SoundVisualizer.Tests (xUnit tests, including reference outputs for each AI step).
SoundVisualizer is distributed under the AGPL v3 license to encourage a virtuous cycle in the open-source community. Anyone is welcome to modify the code and share custom UI themes or streaming plugins. See the LICENSE file for more information.
For licensing and copyright information regarding third-party libraries (NAudio, ONNX Runtime, etc.) and AI models used in this project, please refer to the THIRD_PARTY.md file.