DeskAide

A local-first AI desktop assistant for Windows.

Talk or type, let the model plan, and keep every system action visible and under your control.

English · 简体中文

Important

DeskAide is currently alpha software. It can launch applications, manage files, access the clipboard, and run PowerShell commands. Review the permission policy and confirmation prompt before allowing sensitive actions.

Why DeskAide?

Most AI assistants stop at text. DeskAide connects a model to carefully controlled Windows tools, while keeping the user in the loop. It runs locally, has no required DeskAide backend, and lets you bring your own compatible model provider.

• Local-first — settings, credentials, conversations, and execution history stay on your computer.
• Action-oriented — open apps and URLs, search or organize files, use the clipboard, and run PowerShell tasks.
• User-controlled — configure Ask, Allow, or Block for each sensitive behavior category.
• Visible execution — animated activity badges show queued, running, completed, failed, and interrupted operations.
• Personal — replace the floating orb with a PNG, JPG, or GIF and adjust its size, crop, stretch, color, and screen position.

Features

AI agent

• OpenAI-compatible Chat Completions and Anthropic Messages APIs.
• Provider groups with independent protocol, Base URL, custom full API path, credentials, and model lists.
• Streaming responses, structured tool calls, tool-result feedback, and configurable agent rounds (1–999).
• Full conversation context, grouped local history, new conversations, and model switching without restarting.
• AI-generated follow-up actions with safe parsing and a collapsible suggestions area.
• Markdown rendering, selectable messages, timestamps, context menus, one-click copy, retry, and stop generation.

Voice and wake-up

• Text and voice are separate input modes.
• The default mode on each assistant invocation can be set to text or voice; voice mode starts listening immediately.
• Local whisper.cpp transcription is the default; users can select a GGML model or download the multilingual base model.
• vivo real-time ASR remains available as a temporary compatibility fallback.
• Partial transcription, local noise calibration, voice activity detection, silence-based auto-submit, cancel, and retry.
• System tray, configurable global shortcut (Ctrl+Alt+Space by default), and optional experimental Windows SAPI hotword.

Windows tools

• Launch or focus applications, open files, and open URLs.
• Search, copy, move, and delete files to the Recycle Bin.
• Read and write the clipboard.
• Run PowerShell commands asynchronously, with optional UAC elevation.
• Categorized controls for app launch, clipboard, file modification, deletion, registry, power, process control, services, network, system settings, PowerShell, and elevation.

Desktop experience

• Always-on-top floating pet with persistent multi-monitor position.
• Resizable assistant window, full screen, pinning, and native window resizing.
• Adaptive light/dark appearance with configurable accent presets.
• Custom pet editor with proportional or free resizing, contain/crop/stretch modes, zoom, composition dragging, and non-interactive live desktop preview.
• System tray lifecycle, configurable post-task behavior, start on login, settings auto-save, and an Apple-inspired QML interface.
• GitHub Release update checks at startup or on demand, semantic-version handling for alpha/beta builds, release notes, version skipping, and guided browser downloads.

Privacy and security

• API keys are stored in Windows Credential Manager.
• Conversation data is stored in SQLite and sensitive history content is protected with current-user DPAPI.
• Model requests never access the operating system directly; every tool call passes through a local policy and validation layer.
• Destructive and privileged operations can require explicit confirmation and still follow Windows UAC.
• Configuration export includes plaintext credentials only after Windows reauthentication. Treat exported JSON files as secrets.

Architecture

QML interface
    │
    ▼
AppController ── SettingsStore / HistoryStore
    │
    ▼
AgentService ── Permission policy ── ToolExecutor ── Windows
    │
    ▼
ModelClient ── ModelProtocol / StreamDecoder ── User API provider

VivoAsrClient / HotwordService / GlobalHotkey ──► AppController

Component	Responsibility
AppController	Application lifecycle, tray, windows, QML-facing state, and service coordination
AgentService	Agent loop, permission decisions, confirmations, and asynchronous tool orchestration
ModelClient	Streaming requests, conversation context, tool-result loop, and cancellation
ModelProtocol / StreamDecoder	OpenAI/Anthropic payload adaptation and SSE decoding
ToolExecutor	Validated Windows application, file, clipboard, PowerShell, and elevation operations
SettingsStore / HistoryStore	Preferences, Credential Manager secrets, SQLite conversations, and DPAPI protection
VivoAsrClient / HotwordService / GlobalHotkey	Independent voice and platform input services
qml/	Reusable controls, assistant, pet, settings, and avatar editor windows

See PLAN.md for the product decisions, security boundaries, and implementation status.

Requirements

Runtime

• Windows 10 version 1809 or later.
• Windows 11 is recommended.
• A model endpoint compatible with OpenAI Chat Completions or Anthropic Messages.
• Local voice input requires a whisper.cpp GGML model; a vivo AppKey is needed only when selecting the temporary cloud fallback.
• A microphone and a matching Windows speech package are required for their respective voice features.

Build environment

• Qt 6.8.x with the following modules:
  • Core, Gui, Widgets, Quick, QML
  • Network, WebSockets, Multimedia
  • SQL and Concurrent
• A C++20 compiler. The project is currently verified with Qt 6.8.3 MinGW 64-bit.
• CMake 3.21 or later.
• Ninja or another CMake-supported generator.

Build from source

Qt Creator

1. Install Qt 6.8.x and the matching MinGW 64-bit kit.
2. Open the root CMakeLists.txt in Qt Creator.
3. Select a Desktop Qt 6.8 MinGW 64-bit kit.
4. Configure, build, and run the DeskAide target.

Tip

DeskAide normally remains active in the system tray. Exit it from the tray or stop it in Qt Creator before rebuilding, otherwise Windows may keep DeskAide.exe locked.

Command line

Run from a Qt-enabled terminal, or replace <QT_DIR> and <MINGW_DIR> with your installation paths:

$env:PATH = "<QT_DIR>\bin;<MINGW_DIR>\bin;$env:PATH"

cmake -S . -B build -G Ninja `
  -DCMAKE_PREFIX_PATH="<QT_DIR>" `
  -DCMAKE_BUILD_TYPE=Release

cmake --build build --parallel

The resulting executable is build/DeskAide.exe for a single-configuration Ninja build.

First run

1. Open Settings → Models.
2. Add a provider group and choose OpenAI compatible or Anthropic.
3. Enter the Base URL, optional full API path, API key, and one or more model names.
4. Review Permissions before running local actions.
5. Optionally configure voice input, shortcut, hotword, appearance, history, and startup behavior.
6. Wake DeskAide from the floating pet, tray, or global shortcut and enter a task.

Example requests:

Open Notepad.
Copy the latest PDF from Downloads to my Desktop.
Show me what is currently in the clipboard.
Find large ZIP files in Downloads, but do not delete anything.

Configuration and data

DeskAide does not require a project-operated cloud backend. It connects directly to the endpoints configured by the user.

Data	Storage
Preferences and model groups	Windows user settings
API and voice credentials	Windows Credential Manager
Conversations and tool activities	Local SQLite database
Sensitive history payloads	Current-user DPAPI protection
Application logs	Local application data; credentials are intentionally excluded

Configuration import/export includes credentials for migration purposes. Exported files contain secrets in plaintext and should never be committed, shared, or stored in an untrusted location.

Current limitations

• Windows only.
• Alpha quality; end-to-end behavior varies between compatible API providers.
• No built-in web search or browser session automation.
• No text-to-speech or screen-vision support in the current version.
• The bundled model downloader requires network access once; local whisper.cpp transcription can then run offline.
• Windows Hello availability depends on the Windows version and device configuration; Windows password verification is used as a fallback for protected export.

Roadmap

• Broader provider compatibility and protocol diagnostics.
• Signed installers and optional in-app download/install after the existing GitHub Release update check.
• Expanded automated tests for model streams, permissions, and Windows tools.
• Improved accessibility, reduced-motion behavior, and localization.
• Optional additional speech recognition and text-to-speech providers.

The roadmap is intentionally conservative: security and reliable local execution take priority over adding more autonomous behavior.

Contributing

Issues and pull requests are welcome.

1. Fork the repository.
2. Create a focused branch: git switch -c feature/your-feature.
3. Keep UI, protocol, tool, and persistence changes within their existing architectural boundaries.
4. Do not commit credentials, exported configurations, personal conversations, build output, or diagnostic logs.
5. Build the project and verify both light and dark modes before opening a pull request.

For security-sensitive reports, avoid publishing credentials, personal paths, command output, or private conversation content in a public issue.

License

DeskAide is released under the MIT License.

Acknowledgements

• Qt for the cross-platform application and QML framework.
• OpenAI-compatible and Anthropic-compatible API ecosystems.
• vivo speech services for the current real-time ASR integration.

---

Built by ProgramCX · Project home