🔥 The_Colonel

An enhanced Open Interpreter fork with robust Open WebUI integration and advanced computer control capabilities

The_Colonel is a powerful, battle-tested fork of Open Interpreter designed for serious automation and AI-powered computer control. With comprehensive Open WebUI integration, advanced streaming capabilities, and robust error handling, it bridges the gap between conversational AI and practical system automation.

✨ Key Features

• 🌐 Seamless Open WebUI Integration: Production-ready API server with OpenAI-compatible streaming endpoints
• 🛠️ Comprehensive Tool Arsenal: 12+ specialized endpoints for code execution, file operations, and computer control
• 🖥️ Advanced Computer Control: Direct mouse/keyboard interaction, screenshots, and window management
• 📁 Intelligent File Management: Advanced file operations with upload/download capabilities
• 👤 Dynamic Profile System: Hot-swappable configurations for different AI models and use cases
• 🔐 Enterprise-Ready Security: Bearer token authentication with localhost development mode
• ⚡ Optimized Streaming: Robust chunk processing with real-time response streaming
• 🔧 Error-Resilient Architecture: Advanced error handling and recovery mechanisms
• 📋 OpenAPI 3.1 Specification: Full API documentation with tagged tool categorization

🚀 Quick Start

Installation

# Clone the repository
git clone https://github.com/Unicorn-Commander/The_Colonel.git
cd The_Colonel

# Set up virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install in development mode
pip install -e .

Basic Usage

Traditional Terminal Interface:

interpreter

Open WebUI Server Mode:

interpreter --openwebui_server --host localhost --port 8264

With Custom Profile:

interpreter --openwebui_server --profile The_Colonel.py

Remote Access with Authentication:

interpreter --openwebui_server --host 0.0.0.0 --port 8264 --auth_token your_secure_token

Quick Start Scripts:

# Local development (no authentication)
./start_server.sh

# Remote access with authentication
./start_server_auth.sh

🔧 Open WebUI Integration

Individual Tool Servers

The_Colonel provides 4 specialized tool servers that can be added individually to Open WebUI:

🐍 Python Code Executor:

• Server: http://your-ip:8264/python
• OpenAPI: http://your-ip:8264/python/openapi.json
• Execute Python code, data analysis, calculations

🔧 Shell Command Executor:

• Server: http://your-ip:8264/shell
• OpenAPI: http://your-ip:8264/shell/openapi.json
• Run bash/shell commands, system operations

📁 File Operations:

• Server: http://your-ip:8264/files
• OpenAPI: http://your-ip:8264/files/openapi.json
• Read, write, and manage files

🖥️ Computer Control:

• Server: http://your-ip:8264/computer
• OpenAPI: http://your-ip:8264/computer/openapi.json
• Screenshots, mouse clicks, keyboard input

Legacy Endpoints

• GET /openapi.json - Complete API specification
• POST /v1/chat/completions - OpenAI-compatible chat endpoint

Open WebUI Configuration

LLM Connection:

• Base URL: http://localhost:8264/v1
• Model: the-colonel

Individual Tool Setup: Add each tool separately to Open WebUI with:

• Tool Server URL: http://your-ip:8264/{tool-name}
• API Key: your-auth-token
• OpenAPI JSON: http://your-ip:8264/{tool-name}/openapi.json

📖 Complete Tool Documentation: See Tool Reference Guide for detailed setup instructions, examples, and usage patterns.

Profile Selection

Use the profile query parameter for chat completions:

curl -X POST "http://localhost:8264/v1/chat/completions?profile=The_Colonel" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello"}]}'

🎯 Use Cases

For Developers:

• Code execution and debugging through web interface
• File management and project manipulation
• Automated testing and deployment scripts

For Power Users:

• Computer automation via API calls
• Screen capture and GUI automation
• Advanced file processing workflows

For Integration:

• Embed in existing web applications
• Build custom frontends with the API
• Create automated workflows and pipelines

🛡️ Security & Safety

Local Development:

• No authentication required for localhost access
• Safe for development and testing

Remote Access:

• Bearer token authentication required
• Use strong tokens for production deployments

Code Execution:

• All code execution happens in your local environment
• Review and approve code before execution in interactive mode
• Use auto_run carefully in production environments

📚 Advanced Usage

Python API

from interpreter import interpreter

# Basic chat
interpreter.chat("Analyze this dataset and create visualizations")

# Streaming responses
for chunk in interpreter.chat("Process these files", stream=True):
    print(chunk)

# Custom configuration
interpreter.llm.model = "gpt-4"
interpreter.auto_run = True
interpreter.os = True  # Enable OS control features

Profile System

Create custom profiles in interpreter/terminal_interface/profiles/defaults/:

# my_profile.py
from interpreter import interpreter

interpreter.llm.model = "gpt-4"
interpreter.llm.temperature = 0.1
interpreter.auto_run = True
interpreter.os = True
interpreter.computer.import_computer_api = True

Use with:

interpreter --profile my_profile.py

⚡ Streaming & Performance

Robust Chunk Processing

The_Colonel features advanced streaming response handling that properly processes various chunk formats from the interpreter:

• Error-Resilient: Individual chunk processing errors don't crash the entire conversation
• Format-Agnostic: Handles dictionary chunks, string chunks, and malformed data gracefully
• Type-Safe: Uses .get() methods instead of direct key access to prevent KeyErrors
• Debug-Friendly: Comprehensive logging for troubleshooting chunk processing issues

Technical Architecture

Streaming Response Flow:

1. OpenAI-compatible request received
2. Profile configuration loaded dynamically
3. Messages converted to interpreter format
4. Interpreter streams chunks in real-time
5. Chunks processed through robust error handling
6. Server-Sent Events formatted for Open WebUI
7. Real-time display in web interface

🛠️ Troubleshooting

Common Issues

1. API Key Authentication Errors

# Check your API key is set correctly
echo $OPENAI_API_KEY

# Update .env file if needed
nano .env

2. Streaming Response Issues

• "Error: 'type'" on second messages → Fixed with improved message state management
• Messages appear in terminal but not GUI → Resolved with proper SSE formatting
• KeyError 'type' exceptions → Handled with robust chunk processing

3. Profile Loading Problems

# Clear Python cache if profiles aren't reloading
find . -name "*.pyc" -delete
find . -name "__pycache__" -type d -exec rm -rf {} +

4. Port Already in Use

# Find and kill process using port 8264
lsof -ti:8264 | xargs kill -9

Debug Mode

Enable detailed logging:

interpreter --openwebui_server --verbose

🔧 Configuration

Environment Variables

Copy .env.example to .env and configure your settings:

cp .env.example .env
# Edit with your actual API keys and settings

# API Keys
OPENAI_API_KEY=your_openai_key_here
ANTHROPIC_API_KEY=your_anthropic_key_here
XAI_API_KEY=your_xai_key_here
DEEPSEEK_API_KEY=your_deepseek_key_here

# Server Configuration  
DEFAULT_PROFILE=The_Colonel.py
SERVER_HOST=0.0.0.0
SERVER_PORT=8264
AUTH_TOKEN=your_secure_random_token

# General Settings
DISABLE_TELEMETRY=true
AUTO_RUN=false
SAFE_MODE=off

Profile Customization

Advanced profile example with environment-based API key:

# custom_profile.py
import os
from interpreter import interpreter

# Model Configuration
interpreter.llm.model = "gpt-4o-mini"
interpreter.llm.api_key = os.getenv("OPENAI_API_KEY")
interpreter.llm.temperature = 0.1
interpreter.llm.context_window = 128000
interpreter.llm.max_tokens = 16384

# Capabilities
interpreter.auto_run = True
interpreter.os = True
interpreter.computer.import_computer_api = True

# Custom instructions for better computer control
interpreter.custom_instructions = """
When taking screenshots, if you encounter errors with pywinctl, 
try using computer.display.screenshot(active_app_only=False) instead.
"""

🔨 Development

Project Structure

The_Colonel/
├── interpreter/
│   ├── core/
│   │   ├── openwebui_server.py    # Open WebUI integration
│   │   └── ...
│   ├── computer_use/
│   │   └── tools/                 # Computer control tools
│   └── terminal_interface/
│       └── profiles/              # Configuration profiles
├── openapi.json                   # API specification
├── documentation/                 # Project documentation
└── examples/                      # Usage examples

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Make your changes and test thoroughly
4. Submit a pull request with detailed description

Testing

# Test server functionality
source venv/bin/activate
python -c "from interpreter.core.openwebui_server import create_openwebui_server; print('✅ Server tests passed')"

# Start test server
interpreter --openwebui_server --host localhost --port 8264

# Test endpoints
curl http://localhost:8264/openapi.json

📖 Documentation

• Tool Reference Guide - Complete tool documentation with examples
• Technical Implementation Guide - Architecture and technical details
• Project Summary - Project overview and current status
• Setup Guide - Installation and configuration
• OpenAPI Spec - Complete API documentation

🏗️ Based on Open Interpreter

The_Colonel builds upon the excellent foundation of Open Interpreter, adding:

• Open WebUI compatibility
• Enhanced tool endpoints
• Computer control capabilities
• Improved profile system
• Production-ready API server

📜 License

This project is licensed under the AGPL License - see the LICENSE file for details.

🤝 Acknowledgments

• Open Interpreter Team - For the incredible foundation
• Open WebUI Community - For the inspiration and integration target
• Contributors - For making this project better

---

⚠️ Note: The_Colonel executes code in your local environment. Always review and understand code before execution, especially when using auto_run mode or in production environments.