Go to file
Wirasm 1a78a8e287
feat: TanStack Query Migration Phase 2 - Cleanup and Test Reorganization (#588)
* refactor: migrate layouts to TanStack Query and Radix UI patterns

- Created new modern layout components in src/components/layout/
- Migrated from old MainLayout/SideNavigation to new system
- Added BackendStatus component with proper separation of concerns
- Fixed horizontal scrollbar issues in project list
- Renamed old layouts folder to agent-chat for unused chat panel
- Added layout directory to Biome configuration
- Fixed all linting and TypeScript issues in new layout code
- Uses TanStack Query for backend health monitoring
- Temporarily imports old settings/credentials until full migration

* test: reorganize test infrastructure with colocated tests in subdirectories

- Move tests into dedicated tests/ subdirectories within each feature
- Create centralized test utilities in src/features/testing/
- Update all import paths to match new structure
- Configure tsconfig.prod.json to exclude test files
- Remove legacy test files from old test/ directory
- All 32 tests passing with proper provider wrapping

* fix: use error boundary wrapper for ProjectPage

- Export ProjectsViewWithBoundary from projects feature module
- Update ProjectPage to use boundary-wrapped version
- Provides proper error containment and recovery with TanStack Query integration

* cleanup: remove unused MCP client components

- Remove ToolTestingPanel, ClientCard, and MCPClients components
- These were part of an unimplemented MCP clients feature
- Clean up commented import in MCPPage
- Preparing for proper MCP feature migration to features directory

* cleanup: remove unused mcpService.ts

- Remove duplicate/unused mcpService.ts (579 lines)
- Keep mcpServerService.ts which is actively used by MCPPage and useMCPQueries
- mcpService was never imported or used anywhere in the codebase

* cleanup: remove unused mcpClientService and update deprecation comments

- Remove mcpClientService.ts (445 lines) - no longer used after removing MCP client components
- Update deprecation comments in mcpServerService to remove references to deleted service
- This completes the MCP service cleanup

* fix: correct test directory exclusion in coverage config

Update coverage exclusion from 'test/' to 'tests/' to match actual
project structure and ensure proper test file exclusion from coverage.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: fix ArchonChatPanel import path in agent-chat.mdx

Update import from deprecated layouts to agent-chat directory.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: improve backend health hook and types

- Use existing ETag infrastructure in useBackendHealth for 70% bandwidth reduction
- Honor React Query cancellation signals with proper timeout handling
- Remove duplicate HealthResponse interface, import from shared types
- Add React type import to fix potential strict TypeScript issues

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: remove .d.ts exclusion from production TypeScript config

Removing **/*.d.ts exclusion to fix import.meta.env type errors in
production builds. The exclusion was preventing src/env.d.ts from
being included, breaking ImportMetaEnv interface definitions.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: implement modern MCP feature architecture

- Add new /features/mcp with TanStack Query integration
- Components: McpClientList, McpStatusBar, McpConfigSection
- Services: mcpApi with ETag caching
- Hooks: useMcpStatus, useMcpConfig, useMcpClients, useMcpSessionInfo
- Views: McpView with error boundary wrapper
- Full TypeScript types for MCP protocol

Part of TanStack Query migration phase 2.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: complete MCP modernization and cleanup

- Remove deprecated mcpServerService.ts (237 lines)
- Remove unused useMCPQueries.ts hooks (77 lines)
- Simplify MCPPage.tsx to use new feature architecture
- Export useSmartPolling from ui/hooks for MCP feature
- Add Python MCP API routes for backend integration

This completes the MCP migration to TanStack Query with:
- ETag caching for 70% bandwidth reduction
- Smart polling with visibility awareness
- Vertical slice architecture
- Full TypeScript type safety

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: correct MCP transport mode display and complete cleanup

- Fix backend API to return correct "streamable-http" transport mode
- Update frontend to dynamically display transport type from config
- Remove unused MCP functions (startMCPServer, stopMCPServer, getMCPServerStatus)
- Clean up unused MCPServerResponse interface
- Update log messages to show accurate transport mode
- Complete aggressive MCP cleanup with 75% code reduction (617 lines removed)

Backend changes:
- python/src/server/api_routes/mcp_api.py: Fix transport and logs
- Reduced from 818 to 201 lines while preserving all functionality

Frontend changes:
- McpStatusBar: Dynamic transport display based on config
- McpView: Pass config to status bar component
- api.ts: Remove unused MCP management functions

All MCP tools tested and verified working after cleanup.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* simplify MCP API to status-only endpoints

- Remove Docker container management functionality
- Remove start/stop/restart endpoints
- Simplify to status and config endpoints only
- Container is now managed entirely via docker-compose

* feat: complete MCP feature migration to TanStack Query

- Add MCP feature with TanStack Query hooks and services
- Create useMcpQueries hook with smart polling for status/config
- Implement mcpApi service with streamable-http transport
- Add MCP page component with real-time updates
- Export MCP hooks from features/ui for global access
- Fix logging bug in mcp_api.py (invalid error kwarg)
- Update docker command to v2 syntax (docker compose)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: clean up unused CSS and unify Tron-themed scrollbars

- Remove 200+ lines of unused CSS classes (62% file size reduction)
- Delete unused: glass classes, neon-dividers, card animations, screensaver animations
- Remove unused knowledge-item-card and hide-scrollbar styles
- Remove unused flip-card and card expansion animations
- Update scrollbar-thin to match Tron theme with blue glow effects
- Add gradient and glow effects to thin scrollbars for consistency
- Keep only actively used styles: neon-grid, scrollbars, animation delays

File reduced from 11.2KB to 4.3KB with no visual regressions

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: address CodeRabbit CSS review feedback

- Fix neon-grid Tailwind @apply with arbitrary values (breaking build)
- Convert hardcoded RGBA colors to HSL tokens using --blue-accent
- Add prefers-reduced-motion accessibility support
- Add Firefox dark mode scrollbar-color support
- Optimize transitions to specific properties instead of 'all'

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: properly close Docker client to prevent resource leak

- Add finally block to ensure Docker client is closed
- Prevents resource leak in get_container_status function
- Fix linting issues (whitespace and newline)

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-06 13:43:53 +03:00
.claude add archon-coderabbit-helper slash command 2025-09-01 18:16:08 +03:00
.github CI fails now when unit tests for backend fail (#536) 2025-08-30 12:52:34 -05:00
archon-ui-main feat: TanStack Query Migration Phase 2 - Cleanup and Test Reorganization (#588) 2025-09-06 13:43:53 +03:00
docs feat: TanStack Query Migration Phase 2 - Cleanup and Test Reorganization (#588) 2025-09-06 13:43:53 +03:00
migration Fix race condition in concurrent crawling with unique source IDs (#472) 2025-08-29 14:54:16 +03:00
PRPs refactor: Remove Socket.IO and implement HTTP polling architecture (#514) 2025-09-02 14:41:35 -05:00
python feat: TanStack Query Migration Phase 2 - Cleanup and Test Reorganization (#588) 2025-09-06 13:43:53 +03:00
.dockerignore
.env.example POC: TanStack Query POC implementation (#567) 2025-09-05 11:30:05 +03:00
.gitignore POC: TanStack Query POC implementation (#567) 2025-09-05 11:30:05 +03:00
AGENTS.md POC: TanStack Query POC implementation (#567) 2025-09-05 11:30:05 +03:00
check-env.js Improve development environment with Docker Compose profiles (#435) 2025-08-22 17:18:10 +03:00
CLAUDE.md POC: TanStack Query POC implementation (#567) 2025-09-05 11:30:05 +03:00
CONTRIBUTING.md Improve development environment with Docker Compose profiles (#435) 2025-08-22 17:18:10 +03:00
docker-compose.docs.yml
docker-compose.yml POC: TanStack Query POC implementation (#567) 2025-09-05 11:30:05 +03:00
LICENSE
Makefile CI fails now when unit tests for backend fail (#536) 2025-08-30 12:52:34 -05:00
README.md Spacing updates for Make installation in README 2025-08-30 17:00:05 -05:00

Archon Main Graphic

Power up your AI coding assistants with your own custom knowledge base and task management as an MCP server

Quick StartUpgradingWhat's IncludedArchitectureTroubleshooting


🎯 What is Archon?

Archon is currently in beta! Expect things to not work 100%, and please feel free to share any feedback and contribute with fixes/new features! Thank you to everyone for all the excitement we have for Archon already, as well as the bug reports, PRs, and discussions. It's a lot for our small team to get through but we're committed to addressing everything and making Archon into the best tool it possibly can be!

Archon is the command center for AI coding assistants. For you, it's a sleek interface to manage knowledge, context, and tasks for your projects. For the AI coding assistant(s), it's a Model Context Protocol (MCP) server to collaborate on and leverage the same knowledge, context, and tasks. Connect Claude Code, Kiro, Cursor, Windsurf, etc. to give your AI agents access to:

  • Your documentation (crawled websites, uploaded PDFs/docs)
  • Smart search capabilities with advanced RAG strategies
  • Task management integrated with your knowledge base
  • Real-time updates as you add new content and collaborate with your coding assistant on tasks
  • Much more coming soon to build Archon into an integrated environment for all context engineering

This new vision for Archon replaces the old one (the agenteer). Archon used to be the AI agent that builds other agents, and now you can use Archon to do that and more.

It doesn't matter what you're building or if it's a new/existing codebase - Archon's knowledge and task management capabilities will improve the output of any AI driven coding.

Quick Start

Prerequisites

Setup Instructions

  1. Clone Repository:

    git clone https://github.com/coleam00/archon.git
    
    cd archon
    
  2. Environment Configuration:

    cp .env.example .env
    # Edit .env and add your Supabase credentials:
    # SUPABASE_URL=https://your-project.supabase.co
    # SUPABASE_SERVICE_KEY=your-service-key-here
    

    IMPORTANT NOTES:

    • For cloud Supabase: they recently introduced a new type of service role key but use the legacy one (the longer one).
    • For local Supabase: set SUPABASE_URL to http://host.docker.internal:8000 (unless you have an IP address set up).
  3. Database Setup: In your Supabase project SQL Editor, copy, paste, and execute the contents of migration/complete_setup.sql

  4. Start Services (choose one):

    Full Docker Mode (Recommended for Normal Archon Usage)

    docker compose up --build -d
    

    This starts all core microservices in Docker:

    • Server: Core API and business logic (Port: 8181)
    • MCP Server: Protocol interface for AI clients (Port: 8051)
    • UI: Web interface (Port: 3737)

    Ports are configurable in your .env as well!

  5. Configure API Keys:

    • Open http://localhost:3737
    • You'll automatically be brought through an onboarding flow to set your API key (OpenAI is default)

Quick Test

Once everything is running:

  1. Test Web Crawling: Go to http://localhost:3737 → Knowledge Base → "Crawl Website" → Enter a doc URL (such as https://ai.pydantic.dev/llms-full.txt)
  2. Test Document Upload: Knowledge Base → Upload a PDF
  3. Test Projects: Projects → Create a new project and add tasks
  4. Integrate with your AI coding assistant: MCP Dashboard → Copy connection config for your AI coding assistant

Installing Make

🛠️ Make installation (OPTIONAL - For Dev Workflows)

Windows

# Option 1: Using Chocolatey
choco install make

# Option 2: Using Scoop
scoop install make

# Option 3: Using WSL2
wsl --install
# Then in WSL: sudo apt-get install make

macOS

# Make comes pre-installed on macOS
# If needed: brew install make

Linux

# Debian/Ubuntu
sudo apt-get install make

# RHEL/CentOS/Fedora
sudo yum install make
🚀 Quick Command Reference for Make
Command Description
make dev Start hybrid dev (backend in Docker, frontend local)
make dev-docker Everything in Docker
make stop Stop all services
make test Run all tests
make lint Run linters
make install Install dependencies
make check Check environment setup
make clean Remove containers and volumes (with confirmation)

🔄 Database Reset (Start Fresh if Needed)

If you need to completely reset your database and start fresh:

⚠️ Reset Database - This will delete ALL data for Archon!
  1. Run Reset Script: In your Supabase SQL Editor, run the contents of migration/RESET_DB.sql

    ⚠️ WARNING: This will delete all Archon specific tables and data! Nothing else will be touched in your DB though.

  2. Rebuild Database: After reset, run migration/complete_setup.sql to create all the tables again.

  3. Restart Services:

    docker compose --profile full up -d
    
  4. Reconfigure:

    • Select your LLM/embedding provider and set the API key again
    • Re-upload any documents or re-crawl websites

The reset script safely removes all tables, functions, triggers, and policies with proper dependency handling.

📚 Documentation

Core Services

Service Container Name Default URL Purpose
Web Interface archon-ui http://localhost:3737 Main dashboard and controls
API Service archon-server http://localhost:8181 Web crawling, document processing
MCP Server archon-mcp http://localhost:8051 Model Context Protocol interface
Agents Service archon-agents http://localhost:8052 AI/ML operations, reranking

Upgrading

To upgrade Archon to the latest version:

  1. Pull latest changes:

    git pull
    
  2. Check for migrations: Look in the migration/ folder for any SQL files newer than your last update. Check the file created dates to determine if you need to run them. You can run these in the SQL editor just like you did when you first set up Archon. We are also working on a way to make handling these migrations automatic!

  3. Rebuild and restart:

    docker compose up -d --build
    

This is the same command used for initial setup - it rebuilds containers with the latest code and restarts services.

What's Included

🧠 Knowledge Management

  • Smart Web Crawling: Automatically detects and crawls entire documentation sites, sitemaps, and individual pages
  • Document Processing: Upload and process PDFs, Word docs, markdown files, and text documents with intelligent chunking
  • Code Example Extraction: Automatically identifies and indexes code examples from documentation for enhanced search
  • Vector Search: Advanced semantic search with contextual embeddings for precise knowledge retrieval
  • Source Management: Organize knowledge by source, type, and tags for easy filtering

🤖 AI Integration

  • Model Context Protocol (MCP): Connect any MCP-compatible client (Claude Code, Cursor, even non-AI coding assistants like Claude Desktop)
  • MCP Tools: Comprehensive yet simple set of tools for RAG queries, task management, and project operations
  • Multi-LLM Support: Works with OpenAI, Ollama, and Google Gemini models
  • RAG Strategies: Hybrid search, contextual embeddings, and result reranking for optimal AI responses
  • Real-time Streaming: Live responses from AI agents with progress tracking

📋 Project & Task Management

  • Hierarchical Projects: Organize work with projects, features, and tasks in a structured workflow
  • AI-Assisted Creation: Generate project requirements and tasks using integrated AI agents
  • Document Management: Version-controlled documents with collaborative editing capabilities
  • Progress Tracking: Real-time updates and status management across all project activities

🔄 Real-time Collaboration

  • WebSocket Updates: Live progress tracking for crawling, processing, and AI operations
  • Multi-user Support: Collaborative knowledge building and project management
  • Background Processing: Asynchronous operations that don't block the user interface
  • Health Monitoring: Built-in service health checks and automatic reconnection

Architecture

Microservices Structure

Archon uses true microservices architecture with clear separation of concerns:

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Frontend UI   │    │  Server (API)   │    │   MCP Server    │    │ Agents Service  │
│                 │    │                 │    │                 │    │                 │
│  React + Vite   │◄──►│    FastAPI +    │◄──►│    Lightweight  │◄──►│   PydanticAI    │
│  Port 3737      │    │    SocketIO     │    │    HTTP Wrapper │    │   Port 8052     │
│                 │    │    Port 8181    │    │    Port 8051    │    │                 │
└─────────────────┘    └─────────────────┘    └─────────────────┘    └─────────────────┘
         │                        │                        │                        │
         └────────────────────────┼────────────────────────┼────────────────────────┘
                                  │                        │
                         ┌─────────────────┐               │
                         │    Database     │               │
                         │                 │               │
                         │    Supabase     │◄──────────────┘
                         │    PostgreSQL   │
                         │    PGVector     │
                         └─────────────────┘

Service Responsibilities

Service Location Purpose Key Features
Frontend archon-ui-main/ Web interface and dashboard React, TypeScript, TailwindCSS, Socket.IO client
Server python/src/server/ Core business logic and APIs FastAPI, service layer, Socket.IO broadcasts, all ML/AI operations
MCP Server python/src/mcp/ MCP protocol interface Lightweight HTTP wrapper, MCP tools, session management
Agents python/src/agents/ PydanticAI agent hosting Document and RAG agents, streaming responses

Communication Patterns

  • HTTP-based: All inter-service communication uses HTTP APIs
  • Socket.IO: Real-time updates from Server to Frontend
  • MCP Protocol: AI clients connect to MCP Server via SSE or stdio
  • No Direct Imports: Services are truly independent with no shared code dependencies

Key Architectural Benefits

  • Lightweight Containers: Each service contains only required dependencies
  • Independent Scaling: Services can be scaled independently based on load
  • Development Flexibility: Teams can work on different services without conflicts
  • Technology Diversity: Each service uses the best tools for its specific purpose

🔧 Configuring Custom Ports & Hostname

By default, Archon services run on the following ports:

  • archon-ui: 3737
  • archon-server: 8181
  • archon-mcp: 8051
  • archon-agents: 8052
  • archon-docs: 3838 (optional)

Changing Ports

To use custom ports, add these variables to your .env file:

# Service Ports Configuration
ARCHON_UI_PORT=3737
ARCHON_SERVER_PORT=8181
ARCHON_MCP_PORT=8051
ARCHON_AGENTS_PORT=8052
ARCHON_DOCS_PORT=3838

Example: Running on different ports:

ARCHON_SERVER_PORT=8282
ARCHON_MCP_PORT=8151

Configuring Hostname

By default, Archon uses localhost as the hostname. You can configure a custom hostname or IP address by setting the HOST variable in your .env file:

# Hostname Configuration
HOST=localhost  # Default

# Examples of custom hostnames:
HOST=192.168.1.100     # Use specific IP address
HOST=archon.local      # Use custom domain
HOST=myserver.com      # Use public domain

This is useful when:

  • Running Archon on a different machine and accessing it remotely
  • Using a custom domain name for your installation
  • Deploying in a network environment where localhost isn't accessible

After changing hostname or ports:

  1. Restart Docker containers: docker compose down && docker compose --profile full up -d
  2. Access the UI at: http://${HOST}:${ARCHON_UI_PORT}
  3. Update your AI client configuration with the new hostname and MCP port

🔧 Development

Quick Start

# Install dependencies
make install

# Start development (recommended)
make dev        # Backend in Docker, frontend local with hot reload

# Alternative: Everything in Docker
make dev-docker # All services in Docker

# Stop everything (local FE needs to be stopped manually)
make stop

Development Modes

Best for active development with instant frontend updates:

  • Backend services run in Docker (isolated, consistent)
  • Frontend runs locally with hot module replacement
  • Instant UI updates without Docker rebuilds

Full Docker Mode - make dev-docker

For all services in Docker environment:

  • All services run in Docker containers
  • Better for integration testing
  • Slower frontend updates

Testing & Code Quality

# Run tests
make test       # Run all tests
make test-fe    # Run frontend tests
make test-be    # Run backend tests

# Run linters
make lint       # Lint all code
make lint-fe    # Lint frontend code
make lint-be    # Lint backend code

# Check environment
make check      # Verify environment setup

# Clean up
make clean      # Remove containers and volumes (asks for confirmation)

Viewing Logs

# View logs using Docker Compose directly
docker compose logs -f              # All services
docker compose logs -f archon-server # API server
docker compose logs -f archon-mcp    # MCP server
docker compose logs -f archon-ui     # Frontend

Note: The backend services are configured with --reload flag in their uvicorn commands and have source code mounted as volumes for automatic hot reloading when you make changes.

Troubleshooting

Common Issues and Solutions

Port Conflicts

If you see "Port already in use" errors:

# Check what's using a port (e.g., 3737)
lsof -i :3737

# Stop all containers and local services
make stop

# Change the port in .env

Docker Permission Issues (Linux)

If you encounter permission errors with Docker:

# Add your user to the docker group
sudo usermod -aG docker $USER

# Log out and back in, or run
newgrp docker

Windows-Specific Issues

  • Make not found: Install Make via Chocolatey, Scoop, or WSL2 (see Installing Make)
  • Line ending issues: Configure Git to use LF endings:
    git config --global core.autocrlf false
    

Frontend Can't Connect to Backend

  • Check backend is running: curl http://localhost:8181/health
  • Verify port configuration in .env
  • For custom ports, ensure both ARCHON_SERVER_PORT and VITE_ARCHON_SERVER_PORT are set

Docker Compose Hangs

If docker compose commands hang:

# Reset Docker Compose
docker compose down --remove-orphans
docker system prune -f

# Restart Docker Desktop (if applicable)

Hot Reload Not Working

  • Frontend: Ensure you're running in hybrid mode (make dev) for best HMR experience
  • Backend: Check that volumes are mounted correctly in docker-compose.yml
  • File permissions: On some systems, mounted volumes may have permission issues

📈 Progress

Star History Chart

📄 License

Archon Community License (ACL) v1.2 - see LICENSE file for details.

TL;DR: Archon is free, open, and hackable. Run it, fork it, share it - just don't sell it as-a-service without permission.