Fixed TypeError when passing custom fields to Python logger by using the 'extra' parameter instead of direct keyword arguments. This resolves embedding creation failures during crawl operations.
* Add improved development environment with backend in Docker and frontend locally
- Created dev.bat script to run backend services in Docker and frontend locally
- Added docker-compose.backend.yml for backend-only Docker setup
- Updated package.json to run frontend on port 3737
- Fixed api.ts to use default port 8181 instead of throwing error
- Script automatically stops production containers to avoid port conflicts
- Provides instant HMR for frontend development
* Refactor development environment setup: replace dev.bat with Makefile for cross-platform support and enhanced commands
* Enhance development environment: add environment variable checks and update test commands for frontend and backend
* Improve development environment with Docker Compose profiles
This commit enhances the development workflow by replacing the separate
docker-compose.backend.yml file with Docker Compose profiles, fixing
critical service discovery issues, and adding comprehensive developer
tooling through an improved Makefile system.
Key improvements:
- Replace docker-compose.backend.yml with cleaner profile approach
- Fix service discovery by maintaining consistent container names
- Fix port mappings (3737:3737 instead of 3737:5173)
- Add make doctor for environment validation
- Fix port configuration and frontend HMR
- Improve error handling with .SHELLFLAGS in Makefile
- Add comprehensive port configuration via environment variables
- Simplify make dev-local to only run essential services
- Add logging directory creation for local development
- Document profile strategy in docker-compose.yml
These changes provide three flexible development modes:
- Hybrid mode (default): Backend in Docker, frontend local with HMR
- Docker mode: Everything in Docker for production-like testing
- Local mode: API server and UI run locally
Co-authored-by: Zak Stam <zaksnet@users.noreply.github.com>
* Fix make stop command to properly handle Docker Compose profiles
The stop command now explicitly specifies all profiles to ensure
all containers are stopped regardless of how they were started.
* Fix README to document correct make commands
- Changed 'make lint' to 'make lint-frontend' and 'make lint-backend'
- Removed non-existent 'make logs-server' command
- Added 'make watch-mcp' and 'make watch-agents' commands
- All documented make commands now match what's available in Makefile
* fix: Address critical issues from code review #435
- Create robust environment validation script (check-env.js) that properly parses .env files
- Fix Docker healthcheck port mismatch (5173 -> 3737)
- Remove hard-coded port flags from package.json to allow environment configuration
- Fix Docker detection logic using /.dockerenv instead of HOSTNAME
- Normalize container names to lowercase (archon-server, archon-mcp, etc.)
- Improve stop-local command with port-based fallback for process killing
- Fix API configuration fallback chain to include VITE_PORT
- Fix Makefile shell variable expansion using runtime evaluation
- Update .PHONY targets with comprehensive list
- Add --profile flags to Docker Compose commands in README
- Add VITE_ARCHON_SERVER_PORT to docker-compose.yml
- Add Node.js 18+ to prerequisites
- Use dynamic ports in Makefile help messages
- Add lint alias combining frontend and backend linting
- Update .env.example documentation
- Scope .gitignore logs entry to /logs/
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix container name resolution for MCP server
- Add dynamic container name resolution with three-tier strategy
- Support environment variables for custom container names
- Add service discovery labels to docker-compose services
- Update BackendStartupError with correct container name references
* Fix frontend test failures in API configuration tests
- Update environment variable names to use VITE_ prefix that matches production code
- Fix MCP client service tests to use singleton instance export
- Update default behavior tests to expect fallback to port 8181
- All 77 frontend tests now pass
* Fix make stop-local to avoid Docker daemon interference
Replace aggressive kill -9 with targeted process termination:
- Filter processes by command name (node/vite/python/uvicorn) before killing
- Use graceful SIGTERM instead of SIGKILL
- Add process verification to avoid killing Docker-related processes
- Improve logging with descriptive step messages
* refactor: Simplify development workflow based on comprehensive review
- Reduced Makefile from 344 lines (43 targets) to 83 lines (8 essential targets)
- Removed unnecessary environment variables (*_CONTAINER_NAME variables)
- Fixed Windows compatibility by removing Unix-specific commands
- Added security fixes to check-env.js (path validation)
- Simplified MCP container discovery to use fixed container names
- Fixed 'make stop' to properly handle Docker Compose profiles
- Updated documentation to reflect simplified workflow
- Restored original .env.example with comprehensive Supabase key documentation
This addresses all critical issues from code review:
- Cross-platform compatibility ✅
- Security vulnerabilities fixed ✅
- 81% reduction in complexity ✅
- Maintains all essential functionality ✅
All tests pass: Frontend (77/77), Backend (267/267)
* feat: Add granular test and lint commands to Makefile
- Split test command into test-fe and test-be for targeted testing
- Split lint command into lint-fe and lint-be for targeted linting
- Keep original test and lint commands that run both
- Update help text with new commands for better developer experience
* feat: Improve Docker Compose detection and prefer modern syntax
- Prefer 'docker compose' (plugin) over 'docker-compose' (standalone)
- Add better error handling in Makefile with proper exit on failures
- Add Node.js check before running environment scripts
- Pass environment variables correctly to frontend in hybrid mode
- Update all documentation to use modern 'docker compose' syntax
- Auto-detect which Docker Compose version is available
* docs: Update CONTRIBUTING.md to reflect simplified development workflow
- Add Node.js 18+ as prerequisite for hybrid development
- Mark Make as optional throughout the documentation
- Update all docker-compose commands to modern 'docker compose' syntax
- Add Make command alternatives for testing (make test, test-fe, test-be)
- Document make dev for hybrid development mode
- Remove linting requirements until codebase errors are resolved
* fix: Rename frontend service to archon-frontend for consistency
Aligns frontend service naming with other services (archon-server, archon-mcp, archon-agents) for better consistency in Docker image naming patterns.
---------
Co-authored-by: Zak Stam <zakscomputers@hotmail.com>
Co-authored-by: Zak Stam <zaksnet@users.noreply.github.com>
- Transform URLs to raw content (e.g., GitHub blob -> raw) before sending to crawler
- Maintain mapping dictionary to preserve original URLs in results
- Align progress callback signatures between batch and recursive strategies
- Add safety guards for missing links attribute
- Remove unused loop counter in batch strategy
- Optimize binary file checks to avoid duplicate calls
This ensures GitHub files are crawled as raw content instead of HTML pages,
fixing the issue where content extraction was degraded due to HTML wrapping.
Remove wait_for='body' selector from documentation site crawling config.
The body element exists immediately in HTML, causing unnecessary timeouts
for JavaScript-rendered content. Now relies on domcontentloaded event
and delay_before_return_html for proper JavaScript execution.
- Fixed test_update_task_status to use individual parameters
- Added test_update_task_no_fields for validation testing
- All MCP tests passing (44 tests)
Resolves#420 - Tasks being duplicated instead of updated
Changes:
1. Fixed update_task function signature to use individual optional parameters
- Changed from TypedDict to explicit parameters (title, status, etc.)
- Consistent with update_project and update_document patterns
- Builds update_fields dict internally from provided parameters
2. Updated MCP instructions with correct function names
- Replaced non-existent manage_task with actual functions
- Added complete function signatures for all tools
- Improved workflow documentation with concrete examples
This fixes the issue where AI agents were confused by:
- Wrong function names in instructions (manage_task vs update_task)
- Inconsistent parameter patterns across update functions
- TypedDict magic that wasn't clearly documented
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed missing knowledge_type and tags parameters in DocumentStorageService.upload_document()
- Added source_type='file' to document chunk metadata for proper categorization
- Enhanced source metadata creation to include source_type based on source_id pattern
- Fixed metadata spread order in knowledge_item_service to prevent source_type override
- Business documents now correctly show pink color theme and appear in Business Documents section
Fixes issue where business documents were incorrectly stored as technical knowledge
and appeared with blue color theme instead of pink.
* fix: Allow HTTP for all private network ranges in Supabase URLs
- Extend HTTP support to all RFC 1918 private IP ranges
- Class A: 10.0.0.0 to 10.255.255.255 (10.0.0.0/8)
- Class B: 172.16.0.0 to 172.31.255.255 (172.16.0.0/12)
- Class C: 192.168.0.0 to 192.168.255.255 (192.168.0.0/16)
- Also includes link-local (169.254.0.0/16) addresses
- Uses Python's ipaddress module for robust IP validation
- Maintains HTTPS requirement for public/production URLs
- Backwards compatible with existing localhost exceptions
* security: Fix URL validation vulnerabilities
- Replace substring matching with exact hostname matching to prevent bypass attacks
- Exclude unspecified address (0.0.0.0) from allowed HTTP hosts
- Add support for .localhost domains per RFC 6761
- Improve error messages with hostname context for better debugging
Addresses security concerns raised in PR review regarding:
- Malicious domains like 'localhost.attacker.com' bypassing HTTPS requirements
- Unspecified address being incorrectly allowed as valid connection target
---------
Co-authored-by: tazmon95 <tazmon95@users.noreply.github.com>
Co-authored-by: root <root@supatest2.jtpa.net>
Applied automated linting and formatting:
- Fixed missing newlines at end of files
- Adjusted line wrapping for better readability
- Fixed multi-line string formatting in tests
- No functional changes, only style improvements
All 43 tests still passing after formatting changes.
Based on latest PR #306 review feedback:
Fixed Issues:
- Replaced last remaining basic error handling with MCPErrorFormatter
in version_tools.py get_version function
- Added proper error handling for invalid env vars in get_max_polling_attempts
- Improved type hints with TaskUpdateFields TypedDict for better validation
- All tools now consistently use get_default_timeout() (verified with grep)
Test Improvements:
- Added comprehensive tests for MCPErrorFormatter utility (10 tests)
- Added tests for timeout_config utility (13 tests)
- All 43 MCP tests passing with new utilities
- Tests verify structured error format and timeout configuration
Type Safety:
- Created TaskUpdateFields TypedDict to specify exact allowed fields
- Documents valid statuses and assignees in type comments
- Improves IDE support and catches type errors at development time
This completes all priority actions from the review:
✅ Fixed inconsistent timeout usage (was already done)
✅ Fixed error handling inconsistency
✅ Improved type hints for update_fields
✅ Added tests for utility modules
Comprehensive update to MCP server error handling:
Error Handling Improvements:
- Applied MCPErrorFormatter to all remaining MCP tool files
- Replaced all hardcoded timeout values with configurable timeout system
- Converted all simple string errors to structured error format
- Added proper httpx exception handling with detailed context
Tools Updated:
- document_tools.py: All 5 document management tools
- version_tools.py: All 4 version management tools
- feature_tools.py: Project features tool
- project_tools.py: Remaining 3 project tools (get, list, delete)
- task_tools.py: Remaining 4 task tools (get, list, update, delete)
Test Improvements:
- Removed backward compatibility checks from all tests
- Tests now enforce structured error format (dict not string)
- Any string error response is now considered a bug
- All 20 tests passing with new strict validation
This completes the error handling refactor for all MCP tools,
ensuring consistent client experience and better debugging.
Critical improvements to MCP server reliability and client experience:
Error Handling:
- Created MCPErrorFormatter for consistent error responses across all tools
- Provides structured errors with type, message, details, and actionable suggestions
- Helps clients (like Claude Code) understand and handle failures gracefully
- Categorizes errors (connection_timeout, validation_error, etc.) for better debugging
Timeout Configuration:
- Centralized timeout config with environment variable support
- Different timeouts for regular operations vs polling operations
- Configurable via MCP_REQUEST_TIMEOUT, MCP_CONNECT_TIMEOUT, etc.
- Prevents indefinite hangs when services are unavailable
Module Registration:
- Distinguishes between ImportError (acceptable) and code errors (must fix)
- SyntaxError/NameError/AttributeError now halt execution immediately
- Prevents broken code from silently failing in production
Polling Safety:
- Fixed project creation polling with exponential backoff
- Handles API unavailability with proper error messages
- Maximum attempts configurable via MCP_MAX_POLLING_ATTEMPTS
Response Normalization:
- Fixed inconsistent response handling in list_tasks
- Validates and normalizes different API response formats
- Clear error messages when response format is unexpected
These changes address critical issues from PR review while maintaining
backward compatibility. All 20 existing tests pass.
- Modified validate_supabase_url() to allow HTTP for local development
- HTTP is now allowed for localhost, 127.0.0.1, host.docker.internal, and 0.0.0.0
- HTTPS is still required for production/non-local environments
- Fixes server startup failure when using local Supabase with Docker
- Create test structure mirroring features folder organization
- Add tests for document tools (create, list, update, delete)
- Add tests for version tools (create, list, restore, invalid field handling)
- Add tests for task tools (create with sources, list with filters, update, delete)
- Add tests for project tools (create with polling, list, get)
- Add tests for feature tools (get features with various structures)
- Mock HTTP client for all external API calls
- Test both success and error scenarios
- 100% test coverage for critical tool functions
- Add explicit type annotations for params dictionaries to resolve mypy errors
- Remove trailing whitespace from blank lines (W293 ruff warnings)
- Ensure type safety in task_tools.py and document_tools.py
Update MCP server to use the new modular tool structure:
- Projects and tasks from existing modules
- Documents and versions from new modules
- Feature management from standalone module
Remove all feature flag logic as separated tools are now default.
Extract get_project_features as a standalone tool with enhanced
documentation explaining feature structures and usage patterns.
Features track functional components like auth, api, and database.
Extract document management functionality into focused tools:
- create_document: Create new documents with metadata
- list_documents: List all documents in a project
- get_document: Retrieve specific document details
- update_document: Modify existing documents
- delete_document: Remove documents from projects
Extract version control functionality:
- create_version: Create immutable snapshots
- list_versions: View version history
- get_version: Retrieve specific version content
- restore_version: Rollback to previous versions
Includes improved documentation and error messages based on testing.
The consolidated project module contained all project, task, document,
version, and feature management in a single 922-line file. This has been
replaced with focused, single-purpose tools in separate modules.
Resolves issue #282 by adding feature field to task dictionary in
TaskService.list_tasks() method. The project tasks API endpoint was
excluding the feature field while individual task API included it,
causing frontend to default to 'General' instead of showing custom
feature values.
Changes:
- Add feature field to task response in list_tasks method
- Maintains compatibility with existing API consumers
- All 212 tests pass with this change
- Rename src/mcp to src/mcp_server for clarity
- Update all internal imports to use new path
- Create features/projects directory for modular tool organization
- Add separate, simple project tools (create, list, get, delete, update)
- Keep consolidated tools for backward compatibility (via env var)
- Add USE_SEPARATE_PROJECT_TOOLS env var to toggle between approaches
The new separate tools:
- Solve the async project creation context loss issue
- Provide clearer, single-purpose interfaces
- Remove complex PRP examples for simplicity
- Handle project creation polling automatically
- Fix parameter naming confusion in RAG tools (source → source_domain)
- Add clarification that source_domain expects domain names not IDs
- Improve manage_versions documentation with clear examples
- Add better error messages for validation failures
- Enhance manage_document with non-PRP examples
- Add comprehensive documentation to get_project_features
- Fix content parameter type in manage_versions to accept Any type
These changes address usability issues discovered during testing without
breaking existing functionality.
- Consolidated multiple MCP modules into unified project_module
- Removed redundant project, task, document, and version modules
- Identified critical issue with async project creation losing context
- Updated CLAUDE.md with project instructions
This commit captures the current state before refactoring to split
consolidated tools into separate operations for better clarity and
to solve the async project creation context issue.
- Use python-jose (already in dependencies) instead of PyJWT for JWT decoding
- Make unknown Supabase key roles fail fast per alpha principles
- Skip all JWT validations (not just signature) when checking role
- Update tests to expect failure for unknown roles
Fixes:
- No need to add PyJWT dependency - python-jose provides JWT functionality
- Unknown key types now raise ConfigurationError instead of warning
- JWT decode properly skips all validations to only check role claim
- Add backend validation to detect and warn about anon vs service keys
- Prevent startup with incorrect Supabase key configuration
- Consolidate frontend state management following KISS principles
- Remove duplicate state tracking and sessionStorage polling
- Add clear error display when backend fails to start
- Improve .env.example documentation with detailed key selection guide
- Add comprehensive test coverage for validation logic
- Remove unused test results checking to eliminate 404 errors
The implementation now warns users about key misconfiguration while
maintaining backward compatibility. Frontend state is simplified with
MainLayout as the single source of truth for backend status.
- Set hpack and httpcore loggers to WARNING level
- These libraries produce excessive protocol-level debug output
- Improves signal-to-noise ratio in logs
- Read LOG_LEVEL from environment with INFO as default
- Use getattr to safely convert string to logging level constant
- Supports DEBUG, INFO, WARNING, ERROR, CRITICAL levels
- Falls back to INFO if invalid level specified
This minimal change allows debug logs to appear when LOG_LEVEL=DEBUG
is set in the .env file, fixing the issue where debug messages were
being filtered out.
- Add is_binary_file() method to URLHandler to detect 40+ binary extensions
- Update RecursiveCrawlStrategy to filter binary URLs before crawl queue
- Add comprehensive unit tests for binary file detection
- Prevents net::ERR_ABORTED errors when crawler encounters ZIP, PDF, etc.
This fixes the issue where the crawler was treating binary file URLs
(like .zip downloads) as navigable web pages, causing errors in crawl4ai.
- Implement fail-fast error handling for configuration errors
- Distinguish between critical config errors (fail) and network issues (use defaults)
- Add detailed error logging with stack traces for debugging
- Document new crawler settings in .env.example
- Add inline comments explaining safe defaults
Critical configuration errors (ValueError, KeyError, TypeError) now fail fast
as per alpha principles, while transient errors still fall back to safe defaults
with prominent error logging.
Consolidate concurrent crawling limits to use single database setting
instead of hardcoded special case for documentation sites.
Changes:
- Remove hardcoded 20 concurrent limit for documentation sites
- Let strategies use CRAWL_MAX_CONCURRENT from database (default: 10)
- Apply consistent concurrency across all site types
- Improve code formatting and consistency
This fixes Playwright browser crashes caused by excessive concurrent
pages on documentation sites and provides single configuration point
for tuning crawler performance.
