Web Intelligence MCP Server

Central hub for web crawling, scraping, research, and content extraction capabilities in Nexus.

Purpose

Provide AI agents and Claude with robust web research capabilities: - URL crawling and content extraction - HTML to markdown conversion - Link discovery and management - Integration with Links environment for bookmarking - Content staging for processing pipelines

Status: Building (Operation Webweaver)

Ports

• Vault: 6670 (password protected)
• Operational: 6671 (read replica)

Key Technologies

• spider_rs (Python bindings for Rust spider crate) - already installed
• html-to-markdown conversion
• Links MCP integration
• Temp/Staging environment integration