Deployment

The platform runs as a set of containerized services. The spike runs locally; production needs a different shape.

Components

Component	Purpose	Spike	Production
Crawler CLI	Run a crawl	Local	Containerized job
Scheduler	Trigger crawls on a schedule	None	Cron / k8s CronJob
SQLite store	Primary data store	Local file	Shared volume or managed
MCP server	Serve data to agents	Stdio	HTTP/SSE
Backup job	Snapshot SQLite	None	Daily to object storage

Container Layout

# Dockerfile
FROM python:3.12-slim

# Install uv
COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv

# Copy project
WORKDIR /app
COPY pyproject.toml uv.lock ./
RUN uv sync --frozen

COPY . .

# Default: serve the MCP server
CMD ["uv", "run", "python", "crawler.py", "serve"]

The same image runs as the MCP server, the CLI, and the scheduler. Different CMD per use case.

Volumes

The SQLite database lives on a persistent volume. Backups write to object storage (S3, GCS, etc.).

# docker-compose.yml (development)
services:
  intelligence:
    build: .
    volumes:
      - ./data:/app/data
    command: uv run python crawler.py serve

Scheduler

A k8s CronJob (or equivalent) runs the CLI on each vertical's schedule:

# k8s cronjob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: intelligence-dmc-weekly
spec:
  schedule: "0 3 * * 0"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: intelligence
            image: intelligence:latest
            command: ["uv", "run", "python", "crawler.py", "crawl", "--source", "all"]
            volumeMounts:
            - name: data
              mountPath: /app/data
          restartPolicy: OnFailure
  volume:
    - name: data
      persistentVolumeClaim:
        claimName: intelligence-data

The same template, one CronJob per vertical.

MCP Server Transport

The spike uses stdio. Production needs HTTP so remote agents can connect.

# Production: run with HTTP transport
if __name__ == "__main__":
    mcp.run(transport="http", host="0.0.0.0", port=8000)

The MCP server runs as a long-lived service behind a load balancer. Agents connect over HTTP/SSE.

Configuration

All config via environment variables:

INTELLIGENCE_DATA_DIR=/app/data
INTELLIGENCE_LOG_LEVEL=info
INTELLIGENCE_MCP_TRANSPORT=http
INTELLIGENCE_MCP_PORT=8000
INTELLIGENCE_SCHEDULE_CONFIG=/app/config/schedule.yaml

No secrets needed (the platform crawls public sources, not authenticated APIs). Future verticals with API access (Experiences, Hotels) will need secrets management per the Integrations Architecture.

Health & Readiness

• /healthz — process is up
• /readyz — SQLite is reachable, last crawl wasn't catastrophically broken
• /metrics — Prometheus

CI/CD

1. Lint (ruff)
2. Type check (mypy)
3. Unit tests
4. Integration tests (against a sandbox source, where possible)
5. Build image
6. Push to registry
7. Deploy to staging
8. Smoke test against staging
9. Manual approval for prod
10. Deploy to prod

Cloud

Container-agnostic. Any container host works (ECS, Cloud Run, k8s, Fly). The persistent volume is the only stateful piece.

Local Development

git clone ...
cd intelligence
uv sync
uv run python crawler.py crawl --source inside_travel -n 5 --no-deep-crawl
uv run python crawler.py search --destination "Greece"
uv run python crawler.py serve

Local dev uses the same SQLite file. The data/ directory is gitignored.