Orchestrator Core Engine

The central stateful service that owns conversational sessions, decomposes requests, dispatches work, and synthesizes results. The Orchestrator is the only component with access to full member context and the tool cascade.

---

Epic: Session State Management ✓

Plan: Build a two-tier state system: fast in-memory working state for the current turn, and persistent storage for cross-turn continuity. In-memory state is a cache, not the source of truth. The Orchestrator is designed to reconstruct itself from external state on every turn.

Architectural Context: The Orchestrator is the only stateful service in the system. It owns a user's conversational session and is the sole gate for member and trip data. When an Orchestrator instance dies, the next request hits a new instance which reconstructs state from DB/Redis. This design enables horizontal scaling via session sharding.

Build Brief · Build Summary

Tasks

• Design session state schema: current turn context (user message, attached context), pending jobs (job_id → agent_type mapping), intermediate results (partial results as they arrive)
• Implement in-memory working copy: Map<session_id, SessionState> with LRU eviction
• Define Postgres schema: members table (id, name, tier, preferences JSONB), trips table (id, member_id, destination, dates, status, budget), conversation_history table (id, session_id, role, content, timestamp)
• Set up Redis session store: session_id → serialized state with TTL (30 min)
• Implement session write-through: every state change writes to Redis immediately, batch writes to Postgres every 5 seconds
• Build session reconstruction: on startup or new request, load from Redis (fast) or Postgres (fallback)
• Implement session TTL: expire inactive sessions after 30 minutes
• Build session cleanup: periodic job to archive old conversations to Postgres
• Implement sticky routing on session_id: consistent hash ring maps session → Orchestrator instance
• Build session migration: on instance failure, reassign session to healthy instance
• Add session metrics: active sessions per instance, reconstruction rate, TTL expiration rate

---

Epic: Tool Cascade (Internal Context Loading)

Plan: Build a three-tier tool cascade that progressively narrows context from broad member data to specific curated recommendations. Each tier feeds the next, producing a focused brief for the Research Agents. This is the security and privacy boundary — member data is gated here, workers only receive pre-narrowed briefs.

Architectural Context: The tool cascade is exclusively on the Orchestrator; workers cannot access it. Tier 1 loads member and trip context, Tier 2 loads destination and situational context, Tier 3 loads curated lists and partner inventory. The cascade runs synchronously before dispatch, producing a narrow brief that constrains worker output.

Tasks

• Define tool interface contract: input schema (tool_name, params), output schema (data, metadata), error handling (skip tier, proceed with partial context)
• Implement Tier 1 tools:
• get_member_profile: identity, tier, preferences, loyalty, dietary, accessibility
• get_trip: dates, destination(s), companions, budget, itinerary state
• get_trip_history: past trips, prior reservations, feedback, no-go lists
• get_member_preferences: cuisine likes/dislikes, price comfort zone, party-size patterns
• Implement Tier 2 tools:
• get_destination_info: curated city/region info, neighborhoods, transit, cultural notes
• get_seasonal_context: what's in season, local events, holidays affecting availability
• get_currency_and_payment: FX rates, payment norms, tipping expectations
• Implement Tier 3 tools:
• get_curated_dining_list: concierge-vetted restaurants with tags (signature, hidden gem, splurge, casual)
• get_blacklist: members/trips flagged for known issues
• get_partner_inventory: real-time availability from partner systems (OpenTable, SevenRooms)
• Build cascade orchestrator: chain Tier 1 → Tier 2 → Tier 3, collect outputs
• Implement brief generation: combine cascade outputs into a structured brief for dispatch
• Add caching: cache Tier 1 (member data) for session duration, Tier 2 (destination) for 24 hours, Tier 3 (curated) for 1 hour
• Build cascade error handling: if a tier fails, log warning and proceed with partial context
• Add cascade metrics: call count per tier, latency per tier, error rate per tier

---

Epic: LLM Orchestration Loop

Plan: Build the decompose-dispatch-synthesize loop that takes a user request, breaks it into subtasks, sends them to Research Agents via the job queue, and combines results into a coherent response. Include re-planning logic for when worker outputs conflict.

Architectural Context: The LLM loop runs on the Orchestrator and follows a plan-then-execute pattern. Decompose: break user request into scoped tasks with agent_type, task description, and constraints. Dispatch: enqueue one job per specialist to Redis (LPUSH jobs:{agent_type}). Synthesize: aggregate results as they arrive, with per-job timeout. If results conflict (e.g., all options exceed budget), the Orchestrator re-plans rather than forcing a bad answer.

Tasks

• Define decompose prompt: system prompt that breaks user requests into subtasks with agent_type, task, constraints, result_format
• Implement decompose step: LLM call that returns subtask list as structured JSON
• Build dispatch logic: for each subtask, create job payload and enqueue to Redis (LPUSH jobs:{agent_type} {payload})
• Implement parallel dispatch: independent jobs go in parallel, dependent jobs go sequentially
• Build synthesis step: LLM call that combines agent results into coherent user response
• Implement re-planning logic:
• Detect conflicts (e.g., all results exceed budget, contradictory recommendations)
• Tighten constraints and re-dispatch (e.g., lower max_price, add neighborhood restrictions)
• Surface trade-off to user if no good options exist
• Implement per-job timeout: configurable per agent type (e.g., 30 seconds for dining, 60 seconds for experiences)
• Build partial result support: if some jobs complete and others timeout, synthesize with available results
• Implement streaming response: stream partial results to user as they arrive
• Build conversation history integration: include previous turns in decompose context
• Add LLM metrics: token count per call, latency per call, cost per call, decompose/synthesize ratio