Research Agent Framework

The stateless specialist workers that handle domain-specific research tasks. Each agent is domain-pure, receives a pre-narrowed brief from the Orchestrator, and returns a structured result. Workers cannot ask clarifying questions, access member data, or communicate with each other.

---

Epic: Worker Process Lifecycle

Plan: Build a robust worker process that pulls jobs from Redis via blocking pop, executes them with LLM + domain tools, and writes structured results back. Handle failures gracefully with visibility timeouts, retries, and dead-letter queues.

Architectural Context: Workers are the only components that pull from Redis (via BRPOP). They write results to result:{job_id} and notify the Orchestrator via pub/sub. Workers are stateless and can be killed/replaced at any time without data loss. The visibility timeout mechanism ensures jobs are re-delivered if a worker dies mid-execution.

Tasks

• Define job payload schema matching architecture spec (job_id, session_id, orchestrator_id, agent_type, task, constraints, curated_shortlist, result_format, ttl_seconds)
• Implement Redis connection pool with health checks and reconnection logic
• Implement blocking pop from Redis with configurable timeout (BRPOP jobs:{agent_type} timeout)
• Build job deserialization and schema validation
• Build job execution pipeline: parse payload → select tools → LLM call → tool execution → structured result
• Implement result write-back to result:{job_id} in Redis with TTL matching job TTL
• Build pub/sub notification on result completion (PUBLISH agent:{type}:completed {job_id})
• Implement ack flow: delete job from queue on successful completion
• Implement nack flow: move to dead-letter queue on failure (dead:{agent_type})
• Add job visibility timeout: if worker doesn't ack within timeout, job returns to queue
• Implement graceful shutdown: finish current job, drain, then exit
• Build health check endpoint (/healthz) that verifies Redis connectivity
• Build readiness check endpoint (/readyz) that verifies all dependencies

---

Epic: Agent Abstraction Layer

Plan: Define a standard contract for all Research Agents so new verticals can be added by implementing the interface, not rewriting infrastructure. Each agent type is an independent service with its own worker pool, LLM config, tools, and scaling.

Architectural Context: The Orchestrator treats every Research Agent as opaque. It knows only the agent's name, expected input shape (job payload), and expected output shape (result format). This means the Dining agent could be Python/Claude while the Experiences agent is Node/fine-tuned small model — they all look the same to the queue. Workers do NOT have member profile data, trip context, other workers, or the Orchestrator's tool cascade.

Tasks

• Define input contract: job payload with task (string), constraints (object), curated_shortlist (string[]), context (object)
• Define output contract: result with status (success/partial/failed), data (structured), confidence (0-1), sources (string[]), gaps (string[]), metadata (object)
• Build worker process pool manager: spawn N workers per agent type, monitor health, restart on failure
• Implement agent registration: agents register name, input schema, output schema, capabilities at startup
• Implement per-agent LLM configuration: model selection (GPT-4, Claude, etc.), temperature, max tokens, system prompt
• Implement per-agent tool configuration: which tools each agent can access (APIs, databases, search)
• Build agent discovery: Orchestrator queries available agents and their capabilities
• Add agent health monitoring: heartbeat mechanism, automatic restart on failure
• Build agent versioning: support multiple versions of same agent for gradual rollouts
• Implement agent isolation: each agent type runs in its own process/container with its own resource limits

---

Epic: Dining Research Agent (Vertical 1)

Plan: Build the first Research Agent for restaurant search. Integrate multiple data sources (OpenTable, Resy, Yelp, Tock), enforce a 5-15 second latency budget, and return structured results with availability windows, price estimates, and ratings.

Architectural Context: The Dining Agent is the canonical example of a Research Agent. It receives a brief from the Orchestrator with cuisines, neighborhoods, budget, party size, dietary restrictions, and a curated shortlist. It queries multiple restaurant APIs in parallel, deduplicates results, and returns a RestaurantOption[] with availability. The agent cannot access member data directly — it only sees the narrowed brief.

Tasks

• Set up Dining Agent container with Node.js/Python runtime
• Integrate OpenTable API: search restaurants, check availability, get pricing
• Integrate Resy API: search restaurants, check availability
• Integrate Yelp API: get reviews, ratings, price tier, cuisine tags
• Integrate Tock API: search experiences, prix fixe menus
• Build source aggregation: query multiple APIs in parallel, merge results
• Implement deduplication: match restaurants across sources by name + location + cuisine
• Define RestaurantOption output schema: name, cuisine, price_tier, neighborhoods, hours, rating, review_count, availability_windows, source_urls, notes
• Build dining-specific system prompt for LLM (restaurant search, availability checking, recommendation)
• Implement 5-15 second latency budget: timeout on slow sources, return partial results
• Build caching layer: cache frequently searched restaurants (Redis, TTL 1 hour)
• Implement rate limiting per API source (respect API quotas)
• Add source health monitoring: track success rate per API, fallback to other sources on failure
• Build output validation: ensure result matches RestaurantOption[] schema before writing to Redis

---

Epic: Experiences Research Agent

Plan: Build a Research Agent for bookable experiences (tours, private guides, excursions, classes). Integrate Viator, GetYourGuide, and Klook. Return structured results with duration, pricing, ratings, and availability.

Architectural Context: Experiences are a core part of luxury travel planning. The Experiences Agent receives a brief with destination, activity type, date range, party size, and interests. It queries experience APIs and returns options that match the member's profile.

Tasks

• Set up Experiences Agent container
• Integrate Viator API: search experiences, check availability, get pricing
• Integrate GetYourGuide API: search experiences, check availability
• Integrate Klook API: search experiences (Asia-Pacific focus)
• Handle partner API authentication requirements (OAuth, API keys)
• Define ExperienceOption output schema: name, type, duration, price, rating, review_count, location, availability, booking_url, notes
• Build experiences-specific system prompt
• Implement parallel API querying with timeout
• Build deduplication across experience sources
• Implement caching for frequently searched experiences
• Add source health monitoring and fallback logic

---

Epic: Transportation Research Agent

Plan: Build a Research Agent for ground transportation (black car services, private drivers, car rentals, helicopter transfers). Integrate Uber, Lyft, Blacklane, and car rental APIs. This is the highest complexity vertical due to OAuth requirements and real-time pricing.

Architectural Context: Transportation research is time-sensitive and location-dependent. The agent receives a brief with pickup/dropoff locations, vehicle type preference, timing, and budget. It must return real-time pricing and availability.

Tasks

• Set up Transportation Agent container
• Integrate Uber API: search vehicle types, get pricing estimates
• Integrate Lyft API: search vehicle types, get pricing estimates
• Integrate Blacklane API: search black car services, get pricing
• Build car rental API integrations (Hertz, Avis, Enterprise)
• Handle OAuth requirements for partner APIs
• Define TransportOption output schema: provider, vehicle_type, base_price, estimated_total, pickup_time, coverage_area, booking_url, notes
• Build transportation-specific system prompt
• Implement real-time pricing queries with timeout
• Build location-based search (pickup/dropoff geocoding)
• Add provider health monitoring and fallback logic
• Implement caching for static pricing (vehicle types, base rates)

---

Epic: Flights Research Agent

Plan: Build a Research Agent for flight search. Integrate Amadeus, Skyscanner, and Google Flights. Return structured results with routes, schedules, pricing, and cabin class options.

Architectural Context: Flights are often the first thing booked for a trip. The agent receives a brief with origin, destination, dates, party size, cabin class preference, and loyalty programs. It must search multiple sources and return options with pricing.

Tasks

• Set up Flights Agent container
• Integrate Amadeus API: search flights, get pricing, check availability
• Integrate Skyscanner API: search flights, get pricing
• Build Google Flights crawler (if API not available)
• Handle GDS-dependent partner relationships
• Define FlightOption output schema: airline, route, departure, arrival, duration, stops, cabin_class, price, booking_url, baggage_allowance, notes
• Build flights-specific system prompt
• Implement parallel API querying with timeout
• Build fare comparison across sources
• Implement caching for flight searches (short TTL, 15-30 minutes)
• Add provider health monitoring and fallback logic

---

Epic: DMC Research Agent

Plan: Build a Research Agent for Destination Management Company matching. Integrate the DMC Finder database (10,000+ DMCs, 120+ countries) and Traveller Made network. Return DMC options with specialties, coverage, and contact info.

Architectural Context: DMCs are local ground operators who handle on-the-ground logistics. The agent receives a brief with destination, trip type, group size, and special requirements. It matches against the DMC database and returns options.

Tasks

• Set up DMC Agent container
• Integrate DMC Finder database (10,000+ DMCs)
• Integrate Traveller Made network
• Integrate Virtuoso advisor network (20,000+ luxury advisors)
• Define DMCOption output schema: name, coverage_countries, specialties, languages, certifications, contact_info, rating, notes
• Build DMC-specific system prompt
• Implement destination-based search with country/region matching
• Build specialty matching (luxury, adventure, family, etc.)
• Implement caching for DMC data (long TTL, 24 hours)
• Add data freshness monitoring (re-crawl periodically)