DRAFT Agentic Design Patterns - Routing

Build intelligent routing systems that dynamically direct queries to specialized agents, moving beyond linear workflows to create adaptive, context-aware AI applications.

Mental Model: Traffic Controller for AI Agents

Think of the routing pattern like an intelligent traffic controller at a busy intersection. Just as a traffic controller analyzes incoming vehicles (size, destination, urgency) and directs them to the optimal lane, a routing agent examines incoming requests (intent, complexity, context) and directs them to the most suitable processing agent. In Next.js 15, this means your API routes act as smart dispatchers, using Langchain/Langraph to evaluate requests and route them to specialized handlers - much like how Vercel's Edge Middleware routes requests based on headers, but with AI-powered decision making instead of static rules.

Basic Example: Intent-Based Customer Support Router

1. Create the Router Agent

// lib/agents/router.ts
import { ChatGoogleGenerativeAI } from '@langchain/google-genai';
import { PromptTemplate } from '@langchain/core/prompts';
import { z } from 'zod';
import { memoize } from 'es-toolkit';

const RouteSchema = z.object({
  route: z.enum(['technical', 'billing', 'general']),
  confidence: z.number(),
  reasoning: z.string()
});

export class CustomerSupportRouter {
  private model: ChatGoogleGenerativeAI;
  
  constructor() {
    this.model = new ChatGoogleGenerativeAI({
      model: 'gemini-2.5-flash',
      temperature: 0,
    });
  }
  
  // Memoize for identical queries within 5 minutes
  route = memoize(
    async (query: string) => {
      const prompt = PromptTemplate.fromTemplate(`
        Analyze this customer query and route it to the appropriate department.
        
        Query: {query}
        
        Available routes:
        - technical: For product issues, bugs, technical questions
        - billing: For payment, subscription, refund issues  
        - general: For all other inquiries
        
        Respond with JSON: {{ "route": "...", "confidence": 0.0-1.0, "reasoning": "..." }}
      `);
      
      const formatted = await prompt.format({ query });
      const response = await this.model.invoke(formatted);
      
      return RouteSchema.parse(JSON.parse(response.content as string));
    },
    { ttl: 300000 } // 5 minute cache
  );
}

Creates a router that analyzes customer queries using Gemini Flash and returns structured routing decisions with confidence scores.

2. Implement Specialized Agents

// lib/agents/specialized.ts
import { ChatGoogleGenerativeAI } from '@langchain/google-genai';

export class TechnicalSupportAgent {
  private model = new ChatGoogleGenerativeAI({
    model: 'gemini-2.5-pro',
    temperature: 0.3,
  });
  
  async handle(query: string) {
    const response = await this.model.invoke(
      `You are a technical support specialist. Help with: ${query}`
    );
    return { type: 'technical', response: response.content };
  }
}

export class BillingSupportAgent {
  private model = new ChatGoogleGenerativeAI({
    model: 'gemini-2.5-flash',
    temperature: 0,
  });
  
  async handle(query: string) {
    const response = await this.model.invoke(
      `You are a billing specialist. Resolve: ${query}`
    );
    return { type: 'billing', response: response.content };
  }
}

export class GeneralSupportAgent {
  private model = new ChatGoogleGenerativeAI({
    model: 'gemini-2.5-flash',
    temperature: 0.5,
  });
  
  async handle(query: string) {
    const response = await this.model.invoke(
      `You are a helpful assistant. Answer: ${query}`
    );
    return { type: 'general', response: response.content };
  }
}

Each specialized agent uses different models and temperatures optimized for their specific domain.

3. Create API Route

// app/api/support/route.ts
export const runtime = 'nodejs';
export const maxDuration = 60;

import { CustomerSupportRouter } from '@/lib/agents/router';
import { 
  TechnicalSupportAgent, 
  BillingSupportAgent, 
  GeneralSupportAgent 
} from '@/lib/agents/specialized';
import { NextResponse } from 'next/server';

const router = new CustomerSupportRouter();
const agents = {
  technical: new TechnicalSupportAgent(),
  billing: new BillingSupportAgent(),
  general: new GeneralSupportAgent(),
};

export async function POST(req: Request) {
  try {
    const { query } = await req.json();
    
    // Route the query
    const routing = await router.route(query);
    
    // Execute with selected agent
    const agent = agents[routing.route];
    const result = await agent.handle(query);
    
    return NextResponse.json({
      ...result,
      routing: {
        selected: routing.route,
        confidence: routing.confidence,
        reasoning: routing.reasoning
      }
    });
  } catch (error) {
    console.error('Routing error:', error);
    return NextResponse.json(
      { error: 'Failed to process request' },
      { status: 500 }
    );
  }
}

API endpoint that routes queries to specialized agents based on the router's decision.

4. Frontend Component with TanStack Query

// components/SupportChat.tsx
'use client';

import { useState } from 'react';
import { useMutation } from '@tanstack/react-query';

export default function SupportChat() {
  const [message, setMessage] = useState('');
  
  const submitQuery = useMutation({
    mutationFn: async (query: string) => {
      const res = await fetch('/api/support', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ query }),
      });
      
      if (!res.ok) throw new Error('Request failed');
      return res.json();
    },
  });

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (message.trim()) {
      submitQuery.mutate(message);
      setMessage('');
    }
  };

  return (
    <div className="card w-full bg-base-100 shadow-xl">
      <div className="card-body">
        <h2 className="card-title">Customer Support</h2>
        
        <form onSubmit={handleSubmit}>
          <input
            type="text"
            className="input input-bordered w-full"
            placeholder="How can we help?"
            value={message}
            onChange={(e) => setMessage(e.target.value)}
            disabled={submitQuery.isPending}
          />
          
          <button
            type="submit"
            className="btn btn-primary mt-4"
            disabled={submitQuery.isPending || !message.trim()}
          >
            {submitQuery.isPending ? (
              <span className="loading loading-spinner"></span>
            ) : 'Send'}
          </button>
        </form>
        
        {submitQuery.data && (
          <div className="alert mt-4">
            <div>
              <div className="badge badge-secondary">
                {submitQuery.data.routing.selected}
              </div>
              <p className="mt-2">{submitQuery.data.response}</p>
            </div>
          </div>
        )}
      </div>
    </div>
  );
}

React component that displays routing decisions and agent responses using TanStack Query for state management.

Advanced Example: Multi-Stage Document Processing Pipeline

1. Install Additional Dependencies

npm install @langchain/langgraph @upstash/redis pdf-parse

Adds Langraph for stateful workflows and Upstash Redis for distributed state management.

2. Define Routing State Machine with Langraph

// lib/workflows/document-router.ts
import { StateGraph, Annotation } from '@langchain/langgraph';
import { ChatGoogleGenerativeAI } from '@langchain/google-genai';
import { BaseMessage } from '@langchain/core/messages';
import { groupBy, chunk } from 'es-toolkit';

const DocumentState = Annotation.Root({
  documentId: Annotation<string>(),
  content: Annotation<string>(),
  documentType: Annotation<string>(),
  confidence: Annotation<number>(),
  processingStage: Annotation<string>(),
  extractedData: Annotation<Record<string, any>>(),
  errors: Annotation<string[]>(),
});

export function createDocumentRoutingWorkflow() {
  const model = new ChatGoogleGenerativeAI({
    model: 'gemini-2.5-pro',
    temperature: 0,
  });
  
  const workflow = new StateGraph(DocumentState)
    // Classification node
    .addNode('classify', async (state) => {
      const response = await model.invoke(
        `Classify this document type: ${state.content.substring(0, 1000)}`
      );
      
      // Parse classification result
      const type = extractDocumentType(response.content as string);
      const confidence = calculateConfidence(response.content as string);
      
      return {
        documentType: type,
        confidence: confidence,
        processingStage: 'classified',
      };
    })
    
    // Invoice processor
    .addNode('process_invoice', async (state) => {
      const invoiceData = await extractInvoiceData(state.content);
      return {
        extractedData: invoiceData,
        processingStage: 'completed',
      };
    })
    
    // Contract processor
    .addNode('process_contract', async (state) => {
      const contractData = await extractContractData(state.content);
      return {
        extractedData: contractData,
        processingStage: 'completed',
      };
    })
    
    // General processor
    .addNode('process_general', async (state) => {
      const generalData = await extractGeneralData(state.content);
      return {
        extractedData: generalData,
        processingStage: 'completed',
      };
    })
    
    // Human review node
    .addNode('human_review', async (state) => {
      await notifyHumanReviewer(state);
      return {
        processingStage: 'pending_review',
      };
    });
  
  // Add conditional routing
  workflow.addConditionalEdges('classify', (state) => {
    if (state.confidence < 0.7) {
      return 'human_review';
    }
    
    switch (state.documentType) {
      case 'invoice':
        return 'process_invoice';
      case 'contract':
        return 'process_contract';
      default:
        return 'process_general';
    }
  });
  
  // Set entry point
  workflow.setEntryPoint('classify');
  
  return workflow.compile();
}

Langraph workflow that classifies documents and routes them to specialized processors based on confidence.

3. Implement Streaming API with State Management

// app/api/documents/process/route.ts
export const runtime = 'nodejs';
export const maxDuration = 300;

import { createDocumentRoutingWorkflow } from '@/lib/workflows/document-router';
import { Redis } from '@upstash/redis';

const redis = Redis.fromEnv();
const workflow = createDocumentRoutingWorkflow();

export async function POST(req: Request) {
  const { documentId, content } = await req.json();
  
  const encoder = new TextEncoder();
  const stream = new TransformStream();
  const writer = stream.writable.getWriter();
  
  // Process in background
  (async () => {
    try {
      // Initialize state
      const initialState = {
        documentId,
        content,
        documentType: '',
        confidence: 0,
        processingStage: 'pending',
        extractedData: {},
        errors: [],
      };
      
      // Store initial state in Redis
      await redis.set(
        `doc:${documentId}:state`,
        JSON.stringify(initialState),
        { ex: 3600 } // 1 hour TTL
      );
      
      // Stream workflow events
      const eventStream = await workflow.stream(initialState);
      
      for await (const event of eventStream) {
        const state = event[Object.keys(event)[0]];
        
        // Update Redis state
        await redis.set(
          `doc:${documentId}:state`,
          JSON.stringify(state),
          { ex: 3600 }
        );
        
        // Stream to client
        await writer.write(
          encoder.encode(`data: ${JSON.stringify({
            stage: state.processingStage,
            type: state.documentType,
            confidence: state.confidence,
            hasData: !!state.extractedData,
          })}\n\n`)
        );
        
        if (state.processingStage === 'completed' || 
            state.processingStage === 'pending_review') {
          break;
        }
      }
      
      await writer.write(
        encoder.encode(`data: ${JSON.stringify({ done: true })}\n\n`)
      );
    } catch (error) {
      console.error('Workflow error:', error);
      await writer.write(
        encoder.encode(`data: ${JSON.stringify({ 
          error: 'Processing failed' 
        })}\n\n`)
      );
    } finally {
      await writer.close();
    }
  })();
  
  return new Response(stream.readable, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive',
    },
  });
}

Streaming API that processes documents through the routing workflow and stores state in Redis.

4. Create Hook for Streaming Updates

// hooks/useDocumentProcessing.ts
import { useState, useCallback } from 'react';
import { useMutation } from '@tanstack/react-query';

interface ProcessingUpdate {
  stage?: string;
  type?: string;
  confidence?: number;
  done?: boolean;
  error?: string;
}

export function useDocumentProcessing() {
  const [updates, setUpdates] = useState<ProcessingUpdate[]>([]);
  const [isProcessing, setIsProcessing] = useState(false);
  
  const processDocument = useCallback(async (
    documentId: string,
    content: string
  ) => {
    setIsProcessing(true);
    setUpdates([]);
    
    try {
      const response = await fetch('/api/documents/process', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ documentId, content }),
      });
      
      if (!response.ok) throw new Error('Processing failed');
      
      const reader = response.body?.getReader();
      const decoder = new TextDecoder();
      
      while (reader) {
        const { done, value } = await reader.read();
        if (done) break;
        
        const chunk = decoder.decode(value);
        const lines = chunk.split('\n');
        
        for (const line of lines) {
          if (line.startsWith('data: ')) {
            try {
              const update = JSON.parse(line.slice(6));
              setUpdates(prev => [...prev, update]);
              
              if (update.done || update.error) {
                setIsProcessing(false);
                return update;
              }
            } catch {}
          }
        }
      }
    } catch (error) {
      setIsProcessing(false);
      throw error;
    }
  }, []);
  
  return {
    processDocument,
    updates,
    isProcessing,
    currentStage: updates[updates.length - 1]?.stage,
    documentType: updates[updates.length - 1]?.type,
    confidence: updates[updates.length - 1]?.confidence,
  };
}

Custom hook that manages document processing state and streaming updates.

5. Build Document Processing UI

// components/DocumentProcessor.tsx
'use client';

import { useDocumentProcessing } from '@/hooks/useDocumentProcessing';
import { useState } from 'react';

export default function DocumentProcessor() {
  const [file, setFile] = useState<File | null>(null);
  const { 
    processDocument, 
    updates, 
    isProcessing,
    currentStage,
    documentType,
    confidence
  } = useDocumentProcessing();
  
  const handleFileSelect = (e: React.ChangeEvent<HTMLInputElement>) => {
    const selectedFile = e.target.files?.[0];
    if (selectedFile) {
      setFile(selectedFile);
    }
  };
  
  const handleProcess = async () => {
    if (!file) return;
    
    const content = await file.text();
    await processDocument(file.name, content);
  };
  
  return (
    <div className="card w-full bg-base-100 shadow-xl">
      <div className="card-body">
        <h2 className="card-title">Document Processor</h2>
        
        <div className="form-control">
          <label className="label">
            <span className="label-text">Select document</span>
          </label>
          <input
            type="file"
            className="file-input file-input-bordered"
            onChange={handleFileSelect}
            disabled={isProcessing}
            accept=".pdf,.txt,.docx"
          />
        </div>
        
        <button
          className="btn btn-primary"
          onClick={handleProcess}
          disabled={!file || isProcessing}
        >
          {isProcessing ? (
            <>
              <span className="loading loading-spinner"></span>
              Processing...
            </>
          ) : 'Process Document'}
        </button>
        
        {updates.length > 0 && (
          <div className="mt-6">
            <h3 className="font-semibold mb-4">Processing Steps</h3>
            
            <ul className="steps steps-vertical">
              {['classify', 'process', 'complete'].map((step) => (
                <li
                  key={step}
                  className={`step ${
                    updates.some(u => u.stage === step) ? 'step-primary' : ''
                  }`}
                >
                  <div className="text-left">
                    <div className="font-medium capitalize">{step}</div>
                    {step === 'classify' && documentType && (
                      <div className="text-sm opacity-70">
                        Type: {documentType} ({Math.round((confidence || 0) * 100)}%)
                      </div>
                    )}
                  </div>
                </li>
              ))}
            </ul>
            
            {currentStage === 'pending_review' && (
              <div className="alert alert-warning mt-4">
                <span>Document sent for human review due to low confidence</span>
              </div>
            )}
            
            {currentStage === 'completed' && (
              <div className="alert alert-success mt-4">
                <span>Processing complete! Document type: {documentType}</span>
              </div>
            )}
          </div>
        )}
      </div>
    </div>
  );
}

UI component that displays real-time routing decisions and processing progress.

6. Add Semantic Routing with Embeddings

// lib/routing/semantic-router.ts
import { GoogleGenerativeAIEmbeddings } from '@langchain/google-genai';
import { cosineSimilarity } from '@langchain/core/utils/math';
import { memoize } from 'es-toolkit';

interface Route {
  name: string;
  description: string;
  examples: string[];
  handler: string;
}

export class SemanticRouter {
  private embeddings: GoogleGenerativeAIEmbeddings;
  private routeEmbeddings: Map<string, number[]> = new Map();
  
  constructor(private routes: Route[]) {
    this.embeddings = new GoogleGenerativeAIEmbeddings({
      model: 'embedding-001',
    });
    this.initialize();
  }
  
  private async initialize() {
    // Generate embeddings for each route
    for (const route of this.routes) {
      const description = `${route.description} ${route.examples.join(' ')}`;
      const embedding = await this.embeddings.embedQuery(description);
      this.routeEmbeddings.set(route.name, embedding);
    }
  }
  
  // Memoize for performance
  findBestRoute = memoize(
    async (query: string): Promise<{ route: Route; similarity: number }> => {
      const queryEmbedding = await this.embeddings.embedQuery(query);
      
      let bestRoute: Route | null = null;
      let bestSimilarity = -1;
      
      for (const route of this.routes) {
        const routeEmbedding = this.routeEmbeddings.get(route.name)!;
        const similarity = cosineSimilarity(
          [queryEmbedding],
          [routeEmbedding]
        )[0][0];
        
        if (similarity > bestSimilarity) {
          bestSimilarity = similarity;
          bestRoute = route;
        }
      }
      
      return {
        route: bestRoute!,
        similarity: bestSimilarity,
      };
    },
    { ttl: 60000 } // 1 minute cache
  );
}

Semantic router that uses embeddings to find the most similar route based on meaning rather than keywords.

7. Implement Adaptive Routing with Learning

// lib/routing/adaptive-router.ts
import { Redis } from '@upstash/redis';
import { pick, omit } from 'es-toolkit';

interface RoutingDecision {
  query: string;
  selectedRoute: string;
  confidence: number;
  timestamp: number;
  outcome?: 'success' | 'failure';
}

export class AdaptiveRouter {
  private redis = Redis.fromEnv();
  
  async recordDecision(decision: RoutingDecision) {
    const key = `routing:history`;
    await this.redis.lpush(key, JSON.stringify(decision));
    await this.redis.ltrim(key, 0, 999); // Keep last 1000 decisions
  }
  
  async updateOutcome(
    query: string, 
    route: string, 
    outcome: 'success' | 'failure'
  ) {
    // Find and update the decision
    const history = await this.redis.lrange('routing:history', 0, 99);
    
    for (let i = 0; i < history.length; i++) {
      const decision = JSON.parse(history[i] as string) as RoutingDecision;
      
      if (decision.query === query && decision.selectedRoute === route) {
        decision.outcome = outcome;
        await this.redis.lset('routing:history', i, JSON.stringify(decision));
        break;
      }
    }
    
    // Update route statistics
    const statKey = `routing:stats:${route}`;
    await this.redis.hincrby(statKey, outcome, 1);
  }
  
  async getRoutePerformance(route: string) {
    const stats = await this.redis.hgetall(`routing:stats:${route}`);
    const success = parseInt(stats.success || '0');
    const failure = parseInt(stats.failure || '0');
    const total = success + failure;
    
    return {
      successRate: total > 0 ? success / total : 0.5,
      totalRequests: total,
    };
  }
  
  async selectBestRoute(candidates: string[]): Promise<string> {
    const performances = await Promise.all(
      candidates.map(async (route) => ({
        route,
        performance: await this.getRoutePerformance(route),
      }))
    );
    
    // Select route with best success rate (with exploration)
    const explorationRate = 0.1;
    
    if (Math.random() < explorationRate) {
      // Explore: randomly select
      return candidates[Math.floor(Math.random() * candidates.length)];
    } else {
      // Exploit: select best performer
      return performances.reduce((best, current) =>
        current.performance.successRate > best.performance.successRate
          ? current
          : best
      ).route;
    }
  }
}

Adaptive router that learns from outcomes and improves routing decisions over time using reinforcement learning principles.

Conclusion

The routing pattern transforms static, linear AI workflows into dynamic, intelligent systems that adapt to context and learn from outcomes. By implementing routing with Langchain and Langraph in Next.js 15, you can build production-ready applications that efficiently distribute work across specialized agents, reduce costs through smart model selection, and continuously improve through feedback loops. The combination of semantic understanding, state management, and adaptive learning creates AI systems that become more effective over time while maintaining the scalability and developer experience benefits of Vercel's serverless platform.