DRAFT Agentic Design Patterns - Learning and Adaptation

Learn how to build agents that genuinely improve through experience using TypeScript, LangGraph, and modern memory architectures optimized for Vercel's serverless platform.

Mental Model: The Evolving Restaurant

Think of a learning agent like a restaurant that adapts to its customers. Initially, it serves generic dishes (baseline responses). Over time, it learns regular customers' preferences (user-specific adaptation), discovers popular combinations (pattern recognition), adjusts recipes based on feedback (reinforcement learning), and even predicts what new customers might enjoy based on similar past patrons (transfer learning). Just as a successful restaurant evolves from a static menu to a dynamic, customer-aware experience, your agents transform from simple responders to intelligent systems that improve with every interaction.

Basic Example: Adaptive Chat Agent with Memory

Let's build a simple agent that remembers user preferences and adapts its responses accordingly.

// app/api/adaptive-agent/route.ts
import { StateGraph, MemorySaver } from "@langchain/langgraph";
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";
import { HumanMessage, AIMessage } from "@langchain/core/messages";
import { Redis } from "@upstash/redis";
import { groupBy, maxBy, sortBy } from "es-toolkit";
import { z } from "zod";

// Define adaptive state schema
const AdaptiveStateSchema = z.object({
  messages: z.array(z.any()),
  userPreferences: z.object({
    style: z.enum(['concise', 'detailed', 'technical']).optional(),
    topics: z.array(z.string()).optional(),
  }).default({}),
  interactionCount: z.number().default(0),
  feedbackScores: z.array(z.number()).default([]),
});

type AdaptiveState = z.infer<typeof AdaptiveStateSchema>;

// Initialize Redis for persistent memory
const redis = new Redis({
  url: process.env.UPSTASH_REDIS_URL!,
  token: process.env.UPSTASH_REDIS_TOKEN!,
});

// Create the adaptive agent
async function createAdaptiveAgent() {
  const model = new ChatGoogleGenerativeAI({
    temperature: 0.7,
    modelName: "gemini-pro",
  });

  // Memory saver for conversation persistence
  const checkpointer = new MemorySaver();

  // Define the agent logic
  async function agentNode(state: AdaptiveState) {
    const { messages, userPreferences } = state;
    
    // Adapt prompt based on learned preferences
    const systemPrompt = `You are a helpful assistant.
      ${userPreferences.style ? `Respond in a ${userPreferences.style} manner.` : ''}
      ${userPreferences.topics?.length ? `The user is interested in: ${userPreferences.topics.join(', ')}` : ''}
      Interaction count: ${state.interactionCount}`;

    const response = await model.invoke([
      { role: "system", content: systemPrompt },
      ...messages
    ]);

    return {
      messages: [...messages, response],
      interactionCount: state.interactionCount + 1,
    };
  }

  // Build the graph
  const workflow = new StateGraph<AdaptiveState>({
    channels: AdaptiveStateSchema.shape,
  })
    .addNode("agent", agentNode)
    .setEntryPoint("agent")
    .setFinishPoint("agent");

  return workflow.compile({ checkpointer });
}

// API handler
export async function POST(req: Request) {
  const { message, sessionId, feedback } = await req.json();

  // Retrieve or initialize user state
  const storedState = await redis.get(`session:${sessionId}`) as AdaptiveState | null;
  const initialState: AdaptiveState = storedState || {
    messages: [],
    userPreferences: {},
    interactionCount: 0,
    feedbackScores: [],
  };

  // Process feedback if provided
  if (feedback) {
    initialState.feedbackScores.push(feedback);
    
    // Simple preference learning
    if (initialState.feedbackScores.length > 3) {
      const avgScore = initialState.feedbackScores.reduce((a, b) => a + b) / initialState.feedbackScores.length;
      if (avgScore < 3) {
        initialState.userPreferences.style = 'detailed';
      } else if (avgScore > 4) {
        initialState.userPreferences.style = 'concise';
      }
    }
  }

  // Run the agent
  const agent = await createAdaptiveAgent();
  const result = await agent.invoke(
    {
      ...initialState,
      messages: [...initialState.messages, new HumanMessage(message)],
    },
    {
      configurable: { thread_id: sessionId },
    }
  );

  // Save updated state
  await redis.set(`session:${sessionId}`, result, {
    ex: 86400 * 7, // 7 days TTL
  });

  return Response.json({
    response: result.messages[result.messages.length - 1].content,
    preferences: result.userPreferences,
    interactionCount: result.interactionCount,
  });
}

This basic agent tracks user interactions, learns from feedback scores, and adapts its response style. It uses Upstash Redis for serverless-compatible persistent storage.

// components/AdaptiveChatInterface.tsx
import { useState } from 'react';
import { useMutation } from '@tanstack/react-query';
import { debounce } from 'es-toolkit';

interface ChatResponse {
  response: string;
  preferences: any;
  interactionCount: number;
}

export function AdaptiveChatInterface() {
  const [message, setMessage] = useState('');
  const [sessionId] = useState(() => crypto.randomUUID());
  const [lastResponse, setLastResponse] = useState<string>('');

  const chatMutation = useMutation({
    mutationFn: async (params: { message: string; feedback?: number }) => {
      const response = await fetch('/api/adaptive-agent', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ ...params, sessionId }),
      });
      return response.json() as Promise<ChatResponse>;
    },
    onSuccess: (data) => {
      setLastResponse(data.response);
    },
  });

  // Debounced feedback handler
  const handleFeedback = debounce((score: number) => {
    chatMutation.mutate({ message: '', feedback: score });
  }, 500);

  return (
    <div className="card bg-base-100 shadow-xl">
      <div className="card-body">
        <h2 className="card-title">Adaptive Chat Agent</h2>
        
        {lastResponse && (
          <div className="alert alert-info">
            <span>{lastResponse}</span>
          </div>
        )}

        <div className="form-control">
          <input
            type="text"
            placeholder="Ask anything..."
            className="input input-bordered"
            value={message}
            onChange={(e) => setMessage(e.target.value)}
            onKeyPress={(e) => {
              if (e.key === 'Enter') {
                chatMutation.mutate({ message });
                setMessage('');
              }
            }}
          />
        </div>

        {lastResponse && (
          <div className="rating rating-lg">
            {[1, 2, 3, 4, 5].map((score) => (
              <input
                key={score}
                type="radio"
                name="rating"
                className="mask mask-star-2 bg-orange-400"
                onClick={() => handleFeedback(score)}
              />
            ))}
          </div>
        )}

        <div className="stats shadow">
          <div className="stat">
            <div className="stat-title">Interactions</div>
            <div className="stat-value text-primary">
              {chatMutation.data?.interactionCount || 0}
            </div>
          </div>
        </div>
      </div>
    </div>
  );
}

The frontend collects user feedback through a star rating system and displays interaction statistics, creating a complete feedback loop.

Advanced Example: Multi-Agent Learning System with Experience Replay

Now let's build a sophisticated system with multiple specialized agents that learn from shared experiences.

// lib/memory/experience-store.ts
import { PineconeStore } from "@langchain/pinecone";
import { GoogleGenerativeAIEmbeddings } from "@langchain/google-genai";
import { Pinecone } from "@pinecone-database/pinecone";
import { chunk, sortBy, take, groupBy, maxBy } from "es-toolkit";
import { z } from "zod";

// Experience schema
const ExperienceSchema = z.object({
  id: z.string(),
  interaction: z.object({
    input: z.string(),
    context: z.record(z.any()),
    agentType: z.string(),
  }),
  outcome: z.object({
    response: z.string(),
    success: z.boolean(),
    metrics: z.object({
      latency: z.number(),
      tokenCount: z.number(),
      userSatisfaction: z.number().optional(),
    }),
  }),
  timestamp: z.string(),
  embedding: z.array(z.number()).optional(),
});

type Experience = z.infer<typeof ExperienceSchema>;

export class ExperienceReplayBuffer {
  private vectorStore: PineconeStore;
  private embeddings: GoogleGenerativeAIEmbeddings;
  private bufferSize = 1000;
  private priorityAlpha = 0.6; // Priority exponent
  
  constructor() {
    const pinecone = new Pinecone({
      apiKey: process.env.PINECONE_API_KEY!,
    });

    this.embeddings = new GoogleGenerativeAIEmbeddings({
      modelName: "embedding-001",
    });

    this.vectorStore = new PineconeStore(this.embeddings, {
      pineconeIndex: pinecone.index(process.env.PINECONE_INDEX!),
      namespace: "experiences",
    });
  }

  async store(experience: Experience): Promise<void> {
    // Calculate priority based on TD error or outcome metrics
    const priority = this.calculatePriority(experience);
    
    await this.vectorStore.addDocuments([
      {
        pageContent: JSON.stringify({
          interaction: experience.interaction,
          outcome: experience.outcome,
        }),
        metadata: {
          id: experience.id,
          timestamp: experience.timestamp,
          priority,
          agentType: experience.interaction.agentType,
          success: experience.outcome.success,
          userSatisfaction: experience.outcome.metrics.userSatisfaction,
        },
      },
    ]);

    // Maintain buffer size
    await this.pruneOldExperiences();
  }

  async sample(
    context: any, 
    k: number = 5, 
    strategy: 'uniform' | 'prioritized' = 'prioritized'
  ): Promise<Experience[]> {
    const query = JSON.stringify(context);
    const results = await this.vectorStore.similaritySearch(query, k * 2);
    
    if (strategy === 'prioritized') {
      // Prioritized experience replay
      const experiences = results.map(doc => ({
        ...JSON.parse(doc.pageContent),
        priority: doc.metadata.priority || 1,
      }));

      // Sample based on priority
      const sorted = sortBy(experiences, exp => -exp.priority);
      return take(sorted, k);
    }

    return take(results.map(doc => JSON.parse(doc.pageContent)), k);
  }

  private calculatePriority(experience: Experience): number {
    // Higher priority for surprising outcomes
    const successWeight = experience.outcome.success ? 0.3 : 0.7;
    const satisfactionWeight = experience.outcome.metrics.userSatisfaction 
      ? (5 - experience.outcome.metrics.userSatisfaction) / 5 
      : 0.5;
    
    return Math.pow(successWeight + satisfactionWeight, this.priorityAlpha);
  }

  private async pruneOldExperiences(): Promise<void> {
    // Implementation for maintaining buffer size
    // Remove oldest experiences when buffer exceeds limit
  }
}

// lib/agents/specialized-agents.ts
import { StateGraph } from "@langchain/langgraph";
import { ChatGoogleGenerativeAI } from "@langchain/google-genai";
import { ExperienceReplayBuffer } from "./memory/experience-store";
import { filter, map, reduce } from "es-toolkit";

interface LearningAgentState {
  messages: any[];
  experiences: any[];
  adaptationMetrics: {
    successRate: number;
    avgLatency: number;
    confidenceScore: number;
  };
}

class SpecializedLearningAgent {
  private model: ChatGoogleGenerativeAI;
  private experienceBuffer: ExperienceReplayBuffer;
  private agentType: string;
  private learningRate = 0.1;
  
  constructor(agentType: 'researcher' | 'coder' | 'analyst') {
    this.agentType = agentType;
    this.model = new ChatGoogleGenerativeAI({
      temperature: 0.3,
      modelName: "gemini-pro",
    });
    this.experienceBuffer = new ExperienceReplayBuffer();
  }

  async createWorkflow() {
    const workflow = new StateGraph<LearningAgentState>({
      channels: {
        messages: { value: (x: any[], y: any[]) => [...x, ...y] },
        experiences: { value: (x: any[], y: any[]) => [...x, ...y] },
        adaptationMetrics: { 
          value: (x: any, y: any) => ({ ...x, ...y }) 
        },
      },
    });

    // Retrieve relevant experiences
    workflow.addNode("retrieve_experiences", async (state) => {
      const context = state.messages[state.messages.length - 1];
      const experiences = await this.experienceBuffer.sample(context, 5);
      
      return { experiences };
    });

    // Adapt behavior based on experiences
    workflow.addNode("adapt_strategy", async (state) => {
      const successfulExperiences = filter(
        state.experiences,
        exp => exp.outcome.success
      );

      // Calculate adaptation metrics
      const successRate = successfulExperiences.length / state.experiences.length;
      const avgLatency = reduce(
        state.experiences,
        (acc, exp) => acc + exp.outcome.metrics.latency,
        0
      ) / state.experiences.length;

      // Determine optimal strategy
      const strategy = this.determineStrategy(successfulExperiences);

      return {
        adaptationMetrics: {
          successRate,
          avgLatency,
          confidenceScore: this.calculateConfidence(state.experiences),
        },
      };
    });

    // Execute with learned behavior
    workflow.addNode("execute", async (state) => {
      const systemPrompt = this.buildAdaptivePrompt(
        state.experiences,
        state.adaptationMetrics
      );

      const startTime = Date.now();
      const response = await this.model.invoke([
        { role: "system", content: systemPrompt },
        ...state.messages,
      ]);
      const latency = Date.now() - startTime;

      // Store this interaction as new experience
      const experience = {
        id: crypto.randomUUID(),
        interaction: {
          input: state.messages[state.messages.length - 1].content,
          context: state.adaptationMetrics,
          agentType: this.agentType,
        },
        outcome: {
          response: response.content,
          success: true, // Will be updated based on feedback
          metrics: {
            latency,
            tokenCount: response.usage?.total_tokens || 0,
          },
        },
        timestamp: new Date().toISOString(),
      };

      await this.experienceBuffer.store(experience);

      return {
        messages: [response],
      };
    });

    // Connect the nodes
    workflow
      .addEdge("retrieve_experiences", "adapt_strategy")
      .addEdge("adapt_strategy", "execute")
      .setEntryPoint("retrieve_experiences")
      .setFinishPoint("execute");

    return workflow.compile();
  }

  private determineStrategy(experiences: any[]): string {
    // Group experiences by patterns
    const patterns = groupBy(experiences, exp => 
      exp.interaction.input.split(' ')[0].toLowerCase()
    );

    // Find most successful pattern
    const bestPattern = maxBy(
      Object.entries(patterns),
      ([_, exps]) => filter(exps, e => e.outcome.success).length
    );

    return bestPattern ? bestPattern[0] : 'default';
  }

  private calculateConfidence(experiences: any[]): number {
    if (experiences.length === 0) return 0.5;
    
    const weights = experiences.map((_, idx) => 
      Math.exp(-idx * 0.5) // Exponential decay for recency
    );
    
    const weightedSuccess = reduce(
      experiences,
      (acc, exp, idx) => acc + (exp.outcome.success ? weights[idx] : 0),
      0
    );

    return weightedSuccess / reduce(weights, (a, b) => a + b, 0);
  }

  private buildAdaptivePrompt(
    experiences: any[], 
    metrics: any
  ): string {
    const successfulPatterns = filter(
      experiences, 
      exp => exp.outcome.success
    ).map(exp => exp.interaction.input);

    return `You are a specialized ${this.agentType} agent.
      
      Based on past interactions:
      - Success rate: ${(metrics.successRate * 100).toFixed(1)}%
      - Average response time: ${metrics.avgLatency}ms
      - Confidence level: ${(metrics.confidenceScore * 100).toFixed(1)}%
      
      Successful patterns observed:
      ${successfulPatterns.slice(0, 3).join('\n')}
      
      Adapt your response style and approach based on these learnings.`;
  }
}

// lib/agents/multi-agent-coordinator.ts
export class MultiAgentLearningCoordinator {
  private agents: Map<string, SpecializedLearningAgent>;
  private routingModel: ChatGoogleGenerativeAI;
  
  constructor() {
    this.agents = new Map([
      ['researcher', new SpecializedLearningAgent('researcher')],
      ['coder', new SpecializedLearningAgent('coder')],
      ['analyst', new SpecializedLearningAgent('analyst')],
    ]);
    
    this.routingModel = new ChatGoogleGenerativeAI({
      temperature: 0,
      modelName: "gemini-pro",
    });
  }

  async route(input: string): Promise<string> {
    const routingPrompt = `Determine which specialist agent should handle this request:
      - researcher: For information gathering, fact-checking, research tasks
      - coder: For programming, debugging, code generation
      - analyst: For data analysis, insights, strategic planning
      
      Input: "${input}"
      
      Respond with only the agent name.`;

    const response = await this.routingModel.invoke(routingPrompt);
    return response.content.trim().toLowerCase();
  }

  async process(input: string, sessionId: string): Promise<any> {
    // Determine which agent to use
    const agentType = await this.route(input);
    const agent = this.agents.get(agentType);
    
    if (!agent) {
      throw new Error(`Unknown agent type: ${agentType}`);
    }

    const workflow = await agent.createWorkflow();
    const result = await workflow.invoke({
      messages: [{ role: "user", content: input }],
      experiences: [],
      adaptationMetrics: {
        successRate: 0.5,
        avgLatency: 0,
        confidenceScore: 0.5,
      },
    });

    return {
      response: result.messages[result.messages.length - 1].content,
      agentType,
      metrics: result.adaptationMetrics,
    };
  }
}

// app/api/learning-system/route.ts
import { MultiAgentLearningCoordinator } from "@/lib/agents/multi-agent-coordinator";
import { NextRequest } from "next/server";

const coordinator = new MultiAgentLearningCoordinator();

export async function POST(req: NextRequest) {
  // Disable background callbacks for serverless
  process.env.LANGCHAIN_CALLBACKS_BACKGROUND = "false";
  
  const { message, sessionId, feedback } = await req.json();

  try {
    const result = await coordinator.process(message, sessionId);
    
    return Response.json({
      success: true,
      ...result,
    });
  } catch (error) {
    console.error("Learning system error:", error);
    return Response.json(
      { success: false, error: "Processing failed" },
      { status: 500 }
    );
  }
}

This advanced system implements prioritized experience replay, multi-agent coordination, and adaptive strategy selection based on historical performance.

// components/LearningSystemDashboard.tsx
import { useState } from 'react';
import { useQuery, useMutation } from '@tanstack/react-query';
import { groupBy, sortBy } from 'es-toolkit';
import { LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, Legend } from 'recharts';

interface SystemMetrics {
  agentType: string;
  metrics: {
    successRate: number;
    avgLatency: number;
    confidenceScore: number;
  };
}

export function LearningSystemDashboard() {
  const [sessionId] = useState(() => crypto.randomUUID());
  const [history, setHistory] = useState<SystemMetrics[]>([]);

  const sendMessage = useMutation({
    mutationFn: async (message: string) => {
      const response = await fetch('/api/learning-system', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ message, sessionId }),
      });
      return response.json();
    },
    onSuccess: (data) => {
      setHistory(prev => [...prev, {
        agentType: data.agentType,
        metrics: data.metrics,
      }]);
    },
  });

  // Calculate performance trends
  const performanceData = history.map((item, index) => ({
    interaction: index + 1,
    confidence: item.metrics.confidenceScore * 100,
    success: item.metrics.successRate * 100,
  }));

  // Agent usage distribution
  const agentUsage = Object.entries(
    groupBy(history, item => item.agentType)
  ).map(([agent, items]) => ({
    agent,
    count: items.length,
    avgConfidence: items.reduce((acc, item) => 
      acc + item.metrics.confidenceScore, 0
    ) / items.length * 100,
  }));

  return (
    <div className="container mx-auto p-4">
      <div className="grid grid-cols-1 lg:grid-cols-2 gap-6">
        {/* Chat Interface */}
        <div className="card bg-base-100 shadow-xl">
          <div className="card-body">
            <h2 className="card-title">Multi-Agent Learning System</h2>
            
            <div className="form-control">
              <div className="input-group">
                <input
                  type="text"
                  placeholder="Ask anything..."
                  className="input input-bordered flex-1"
                  onKeyPress={(e) => {
                    if (e.key === 'Enter') {
                      sendMessage.mutate((e.target as HTMLInputElement).value);
                      (e.target as HTMLInputElement).value = '';
                    }
                  }}
                />
                <button 
                  className="btn btn-primary"
                  onClick={() => {
                    const input = document.querySelector('input');
                    if (input?.value) {
                      sendMessage.mutate(input.value);
                      input.value = '';
                    }
                  }}
                >
                  Send
                </button>
              </div>
            </div>

            {sendMessage.data && (
              <div className="alert alert-info mt-4">
                <div>
                  <span className="badge badge-secondary mr-2">
                    {sendMessage.data.agentType}
                  </span>
                  {sendMessage.data.response}
                </div>
              </div>
            )}
          </div>
        </div>

        {/* Performance Metrics */}
        <div className="card bg-base-100 shadow-xl">
          <div className="card-body">
            <h2 className="card-title">Learning Progress</h2>
            
            {performanceData.length > 0 && (
              <LineChart width={400} height={200} data={performanceData}>
                <CartesianGrid strokeDasharray="3 3" />
                <XAxis dataKey="interaction" />
                <YAxis />
                <Tooltip />
                <Legend />
                <Line 
                  type="monotone" 
                  dataKey="confidence" 
                  stroke="#8884d8" 
                  name="Confidence %"
                />
                <Line 
                  type="monotone" 
                  dataKey="success" 
                  stroke="#82ca9d" 
                  name="Success %"
                />
              </LineChart>
            )}
          </div>
        </div>

        {/* Agent Statistics */}
        <div className="card bg-base-100 shadow-xl lg:col-span-2">
          <div className="card-body">
            <h2 className="card-title">Agent Performance</h2>
            
            <div className="overflow-x-auto">
              <table className="table table-zebra">
                <thead>
                  <tr>
                    <th>Agent Type</th>
                    <th>Usage Count</th>
                    <th>Avg Confidence</th>
                  </tr>
                </thead>
                <tbody>
                  {agentUsage.map(agent => (
                    <tr key={agent.agent}>
                      <td className="font-bold">{agent.agent}</td>
                      <td>{agent.count}</td>
                      <td>
                        <progress 
                          className="progress progress-primary w-32" 
                          value={agent.avgConfidence} 
                          max="100"
                        />
                        <span className="ml-2">{agent.avgConfidence.toFixed(1)}%</span>
                      </td>
                    </tr>
                  ))}
                </tbody>
              </table>
            </div>
          </div>
        </div>
      </div>
    </div>
  );
}

The dashboard provides real-time visualization of learning progress, agent performance metrics, and system confidence trends.

Conclusion

Learning and adaptation transform static LLM applications into dynamic systems that improve with every interaction. By combining LangGraph's stateful orchestration, vector-based experience replay, and lightweight reinforcement learning techniques, you can build agents that genuinely learn from experience while maintaining excellent performance on serverless platforms like Vercel. Start with simple preference learning, then progressively add sophisticated features like prioritized experience replay and multi-agent coordination as your system matures.