Skip to content

Persona Memory System

Overview

The Persona Memory System gives personas (like Sage) their own memories, experiences, and personalities that they can naturally reference in conversations. This makes interactions feel more authentic and helps personas continue conversations naturally by sharing about themselves or asking relevant questions.

Key Features

  1. Personal Memories: Personas have their own memories (experiences, preferences, opinions, learned facts)
  2. Conversation Continuation: Personas ask questions or share about themselves ~50% of the time
  3. Privacy-First Learning: Personas learn from conversations but always anonymize user data
  4. Admin Interface: Web UI to manage persona memories and profiles
  5. JSON + mem0 Storage: Memories stored in version-controlled JSON files, synced to mem0 for semantic search

Architecture

Components

app/
├── persona/
│   ├── memories/
│   │   └── sage/
│   │       └── persona_memories.json     # Sage's memory file
│   ├── persona_memory_loader.py          # Loads JSON memories
│   └── passports/
│       └── sage.json                      # Includes continuation settings
├── memory/
│   ├── persona_memory_service.py         # Syncs JSON to mem0
│   └── persona_memory_classifier.py      # Privacy-safe learning
├── messaging/
│   ├── continuation_coordinator.py       # Manages conversation continuations
│   ├── reflex.py                         # Updated with continuation support
│   └── burst_planner.py                  # Updated with continuation support
├── models/
│   └── schemas.py                        # PersonaContext dataclass
└── api/
    └── admin_routes.py                   # Admin API endpoints

scripts/
└── sync_persona_memories.py             # Sync script

web/
└── admin.html                            # Admin interface

Data Flow

  1. Startup: sync_persona_memories.py loads JSON files into mem0
  2. Message Handling:
  3. MessageHandler creates PersonaContext with relevant memories
  4. PersonaEngine includes memories in system prompt
  5. Sage can naturally reference her own experiences
  6. Continuation Logic:
  7. ReflexResponder may add quick question (~25% of banter)
  8. BurstPlanner may add share/question (~50% overall)
  9. ContinuationCoordinator prevents double-questioning
  10. Learning:
  11. PersonaMemoryClassifier analyzes conversation
  12. If interesting (importance >= 7/10), stores anonymized memory
  13. Runtime memories stored in mem0, can be exported to JSON later

persona_memories.json Format

{
  "persona_id": "sage",
  "version": "1.0",
  "metadata": {
    "last_updated": "2025-10-31",
    "total_memories": 8,
    "source": "manual_curation"
  },
  "memories": [
    {
      "id": "sage_mem_001",
      "category": "experience",
      "text": "I spent a summer working at a coffee shop and learned to make latte art",
      "tags": ["work", "coffee", "skills", "summer"],
      "emotional_tone": "nostalgic",
      "timestamp": "2023-06-01",
      "importance": 6,
      "can_reference": true
    }
  ]
}

Memory Categories

  • experience: Something the persona experienced
  • preference: Something the persona likes/dislikes
  • opinion: The persona's opinion on something
  • learned_fact: Something the persona learned
  • interest: The persona's interests/hobbies
  • relationship: Connections to people/places
  • emotional_pattern: Emotional patterns the persona has noticed

Importance Scale (1-10)

  • 1-3: Not worth storing (routine, boring)
  • 4-6: Somewhat interesting but not enough to store
  • 7-8: Interesting, worth storing
  • 9-10: Very impactful, definitely store

Continuation System

Settings (in persona passport)

{
  "continuation": {
    "enabled": true,
    "probability": 0.5,
    "reflex_question_probability": 0.25,
    "types": ["question", "share"],
    "share_from_memories": true
  }
}

How It Works

  1. Reflex Stage (ReflexResponder):
  2. May add a quick question (~25% for banter/sharing)
  3. Examples: "what about u?", "how'd that go?", "u excited?"

  4. Tracked per conversation to prevent duplication

  5. Burst Stage (BurstPlanner):

  6. If reflex didn't ask question, may add continuation (~50%)
  7. Types:
    • Share: References persona's own memories
    • Question: Asks relevant follow-up question

  8. Prefers share if relevant memories exist

  9. Coordination (ContinuationCoordinator):

  10. Single source of truth for continuation decisions
  11. Prevents asking two questions in one response
  12. Uses conversation ID to track state

Example Flow

User: "I tried pineapple pizza for the first time"

Reflex: "ooh how was it?"  (quick question, 25% chance)

Since reflex asked a question, burst skips continuation.

---

User: "I tried pineapple pizza for the first time"

Reflex: "OMG"  (no question)

Burst: [normal bubbles] + "honestly I think pineapple on pizza is amazing"
       (share from Sage's memories, since no reflex question)

Privacy-Safe Learning

PersonaMemoryClassifier

The classifier analyzes conversations to determine if the persona should remember them, with strict privacy protections:

Anonymization Rules

  1. Names → "someone" or "a friend"
  2. Companies → "their workplace" or "a tech company"
  3. Locations → "a city" or omit entirely
  4. Contact Info → REMOVE completely (phone, email, address)
  5. Specifics → Generalize

Examples

User: "John at Google told me about their new AI project" Stored: "I had a fascinating conversation about AI projects"

User: "I live at 123 Main St in San Francisco" Stored: Nothing (not interesting enough + too much PII)

User: "I'm terrified of deep water" Stored: "Someone shared they're afraid of deep water" (generalized but interesting pattern)

What Gets Stored

Only conversations that are: - Highly interesting/impactful (importance >= 7/10) - About topics, ideas, emotions, patterns - Something persona learned or experienced - General insights without specific PII

PersonaContext Dataclass

Bundles all persona-related data to reduce parameter pollution:

@dataclass
class PersonaContext:
    persona_id: str
    passport: dict  # Full persona passport
    memories: List[dict]  # Relevant persona memories
    relationship_stage: str  # User relationship stage
    continuation_settings: dict  # Continuation config

Used throughout: - PersonaEngine (includes memories in prompt) - BurstPlanner (for continuation logic) - MessageHandler (creates and passes context)

Admin Panel

Access

Navigate to http://localhost:8000/admin (or your deployed URL)

Features

  1. Persona Selector: Switch between personas
  2. Memories Tab:
  3. View all memories (JSON + runtime)
  4. Search and filter by category/source
  5. Add new runtime memories
  6. Sync JSON to mem0
  7. Profile Tab: View persona passport (read-only)
  8. Stats Tab: Memory statistics and breakdowns

Memory Management Workflow

  1. Edit JSON Files:
  2. Edit app/persona/memories/{persona_id}/persona_memories.json
  3. Add/edit memories in structured format

  4. Commit to version control

  5. Sync to mem0:

  6. Click "Sync from JSON" in admin panel
  7. Or run: python scripts/sync_persona_memories.py --persona sage

  8. Only new/changed memories are synced (use --force to re-sync all)

  9. Add Runtime Memories:

  10. Use "Add Memory" button in admin panel
  11. Runtime memories stored in mem0 only
  12. Can be exported to JSON later for persistence

API Endpoints

Admin Routes (/admin)

  • GET /admin - Serve admin HTML
  • GET /admin/personas - List all personas with stats
  • GET /admin/personas/{persona_id}/profile - Get persona passport
  • GET /admin/personas/{persona_id}/memories - Get all memories
  • POST /admin/personas/{persona_id}/memories - Add runtime memory
  • DELETE /admin/personas/{persona_id}/memories/{memory_id} - Delete memory
  • POST /admin/personas/{persona_id}/memories/sync - Sync JSON to mem0
  • GET /admin/personas/{persona_id}/memories/stats - Get memory statistics

Sync Script

Usage

# Sync all personas
python scripts/sync_persona_memories.py

# Sync specific persona
python scripts/sync_persona_memories.py --persona sage

# Force re-sync (even if already synced)
python scripts/sync_persona_memories.py --force

When to Sync

  • Startup: Automatically sync all personas (can be configured)
  • After JSON edits: Manually sync to update mem0
  • Development: Sync after adding test memories

Integration with Message Flow

Old Flow (Before Persona Memories)

1. User message → MessageHandler
2. Search user memories (about user)
3. Generate response with PersonaEngine
4. Send response

New Flow (With Persona Memories)

1. User message → MessageHandler
2. Search user memories (about user)
3. Create PersonaContext:
   - Load persona passport
   - Search persona memories (about Sage)
   - Get continuation settings
4. Generate response with PersonaEngine + PersonaContext:
   - System prompt includes "ABOUT YOU" section
   - Sage can reference her own memories
5. Reflex + Burst coordination:
   - May add continuation (question/share)
6. Send response
7. Classify conversation:
   - If interesting, store anonymized persona memory

Example: Sage's Memories

{
  "id": "sage_mem_002",
  "category": "opinion",
  "text": "I think pineapple on pizza is actually amazing and people are too judgy about it",
  "tags": ["food", "opinions", "controversial"],
  "emotional_tone": "playful_defensive",
  "importance": 4,
  "can_reference": true
}

How it appears in conversation:

User: "I tried pineapple pizza for the first time"
Sage: "ooh how was it?"
User: "actually pretty good!"
Sage: "RIGHT?? honestly i think pineapple on pizza is amazing and people are too judgy about it"

Best Practices

Creating Memories

  1. Be specific but general: "I went to Venice" not "I went to Venice with John on June 15, 2024"
  2. Use natural language: Write as Sage would say it
  3. Set appropriate importance: Most memories should be 5-7
  4. Add relevant tags: Helps with search and filtering
  5. Use can_reference: false for sensitive memories you don't want referenced

Privacy

  1. Never store user PII in persona memories
  2. Always anonymize in PersonaMemoryClassifier
  3. Review runtime memories periodically to ensure no leaks
  4. Use importance >= 7 threshold for storing

Continuation Tuning

Adjust in persona passport: - probability: Overall continuation rate (default 0.5) - reflex_question_probability: Quick questions rate (default 0.25) - types: ["question", "share"] or just one - share_from_memories: Whether to use memories for shares

Performance

  • mem0 search is fast: Semantic search across all memories
  • JSON keyword search: Fallback if needed
  • Cache: PersonaMemoryLoader caches JSON files
  • Sync is incremental: Only new memories synced by default

Testing

Manual Testing

  1. Start server: python -m uvicorn app.main:app --reload
  2. Sync memories: python scripts/sync_persona_memories.py
  3. Access admin: http://localhost:8000/admin
  4. Send test messages via Telegram or API
  5. Check continuation: Observe Sage asking questions or sharing

Verification

# Check memories loaded
from app.persona.persona_memory_loader import get_persona_memory_loader
loader = get_persona_memory_loader()
memories = loader.get_all_memories("sage")
print(f"Loaded {len(memories)} memories for Sage")

# Check mem0 sync
from app.memory.persona_memory_service import get_persona_memory_service
service = get_persona_memory_service()
status = service.get_sync_status("sage")
print(f"Last sync: {status}")

# Search memories
results = service.search_persona_memories("sage", "pizza", limit=3)
print(f"Found {len(results)} memories about pizza")

Troubleshooting

Memories not appearing

  1. Check JSON file exists: app/persona/memories/sage/persona_memories.json
  2. Validate JSON syntax
  3. Run sync script: python scripts/sync_persona_memories.py --persona sage --force
  4. Check logs for errors

Continuation not working

  1. Check persona passport has continuation settings
  2. Verify continuation.enabled = true
  3. Check logs for coordinator decisions
  4. Try multiple messages (it's probabilistic)

Privacy concerns

  1. Review runtime memories in admin panel
  2. Check PersonaMemoryClassifier logs
  3. Verify anonymization is working
  4. Set higher importance threshold if needed

Future Enhancements

  • Photo upload: Process images to create memories
  • Conversation import: Bulk import from chat logs
  • Memory relationships: Link related memories
  • Memory decay: Reduce importance over time
  • Export runtime memories: Save to JSON for persistence
  • Multi-persona sharing: Personas share certain memories
  • Memory conflicts: Detect and resolve contradictions

File Locations

  • JSON Memories: app/persona/memories/{persona_id}/persona_memories.json
  • Persona Passports: app/persona/passports/{persona_id}.json
  • Admin Panel: web/admin.html
  • Sync Script: scripts/sync_persona_memories.py
  • Services: app/memory/persona_memory_service.py
  • Classifier: app/memory/persona_memory_classifier.py
  • Coordinator: app/messaging/continuation_coordinator.py

Summary

The Persona Memory System gives Sage a realistic personality with her own memories and experiences. She can:

  • Reference her past: "I spent a summer working at a coffee shop"
  • Share opinions: "I think pineapple on pizza is amazing"
  • Continue conversations: Asks questions or shares naturally
  • Learn safely: Forms new memories while protecting user privacy

This makes interactions feel more natural and authentic, as Sage becomes a character with her own history rather than just a response generator.