Proactive Notifications System - Setup & Deployment Guide

🎯 Overview

The proactive notification system enables Sage to automatically alert you about: - ✅ Urgent emails requiring immediate attention - ✅ Calendar event changes and conflicts - ✅ Flight departure reminders and travel updates - ✅ Cancellation deadlines to avoid fees - ✅ Morning calendar previews - ✅ Evening prep reminders

✨ What Was Fixed

Issue #1: Workflows Not Registered ❌ → ✅

Problem: The proactive workflow files existed but were never imported, so they never registered with the system.

Solution: - Updated app/superpowers/catalog/__init__.py to import all 6 proactive workflows - Updated app/superpowers/catalog/proactive/__init__.py to export all workflows

Files Changed:

app/superpowers/catalog/__init__.py
app/superpowers/catalog/proactive/__init__.py

Issue #2: Scheduler Disabled by Default ❌ → ✅

Problem: The proactive scheduler was disabled by default and required an environment variable to enable.

Solution: Set ENABLE_PROACTIVE_SCHEDULER=true in Railway environment variables

Location: app/main.py:115-124

Issue #3: Outdated LLM Usage ⚠️ → ✅

Problem: Workflows were using llm_model_default (GPT-5) instead of llm_model_fast (GPT-5 Mini) for fast analysis.

Solution: Updated all workflows to use GPT-5 Mini for cost-efficient, fast analysis.

Files Changed:

app/superpowers/catalog/proactive/email_urgency.py
app/superpowers/catalog/proactive/calendar_events.py
app/superpowers/catalog/proactive/evening_prep.py

Issue #4: Direct MessageDelivery Usage ⚠️ → ✅

Problem: Some workflows were calling MessageDelivery directly (Telegram-only) instead of ProactiveSender (multi-platform).

Solution: Updated workflows to use ProactiveSender which auto-detects platform (Telegram/iMessage).

Files Changed:

app/superpowers/catalog/proactive/email_urgency.py
app/superpowers/catalog/proactive/calendar_events.py
app/superpowers/catalog/proactive/evening_prep.py

Issue #5: Missing Error Handling ⚠️ → ✅

Problem: LLM JSON parsing had no handling for markdown code blocks, causing silent failures.

Solution: Added robust JSON parsing with markdown extraction and error logging.

Improvements: - Handle markdown code blocks (json...) - Log parsing errors with context - Graceful fallbacks on failures - Try/except blocks around all sends

---

🚀 Deployment Steps

Step 1: Enable the Scheduler

The scheduler is disabled by default. You need to enable it via Railway environment variable:

Option A: Via Railway Dashboard (Recommended) 1. Go to Railway project: https://railway.app/ 2. Select the archety-backend service 3. Go to Variables tab 4. Click + New Variable 5. Add: ENABLE_PROACTIVE_SCHEDULER = true 6. Click Deploy to restart with new variable

Option B: Via Railway CLI

railway variables set ENABLE_PROACTIVE_SCHEDULER=true --service archety-backend

Step 2: Commit and Push Changes

# Review changes
git status
git diff

# Commit all changes
git add .
git commit -m "fix: Enable proactive notifications system with latest improvements

- Import all 6 proactive workflows in catalog
- Use GPT-5 Mini for fast, cost-efficient analysis
- Use ProactiveSender for multi-platform support (Telegram/iMessage)
- Add robust JSON parsing with markdown handling
- Add comprehensive error logging
- Update email_urgency, calendar_events, and evening_prep workflows"

# Push to Railway for deployment
git push origin claude/email-calendar-alerts-011CUxuEseqDZYvQnTQsYrXT

Step 3: Verify Deployment

After deploying, check the logs to confirm the scheduler started:

# Via Railway dashboard: Deployments → View Logs

# Or via Railway CLI:
railway logs --service archety-backend

Expected log output:

Starting up Persona Passport Orchestrator...
Proactive scheduler started
Scheduler service started with 6 proactive workflows

If scheduler is still disabled, you'll see:

Proactive scheduler DISABLED (set ENABLE_PROACTIVE_SCHEDULER=true to enable)

---

📊 Proactive Workflows Schedule

Once enabled, these workflows run automatically:

Workflow	Schedule	Purpose
Email Urgency Scan	Every 15 min, 8am-8pm	Scan for urgent emails requiring attention
Calendar Events Monitor	Every 30 min, 8am-10pm	Detect calendar changes and conflicts
Morning Calendar Check	7:30 AM daily	Preview today's schedule
Evening Prep	5:30 PM daily	Prep for tomorrow with schedule + tasks
Travel Brain	Every 20 min, 8am-11pm	Flight departure reminders, travel alerts
Cancellation Protector	4x daily (8am, noon, 4pm, 8pm)	Warn about cancellation deadlines

---

🔧 Technical Architecture

Workflow Registration Flow

1. app/main.py imports app/superpowers/catalog
2. catalog/__init__.py imports catalog/proactive
3. Each proactive/*.py file calls register_workflow()
4. WorkflowRegistry stores all workflows
5. SchedulerService queries registry and schedules jobs

Scheduler Service Flow

1. APScheduler triggers workflow at scheduled time
2. SchedulerService.run_*() gets active users from DB
3. For each user:
   - Execute workflow via WorkflowEngine
   - Workflow scans Gmail/Calendar (OAuth required)
   - LLM analyzes for urgency/importance
   - ProactiveSender routes to Telegram or iMessage
   - Log notification to database

ProactiveSender Platform Detection

# Auto-detects user's platform
def send_proactive_message(user_phone, message_text, priority):
    platform = _get_user_platform(user_phone)  # Check DB + edge agent

    if platform == 'imessage':
        return _send_via_imessage(...)  # Via EdgeAgentManager
    elif platform == 'telegram':
        return _send_via_telegram(...)  # Via MessageDelivery
    else:
        return _send_via_messenger(...)  # Generic fallback

---

🧪 Testing Proactive Notifications

Test Email Urgency

1. Send yourself an email with "URGENT" in subject
2. Mark it as unread
3. Wait up to 15 minutes for next scan
4. Check if you receive a Telegram/iMessage alert

Test Travel Brain

1. Forward a flight confirmation email to yourself
2. Ensure it's from last 14 days
3. Wait up to 20 minutes for scan
4. If flight is within 4-6 hours, you'll get a reminder

Test Calendar Events

1. Change an existing calendar event time
2. Wait up to 30 minutes for scan
3. LLM will analyze if change is conversation-worthy
4. You'll get alert if change is significant

Manual Trigger (Development)

# Run a workflow manually for testing
from app.superpowers.engine import WorkflowEngine
from app.superpowers.workflow import get_workflow_registry

engine = WorkflowEngine()
result = await engine.execute_workflow(
    workflow_id="proactive_email_urgency",
    user_id="+16463831222",  # Your phone/chat ID
    trigger_data={'scan_hours': 2, 'user_phone': '+16463831222'}
)

---

📋 OAuth Requirements

Gmail & Calendar access required for most workflows:

1. User must authorize OAuth via link: https://yourdomain.com/auth/google
2. Grants read-only access to Gmail + Calendar
3. Tokens stored in oauth_tokens table
4. Scheduler checks for valid tokens before running

Without OAuth: - Email urgency: Won't run - Calendar events: Won't run - Morning calendar: Won't run - Evening prep: Partial (no email/calendar data) - Travel brain: Won't find flights - Cancellation protector: Won't find deadlines

To check OAuth status:

SELECT user_phone, scope, expires_at
FROM oauth_tokens
WHERE expires_at > NOW();

---

🐛 Troubleshooting

Scheduler Not Starting

Symptom: Logs show "Proactive scheduler DISABLED"

Solution: 1. Confirm ENABLE_PROACTIVE_SCHEDULER=true is set in Railway 2. Restart the service to pick up the new variable 3. Check logs again for "Proactive scheduler started"

Workflows Not Triggering

Symptom: No notifications received after 24 hours

Possible causes: 1. No OAuth tokens - User hasn't authorized Gmail/Calendar 2. No active users - _get_active_users() returns empty list 3. Circuit breaker open - Workflow failed 3+ times and is paused 4. LLM threshold not met - Emails/events not urgent enough

Debug steps:

# Check active users with OAuth tokens
SELECT DISTINCT user_phone FROM oauth_tokens WHERE expires_at > NOW();

# Check recent proactive notifications
SELECT * FROM proactive_notifications
WHERE notified_at > NOW() - INTERVAL '24 hours'
ORDER BY notified_at DESC;

# Check workflow execution logs
# Look for: "Running email urgency scan..." in Railway logs

JSON Parsing Failures

Symptom: Logs show "LLM response parsing failed"

This is now handled gracefully: - Workflow continues without crash - Error logged with response snippet - Returns should_notify: false to skip send - Circuit breaker prevents repeated failures

To investigate:

railway logs --service archety-backend | grep "parsing failed"

Messages Not Delivering

Symptom: Workflow runs but no message received

Check ProactiveSender logs:

railway logs --service archety-backend | grep "ProactiveSender"

Possible issues: 1. Platform detection failed → Check users table platform column 2. Telegram bot token invalid → Check TELEGRAM_BOT_TOKEN env var 3. iMessage edge agent offline → Check edge agent status 4. User blocked bot → Check Telegram bot status

---

📈 Monitoring & Metrics

Key Metrics to Track

Via Database:

-- Notifications sent per day
SELECT
    DATE(notified_at) as date,
    notification_type,
    COUNT(*) as count
FROM proactive_notifications
WHERE notified_at > NOW() - INTERVAL '7 days'
GROUP BY DATE(notified_at), notification_type
ORDER BY date DESC;

-- Notification success rate (vs workflow runs)
-- This requires adding workflow execution logging

Via Logs:

# Count workflow executions today
railway logs | grep "Running.*scan" | grep $(date +%Y-%m-%d) | wc -l

# Count successful sends today
railway logs | grep "Sent.*alert" | grep $(date +%Y-%m-%d) | wc -l

Expected Patterns

Normal operation: - Email scans: ~45/day (every 15 min, 8am-8pm) - Calendar scans: ~28/day (every 30 min, 8am-10pm) - Notifications sent: 3-10/day per user (depends on activity) - LLM cost: ~$0.10-0.30/day per active user (GPT-5 Mini)

Abnormal patterns: - Zero notifications for 24+ hours → OAuth expired or no activity - 50+ notifications/day → LLM threshold too low or logic bug - Repeated parsing failures → LLM output format changed

---

💰 Cost Optimization

Current Optimizations

1. GPT-5 Mini Usage: Workflows use llm_model_fast (GPT-5 Mini) instead of GPT-5
2. Cost: ~$0.00015/1K tokens (prompt) vs $0.005/1K tokens

3. 97% cost reduction for analysis tasks

4. Token Limits: Reduced max_tokens across workflows

5. Email urgency: 400 tokens (was 800)
6. Calendar events: 600 tokens

7. Evening prep: 400 tokens

8. Email Scanning: Reduced from 50 to 15 emails per scan

9. Snippet-only (no full body) saves tokens in prompts

10. Memory Context: Reduced from 20 to 5 memories per analysis

11. Smaller prompts = faster + cheaper

12. Scheduling Frequency:

13. Email urgency: Every 15 min (not 5 min) during waking hours only
14. Calendar events: Every 30 min (not every 10 min)

Cost Estimates

Per user per day: - Email scans: 45 runs × \(0.0002 = **\)0.009/day** - Calendar scans: 28 runs × \(0.0003 = **\)0.008/day** - Travel/cancellation: 10 runs × \(0.0001 = **\)0.001/day** - Morning/evening: 2 runs × \(0.0005 = **\)0.001/day**

Total: ~$0.02/user/day or $0.60/user/month

For 100 active users: ~$60/month in LLM costs

---

🔐 Security & Privacy

OAuth Token Security

• Tokens encrypted at rest in database
• Stored in oauth_tokens table with expires_at
• Auto-refresh handled by TokenManager
• Revocation supported via "stop checking my calendar" command

Data Access Principles

• Read-only: No write access to Gmail/Calendar
• Scoped: Only request minimum necessary permissions
• Time-limited: Tokens expire and require re-auth
• Transparent: User told exactly what we're looking at

PII Handling

• Proactive notifications contain user data (email subjects, calendar events)
• Stored in proactive_notifications table with user_phone identifier
• NOT shared across users
• Can be deleted via cleanup endpoint

---

📚 Additional Resources

• PRD: /prd.md (sections on proactive workflows)
• Flow Documentation: /docs/architecture/PROACTIVE_FLOW_COMPLETE.md
• Edge Agent Architecture: /docs/edge/ARCHITECTURE.md (iMessage integration)
• Scheduler Service: /app/scheduler/background_service.py
• Workflow Files: /app/superpowers/catalog/proactive/

---

✅ Checklist for Deployment

• Set ENABLE_PROACTIVE_SCHEDULER=true in Railway
• Commit all code changes
• Push to branch claude/email-calendar-alerts-011CUxuEseqDZYvQnTQsYrXT
• Verify deployment succeeded in Railway logs
• Check "Proactive scheduler started" in logs
• Ensure user has OAuth tokens (check oauth_tokens table)
• Wait 15-30 minutes for first workflow execution
• Verify notification sent to Telegram/iMessage
• Monitor logs for errors or circuit breaker triggers

---

🎉 Success Criteria

You'll know the system is working when: 1. ✅ Railway logs show "Proactive scheduler started with 6 workflows" 2. ✅ Logs show "Running email urgency scan..." every 15 minutes 3. ✅ Logs show "Running calendar events scan..." every 30 minutes 4. ✅ You receive at least 1-2 proactive notifications per day 5. ✅ Notifications are contextually relevant and well-timed 6. ✅ Messages appear in your Telegram or iMessage 7. ✅ Database shows entries in proactive_notifications table

---

Last Updated: 2025-11-09 Implemented by: Claude (Session: 011CUxuEseqDZYvQnTQsYrXT)