Dev/Prod Environment Isolation Audit¶

Date: November 13, 2025 Last Updated: November 13, 2025 (mem0 separation completed) Status: 🟡 MEDIUM PRIORITY ISSUES REMAINING - Critical mem0 issue resolved!

Executive Summary¶

After auditing both Railway environments and implementing fixes:

✅ RESOLVED - Critical Issues Fixed¶

✅ mem0 Memory Storage - Now using separate API keys (FIXED!)
Dev: m0-REDACTED_DEV
Prod: m0-REDACTED_PROD

🔴 REMAINING BLOCKERS for Production¶

Telegram Bot - Same bot token = Can't distinguish dev/prod users
Amplitude Analytics - Same key = Dev events pollute prod analytics

🟡 MEDIUM - Operational Issues¶

Sentry Error Tracking - Same DSN = Dev errors mixed with prod
Relay Webhook Secret - Same secret = Mac mini could talk to wrong environment

🟢 ACCEPTABLE - Low Risk Shared Keys¶

OpenAI, Perplexity, Weather APIs - Can be shared (read-only, no data)
Keywords AI tracing - Can be shared (just observability)

Detailed Analysis¶

✅ Properly Isolated Services¶

Service	Dev Value	Prod Value	Status
mem0 Memory Storage	m0-REDACTED_DEV	m0-REDACTED_PROD	✅ Isolated (FIXED!)
Supabase Database	atqvbecetywpppbdkdhp	tinckcednwwyruzdjfex	✅ Isolated
DATABASE_URL	Dev Supabase pooler	Prod Supabase pooler	✅ Isolated
SUPABASE_URL	https://atqvbecetywpppbdkdhp.supabase.co	https://tinckcednwwyruzdjfex.supabase.co	✅ Isolated
SUPABASE_ANON_KEY	Different key	Different key	✅ Isolated
SUPABASE_SERVICE_KEY	Different key	Different key	✅ Isolated
BASE_URL	archety-backend-dev.up.railway.app	archety-backend-prod.up.railway.app	✅ Isolated
SECRET_KEY	Different	Different	✅ Isolated
FERNET_KEY	Different	Different	✅ Isolated
EDGE_SECRET	Different	Different	✅ Isolated
GOOGLE_CLIENT_ID	Dev OAuth app	Prod OAuth app	✅ Isolated
GOOGLE_CLIENT_SECRET	Dev secret	Prod secret	✅ Isolated
OAUTH_REDIRECT_URI	Dev URL	Prod URL	✅ Isolated
LOG_LEVEL	DEBUG	INFO	✅ Isolated

🔴 CRITICAL: Shared Services (Data Collision Risk)¶

1. mem0 Memory Storage ✅ FIXED (November 13, 2025)¶

Previous State:

# Both environments used SAME key (FIXED!)
Dev:  MEM0_API_KEY=m0-REDACTED_DEV
Prod: MEM0_API_KEY=m0-REDACTED_PROD

Current State (FIXED):

# Now using DIFFERENT keys ✅
Dev:  MEM0_API_KEY=m0-REDACTED_DEV
Prod: MEM0_API_KEY=m0-REDACTED_PROD

Impact (RESOLVED): - ✅ Dev testing now writes to SEPARATE memory project - ✅ Test users' memories completely isolated from production - ✅ No namespace collision possible - ✅ Safe to delete test data in dev - ✅ Production memory recalls only production data

Risk Level: ✅ RESOLVED - Data isolation complete!

Example Scenario:

Dev:  User +15551234567 (test user) creates memories
Prod: User +15551234567 (real user) creates memories
Both write to namespace: "uuid-123_sage"
→ Memories collide and corrupt each other

Solution Required: - Create separate mem0 projects for dev and prod - OR use mem0 organization/project IDs to separate (if supported) - OR use namespace prefixes: dev_{user_id}_{persona_id} vs prod_{user_id}_{persona_id}

2. Telegram Bot 🔴 HIGH PRIORITY¶

Current State:

# Both environments use SAME bot
Dev:  TELEGRAM_BOT_TOKEN=<redacted>
Prod: TELEGRAM_BOT_TOKEN=<redacted>

Impact: - ❌ Can't distinguish between dev and prod users - ❌ Dev testing messages go to same bot as production - ❌ If dev crashes, prod users see errors - ❌ Can't test bot changes safely without affecting prod users - ❌ Webhook conflicts if both environments try to set webhook

Risk Level: 🔴 HIGH - User experience corruption

Example Scenario:

Tester: Sends "test message" to bot
Bot responds with dev environment (maybe broken)
Real user: Sends message to same bot
Bot responds with prod environment
→ Inconsistent user experience, testing pollutes production

Solution Required: - Create separate Telegram bot for development (@archety_dev_bot) - Keep existing bot for production (@archety_bot) - Update dev environment with new bot token

How to Create Dev Bot: 1. Message @BotFather on Telegram 2. Send /newbot 3. Name it "Archety Dev Bot" 4. Username: archety_dev_bot 5. Copy token and add to Railway dev environment

3. Amplitude Analytics 🟡 MEDIUM PRIORITY¶

Current State:

# Both environments use SAME key
Dev:  AMPLITUDE_API_KEY=d56979b4efd0937eea585eefa2c5310c
Prod: AMPLITUDE_API_KEY=d56979b4efd0937eea585eefa2c5310c

Impact: - ❌ Dev testing events pollute production analytics - ❌ Can't distinguish between dev and prod metrics - ❌ User count inflated by test users - ❌ Event analysis shows test data mixed with real data - ❌ A/B tests and analytics become unreliable

Risk Level: 🟡 MEDIUM - Analytics corruption (not data loss)

Example Scenario:

Dev: 100 test events fired during development
Prod: 50 real user events
Amplitude dashboard: Shows 150 events (mixed)
→ Metrics unreliable, can't make product decisions

Solution Required: - Create separate Amplitude project for development - OR use environment property in events to tag dev vs prod - Best practice: Separate projects for clean analytics

Code Change Option (Quick Fix):

# app/analytics/amplitude_service.py
amplitude.track(
    user_id=user_id,
    event_type=event_type,
    event_properties={
        **properties,
        'environment': settings.environment,  # 'development' or 'production'
    }
)

🟡 MEDIUM: Shared Services (Operational Issues)¶

4. Sentry Error Tracking 🟡 MEDIUM PRIORITY¶

Current State:

# Both environments use SAME DSN
Dev:  SENTRY_DSN=https://961e97f54573cc3c886dd677ac8e4c60@o4510292155957248.ingest.us.sentry.io/4510292227391488
Prod: SENTRY_DSN=https://961e97f54573cc3c886dd677ac8e4c60@o4510292155957248.ingest.us.sentry.io/4510292227391488

Impact: - ❌ Dev errors mixed with production errors - ❌ Can't set different alert rules for dev vs prod - ❌ Error rates inflated by dev testing - ❌ Hard to prioritize which errors to fix - ❌ Dev experiments trigger production alerts

Risk Level: 🟡 MEDIUM - Operations impacted, not data loss

Solution Required: - Create separate Sentry projects for dev and prod - Use environment tags in Sentry (quick fix)

Quick Fix Option:

# app/main.py
sentry_sdk.init(
    dsn=settings.sentry_dsn,
    environment=settings.environment,  # Tag errors by environment
)

5. Relay Webhook Secret 🟡 MEDIUM PRIORITY¶

Current State:

# Both environments use SAME secret
Dev:  RELAY_WEBHOOK_SECRET=<REDACTED_HEX_SECRET>
Prod: RELAY_WEBHOOK_SECRET=<REDACTED_HEX_SECRET>

Impact: - ⚠️ Mac mini relay could accidentally send to wrong environment - ⚠️ If secret leaks, both environments compromised - ⚠️ Can't revoke dev secret without affecting prod

Risk Level: 🟡 MEDIUM - Security and routing risk

Solution Required: - Generate separate webhook secret for dev environment - Update Mac mini relay configuration to use correct secret per environment

🟢 ACCEPTABLE: Shared Services (Low Risk)¶

These can remain shared for MVP, but should be separated eventually:

OpenAI API Key¶

Impact: Just costs, no data separation issue
Recommendation: Can share for MVP, separate later for cost tracking

Perplexity API Key¶

Impact: Read-only API, no data storage
Recommendation: Safe to share

OpenWeatherMap API Key¶

Impact: Read-only API, no user data
Recommendation: Safe to share

Parallel AI API Key¶

Impact: Read-only search API
Recommendation: Safe to share

Keywords AI Tracing¶

Impact: Observability tool, can filter by environment
Recommendation: Safe to share (but tag events by environment)

Recommended Actions¶

✅ COMPLETED¶

✅ Separate mem0 Projects - COMPLETED November 13, 2025

# ✅ DONE: Production now uses separate mem0 project
# Dev: m0-REDACTED_DEV
# Prod: m0-REDACTED_PROD

🔴 IMMEDIATE (Before Production Launch)¶

Create Dev Telegram Bot

# Create @archety_dev_bot
# Update Railway dev: TELEGRAM_BOT_TOKEN=<new_dev_token>

Separate Amplitude Projects

# Create "Archety Dev" project in Amplitude
# Update Railway dev: AMPLITUDE_API_KEY=<new_dev_key>

🟡 RECOMMENDED (Before Scale)¶

Separate Sentry Projects

# Create dev project in Sentry
# Update Railway dev: SENTRY_DSN=<new_dev_dsn>

Generate Separate Relay Secrets

# Generate new secret: openssl rand -hex 32
# Update Railway dev: RELAY_WEBHOOK_SECRET=<new_dev_secret>
# Update Mac mini relay config

🟢 OPTIONAL (Cost Tracking)¶

Separate OpenAI Keys
Create separate OpenAI project for dev
Easier cost tracking and rate limiting

Step-by-Step Fix Guide¶

Fix 1: Separate mem0 Projects (CRITICAL)¶

Step 1: Create Dev mem0 Project 1. Go to mem0 dashboard: https://app.mem0.ai 2. Create new project: "Archety Development" 3. Copy the new API key

Step 2: Update Railway Dev Environment

# Link to dev environment
railway link
railway environment development

# Set new mem0 key
railway variables set MEM0_API_KEY=<new_dev_key_here>

Step 3: Verify Isolation

# Test that dev uses separate namespace
curl https://archety-backend-dev.up.railway.app/health
# Should connect to dev mem0 project

Fix 2: Create Dev Telegram Bot¶

Step 1: Create Bot 1. Open Telegram and message @BotFather 2. Send: /newbot 3. Name: Archety Dev Bot 4. Username: archety_dev_bot 5. Copy token (starts with numbers:)

Step 2: Update Railway Dev

railway environment development
railway variables set TELEGRAM_BOT_TOKEN=<new_dev_bot_token>

Step 3: Update Documentation - Update docs/integration/TELEGRAM_TESTING.md with dev bot info - Document which bot to use for dev vs prod testing

Fix 3: Separate Amplitude Projects¶

Step 1: Create Dev Project 1. Go to Amplitude dashboard: https://analytics.amplitude.com 2. Create new project: "Archety Development" 3. Copy the new API key

Step 2: Update Railway Dev

railway environment development
railway variables set AMPLITUDE_API_KEY=<new_dev_key>

Verification Checklist¶

After implementing fixes, verify isolation:

Data Isolation Tests¶

mem0 Test:

# Create memory in dev
# Verify it doesn't appear in prod mem0 project

Telegram Test:

# Send message to dev bot
# Verify prod bot doesn't receive it

Amplitude Test:

# Fire event in dev
# Verify it appears in dev Amplitude project only

Environment Variable Audit¶

All critical services use separate keys
No shared credentials that could cause data collision
Secrets are different between environments
URLs point to correct environment

Current Risk Assessment (Updated: November 13, 2025)¶

Category	Risk Level	Impact
Data Corruption	✅ RESOLVED	mem0 now isolated - no corruption risk
User Experience	🔴 HIGH	Telegram bot confusion affects users
Analytics Accuracy	🟡 MEDIUM	Amplitude data unreliable for decisions
Error Tracking	🟡 MEDIUM	Sentry alerts may be noisy
Security	🟡 MEDIUM	Shared secrets = broader impact if leaked
Cost Tracking	🟢 LOW	Shared OpenAI key makes cost analysis harder

Deployment Recommendation (Updated: November 13, 2025)¶

Current Status: 🟡 SOFT READY - Telegram bot recommended

Resolved: - [x] ✅ mem0 projects separated (dev and prod) - DONE!

Remaining Blockers: - [ ] 🔴 Telegram bot not separated (HIGH PRIORITY - user experience) - [ ] 🟡 Amplitude projects separated (RECOMMENDED - analytics)

Safe to Deploy After: - [x] ✅ mem0 projects separated (dev and prod) - DONE! - [ ] Telegram dev bot created and configured (RECOMMENDED) - [ ] Amplitude projects separated (RECOMMENDED)

Timeline Estimate: - ✅ mem0 separation: COMPLETED - Telegram bot creation: 10 minutes - Amplitude separation: 10 minutes - Remaining: ~20 minutes to resolve remaining issues

Current Assessment: - Can deploy to prod: Yes (mem0 critical blocker resolved) - Should deploy to prod: After creating dev Telegram bot (10 min)

Long-Term Recommendations¶

Environment Strategy¶

Three-Tier Environments:
Local: Developer machines (localhost)
Development: Railway dev environment (auto-deploy from dev branch)
Production: Railway prod environment (manual deploy from master)
Service Separation Matrix:

Service	Local	Dev	Prod	Notes
Supabase	Dev project	Dev project	Prod project	Already separated ✅
mem0	Dev key	Dev key	Prod key	NEEDS SEPARATION 🔴
Telegram Bot	Dev bot	Dev bot	Prod bot	NEEDS SEPARATION 🔴
Amplitude	Dev project	Dev project	Prod project	NEEDS SEPARATION 🟡
Sentry	Dev project	Dev project	Prod project	NEEDS SEPARATION 🟡
OpenAI	Shared	Shared	Prod key	Optional separation

Namespace Prefixes:
Add environment prefix to all namespaced data
Example: dev_user123_sage vs prod_user123_sage
Provides double protection even if keys leak

Summary (Updated: November 13, 2025)¶

✅ What's Working Well¶

✅ mem0 completely isolated (FIXED November 13, 2025!)
✅ Supabase completely isolated (database, auth, storage)
✅ Security keys properly separated (SECRET_KEY, FERNET_KEY, EDGE_SECRET)
✅ OAuth credentials separated (Google)
✅ URLs and routing separated

⚠️ What Needs Fixing¶

RESOLVED: 1. ✅ Separate mem0 projects - COMPLETED!

RECOMMENDED (Before Prod Launch): 2. Create dev Telegram bot (10 min) - High priority for user experience 3. Separate Amplitude projects (10 min) - For clean analytics

OPTIONAL (Before Scale): 4. Separate Sentry projects (10 min) 5. Separate webhook secrets (5 min)

Total Time Remaining: ~20 minutes (recommended fixes)

Last Updated: November 13, 2025 - mem0 separation completed Audit Status: 🟡 Critical blocker resolved, recommended improvements remain Next Action: Create dev Telegram bot (10 min), then ready for production