Stress Testing Guide¶

This guide covers how to run stress tests for the Archety backend, particularly for the Trip Planner MiniApp and venue resolution features.

Prerequisites¶

Access to Claude Code with MCP tools configured
MCP servers: archety-dev (development) or archety-prod (production)

Phone Number Format Requirements¶

CRITICAL: All test phone numbers must use numeric-only format.

Valid Format¶

+1555XXXYYYY

Where: - +1555 - Prefix (US format, 555 is reserved for testing) - XXX - Run/batch identifier (e.g., 000, 001, 002) - YYYY - Scenario/user identifier (e.g., 0001, 0002)

Examples¶

+15550001001  # Batch 000, User 1001
+15550001002  # Batch 000, User 1002
+15550010001  # Batch 001, User 0001

Invalid Formats (Will Fail Validation)¶

+1555abc0001    # Letters not allowed
test-user-001   # Must be phone format
user@email.com  # Email not valid for MCP send_message
+1555lqurfa01   # Mixed alphanumeric not allowed

Running Stress Tests via MCP¶

Method 1: Sequential Testing with Claude Code¶

Use the mcp__archety-dev__send_message tool for each test:

# Step 1: Start a trip
mcp__archety-dev__send_message(
    user_id="+15550001001",
    persona_id="sage",
    message="Let's plan a trip to Tokyo"
)

# Step 2: Add a venue (after trip is created)
mcp__archety-dev__send_message(
    user_id="+15550001001",
    persona_id="sage",
    message="add Shake Shack"
)

Method 2: Parallel Testing¶

Launch multiple send_message calls in parallel for concurrent user simulation:

# Start trips for multiple users simultaneously
mcp__archety-dev__send_message(user_id="+15550001001", persona_id="sage", message="Let's plan a trip to Tokyo")
mcp__archety-dev__send_message(user_id="+15550001002", persona_id="sage", message="Let's plan a trip to Paris")
mcp__archety-dev__send_message(user_id="+15550001003", persona_id="sage", message="Let's plan a trip to New York")

Test Scenarios¶

Scenario 1: Basic Trip + Venue Flow¶

Tests the complete flow of starting a trip and adding a venue.

Step	User ID	Message	Expected Result
1	+15550001001	"Let's plan a trip to Tokyo"	Trip created, session started
2	+15550001001	"add Ichiran Ramen"	Venue resolved or clarification requested

Scenario 2: Venue Disambiguation¶

Tests handling of ambiguous venue names.

Step	User ID	Message	Expected Result
1	+15550002001	"Let's plan a trip to New York"	Trip created
2	+15550002001	"add Shake Shack"	Multiple locations found, clarification requested
3	+15550002001	"the one in Times Square"	Venue resolved with location hint

Scenario 3: Direct Resolution¶

Tests venues with unique names that resolve directly.

Step	User ID	Message	Expected Result
1	+15550003001	"Let's plan a trip to Paris"	Trip created
2	+15550003001	"add Le Comptoir du Pantheon"	Venue added directly (unique name)

Scenario 4: Query/Filter Commands¶

Tests the query engine for filtering venues.

Step	User ID	Message	Expected Result
1	+15550004001	"Let's plan a trip to Tokyo"	Trip created
2	+15550004001	"add Ichiran Ramen"	Venue added
3	+15550004001	"add teamLab Planets"	Venue added
4	+15550004001	"show me restaurants"	Filtered list of restaurants
5	+15550004001	"what's good for dinner in Shibuya?"	Filtered by time + district

Batch Stress Test Template¶

Use this template to run a batch of stress tests:

# Phone number format: +1555{batch_id:03d}{scenario_id:04d}
# Example: batch 1, scenarios 1-5 = +15550010001 to +15550010005

SCENARIOS = [
    # (phone, destination, venue, expected_behavior)
    ("+15550010001", "Tokyo", "Ichiran Ramen", "resolve_or_clarify"),
    ("+15550010002", "Paris", "Le Comptoir du Pantheon", "direct_resolve"),
    ("+15550010003", "New York", "Joe's Pizza", "clarify_multiple"),
    ("+15550010004", "London", "Dishoom", "resolve_or_clarify"),
    ("+15550010005", "San Francisco", "Tartine Bakery", "direct_resolve"),
]

Interpreting Results¶

Success Indicators¶

Trip Created: Response contains "Started planning your {destination} trip"
Venue Added: Response contains "Added" with venue details
Clarification Requested: Response contains "I found X locations matching"

Failure Indicators¶

Validation Error: "Sender must be a valid phone number" - Check phone format
Not Found: "couldn't find" or "No places found" - Check venue name/destination
Session Error: Response about no active trip - Trip wasn't created first

Common Issues and Solutions¶

Issue: "Sender must be a valid phone number or email"¶

Cause: Phone number contains non-numeric characters or invalid format.

Solution: Use format +1555XXXXXXX with only digits after the prefix.

Issue: "couldn't find [venue] in [city]"¶

Cause: Venue resolution failed. Could be: 1. No active trip session (destination not set) 2. Venue name too vague 3. Google Places API key not configured

Solution: 1. Ensure trip is started before adding venues 2. Use more specific venue names 3. Verify GOOGLE_MAPS_API_KEY is set in Railway variables

Issue: Timeout or slow responses¶

Cause: Concurrent load or API rate limits.

Solution: Add delays between requests (1-2 seconds) when running sequential tests.

Monitoring and Debugging¶

Check Railway Logs¶

Use the MCP Railway tools to check logs:

mcp__Railway__get-logs(
    workspacePath="/path/to/archety",
    logType="deploy",
    filter="venue",
    lines=50
)

Check Conversation History¶

mcp__archety-dev__get_conversation_history(
    user_id="+15550001001",
    persona_id="sage",
    limit=10
)

Check Memory State¶

mcp__archety-dev__get_memories(
    user_id="+15550001001",
    persona_id="sage",
    limit=10
)

Performance Benchmarks¶

Metric	Target	Notes
Trip Creation	< 3s	Initial session setup
Venue Resolution	< 5s	Includes Google Places API call
Query/Filter	< 2s	Local filtering operation
Concurrent Users	20+	Tested with parallel MCP calls