AI for Load Test Monitoring¶

Stage 2 is where the AI assistant becomes your real-time co-pilot. During active load tests, the AI monitors metrics as they arrive, alerts you to problems, and helps you determine whether issues are configuration errors (back to Stage 1) or genuine performance problems (forward to Stage 3).

This is the bridge stage, where test case configuration meets performance reality. Things that worked perfectly with one virtual user have a way of falling apart at fifty.

What Makes Stage 2 Unique¶

Stage 2 monitoring is different from configuration (Stage 1) and post-test analysis (Stage 3):

Stage 2 Inherits from Both Stages¶

From Stage 1 (Configuration): - Config errors often only surface under load - Authentication issues may work with 1 user but fail at 50 - Rate limiting triggers that weren't visible during replay - Connection pool exhaustion from misconfigured sessions

From Stage 3 (Performance): - Response time degradation becomes visible in real-time - Throughput plateaus as bottlenecks emerge - Error rates spike under sustained load - Server resource exhaustion manifests gradually

Stage 2 Has Unique Issues¶

Load-specific problems not seen in other stages: - Connection refused (server can't accept more connections) - Timeouts under load (server responds slowly under stress) - Rate limiting triggered (API throttles excessive requests) - Resource exhaustion (memory leaks, thread starvation)

Key insight: The AI's most valuable job during Stage 2 is answering one question: is this a config problem or a performance problem? The fix for each is completely different.

What AI Helps With During Load Tests¶

Real-Time Monitoring¶

AI can: - Watch metrics as the load test runs - Track response time trends during ramp-up - Monitor error rates and types - Compare current metrics to baseline expectations - Alert you when thresholds are exceeded

Example prompts: - "How is the load test performing right now?" - "Are response times within acceptable range?" - "Why did throughput plateau at 75 users?" - "Alert me if error rate exceeds 1%"

Where AI guidance appears: - Monitoring During a Load Test - Running a Load Test

Error Detection and Classification¶

AI can: - Identify when errors spike during ramp-up - Classify error types (4xx client errors, 5xx server errors, timeouts) - Determine if errors are config-related or load-related - Suggest immediate actions to take

Example prompts: - "Why are 401 errors appearing now when replay succeeded?" - "Errors spiked at 50 users - is this a config issue or performance issue?" - "What do these 503 errors mean?" - "Should I stop the test or let it continue?"

Where AI guidance appears: - Monitoring During a Load Test - Load Test Troubleshooting

Performance Degradation Detection¶

AI can: - Identify when response times start degrading - Compare performance at different user levels (10 vs 50 vs 100 users) - Detect sudden spikes or gradual increases - Suggest whether degradation is expected or problematic

Example prompts: - "Response times doubled at 75 users. Is this normal?" - "Why did the checkout page suddenly slow down?" - "Compare response times at 25 users vs 100 users" - "At what user level did performance start degrading?"

Where AI guidance appears: - Monitoring During a Load Test - Throughout Understanding Metrics

Stage Transition Decisions¶

AI can: - Determine if issues are Stage 1 (config) or Stage 3 (performance) problems - Recommend whether to stop and reconfigure or continue and analyze - Identify mixed issues (both config and performance problems) - Guide you to the right troubleshooting approach

Example prompts: - "Are these errors due to misconfiguration or server overload?" - "Should I stop and fix config, or is this a performance bottleneck?" - "Is this a correlation issue or genuine load handling problem?" - "Do I need to go back to Stage 1 or proceed to Stage 3 analysis?"

Where AI guidance appears: - Monitoring During a Load Test - Debugging Failed Replays - Performance Analysis Workflow

Server Resource Monitoring¶

AI can: - Interpret server CPU, memory, and disk metrics - Correlate server metrics with application performance - Identify resource exhaustion patterns - Suggest whether bottleneck is in application or infrastructure

Example prompts: - "Server CPU is at 95% during the test. What does this mean?" - "Memory usage is increasing steadily. Is this a memory leak?" - "Response times are slow but CPU is only 30%. What's the bottleneck?" - "Which server resources are constraining performance?"

Where AI guidance appears: - Server Monitoring Introduction - Server Metrics & Counters - Server Performance Checklist

Common Monitoring Scenarios¶

Scenario 1: Errors Appear During Ramp-Up¶

Your situation: Replay succeeded, but errors appear as virtual users increase.

Ask the AI:

My load test was running clean until 50 users, then 401 errors
started appearing. Replay succeeded with 1 user. What's wrong?

AI will help you: - Determine if session cookies are being reused incorrectly - Check if connection pooling is exhausting sessions - Identify rate limiting or authentication token issues - Decide: Is this Stage 1 (config) or Stage 2 (load-specific)?

Likely diagnosis: A configuration issue that only manifests under load. Back to Stage 1.

Next: Debugging Failed Replays

Scenario 2: Response Times Degrade Under Load¶

Your situation: Test starts fast but slows down as users ramp up.

Ask the AI:

Response times started at 200ms with 10 users but increased to
2000ms at 100 users. Error rate is still 0%. What's causing this?

AI will help you: - Identify if degradation is linear or sudden - Check if specific pages are bottlenecks - Correlate with server resource usage - Determine if this is expected behavior or a problem

Likely diagnosis: A genuine performance issue. Forward to Stage 3 for analysis.

Next: Performance Analysis Workflow

Scenario 3: Throughput Plateaus¶

Your situation: Throughput stops increasing even as more users are added.

Ask the AI:

Throughput increased steadily up to 75 users, then plateaued.
Adding more users doesn't increase throughput. Why?

AI will help you: - Identify if you've hit a bottleneck - Check for connection limits or thread pool exhaustion - Analyze which resources are constraining throughput - Suggest whether the bottleneck is fixable

Likely diagnosis: A performance bottleneck. Forward to Stage 3 for root cause analysis.

Next: Identifying Bottlenecks

Scenario 4: Connection Refused Errors¶

Your situation: Load test starts getting "Connection refused" or timeout errors.

Ask the AI:

I'm seeing "Connection refused" errors as users ramp above 80.
The server is still responding to some requests. What's happening?

AI will help you: - Determine if server hit connection limit - Check if load balancer or firewall is blocking connections - Identify if application crashed or became unresponsive - Suggest immediate actions (stop test, check server status)

Likely diagnosis: A load-specific issue unique to Stage 2. The server is simply overwhelmed.

Next: Load Test Troubleshooting

Scenario 5: Mixed Config and Performance Issues¶

Your situation: Some errors are 401, others are 500, response times are also degrading.

Ask the AI:

I'm seeing 401 errors on some requests, 500 errors on others,
and response times are increasing. Which problem should I fix first?

AI will help you: - Prioritize issues (config errors block valid testing) - Separate correlation problems from server errors - Determine if 500 errors are due to load or application bugs - Guide you through systematic troubleshooting

Likely diagnosis: Mixed issues. Fix the Stage 1 config problems first (they pollute your performance data), then retest and analyze.

Next: Debugging Failed Replays → Performance Analysis Workflow

Scenario 6: Server Resource Exhaustion¶

Your situation: Server CPU or memory maxes out during the test.

Ask the AI:

Server CPU hit 100% at 60 users and stayed there. Response times
spiked to 10+ seconds. Is the server undersized or is there a bug?

AI will help you: - Correlate resource usage with performance degradation - Identify if this is expected capacity limit - Suggest whether issue is infrastructure or application code - Recommend next steps (capacity planning vs optimization)

Likely diagnosis: A performance issue. Forward to Stage 3 for bottleneck analysis.

Next: Identifying Bottlenecks

Real-Time AI Interaction During Load Tests¶

Continuous Monitoring Pattern¶

Start of test:

I'm starting a load test with 100 users ramping over 5 minutes.
What should I watch for?

During ramp-up (if issues appear):

Errors just started appearing at 45 users. What type of errors
and what do they mean?

During steady state:

We're at target load (100 users) but response times are 3x slower
than baseline. Is this acceptable or a problem?

End of test:

Test completed. Should I run analysis now or do I need to retest
with different configuration?

Alert-Driven Pattern¶

Set up alerts:

Alert me if:
- Error rate exceeds 1%
- Response time exceeds 3000ms
- Throughput drops by more than 20%

When alert triggers:

You alerted that error rate exceeded 1%. Show me what's happening
and suggest what to do.

When to Use AI vs Manual Monitoring¶

Use AI when:¶

✅ You need real-time interpretation - "What do these metrics mean right now?"

✅ You see unexpected behavior - "Why did response times suddenly spike?"

✅ You're unsure whether to stop - "Should I stop this test or let it continue?"

✅ You need to triage issues - "Which problem should I investigate first?"

✅ You want automatic alerts - "Notify me if error rate exceeds threshold"

Use Manual Monitoring when:¶

📊 You're tracking expected metrics - Watching normal ramp-up progress

📊 You have specific KPIs to hit - Verifying response time stays under 500ms

📊 You're running known-good tests - Regression testing with stable baselines

📊 You want detailed charts - Visual analysis of metric trends

Effective Monitoring Prompts¶

✅ Good Monitoring Prompts¶

Context-specific and actionable: - "Error rate just spiked from 0% to 5% at 70 users. What type of errors and why?" - "Response times were stable at 300ms up to 50 users, then jumped to 1200ms. What changed?" - "I'm seeing 'Connection refused' errors. Is the server down or just overloaded?"

Comparative and analytical: - "Compare performance now (80 users) with baseline (10 users). What's degraded?" - "Why is the checkout page 10x slower than other pages under load?" - "Server CPU is at 40% but response times doubled. Where's the bottleneck?"

❌ Avoid Vague Prompts¶

Too general: - "How's my test doing?" (What metrics concern you?) - "Is this normal?" (What behavior? Compared to what baseline?) - "Something seems wrong" (What, specifically?)

Missing context: - "Why errors?" (What errors? When did they start? Which pages?) - "Should I stop?" (Why are you considering stopping? What's the issue?)

AI Monitoring Workflow¶

Before Load Test¶

Set expectations with AI:

I'm about to run a load test ramping to 100 users over 10 minutes.
What are typical warning signs to watch for?

During Load Test¶

Monitor key phases:
Early ramp (0-25% of target users): Check for config errors
Mid ramp (25-75%): Monitor for performance degradation
Peak load (75-100%): Watch for resource exhaustion
Steady state: Verify stable performance

Ask AI when issues appear:

[Describe what you're seeing] What's causing this and should I
stop the test?

After Load Test¶

Determine next steps:
If config errors: Stop, fix correlation, retest
If performance issues: Proceed to Stage 3 analysis
If both: Fix config first, then retest and analyze

Next stage: AI for Performance Analysis

Next Steps¶

Learn AI for analysis - Investigate performance after tests complete (Stage 3)
Review AI configuration - Fix config issues found during monitoring (back to Stage 1)
Understand AI limitations - What AI can and cannot do

Or: Return to Load Testing and Server Monitoring sections where AI guidance is embedded: