Skip to content

Cloud Load Testing

Cloud engines let you generate load from anywhere in the world without maintaining your own hardware. Instead of buying and configuring dozens of physical servers (which sit idle 99% of the time), you spin up AWS EC2 instances on-demand for a few dollars per hour. When the test finishes, you shut them down. No maintenance, no idle hardware costs.

Why use cloud engines: You can test from multiple geographic regions simultaneously (Virginia, California, Ireland, Singapore). You can scale instantly: need 50 engines for a major product launch, spin them up in 5 minutes. The cost is $0.50-2.00 per engine per hour, which beats leasing hardware by any measure. And your test traffic comes from thousands of real IPs instead of one office network.

This guide shows you how to:

  • Configure your AWS account for Load Tester
  • Launch cloud engines across multiple regions
  • Run load tests from the cloud
  • Optimize costs (avoid surprise AWS bills)
  • Troubleshoot cloud-specific issues

Prerequisites

Before using cloud engines, you need:

  1. AWS account - Sign up at aws.amazon.com
  2. AWS credentials - Access Key ID and Secret Access Key (IAM user recommended)
  3. Test case configured - Should replay successfully using local engine first
  4. Understanding of AWS billing - Cloud engines accrue charges until terminated

Don't skip local testing. Always verify your test case replays successfully using Load Tester's built-in engine before launching cloud engines. If your test case fails locally due to configuration issues, it will fail in the cloud too. The only difference is that you'll be paying by the hour while you troubleshoot.


How Cloud Engines Work

Load Tester manages cloud engines automatically. You configure AWS credentials once in preferences. When configuring a load test, you request engines ("10 in US-East, 5 in EU-West"). Load Tester launches EC2 instances in the specified regions, the engines connect back to your workstation (which acts as controller), the load test runs with engines generating HTTP traffic to your servers, and Load Tester shuts down the engines when the test completes (or you terminate them manually).

Network architecture:

graph TD
    LT["Your Workstation<br/>Load Tester Controller"]
    subgraph aws["AWS EC2 Engines (multiple regions)"]
        E1["Engine"]
        E2["Engine"]
        E3["Engine"]
    end
    WS["Your Web Servers<br/>Test Target"]

    LT -->|Test configuration<br/>ports 1099/1100| E1
    LT -->|Test configuration<br/>ports 1099/1100| E2
    LT -->|Test configuration<br/>ports 1099/1100| E3
    E1 -->|HTTP traffic| WS
    E2 -->|HTTP traffic| WS
    E3 -->|HTTP traffic| WS

    classDef controller fill:#fff,stroke:#333,stroke-width:1.5px,color:#333
    classDef engine fill:#f5f5f5,stroke:#666,stroke-width:1.5px,color:#333
    classDef server fill:#fff,stroke:#c0392b,stroke-width:1.5px,color:#333
    class LT controller
    class E1,E2,E3 engine
    class WS server

Key point: Your workstation must be able to reach cloud engines on ports 1099 and 1100 (RMI communication). If your corporate firewall blocks outbound connections to those ports, you'll need firewall rules or Elastic IPs (see Elastic IPs for Firewall Traversal).

One-Button Cloud Setup

The reason this page is short on operational steps is that there aren't many. The hard parts of distributed cloud load testing are solved problems from the operator's perspective. Click Run Load Test with cloud engines configured, and Load Tester handles:

  • Provisioning the EC2 instances in the regions and sizes you specified
  • Installing the load generator software on each instance (the engine image is preconfigured, you don't install anything on it by hand)
  • Uploading every test case to every engine, including mixed-workload tests that run multiple test cases simultaneously
  • Distributing the datasets so each engine has its slice of usernames, passwords, search terms, and other per-user data
  • Load-balancing virtual users across engines based on each engine's measured Capacity (the default policy is Distribute evenly (respect capacity), visible in the Engines View toolbar). Engines with more headroom get more users, engines with less get fewer, and no single generator gets pushed to where its own resource consumption distorts measurements. See the live Engines View for the per-engine Capacity numbers in action.
  • Terminating the instances when the test finishes (or after an idle timeout, configurable), so you only pay for the minutes you actually ran

Load balancing across generators: solved. Software distribution: solved. Dataset distribution: solved. Multi-test-case orchestration: transparent. The complicated parts of running a hundred-load-generator distributed test happen below the surface. You don't manage them, you don't see them, you don't have to be right about them.

What's left for you to do is configure AWS credentials once, decide how many engines you want and where, and click run. The remaining sections of this page walk through those three steps.


Step 1: Configure AWS Credentials

Load Tester uses AWS SDK v2.0 to manage EC2 instances. You configure credentials once, then launch engines whenever you need them.

Preferences → AWS Load Generation pane listing configured cloud accounts across five regions, with Add / Remove / Edit / Duplicate buttons

Create IAM User for Load Tester

Best practice: Create a dedicated IAM user for Load Tester instead of using root credentials.

  1. Log into AWS ConsoleIAMUsersCreate User
  2. Username: LoadTesterEngines (or any meaningful name)
  3. Access type: Programmatic access (generates Access Key ID + Secret)
  4. Permissions: Attach AmazonEC2FullAccess policy (or create custom policy limiting to ec2:DescribeInstances, ec2:RunInstances, ec2:TerminateInstances)
  5. Save credentials: Download CSV with Access Key ID and Secret Access Key

Why a dedicated IAM user: If credentials are compromised, you revoke this one user without affecting your root account. You can also set spending limits and restrict permissions to EC2 only. Principle of least privilege.


Add AWS Account to Load Tester

  1. Open Load TesterWindow → Preferences → Web Performance → Accounts → AWS Load Generation
  2. Click Add...
  3. The Amazon EC2 Account dialog opens. Configure the account:

Amazon EC2 Account dialog with Name, Access Key ID, Secret Access Key, Region dropdown, VPC/Subnet selectors, Security key dropdown, Create security group checkbox, Custom AMI IDs list, and Verify button

Field Value Notes
Name US-East (or any descriptive name) Helps you identify which region this account targets
Access Key ID From IAM user CSV 20-character alphanumeric string (e.g., AKIAIOSFODNN7EXAMPLE)
Secret Access Key From IAM user CSV 40-character string (keep secret)
Region us-east-1 (or desired region) See Available Regions
VPC (Use account default) Leave on default unless you need engines launched in a specific VPC for network reachability
Subnet (Use default subnet) Auto-selects based on the VPC choice; override only when you need a specific subnet
Security key A unique key (e.g., your name) Used to isolate engines when multiple testers share the same AWS account. See Sharing AWS Accounts Among Multiple Testers.
Create security group if needed Checked (default) Load Tester creates an EC2 security group permitting RMI traffic between controller and engines. Uncheck only if your security policy forbids automatic SG creation, in which case you'll create one manually.
Custom AMI IDs (empty) Advanced: pin engines to a specific AMI per region. Leave blank to use the default Load Tester engine AMI.
  1. Click Verify... to confirm Load Tester can authenticate with AWS using the supplied credentials. Verification hits the EC2 API in the selected region. Fix any errors before saving.
  2. Click OK

Result: Load Tester can now launch engines in the specified AWS region.


Step 2: Configure Multiple Regions

Real-world load comes from multiple geographic locations. Not everyone accesses your site from Virginia. Configure multiple AWS regions to simulate traffic from North America, Europe, and Asia simultaneously.

Same AWS Load Generation pane showing five accounts configured across Montreal, Virginia, California, Calgary, and Ohio. Same image, referenced again here because multi-region setup looks identical to single-account setup in the preferences list.

Duplicate Existing Account

Instead of re-entering AWS credentials, duplicate the existing account and change only the region:

  1. In Cloud Accounts preferences, select your existing account (e.g., US-East)
  2. Click Duplicate button
  3. Update the duplicated account:
  4. Account Name: EU-West (matches new region)
  5. Region: eu-west-1 (or desired region)
  6. Access Key ID / Secret: Same as original (AWS credentials work across all regions)
  7. Click OK
  8. Repeat for additional regions (e.g., us-west-2, ap-southeast-1)

Result: You now have multiple cloud accounts in Load Tester, each targeting a different AWS region. You can mix engines from different regions in a single load test.


Available Regions

Load Tester supports AWS regions worldwide:

Region Code Location Typical Use Case
us-east-1 Virginia (US East) North American users, default AWS region
us-east-2 Ohio (US East) North American users, lower latency to central US
us-west-1 California (US West) Western US users
us-west-2 Oregon (US West) Western US / Pacific Northwest users
eu-west-1 Ireland (EU) European users
eu-central-1 Frankfurt (Germany) Central European users
ap-southeast-1 Singapore (Asia-Pacific) Southeast Asian users
ap-southeast-2 Sydney (Australia) Australian / Pacific users
ap-northeast-1 Tokyo (Japan) Japanese / East Asian users
sa-east-1 São Paulo (South America) South American users

Choosing regions: Match your user distribution. If 60% of users are in North America, 30% in Europe, 10% in Asia, use 12 US engines, 6 EU engines, 2 Asia engines. To test geographic performance, launch identical load from US-East and EU-West and compare response times (this reveals CDN effectiveness and geographic routing issues). For regulatory compliance, test with EU engines to verify that European users are being served from EU datacenters.


Step 3: Launch Cloud Engines

With AWS accounts configured, you can now launch engines for a load test.

Configure Load Engines dialog with Total Engines = 20, a Saved Load Engines section listing the local engine, and a Cloud Engines section with five regions (Montreal, Calgary, Ohio, Virginia, California) each set to 4 engines

Configure Load Test with Cloud Engines

  1. Open a test case in Navigator
  2. File → New → Load Configuration (or open an existing load configuration)
  3. In the Load Configuration Editor, find the Infrastructure panel and open Configure Load Engines
  4. The Configure Load Engines dialog opens. It has two sections:
    • Saved Load Engines lists locally-installed Load Engines (your own machines). Check Use selected engines only if you want to restrict the test to those local engines and skip the cloud entirely.
    • Cloud Engines lists every AWS account you configured in Step 1, with an Engines column where you type the engine count per region.
  5. Set per-region engine counts in the Cloud Engines table. Set each row to the number of engines you want from that region. Setting a row to 0 excludes that region. The Total Engines badge at the top updates live as you type.
  6. Click Save

Engine distribution:

  • Total engines: Sum of every region's engine count (in the screenshot, 5 regions × 4 engines = 20 total)
  • Virtual users per engine: Load Tester automatically distributes the test's total virtual users across the engines (for example, 3,000 VUs / 20 engines = 150 VUs per engine, balanced so no single engine is pushed to the point where its own resource consumption distorts measurements)

Instance startup time: roughly 3-5 minutes per engine (AWS boots the OS, Load Tester engine software starts, engine connects back to your workstation). Use Save & Connect Now if you want Load Tester to provision the engines immediately so they're ready when you click Start Load Test. Plain Save defers provisioning until test start.


Start the Load Test

  1. Click Run (▶️) in Load Test Configuration
  2. Confirm engine launch popup appears: "Launch 15 cloud engines?"
  3. Click Yes

What happens:

  • Load Tester sends EC2 RunInstances API call for each region
  • AWS starts EC2 instances (Medium High-CPU instance type by default)
  • Engines boot (takes 3-5 minutes)
  • Engines connect to Load Tester (appear in Engines View as "Connected")
  • Load test configuration is sent to engines (test case, VU assignment, load profile)
  • Load test starts when all engines are ready

Monitoring engine startup:

  • Open Engines View: Window → Show View → Engines
  • Watch for engines to appear as they connect (shows hostname, region, status)
  • If engine shows "Starting" for 10+ minutes: Likely network issue (firewall blocking ports 1099/1100)

Understanding Cloud Engine Performance

Cloud engines are virtual machines that share physical hardware with other AWS customers. This affects performance and requires different capacity planning than dedicated hardware.

Virtual Users Per Engine

How many VUs can one engine simulate? For typical test cases, 500-1,000 VUs per engine (simple HTTP traffic, minimal SSL). For SSL-heavy workloads, 200-500 VUs per engine because SSL handshakes are CPU-intensive. Load Tester enforces a hard maximum of 3,000 VUs per engine to prevent overload.

Why the range is so wide: downloading large files (videos, PDFs) exhausts bandwidth before CPU. High request rates exhaust CPU before bandwidth. Extractors, validators, and datasets add CPU overhead. And then there's the noisy-neighbor problem, where other AWS customers on the same physical hardware consume resources you were counting on.

Best practice: Use more engines than the minimum required. If you need 5,000 VUs and 1,000 VUs/engine is typical, use 7-8 engines instead of 5. The extra engines provide buffer for variability and improve test realism by generating traffic from more diverse IP addresses.


Bandwidth Expectations

Each cloud engine has a 100 Mbps network connection, but due to virtualization overhead and shared infrastructure, you should plan for:

  • Conservative estimate: 70 Mbps per engine (use this for capacity planning)
  • Peak observed: 100+ Mbps per engine (under ideal conditions)
  • Reality: Variability is normal; some engines may get 60 Mbps, others 90 Mbps

Bandwidth varies because multiple virtual machines share the same physical NIC, other customers' workloads affect available resources, and geographic routing means that US-East to US-West traffic may traverse different network paths with different congestion levels.

Validating bandwidth:

  1. Open Engines View during load test
  2. Check "Bandwidth" column for each engine
  3. If any engine shows bandwidth at/near 100 Mbps: Test results may be invalid (bottleneck is engine bandwidth, not server capacity)
  4. If bandwidth is consistently maxed out: Add more engines to distribute load

Bandwidth Test Wizard: Tools → Bandwidth Test Wizard runs a pre-configured bandwidth test to verify engine capacity before running your actual load test.


Self-Regulating Engines

Load Tester engines monitor their own resource usage and automatically reduce load if they detect overload conditions:

  • CPU utilization > 90%: Engine stops accepting new VUs
  • Memory exhaustion: Engine warns and stops accepting new VUs
  • Bandwidth saturation: Engine reports bandwidth limit (visible in Engines View)

This prevents invalid test results. If an engine is overloaded, response times artificially increase, and you end up measuring the engine's limitations instead of the server's.

If engines self-regulate during your test:

  • Good: Engines prevented invalid data collection
  • Problem: You need more engines to achieve target VU count
  • 🔧 Solution: Add engines and re-run test

Optimizing Cloud Costs

Cloud engines cost $0.50-2.00 per engine per hour depending on instance type and region. If you're not careful, you can rack up hundreds of dollars in unexpected charges. This section exists because people have learned that lesson the hard way.

Amazon Billing Rules

Amazon charges for cloud instances by the hour. Partial hours round up, so a 1-minute test gets billed for a full hour. The clock starts when the instance state changes to "running" (OS booting) and stops only when the instance is terminated, not stopped. Stopping still accrues charges.

Example:

  • Test 1: 10-minute test, terminate engines immediately → billed for 10 engine-hours (1 hour × 10 engines)
  • Test 2: 65-minute test, terminate engines immediately → billed for 20 engine-hours (2 hours × 10 engines, because 65 minutes rounds up to 2 hours)
  • Test 3: Run 3 tests back-to-back (10 min, 15 min, 10 min), keep engines running between tests, terminate after last test → billed for 10 engine-hours (1 hour × 10 engines) instead of 30 engine-hours (3 hours × 10 engines)

The takeaway: If you're running multiple tests in one session, keep engines running between tests instead of terminating and restarting. You're paying for the hour either way.


When to Keep Engines Running

If you plan to run multiple tests within an hour, keep engines running:

  • Example: Running 5 tests over 2 hours → keep engines running entire time → billed for 2 hours
  • Counterexample: Starting/stopping engines for each of 5 tests → billed for 5 hours (5 × 1-hour minimum)

When to terminate engines immediately:

  • Single test: No more tests planned → terminate immediately after test completes
  • End of workday: No more tests for hours/days → terminate to avoid idle charges
  • Uncertain schedule: Don't know when next test will run → terminate to be safe

Automatic idle shutdown (enabled by default) terminates idle engines after 30 minutes of inactivity. This prevents surprise charges if you forget, but always verify in the AWS Console that engines are actually terminated. Trust, but verify.


Terminating Cloud Engines

Load Tester provides multiple ways to terminate engines:

Option 1: End-of-Test Prompt

When a load test finishes, Load Tester prompts: "Terminate cloud engines?"

  • Click Yes: Engines are terminated immediately (stops billing)
  • Click No: Engines remain running (useful if running another test soon)

Terminate cloud load engines prompt: "Would you like to terminate load engines running in the cloud? Press Yes to terminate. Press No to leave cloud engines running (hourly charges will continue until they are terminated)." With a "Remember this choice next time" checkbox and Yes / No buttons.


Option 2: Engines View (Manual Termination)

To terminate engines at any time:

  1. Open Engines View: Window → Show View → Engines
  2. Select engine(s) to terminate (Shift-click to select multiple)
  3. Click Remove (🗑️ icon)
  4. Confirm termination → Engines shut down and are terminated in AWS

Engines View showing two cloud engines (one Terminated, one Disconnected and selected), the Remove button highlighted in the action row, plus the "Distribute evenly (respect capacity)" load-balancing selector and per-engine columns for Users, Capacity, CPU %, VM Memory, OS Memory, and Disk %.

When engines are terminated:

  • Status changes to "Terminating" (takes ~1 minute)
  • Engine disappears from Engines View once terminated
  • AWS Console shows instance as "Terminated"

Option 3: Automatic Idle Shutdown

By default, Load Tester terminates engines that idle for 30 minutes:

  • To configure idle timeout: Tools → Preferences → Cloud AccountsIdle shutdown period
  • Recommended setting: 30 minutes (default)
  • When engines idle: Countdown timer starts; at 0 minutes, engines are terminated automatically

When to disable idle shutdown: If you intentionally leave engines running for hours between tests (rare).


Option 4: AWS Console (Emergency Termination)

If Load Tester crashes or you lose connection to engines:

  1. Log into AWS ConsoleEC2 Dashboard
  2. Select region (top-right dropdown) → Check each region where you launched engines
  3. View running instances → Look for instances with "WebPerformanceLoadEngine" in the name
  4. Select instancesActions → Instance State → Terminate

Make this a habit: Check the AWS Console at the end of every testing session. Don't rely solely on Load Tester's automatic shutdown. The $500 surprise bill only happens once before you start checking.


Testing Multiple Regions

Simulating traffic from multiple geographic regions makes load tests more realistic and can reveal geographic performance issues.

Engines View with two cloud engines connected from different regions: one in US East (Virginia) and one in Canada West (Calgary). Each engine's account and region are visible in the Name column, and each is assigned a distinct Color (red for Virginia, gold for Calgary) used to distinguish their contributions in load-test charts.

Configuring Multi-Region Tests

To run a load test using engines from 3 regions:

  1. In Load Configuration Editor, add engines from multiple cloud accounts:
  2. US-East: 10 engines
  3. EU-West: 5 engines
  4. AP-Southeast: 3 engines
  5. Total: 18 engines across 3 regions
  6. Start load test

What happens:

  • Load Tester launches engines in all 3 regions simultaneously
  • Virtual users are distributed across all engines (e.g., 3600 VUs / 18 engines = 200 VUs per engine)
  • HTTP traffic originates from all 3 regions at the same time
  • Server sees requests from Virginia, Ireland, and Singapore IPs

Analyzing Regional Performance

Different regions will experience different response times due to network latency:

Region Typical Latency to US Server What This Means
US-East 1-10 ms Fast (same datacenter or nearby)
US-West 50-80 ms Moderate (cross-country)
EU-West 80-150 ms High (transatlantic)
AP-Southeast 150-250 ms Very high (halfway around the world)

Analyzing results:

  1. After load test completes, open Load Test Results View
  2. Group results by engine (or by region if you named accounts descriptively)
  3. Compare average response times:
  4. US-East engines: 50 ms average
  5. EU-West engines: 180 ms average
  6. AP-Southeast engines: 280 ms average

What this tells you: European users experience 3.6x slower responses than US users. If you use a CDN, EU and Asia response times should be much closer to US times (the CDN serves content from local edge locations). If EU times are 10x slower than expected, you have a routing problem worth investigating.


Elastic IPs for Firewall Traversal

By default, cloud engines get random public IPs from Amazon's pool. This causes two problems. Load Tester can't connect to engines if your corporate firewall blocks outbound connections to random IPs. And cloud engines can't reach your test servers if your server firewall only allows traffic from known IPs.

Solution: Use Amazon Elastic IPs (static IPs that don't change across engine launches).


Purchasing Elastic IPs

  1. Log into AWS ConsoleEC2 → Elastic IPs
  2. Click "Allocate Elastic IP address"
  3. Select region (must match the region where you'll launch engines)
  4. Click "Allocate"
  5. Repeat for however many engines you plan to run (one Elastic IP per engine)

Cost: ~$0.005 per hour per Elastic IP when associated with a running instance (free when instance is running, small charge when instance is stopped)


Assigning Elastic IPs to Engines

After launching cloud engines:

  1. In AWS Console, go to EC2 → Instances
  2. Identify Load Tester engine instances (named "WebPerformanceLoadEngine...")
  3. Select an instanceActions → Networking → Associate Elastic IP
  4. Choose an Elastic IP from the list → Associate
  5. Repeat for each engine

Now you must reboot engines and restart Load Tester:

  1. In AWS Console, select all engine instances → Actions → Instance State → Reboot (do NOT terminate!)
  2. Wait ~2 minutes for engines to reboot
  3. Close Load Tester and restart it
  4. Engines reconnect using the Elastic IPs

Why reboot and restart: The engine software started listening on the original IP before you assigned the Elastic IP, so it doesn't detect the change automatically. And Load Tester cached the original IP, so it needs to re-query AWS to discover the new Elastic IPs.


Configuring Firewalls for Elastic IPs

Now that engines have fixed IPs, configure your firewalls:

Firewall Rule 1: Allow your workstation to connect to cloud engines

  • Source: Your workstation IP
  • Destination: Elastic IP(s) of cloud engines
  • Ports: TCP 1099, 1100 (RMI communication)
  • Direction: Outbound

Firewall Rule 2: Allow cloud engines to reach test servers

  • Source: Elastic IP(s) of cloud engines
  • Destination: Your test server IPs
  • Ports: TCP 80, 443 (HTTP/HTTPS), or whatever ports your application uses
  • Direction: Inbound (if server firewall is restrictive)

Contact your network administrator to implement these rules.


Troubleshooting Cloud Engines

Problem: Load Tester Can't Connect to Engines

Symptom: Engines show "Starting" in Engines View for 10+ minutes, or "Unable to connect" error appears

Causes:

  1. Firewall blocks ports 1099/1100: Your corporate firewall blocks outbound connections
  2. Incorrect AWS credentials: Load Tester can't launch instances
  3. AWS region mismatch: Engines launched in different region than expected
  4. Network routing issue: Load Tester can't reach AWS public IPs

Diagnostic steps:

  1. Check AWS Console: Are instances actually running? (EC2 → Instances in correct region)
  2. If instances aren't running: Check Load Tester's error log (likely AWS credential issue)
  3. If instances are running: Network/firewall issue

Fixes:

  • Firewall issue: Configure firewall rules for ports 1099/1100, or use Elastic IPs (see Elastic IPs for Firewall Traversal)
  • AWS credentials wrong: Tools → Preferences → Cloud Accounts → Verify Access Key ID and Secret match IAM user
  • Wrong region: Check cloud account region setting matches AWS Console region

Problem: Test Fails with "Unable to establish connection to the server"

Symptom: Load test runs, but transactions fail with "Unable to establish connection to the server: testserver:1080"

Cause: Cloud engines can't resolve internal hostnames because they're outside your network.

Example:

  • Recording used hostname: testserver or testserver.local (internal DNS)
  • Cloud engines can't resolve: They have no access to your internal DNS

Solutions:

Solution 1: Use Public Hostname

If your test server is publicly accessible:

  1. Re-record test case using the public hostname (e.g., www.example.com instead of testserver)
  2. Or: Use mass-edit to replace hostnames (Edit → Find/Replace in Test Case Editor)
  3. Run replay locally to verify (using local engine)
  4. Then run cloud test

Solution 2: Test Non-Public Sites with VPN

If your test server is internal-only:

  • Option A: Use on-premise engines instead of cloud engines (install Load Tester engine software on servers inside your network)
  • Option B: Set up VPN connection from AWS to your network (requires coordination with network admin)
  • Option C: Use AWS Direct Connect (dedicated network link between AWS and your datacenter)

Realistically, most organizations use cloud engines for testing public-facing applications. If your app isn't publicly accessible, on-premise engines are almost always simpler than setting up VPN or Direct Connect.


Problem: Bandwidth Bottleneck

Symptom: Load test runs, but Engines View shows engines at/near 100 Mbps bandwidth consistently

Cause: Engine bandwidth is the bottleneck, not server capacity. Test results are invalid.

Fix:

  1. Add more engines to distribute bandwidth load (e.g., increase from 5 engines to 10)
  2. Reduce bandwidth-heavy operations in test case (if downloading large files, consider removing them for load testing)
  3. Run Bandwidth Test Wizard first to verify each engine's capacity (Tools → Bandwidth Test Wizard)

Preventing this:

  • Plan engine count based on bandwidth: If test case downloads 50 MB per user, and each engine has 70 Mbps capacity → ~11 users per engine max (50 MB × 8 bits/byte = 400 Mbits, 400/70 = ~6 seconds per user, or ~10 users/min per engine)

Problem: Surprise AWS Bill

Symptom: Amazon charged $500 for 200+ hours of instance time, but you only ran a few tests

Cause: Engines weren't terminated after tests finished. They ran for days/weeks idle.

Diagnosis:

  1. Check AWS Console billing → See which instances ran and for how long
  2. Check AWS Console instances → See if engines are still running right now
  3. Terminate any running engines immediately (Actions → Instance State → Terminate)

Prevention: Always check the AWS Console at the end of each testing session. Enable automatic idle shutdown (30-minute timeout). Set up AWS billing alerts so you get notified when charges exceed a threshold. And don't leave Load Tester running unattended overnight with cloud engines active.

Refund policy: Amazon does not provide refunds for forgotten instances. You agreed to their terms when creating the account. No exceptions, regardless of how politely or indignantly you ask.


Advanced Topics

Running Load Tester in the Cloud

You can install Load Tester itself on an AWS Windows instance (not just the engines). This is useful if:

  • Your workstation has unreliable internet: Load test continues even if your connection drops
  • Your workstation shares bandwidth with test servers: Installing Load Tester in AWS avoids network contention
  • You need 24/7 load testing: AWS instance runs continuously

Setup:

  1. Launch Windows instance in AWS (t3.large or larger recommended)
  2. Install Load Tester on the Windows instance
  3. Configure AWS credentials for launching engines (same as local installation)
  4. Connect via RDP to run tests remotely

Don't record test cases on cloud Windows instances. The default IE security settings are overly restrictive and don't represent normal user behavior. Always record locally, then upload the WPT files to the cloud Load Tester instance.


Custom Instance Types

By default, Load Tester uses Medium High-CPU instances (optimized for compute-intensive workloads like load generation).

To use different instance types (larger/smaller):

  • Not currently configurable in UI: Contact support if you need custom instance types
  • Why Medium High-CPU: Balances CPU performance, network throughput, and cost

Sharing AWS Accounts Among Multiple Testers

If multiple people share the same AWS account for cloud engines, you can isolate engines using security keys:

  1. Create unique SSH key pair per tester in AWS Console (EC2 → Key Pairs)
  2. Each tester configures their cloud account in Load Tester with their unique key name
  3. Load Tester only shows engines launched with that tester's key

This prevents the unfortunate scenario where Tester A accidentally terminates Tester B's engines mid-test.


Ask the AI to Optimize Cloud Tests

If you're struggling with cloud engine configuration or costs:

I need to run a load test with 10,000 virtual users across US and EU regions.
Can you help me calculate how many engines I need and estimate the cost?

The AI can:

  • Calculate engine requirements based on VU count and test case characteristics
  • Recommend regional distribution based on user geography
  • Estimate AWS costs (instance hours, Elastic IPs if needed)
  • Optimize engine count to minimize costs while avoiding bottlenecks
  • Troubleshoot "Unable to connect" errors by analyzing firewall/network configuration
  • Recommend bandwidth testing procedures before running expensive large-scale tests

Next Steps

After running cloud load tests:

For load testing configuration:

For cost optimization:

  • AWS billing best practices: Set up billing alerts in AWS Console
  • Terminate engines checklist: Always verify in AWS Console at end of day