Amazon CloudWatch Monitoring

New in v7.0. Native CloudWatch integration lets you monitor EC2 server metrics (CPU, memory, disk, network) directly from AWS CloudWatch. No agent installation needed.

The metrics flow directly into the load-test result alongside response times, errors, and throughput, which is what enables the AI assistant to do deep root-cause analysis: correlating a response-time spike against a CPU or memory pressure event on a specific instance, naming the cause, and explaining the chain. Without server data in the same dataset, the AI's analysis is limited to client-side symptoms and you do the cross-system correlation by hand. See Why This Matters for AI Analysis for the longer version.

CloudWatch monitoring is the right choice when:

• Your infrastructure runs on AWS EC2
• You want server metrics without installing monitoring agents
• You need cross-account access (a consultant monitoring a client's servers, or an internal team monitoring across AWS accounts)
• You want AI-powered analysis of server metrics integrated with load-test results

---

How CloudWatch Monitoring Works

Load Tester connects to AWS CloudWatch APIs using your AWS credentials to collect EC2 metrics during load tests. Metrics flow directly into your load test results, where the AI assistant can analyze them for bottleneck identification and root cause analysis.

What gets collected: - CPU utilization: Instance CPU percentage - Memory utilization: RAM usage (requires CloudWatch agent on EC2 instances) - Disk I/O: Read/write operations and throughput - Network: Data in/out, packets sent/received - Custom metrics: Any CloudWatch metrics your instances publish

Benefits over agent-based monitoring: - No software installation on servers - Works with any EC2 instance (Windows or Linux) - Uses CloudWatch data you're already collecting - Supports cross-account access (consultant/client scenarios) - Integrated with WPI AI assistant for automated analysis

---

Basic Setup: Same-Account Monitoring

If your AWS credentials have access to the EC2 instances you want to monitor, setup is straightforward.

Prerequisites

AWS IAM Permissions - Your IAM user needs these permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "ec2:DescribeInstances",
        "ec2:DescribeVolumes",
        "ec2:DescribeRegions",
        "ec2:DescribeTags"
      ],
      "Resource": "*"
    }
  ]
}

All permissions are read-only. Load Tester cannot start, stop, modify, or terminate any resources.

---

Configure CloudWatch Account

1. Open Preferences:
2. Windows/Linux: Window → Preferences → Web Performance → Accounts → Amazon CloudWatch

3. macOS: Load Tester → Preferences → Web Performance → Accounts → Amazon CloudWatch

4. Click Add to create a new monitoring account

5. Fill in account details:

6. Name: Descriptive name (e.g., "Production Servers")
7. Access Key ID: Your AWS access key
8. Secret Access Key: Your AWS secret key

9. Regions: Check regions where your EC2 instances run

10. Click Verify Credentials to confirm access works

11. Click OK to save

The monitoring account is now available when configuring server monitoring for load tests.

---

Add CloudWatch Servers to Monitor

Servers are added the same way regardless of monitor type. CloudWatch is one of the monitor types available in the Add Server dialog.

1. Open the Servers View (Window → Show View → Servers, if not already visible)
2. Click Add…
3. In the Monitor Server dialog, select AWS CloudWatch (EC2 instance metrics)
4. Account: pick the CloudWatch monitoring account you configured above
5. Check the EC2 instances to monitor. The dialog lists running instances discovered in the regions configured on the account.
6. Click OK

The selected instances now appear in the Servers View. When you start a load test, CloudWatch metrics flow into the test result alongside response times, and live values appear in the Servers tab, updated every sample period (typically 5 to 10 seconds).

---

Cross-Account Monitoring (Consultant/Client)

Cross-account monitoring lets a consultant monitor a client's EC2 instances without the client sharing their AWS credentials. This is the right approach when:

• A consultant or testing engineer monitors client infrastructure
• Client wants to maintain full control over access (can revoke instantly)
• Security policies prohibit sharing AWS access keys

How it works: Client creates a read-only IAM role in their account. Consultant assumes that role using their own AWS credentials. Client can revoke access at any time by deleting the role.

Party	AWS Account	Responsibility
Client	Target account (owns EC2 instances)	Creates IAM role with read-only permissions
Consultant	Services account (runs Load Tester)	Configures monitoring account with Role ARN

---

Step 1: Client Creates IAM Role

The client performs these steps in their AWS account.

1A. Create Trust Policy

In IAM → Roles → Create role:

1. Trusted entity type: AWS account
2. An AWS account: Select "Another AWS account"
3. Account ID: Enter the consultant's AWS account ID (e.g., 898138143696)
4. External ID (optional but recommended): Agree on a shared secret string for additional security

The External ID protects against the "confused deputy" problem. Use it for production setups.

---

1B. Attach Permissions

Create a custom policy with these permissions and attach it to the role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "ec2:DescribeInstances",
        "ec2:DescribeVolumes",
        "ec2:DescribeRegions",
        "ec2:DescribeTags"
      ],
      "Resource": "*"
    }
  ]
}

Security note: All permissions are read-only. The consultant cannot start, stop, modify, or terminate any resources.

---

1C. Name the Role

Use a descriptive name like WPI-CloudWatch-ReadOnly or LoadTester-Monitoring.

---

1D. Share Role ARN with Consultant

After creation, copy the Role ARN from the role summary page:

arn:aws:iam::123456789012:role/WPI-CloudWatch-ReadOnly

Share this ARN with the consultant along with the External ID (if configured).

---

Step 2: Consultant Configures AssumeRole Permission

The consultant needs permission to assume the client's role.

Add AssumeRole Policy to Your IAM User

Ask your AWS administrator to add this policy to your IAM user (or add it yourself if you have permission):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::CLIENT_ACCOUNT_ID:role/ROLE_NAME"
    }
  ]
}

Replace: - CLIENT_ACCOUNT_ID with the client's AWS account ID (from the Role ARN) - ROLE_NAME with the role name (from the Role ARN)

Example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::123456789012:role/WPI-CloudWatch-ReadOnly"
    }
  ]
}

---

Step 3: Consultant Configures Monitoring Account

1. Open Preferences:
2. Windows/Linux: Window → Preferences → Web Performance → Accounts → Amazon CloudWatch

3. macOS: Load Tester → Preferences → Web Performance → Accounts → Amazon CloudWatch

4. Click Add

5. Fill in account details:

6. Name: Descriptive name (e.g., "ClientCo Production Servers")
7. Access Key ID: Your AWS access key (from your services account)
8. Secret Access Key: Your AWS secret key (from your services account)
9. Role ARN: The client's role ARN (e.g., arn:aws:iam::123456789012:role/WPI-CloudWatch-ReadOnly)
10. External ID: The shared secret (if client configured one)

11. Regions: Check regions where client's EC2 instances run

12. Click Verify Credentials to confirm cross-account access works

13. Click OK

---

Step 4: Add CloudWatch Servers to Monitor

Same workflow as same-account monitoring. The only difference is which monitoring account you pick in step 4.

1. Open the Servers View (Window → Show View → Servers, if not already visible)
2. Click Add…
3. In the Monitor Server dialog, select AWS CloudWatch (EC2 instance metrics)
4. Account: pick the cross-account monitoring account you configured in Step 3
5. Check the EC2 instances from the client's account
6. Click OK

When you start a load test, server metrics from the client's account flow into your load test results, visible in the Servers tab.

---

Security & Access Control

What Cross-Account Access Provides

✅ Consultant CAN: - View CloudWatch metrics for EC2 instances - List EC2 instances and their metadata (instance ID, type, tags) - View instance state (running, stopped, etc.)

❌ Consultant CANNOT: - Start, stop, reboot, or terminate instances - Modify instance configuration - Access instance file systems or applications - Create, modify, or delete any resources - See billing information

---

Revoking Access

The client can revoke consultant access instantly:

Option 1: Delete the role 1. Go to IAM → Roles 2. Find the role (e.g., WPI-CloudWatch-ReadOnly) 3. Delete the role

Option 2: Modify trust policy 1. Go to IAM → Roles → [role name] → Trust relationships 2. Edit trust policy to remove consultant's account ID 3. Save

Access is revoked immediately. The consultant cannot assume the role anymore.

---

Temporary Credentials

When Load Tester assumes the client's role, AWS issues temporary credentials that: - Expire automatically (default 1 hour) - Are refreshed automatically by Load Tester during load tests - Cannot be used to assume other roles - Are scoped to the permissions in the role policy only

The client's access keys are never shared. The consultant authenticates with their own credentials and assumes the role. Clean separation.

---

Troubleshooting

Error	Cause	Fix
"User is not authorized to perform sts:AssumeRole"	Consultant's IAM user lacks AssumeRole permission	Add the sts:AssumeRole policy to consultant's IAM user (see Step 2)
"Access denied" on Verify Credentials	Trust policy doesn't allow consultant's account	Verify the Account ID in role's trust policy matches consultant's AWS account ID
"Missing required permissions: cloudwatch:GetMetricStatistics, ec2:DescribeTags"	Role policy is incomplete	Update role policy with all 7 required permissions (see Step 1B)
"IAM permission validation failed for region"	Wrong region selected, or role doesn't cover that region	Verify client's instances are in selected regions; IAM roles are global but instances are regional
No instances discovered	Instances in different region, not running, or wrong account	Check region selection; confirm instances are "running"; verify Role ARN is correct
"External ID mismatch"	External ID in Load Tester doesn't match role configuration	Verify External ID matches exactly what client configured in trust policy

---

Diagnostic Steps

If CloudWatch monitoring isn't working:

1. Verify IAM permissions:
2. Consultant: Confirm sts:AssumeRole policy is attached to your IAM user

3. Client: Confirm role has all 7 required CloudWatch/EC2 permissions

4. Test role assumption manually (consultant):

   aws sts assume-role \
     --role-arn arn:aws:iam::CLIENT_ACCOUNT_ID:role/ROLE_NAME \
     --role-session-name test \
     --external-id EXTERNAL_ID_VALUE
   

Should return temporary credentials. If this fails, the issue is with IAM configuration, not Load Tester.

1. Verify instances are running (client):

   aws ec2 describe-instances --region us-east-1
   

Confirm instances are in "running" state and in the expected region.

1. Check CloudWatch agent (if monitoring memory):
2. Memory metrics require CloudWatch agent installed on EC2 instances
3. CPU, disk, and network metrics are available without agent
4. See AWS CloudWatch Agent documentation

---

CloudWatch vs. WP Server Monitoring Agent

Both approaches collect server metrics, but have different trade-offs:

Feature	CloudWatch Integration	WP Server Monitoring Agent
Agent installation	Not required (EC2 only)	Required (any server)
Supported platforms	AWS EC2 only	Windows, Linux (any host)
Number of metrics	Limited (CPU, disk, network, custom CloudWatch metrics)	Significantly more comprehensive (50+ metrics in Advanced mode: CPU details, memory breakdown, disk I/O, network stats, DB connections, thread pools, JMX, Windows perf counters)
Cross-account access	Built-in (IAM roles)	Not available
Data collection	Automatic via AWS APIs (no manual steps)	Typically requires manual file collection after test (or open firewall ports for live streaming)
Firewall requirements	None (uses AWS APIs)	Ports 1099/1100 for live streaming, OR manual file collection (no ports needed)
Cost	AWS CloudWatch API charges	Included (Basic: CPU/memory), paid upgrade (Advanced: all metrics)
AI analysis	✅ Full integration	✅ Full integration

Use CloudWatch when: - Infrastructure runs on AWS EC2 - You want automatic metrics collection (no manual file retrieval) - Cross-account access is needed (consultant/client) - Basic infrastructure metrics (CPU, disk, network) are sufficient - Firewall restrictions prevent agent communication

Use WP Server Monitoring Agent when: - Servers are not on AWS EC2 (on-premise, other clouds) - You need comprehensive metrics (50+ counters including DB connections, thread pools, GC pauses, detailed memory breakdown) - You want to avoid AWS API charges - CloudWatch agent installation on EC2 is not permitted - Manual file collection after tests is acceptable

Trade-off summary: - CloudWatch: Easier setup, automatic collection, fewer metrics - WP Agent: Far more metrics (especially Advanced mode), but typically requires manual file collection from the server after each test

Both approaches integrate with WPI AI assistant for automated bottleneck analysis.

---

Best Practices

Region Selection

Only select regions where your EC2 instances actually run. Each selected region requires API calls to ec2:DescribeInstances, which increases credential verification time, adds to your AWS API charges (minor, but unnecessary), and clutters the instance selection dialog with empty regions.

Enable only the 2-3 regions you actually use.

---

Instance Tagging

Use EC2 instance tags to identify servers in the instance selection dialog:

Tag	Purpose	Example
Name	Instance display name	prod-web-01
Role	Server function	webserver, database, appserver
Environment	Deployment stage	production, staging, dev

Load Tester displays instance tags in the selection dialog, making it easy to find the right servers.

---

Monitoring Account Naming

Use descriptive names that indicate: - Client/project (for consultants): "Acme Corp Production", "ClientCo Staging" - Environment: "Production Servers", "Staging Environment" - Region (if multiple): "Production US-East-1", "Production EU-West-1"

Avoid generic names like "CloudWatch" or "AWS Account". Future-you will thank present-you when you're managing ten clients.

---

Security: External ID Usage

Always configure an External ID when setting up cross-account access for production environments. The External ID: - Prevents "confused deputy" attacks (accidental role assumption) - Acts as a second authentication factor - Can be revoked by changing it in the trust policy (forces re-configuration)

Use a randomly generated string (not a password you use elsewhere):

# Generate External ID (Linux/macOS):
openssl rand -hex 16

---

Related Topics

• Server Monitoring Introduction - Overview of monitoring approaches and when to use CloudWatch
• Server Monitoring Agent - Alternative agent-based monitoring for non-AWS environments
• Metrics & Counters - Understanding server metrics and their meaning
• Monitoring Through Firewalls - Firewall configuration for agent-based monitoring
• Server Performance Checklist - Systematic bottleneck identification