Skip to content

AI Provider Setup

Before you can use the AI assistant, you need to tell Load Tester which AI provider to use and give it an API key. Two minutes, tops.

Load Tester supports four providers: Anthropic (for Claude models), AWS Bedrock, OpenAI, and Azure OpenAI. Pick whichever your organization already uses, or whichever you have an API key for. There is no wrong answer here.

Opening the AI Preferences

The AI provider settings live in Load Tester's preferences dialog.

On Windows:

  1. Select Window -> Preferences
  2. Expand Web Performance -> Accounts -> AI Assistant

On macOS:

  1. Select Load Tester -> Preferences
  2. Expand Web Performance -> Accounts -> AI Assistant

You'll see fields for provider, API key, model, and region (for Bedrock). Azure OpenAI shows its own fields for the resource endpoint, deployment name, and an optional service version. Everything you need is on this one page.

Choosing a Provider

Select your provider from the Provider dropdown at the top of the preferences page.

Anthropic (Direct API)

Connects directly to Anthropic's Claude API. This is the simplest option: one API key, no infrastructure to configure.

Requirements:

Available models:

  • Claude Sonnet 4.6
  • Claude Opus 4.6
  • Claude Haiku 4.5

AWS Bedrock

Accesses the same Claude models through your existing AWS account. Bedrock pulls IAM credentials from the AWS Load Generation settings you have already configured, so there is no separate API key to manage.

Requirements:

  • AWS credentials configured in Preferences -> Web Performance -> Accounts -> AWS Load Generation
  • Bedrock model access enabled in your AWS account

Region: Defaults to us-east-1. Change it if your Bedrock endpoint is in a different region.

Available models:

  • Claude Sonnet 4.6
  • Claude Opus 4.6
  • Claude Haiku 4.5

Already Using AWS for Load Generation?

If you've configured AWS credentials for cloud load testing, Bedrock is the easiest path. No new API key needed.

OpenAI

Connects to OpenAI's API for GPT models. Use this if your organization standardizes on OpenAI.

Requirements:

Available models:

  • GPT-5.4
  • GPT-4.1
  • GPT-4.1-mini

Azure OpenAI

Connects to a model your organization has deployed in its own Azure subscription. The AI Assistant talks straight to your Azure endpoint over HTTPS, so every request stays inside your Azure tenant. Use this if your organization standardizes on Azure for AI.

You don't have to be an Azure expert to set this up. You need four values, and the person who administers your organization's Azure account can give you all of them:

  • Resource endpoint: the address of your Azure OpenAI resource. It looks like https://your-resource-name.openai.azure.com. It must be an openai.azure.com address; Load Tester rejects anything else so a typo can't send your key to the wrong place.
  • Deployment name: the name your administrator gave the model when they deployed it, for example gpt-4o. On Azure the deployment name is how you choose the model, so there is no separate model dropdown.
  • Access key: the key that authorizes the requests. Unlike the other providers, an Azure key is not prefixed with sk-. It is usually a long string of letters and numbers.
  • Service version (optional): leave this blank for current Azure resources. Enter a dated value (your administrator will give you one, something like 2024-10-21) only if your resource is older and your administrator tells you it needs one. A blank service version targets Azure's modern endpoint; a dated value targets the older one.

Enter the endpoint, deployment name, and key in their fields, leave the service version blank unless told otherwise, and save. If you aren't sure what to ask for, send your administrator one line: "I need the resource endpoint, the deployment name, and an access key for our Azure OpenAI resource, and the service version if it needs one."

Requirements:

  • An Azure OpenAI resource with a deployed chat-capable model (a GPT-5-class deployment is recommended; see below)
  • The resource endpoint, deployment name, and access key for that resource

Setting up the deployment (for your Azure administrator):

If you administer the Azure resource, three choices make the assistant work well:

  • Deploy a GPT-5-class model. gpt-5.4 is a good, cost-effective pick: it costs about half what gpt-5.5 does and clears our hardest correlation cases. A gpt-4o deployment handles simple setup but stalls on complex correlation.
  • Use the GlobalStandard deployment type. It is pay-per-token with no idle cost.
  • Give it enough capacity. The assistant's correlation work is token-heavy and bursty, so a small deployment (say 200K tokens per minute) can hit rate-limit (429) errors mid-task. Provision a generous rate, up to your quota; 1M tokens per minute works comfortably. On GlobalStandard the capacity is only a rate ceiling, so raising it does not raise the bill.

New models may need a quota request

The newest Azure models (for example gpt-5.5) can default to zero quota in a region, so the deployment fails with an "insufficient quota" error until you request an increase on the Azure portal's Quotas page. Slightly older GPT-5-class models (gpt-5.4, gpt-5.2, gpt-5.1) usually already have quota, so they deploy with no wait.

The AI assistant does real multi-step work: detecting platforms, correlating dynamic values, fixing replay errors, and re-running the recording until it replays clean. Capable models do that reliably; smaller ones can stall partway and leave a case half-configured. We test the assistant against a small set of models so we can point you at the ones that hold up.

Provider Recommended model
Anthropic Claude Sonnet 4.6
AWS Bedrock Claude Sonnet 4.6
OpenAI a GPT-5-class model (for example GPT-5.5)
Azure OpenAI a GPT-5-class deployment (for example gpt-5.4)

Claude Sonnet 4.6 and the GPT-5-class models clear our hardest correlation cases (PKCE and OAuth logins, SAML federation, JavaScript bot challenges) from start to finish. Use one of those for serious configuration and debugging work.

Smaller or older models (Claude Haiku, GPT-4o, the GPT-4.1 family) are cheaper and fine for simple recordings or quick questions, but on a complex site they may not finish the correlation, so the recording can still fail to replay. If the assistant gets a case almost working and then stalls, the model is usually the reason: switch to Sonnet or a GPT-5-class model and run it again.

Configuring Your API Key

For Anthropic, OpenAI, and Azure OpenAI, enter your key in the API Key field. The field is masked, so you won't see the full key after you type it. For Azure, this is the access key from your Azure resource.

Key Security

Your API key is encrypted with AES-256-GCM before it's saved to disk (in ai-assistant.properties). The key never leaves your local machine and is never sent anywhere except directly to your chosen provider's API endpoint.

For AWS Bedrock, there's no API key field. Bedrock uses the IAM credentials from your AWS Load Generation account settings.

Selecting a Model

After choosing a provider and entering your API key, click Refresh Models to pull the current model list from your provider. This ensures you see all available models, including any released after your version of Load Tester shipped.

Select a model from the Model dropdown.

Azure Uses the Deployment Name, Not a Model Dropdown

Azure OpenAI has no model dropdown. The deployment name you entered is the model. Whatever model your administrator deployed under that name is what the Assistant uses, so to change models on Azure you change the deployment name (or ask your administrator to deploy a different model).

Which Model Should I Pick?

For configuration and debugging, use one of the tested models from Recommended Models: Claude Sonnet 4.6 (Anthropic or Bedrock) or a GPT-5-class model (OpenAI or Azure). Those clear the hard multi-step correlation cases reliably. Smaller models like Claude Haiku 4.5 or the GPT-4.1 family are cheaper and good for simple recordings and quick questions, but can stall on complex sites.

Testing the Connection

Click Test Connection to verify everything works. Load Tester sends a small test request to your provider using your API key and selected model.

If it succeeds, you see a confirmation message. You are ready to go.

If it fails, the error message tells you what went wrong:

  • "Invalid API key" - Double-check the key. Make sure there are no leading or trailing spaces. (Copy-paste loves to sneak those in.)
  • "Model not found" - Click Refresh Models to get the current list and select a valid model.
  • "Connection failed" - Check your internet connectivity and firewall rules. Some corporate firewalls block API endpoints.
  • "Access denied" (Bedrock) - Confirm your AWS credentials are configured in Accounts -> AWS Load Generation and that Bedrock model access is enabled in your AWS account.
  • Authentication failed (Azure) - The key or the endpoint is wrong. Azure returns the same failure for either, so check both: confirm the access key, and confirm the endpoint matches your resource exactly.
  • Deployment not found (Azure) - The deployment name doesn't match anything on your resource. Check the spelling with your administrator, and confirm the deployment has finished and is ready.
  • Service version not supported (Azure) - The dated service version you entered isn't valid for your resource. Clear the field to use the modern endpoint, or ask your administrator for the right value.
  • Throttled (Azure) - Your resource hit its rate limit. Wait the period Azure asks for, then try again. This is a quota setting on the Azure side, not a Load Tester problem.

Saving Your Settings

Click Apply and Close to save your configuration.

Once saved, AI features are available throughout Load Tester:

  • The AI panel for interactive questions and debugging
  • The MCP server for connecting external AI tools
  • AI-powered report generation for load test analysis

Troubleshooting

"Invalid API key" after pasting

Copy-paste sometimes grabs invisible whitespace characters. Clear the field completely, then paste again. If the problem persists, try typing the first few characters manually to confirm the field is accepting input.

Refresh Models returns an empty list

This usually means the API key is invalid or the provider endpoint is unreachable. Test the connection first to see the specific error.

Bedrock credentials not working

Bedrock uses the same AWS credentials as cloud load generation. Verify those credentials work by checking Accounts -> AWS Load Generation and confirming you can see your AWS instances. Also confirm that your AWS account has Bedrock model access enabled. This is a separate setting in the AWS console, not something that happens automatically.

Changed providers but old model is still selected

After switching providers, click Refresh Models to load the new provider's model list. The previous model selection doesn't carry over. (Azure is the exception: it has no model dropdown, because the deployment name is the model.)

Not sure which Azure values to enter

Azure splits the configuration across four values, and only your administrator knows them for your organization's resource. Ask for the resource endpoint (the openai.azure.com address), the deployment name, and an access key. Leave the service version blank unless your administrator says your resource needs a specific dated value.

Azure worked yesterday but the deployment now fails

On Azure the deployment name is the model, so a deployment that stops resolving is almost always a rename or removal on the Azure side, not a Load Tester problem. The key and endpoint can be perfectly valid while the name in your settings points at nothing. Confirm with your administrator that the deployment still exists under the same name.

API Key Costs

AI providers charge per request based on the model you select. Opus-class and GPT-5.4 models cost more per query than Sonnet or Haiku. Check your provider's pricing page to understand the cost structure before running extended analysis sessions.

Related Topics: