Define a load test

You add and configure scripts, monitors, and SLAs to a load test definition.

 

Get started

Action How to

Select a project

On the Home page, select a project from the menu bar.

Create a test

Do one of the following:

  • On the Home page, click .
  • On the Load Tests page, click .
Edit a test
  • Highlight a test in the grid.
  • Click Edit or click the test name link in the test grid.
Duplicate a test Click Duplicate to copy a test definition to a new test.

Back to top

Set general test settings

Create or edit a test from the Load Tests page.

Action How to
Test name

Specify the test name.

Description Add a description of your load test.
Email report Select the checkbox to send yourself a report when the load test finishes. Enabled by default.
License Mode

Select a license type.

  • VUH

    Enables you to run API, DEV, and UI Vusers by the hour in load tests on the cloud.

    1 VUH is effectively 1 Vuser executing a load test for 1 hour.

  • VU

    A fixed license enabling you to run Dev, API and UI Vusers.

Add Vusers

Enable adding Vusers during a test run.

Caution: To cover the cost of initializing extra load generators, additional VUH charges will apply.

Delayed start

Select to delay the test run, after initialization, for up to 60 minutes.

This options adds 1 VUH to each Vuser in your test.

Enable logs
  • Generate log files for the test run.

    To download the logs:

    1. Navigate to the Results.
    2. Click the button on the run you want to analyze.
    3. Click the button (More options menu) and select Download Logs.
  • Stream script errors to Splunk.

    Select this check box to enable the streaming of script errors to a Splunk account. For details, see Stream script errors to Splunk.

    Note: When this option is disabled, even if the check box is selected, errors will not be streamed to Splunk.

Enable multiple IP addresses

Enable multiple IPs addresses on a single load generator.

This option is applicable for load generators in an AWS cloud (all regions excluding China) and for Windows on-premises load generators.

IP addresses are assigned to Vusers on a round-robin basis. For example, if an on-premises load generator has 10 IP addresses and 100 Vusers are running on it, the first 10 users will each have a unique IP address and subsequently, each group of 10 Vusers will repeat the IP address pattern.

Supported for the following script types:

  • Web HTTP
  • Mobile Application HTTP
  • Web Services
  • SAP WEB

For details or to enable this feature, contact your account manager or sales representative.

Note:

  • For Windows on-premises load generators, you must ensure that you have allocated the required number of valid IP addresses.
  • For AWS cloud based load generators, you can use a maximum of 30 IP addresses.
  • Allocating multiple IP addresses for AWS cloud based load generators may require an additional cost.
  • Scripts in which WinInet is enabled do not support multiple IP addresses.
Disable Whitelisted IPs Do not use the allocated whitelist IPs for load generators on the cloud.
Group Transactions

When you select this check box, a transaction group is created for all the transactions with the same name in all of the scripts included in the load test. Metrics for this transaction group are displayed in the Dashboard and in Reports, in addition to the metrics for the individual transactions.

Note: This option does not support dynamic transactions.

Enable script assignment

Select this check box if you want to assign scripts in a load test to be run on specific on-premises load generators. For details, see On-premises locations.

Note: This option is applicable only for scripts assigned to an on-premises location.

Accelerate initialization

Select this check box if you want StormRunner Load to try and run the load test's Web HTTP scripts on Linux load generators to speed up the test.

Note: This applies only to Web HTTP scripts.

Back to top

Add scripts from your StormRunner Load repository

  1. Click the Scripts pane.
  2. Do one of the following:
    ActionHow to
    Add from assets

    Select one or multiple scripts to add from the repository.

    1. Click Add from assets.
    2. Highlight one or more scripts in the Scripts dialog box to add to the test.
    3. Filter scripts by a specific label.

      In the Filter panel, highlight the label to search for.

      Use to search for a label name.

    Upload

    Upload a script to the cloud repository from your local drive.

    1. Click Upload.
    2. Select one or more scripts from the file system to upload and add to your test.
    Duplicate

    To use the same script multiple times, duplicate the selected script and change the configuration. For example, change the number of Vusers from 100 to 400.

    1. Highlight an existing script in the grid.
    2. Click Duplicate.
    Copy scheduling

    Copy scheduling parameters from one script to another:

    1. Select a script with the scheduling information you want to copy.
    2. Click the button.
    3. Select the script or scripts to be updated.
    4. Click the button to apply the settings to the selected scripts.
    Runtime settings

    View or edit runtime settings for individual scripts in your test.

    Note: Changes made in runtime settings will affect the script in related tests.

    Actions

    Apply script configuration settings to multiple scripts.

    For details, see Configure the script schedule.

    Copy runtime settings

    Copy runtime settings from one script to another:

    1. Select a script with the runtime settings you want to copy.
    2. Click the Copy runtime settings button.
    3. Select the script or scripts to be updated.

      Note: Only scripts in the same load test, and of the same type as the script you are copying from, are displayed.

    4. Click Apply to apply the settings to the selected scripts.

    Note:

    • Only scripts recorded in VuGen have runtime settings.
    • You can only copy runtime settings between scripts of the same type.
    • Only settings configured directly in the Runtime Settings dialog box, and that are common to both the source and target scripts, are copied.

    • Changes made in runtime settings will affect the script in related tests.

Back to top

Add scripts from your Git repository

You can connect to an existing Git script repository and add those scripts to your StormRunner Load load tests.

For details, see Git integration.

Do the following to add or update a script from your Git repository to StormRunner Load:

Action How to
Access the Git repository

Click and select Upload from Git to open the Git repository dialog box.

Select an agent

From the drop-down menus, select the agent, repository, and branch that connect to the script repository you want to use.

For details on managing your Git agents, see Agents.

Select a script

Select one or more scripts from the Git repository. Click Add.

Sync Git scripts

Do the following to sync Git scripts:

  1. Select .
  2. Select the scripts you want to sync.
  3. Click to sync your scripts with the Git repository.

Back to top

Configure the script schedule

Complete each action to configure the script for the test:

Action How to
Specify the number of VusersA virtual user emulates user actions in your application. Define how many Vusers will perform the business process recorded in your script during the load test.
  1. Click the Vuser column.
  2. Enter the number of Vusers.

Note: For TruClient Native Mobile scripts, only 1 Vuser runs in the test.

Define the Start time

Define the start time of each script in your test.

If you have several scripts associated with your test, you can set a different start time for each script.

If you do not enter a start time, the script will run at the start of the test.

  1. Click the Start time column.
  2. Enter the time to start running script after the test begins to run.
Define the ramp up

Specify the ramp up time to start Vusers at a consistent rate.

  1. Click the Ramp up column.
  2. Enter hours, minutes, seconds (hh:mm:ss).

Note: For TruClient Native Mobile scripts, only 1 Vuser runs, so there is no ramp up.

Set test duration

  1. Click the Duration column.
  2. Enter hours, minutes, and seconds (hh:mm:ss).
Define the tear down

Specify the tear down time to stop Vusers at a consistent rate.

  1. Click the Tear Down column.
  2. Enter hours, minutes, and seconds (hh:mm:ss).

Note: For TruClient Native Mobile scripts, only 1 Vuser runs, so there is no tear down.

Set the location

Location is the location of the load generators that run the script—Cloud, On Premise, or Private Cloud.

Select the required location from the drop-down menu.

For more details on configuring locations, see Configure load generator locations.

Note:

  • Only those locations that support the specific script type are enabled.
  • If you change the location for a script that is configured for On Premise, if the script is assigned to run on a specific load generator it will automatically be unassigned. For details, see On-premises locations.

Configure additional settings

To configure additional settings for a script, click the arrow to the left of the script name and configure the following:

Action How to
Set pacing

Control the time between iterations. The pace tells the Vuser how long to wait between iterations of your actions.

You can control pacing either from the Load test > Scripts tab or from Runtime settings > Pacing.

To set the pacing in the Load test > Scripts tab:

  1. Open the additional settings area for the script.
  2. Select the Local pacing check box.
  3. Enter time in seconds.

    Tip: To set the time in milliseconds, enter the time in decimals. For example, 250 milliseconds = 0.250.

    Pacing defaults to 1 second.

For LoadRunner scripts, if your script contains fixed pacing, pacing defaults to the script value. You can manually change the value.

To set the pacing using runtime settings, see the Runtime settings tab.

Note: If the Local pacing check box is not selected, the runtime settings values are used by default.

Define the ramp up, the rate at which Vusers are added.

Specify how many Vusers are added in time intervals.

  1. Open the additional settings area for the script.
  2. Select the Manaul scheduling check box.
  3. Specify the time interval.
  4. Specify the number of Vusers to start.
Define the tear down, the pace at which Vusers are stopped.

Specify how many Vusers are stopped in time intervals.

  1. Open the additional settings area for the script.
  2. Select the Manaul scheduling check box.
  3. Specify the time interval.
  4. Specify the number of Vusers to stop.
Show snapshots

Snapshots are a visual representation of the script. If snapshots are available for a script, the Show snapshots button is displayed. Click the button to view the available snapshots.

Note: Snapshots are only applicable for TruClient scripts.

Back to top

Dynamically add additional Vusers during the test run

Optionally, configure the test to enable you to add additional Vusers during the test run.

Caution: To cover the cost of initializing extra load generators, additional VUH charges will apply.

  1. From Load tests > Test settings page, select Enable adding Vusers during a test run.
  2. From the Load tests > Scripts > Add Vusers during the test run page, specify the percentage of Vusers you want to add during the test run.

For details on adding Vusers during the test run, see Dynamically increase or decrease running Vusers during the test run.

Back to top

Add monitors

Add monitors to monitor your load test.

  • Each load test can have only one type of monitoring: SiteScope, SiteScope on-premises, or New Relic.
  • If your test requires SiteScope on-premises or New Relic monitoring, you can add only one monitor per load test.

Click the Monitors pane.

Do one of the following:

Action How to

Create

Click Create to open the New Monitor dialog box.

SiteScope monitor:

  1. Select SiteScope as your monitor type.
  2. Define the monitor by specifying the server name/IP address and username.

For details on configuring a SiteScope monitor, see Monitors.

SiteScope-on-Prem monitor:

If your AUT is behind a firewall, use a SiteScope on Prem monitor to monitor your load test.

Make sure that:

  • A Data integration tag has been set up in SiteScope.

  • A StormRunner Load Monitor Agent has been installed and configured for SiteScope, and that it is started. For details, see StormRunner Load agent.

  1. Click SiteScope on-Prem.

    If the SiteScope on prem tab is disable, check:

    • the StormRunner Load agent connectivity to SaaS
    • the StormRunner Load agent connectivity to SiteScope
  2. Give the monitor a name, and select the SiteScope agent.
  3. Select metrics to monitor.

    Tip: To expand the full list of metrics, click the Expand button.

    In addition to the standard monitors, the following Dynamic Docker monitor types are supported:

    • Docker
    • Cluster Swarm

    When you select a Dynamic Docker monitor type, all metrics are added to your test definition.

  4. Save your selection.

New Relic monitor:

  1. Select NewRelic as your monitor type.
  2. Enter your NewRelic API Key.
  3. Select an application or a server to monitor.
  4. Give the monitor a name, and select metrics to monitor.

    • To expand the full list of measurements, click the Expand button.

    • If there are more than 500 metrics, the Expand/Collapse buttons are disabled.

    • If no measurements are listed, make sure that your New Relic monitor is up and running, and that StormRunner is connecting to it.

Dynatrace:

Use a Dynatrace monitor to monitor the applications you are testing and the application servers.

Make sure that:

  • There is a dashboard in Dynatrace containing chart-type dashlets that display the information you want to view in StormRunner.

  • A StormRunner Load Monitor Agent has been installed and configured for Dynatrace, and that it is started. For details, see StormRunner Load agent.

  1. Click Dynatrace.
  2. Give the monitor a unique name, and select the Dynatrace agent.
  3. Select measurements to monitor.

    To expand the full list of measurements, click the Expand button.

    If no measurements are listed, make sure your application is up and running and that the measurements are visible in Dynatrace.

  4. Save your selection.

Add from assets
  1. Click Add from assets.
  2. Select one or more monitors to add to the test definition.
Add topology
  1. Click Add topology.
  2. Select one or more topologies from the Add topology from repository dialog box.
Edit
  1. Select a monitor.
  2. Click Edit and modify server details.

Back to top

Configure load generator locations

For each script in the test, you configure a location—Cloud, On Premise, or Private Cloud. This is the location of the load generators that run the script. For details, see Configure the script schedule.

You configure the settings for each location type in the Distribution pane:

  1. Click the Distribution pane.
  2. Click the tab for the location type you want to configure—Cloud, On Premise, or Private Cloud.

Back to top

Configure SLAs

  1. Click the SLA pane.
  2. Select the percentage of transactions expected to complete successfully. The default value is 90%.

  3. Set the percentile TRT (seconds) (transaction response time) value. This is the expected time it takes the specified percent (from step 2) of transactions to complete.

    For example, in your script you have a transaction named "Login". The 90th percentile TRT(Seconds) value for this transaction is set to 2.50 seconds. If more than 10% of successful transactions that ended during a 5 second window have a response time higher than 2.50 seconds, an SLA warning is recorded.

    The default percentile TRT (Seconds) value is 3.00 seconds.

    Tip: If you have run your test more than once, StormRunner Load will display a suggested percentile TRT value. Click the button to use the suggested value.

  4. Check Stop test to stop the test if the SLA is broken.
  5. You can clear the Enable option if you do not want a transaction to be monitored during the test run.

Tip: Select multiple SLAs at one time and apply one or more actions to the set.

Back to top

Generate a client side breakdown report

When enabled, you can create a client side breakdown report to analyze user experience for a specific business process.

Select from one or more of the following report types:

Report type How to
NV Insights

Generate a comprehensive report based on the selected script that provides information about how your application performs for a specific business process.

Generate an NV Insights report:

  1. Select the Client side breakdown pane.
  2. Under the Micro Focus Analysis > NV Insights tab, enable Run client side breakdown using NV.
  3. From the dropdown, select a script for the report.

See also:

TRT Breakdown

Generate TRT breakdown data:

  1. Select the Client side breakdown pane.
  2. Under the Micro Focus Analysis > Transaction Response Time Breakdown tab, enable Run TRT Breakdown.
  3. From the dialog box select up to five scripts.
  4. To display the results, from the Dashboard > , select the Breakdown widget.

For details, see Transaction response time breakdown data with Micro Focus Analytics.

WebPage

Generate an WebPage report:

  1. Select the Client side breakdown > WebPageTest Report pane.
  2. Enter a URL of your application. You can add up to 3 URLs.
  3. Select a network speed.

    Network speed Description

    Cable

    5/1 Mbps 28ms RTT
    DSL 1.5 Mbps/384 Kbps 50 ms RTT
    FIOS 20/5 Mbps 4 ms RTT
    56K Dial-Up 49/30 Kbps 120 ms RTT

    Mobile 3G

    1.6 Mbps/786Kbps 300ms RTT
    Mobile 3G- Fast 1.6 Mbps/786Kbps 150ms RTT
    Native Connection No traffic shaping

For details, see Client side breakdown with WebPage Test.

Back to top

Schedule a test run

Schedule a test to run on a specific date and time.

Caution: The most up-to-date test settings will be used when the test is launched.

  1. From Load Tests page, click the pane.
  2. Click the button.

    Note: You can configure up to 10 schedules for a load test.

  3. From the date picker, select a date to run the test.
  4. From the time picker, specify a time to run the test.
  5. To set the schedule, click .

    The button appears in the cart section indicating that a schedule is set for this load test.

On the lower part of the page, underneath the set schedules, a list of previously scheduled runs and their launch status is displayed.

Launch Status Description
Launched The test was launched successfully. Click the run ID to view results.
Failed to launch

The test was not launched successfully.

Troubleshooting failed launches:

  • The number of Vusers scheduled in your test exceeds the amount remaining in your license.
  • A load generator or monitor that is defined in your test is down.

Tip: In the Loads Test page, a test with a set schedule has the icon displayed next to its status.

Run the test

Click Run Test.

By default, you will receive a report when your test ends.

Back to top

Notes and Limitations

  • Before running your load test, always make sure that the monitor defined in the test is up and running and accessible so that it can monitor the test and display results in the dashboard.
  • An on-premises load generator can be occupied by one running test at a time.
  • The number of Vusers defined in your test must not exceed the maximum number of Vusers defined in your Vuser license.

  • The SiteScope agent can be occupied by one running test at a time.

Back to top

Next steps:

Analyze results