A load test is a test designed to measure an application's behavior under normal and peak conditions. You add and configure scripts, monitors, and SLAs to a load test definition.
From your Home page, you can select a project or create and edit a test.
Select a project
On the Home page, select a project from the menu bar.
|Create a test||
To create a test, perform one of the following steps:
|Edit a test||
|Duplicate a test||
Click Duplicate to copy a test definition to a new test.
Note: To use this feature, the test name should contain 120 characters or less.
|Give your test a label||
Organize your tests by assigning them labels. For details, see Assign labels.
The Load Tests navigation bar provides the following options:
General. Configure basic, run configuration, log, and load generator settings. For details, see Define general test settings.
|Scripts. Show a list of the scripts and lets you set a script schedule. For details, see Configure a schedule for your script and Manage scripts.|
|Monitors. Show a list of monitors. For details, see Add monitors to a load test.|
|Distribution. Choose the load generator machines for your Vusers. Optionally, assign scripts and the number Vusers for on-premises load generators. For details, see Configure load generator locations.|
|Rendezvous. Set up a rendezvous for your Vusers. For details, see Configure rendezvous settings.|
|SLA. Show the SLA for the test. For details, see Configure SLAs.|
|Single user performance. Collect client side breakdown data. For details, see Generate single user performance data.|
|Streaming. Show a list of the streaming agents. For details, see Data streaming. (This option is only visible when data streaming was enabled for your tenant via a support ticket.)|
|Schedules. Set a schedule for the test run. For details, see Schedule a test run.|
|Runs. Opens the Runs pane listing the runs for the selected test along with run statistics such as run status, regressions, and failed transactions. For details, see Find out the status of my test run.|
|Trends. For details, see Trends.|
In the Scripts page, you configure a schedule for each script.
There are three modes of script schedules: Simple, Manual, and Advanced. For all modes, you configure basic settings, which may differ depending on the run mode that you selected in the General settings.
To configure the Simple, Manual, or Advanced mode settings, click the arrow adjacent to a script to expand its details.
The table below describes the basic settings that you need to set for each script included in the load test.
|Specify the number of Vusers||
Note: For TruClient Native Mobile scripts, only one Vuser runs in the test.
|Specify the percentage of Vusers||
Note: Only visible for load tests configured to run in Goal Oriented mode.
Note: The total for all the scripts in the test must be 100%.
|Define the Start time||
Define the start time of each script in your test.
If you have several scripts associated with your test, you can set a different start time for each script.
If you do not enter a start time, the script runs at the start of the test.
Set test duration
Note: Only visible for load tests configured to run in Duration mode.
Note: You cannot configure the test duration when using the Advanced scheduling mode as the duration is calculated automatically (the total time of all the ramp up, duration, and tear down steps).
|Set the number of iterations||
Note: Only visible for load tests configured to run in Iterations mode.
Configure the number of iterations the script should run during the test.
|Set the maximum duration||
Note: Only visible for load tests configured to run in Iterations mode.
Configure the maximum length of time in which the script should complete the number of configured iterations.
If the script has not completed the number of configured iterations within the maximum duration time, the Vusers for the script are terminated.
|Set the location||
Location is the location of the load generators that run the script—Cloud or On Premise.
Select the required location from the drop-down menu.
For more details on configuring locations, see Define a load test.
You then configure additional settings, either general for all modes or specific to the selected mode, by clicking the arrow to the left of the script name. In the expanded schedule area that opens, select Simple, Manual, or Advanced.
A graph displays a visual representation of the script schedule showing the number of Vusers, duration, ramp up, and tear down. (A smaller version of the graph is also displayed in the row of the selected script.)
Simple mode is the default schedule mode, and enables you to configure a time during which the Vusers are started or stopped, at a consistent rate. For example, if you configure ten Vusers for the script and a Ramp up time of ten minutes, one Vuser is started each minute.
Configure the following additional settings directly in the row of the selected script:
|Define the ramp up||
Specify the ramp up time to start Vusers at a consistent rate.
Note: For TruClient Native Mobile scripts, only one Vuser runs, so there is no ramp up.
|Define the tear down||
Specify the tear down time to stop Vusers at a consistent rate.
Note: For TruClient Native Mobile scripts, only one Vuser runs, so there is no tear down.
Note: This mode is disabled for load tests configured to run in Iterations mode.
Manual mode enables you to simulate a slightly more complex scenario by configuring the ramp up and tear down values for a specific number of users in time intervals. For example, you can configure three Vusers to be started every two minutes until all the Vusers for the test have been started.
Select the Manual radio button and configure the following additional settings in the expanded schedule area:
|Define the ramp up, the rate at which Vusers are added.||
Specify how many Vusers are added and at what time intervals.
|Define the tear down, the pace at which Vusers are stopped.||
Specify how many Vusers are stopped and in what time intervals.
The Advanced mode enables you to configure a schedule that more accurately reflects a real life scenario, with multiple instances of Vusers starting and stopping during the test.
Select the Advanced radio button and then click the Edit button . In the dialog box that opens, configure multiple steps using the following additional settings:
|Define the ramp up, the rate at which Vusers are added.||
Click Add Ramp Up to add a Ramp Up step and specify how many Vusers are added and in what time intervals.
For example, you can configure two Vusers to be started every four minutes until a total of twelve Vusers have been started.
Click Add Duration to add a Duration step and specify the time the test will run before executing the next scheduling step.
Duration does not include ramp up and tear down times.
|Define the tear down, the pace at which Vusers are stopped.||
Click Add Tear Down to add a Tear Down step and specify how many Vusers are stopped and in what time intervals.
For example, you can configure three Vusers to be stopped every two minutes until a total of nine Vusers have been stopped.
- Advanced mode is disabled for:
- scripts in a load test configured for the iterations run mode.
- TruClient native mobile scripts.
- scripts with the General settings > Add Vusers option selected. For details, see Define general test settings.
- The first step must be a Ramp up.
- The last step must be a Tear down.
- The schedule must include at least one Ramp up step, one Duration step, and one Tear down step.
- You can include a maximum of twenty steps in the schedule.
For any of the schedule modes, you can configure the following additional settings:
Controls the time between iterations. The pace tells the Vuser how long to wait between iterations of your actions.
You can control pacing either from the Load test > Scripts tab or from Runtime settings > Pacing.
To set the pacing in the Load test > Scripts tab:
For LoadRunner scripts, if your script contains fixed pacing, pacing defaults to the script value. You can manually change the value.
To set the pacing using runtime settings, see the Scripts.
Note: If the Local pacing check box is not selected, the runtime settings values are used by default.
Snapshots are a visual representation of the script. If snapshots are available for a script, the Show snapshots button is displayed. Click the button to view the available snapshots.
Note: Snapshots are only applicable for TruClient scripts.
You can configure a load test to run in Goal Oriented mode. If you select this option, you configure a goal for the test and when the test runs, it continues until the goal is reached. For more details, see How LoadRunner Cloud works with goals below.
To configure a goal oriented test:
In the load test's General page, select Run Mode > Goal Oriented. For details, see Run mode.
In the load test's Scripts page, configure the % of Vusers and Location for each script. For details, see Configure a schedule for your script.
In the load test's Scripts page, click the Goal settings button . In the Goal Settings dialog box that opens, configure the following:
Setting Description Goal type
Select the goal type:
- Hits per second. The number of hits (HTTP requests) to the Web server per second.
- Transactions per second. The number of transactions completed per second. Only passed transactions are counted.
Transaction name If you selected Transactions per second as the goal type, select or input the transaction to use for the goal. Goal value Enter the number of hits per second or transactions per second to be reached, for the goal to be fulfilled. Vusers Enter the minimum and maximum number of Vusers to be used in the load test. Ramp up
Select the ramp up method and time. This determines the amount of time in which the goal must be reached.
Ramp up automatically with a maximum duration of. Ramp up Vusers automatically to reach the goal as soon as possible. If the goal can’t be reached after the maximum duration, the ramp up will stop.
Reach target number of hits or transactions per second <time> after test is started. Ramp up Vusers to try to reach the goal in the specified duration.
Step up N hits or transactions per second every. Set the number of hits or transactions per second that are added and at what time interval to add them. Hits or transactions per second are added in these increments until the goal is reached.
Action if target cannot be reached
Select what to do if the target is not reached in the time frame set by the Ramp up setting.
Stop test run. Stop the test as soon as the time frame has ended.
Continue test run without reaching goal. Continue running the test for the duration time setting, even though the goal was not reached.
Set a duration time for the load test to continue running after the time frame set by the Ramp up setting has elapsed.
Note: The total test time is the sum of the Ramp up time and the Duration.
The following do not support the Goal Oriented run mode:
Generating single user performance data
The Timeline tab when scheduling a load test
Network emulations for cloud locations
Changing the load in a running load test
TruClient - Native Mobile scripts
LoadRunner Cloud runs Vusers in batches to reach the configured goal value. Each batch has a duration of 2 minutes.
In the first batch, LoadRunner Cloud determines the number of Vusers for each script according to the configured percentages and the minimum and maximum number of Vusers in the goal settings.
After running each batch, LoadRunner Cloud evaluates whether the goal has been reached or not. If the goal has not been reached, LoadRunner Cloud makes a calculation to determine the number of additional Vusers to be added in the next batch.
If the goal has been reached, or there are no remaining Vusers to be added, the ramp up ends. Note that the value can be greater than the defined goal value. LoadRunner Cloud does not remove Vusers to reduce the load.
During run time, if the goal was reached, but subsequently dropped to below the configured value, LoadRunner Cloud will try to add Vusers (if there are any remaining to be added) to reach the goal again.
LoadRunner Cloud does not change think time and pacing configured for scripts.
For each script in the test, you configure a location—Cloud or On Premise. This is the location of the load generators that run the script. For details, see Configure a schedule for your script.
You configure the settings for each location type in the Distribution pane:
- In the Load Tests tab, open the Distribution pane .
- Click the tab for the location type you want to configure—Cloud or On Premise.
For cloud locations, you configure the distribution of Vusers and the network emulation.
In the Cloud tab, click Edit locations and select the locations to add to your test. For details, see Vuser distribution locations.
Tip: Click Group by to group the load generators by location, vendor, vendor region, or geographical area.
- In the Cloud tab, click Edit network emulations and select up to five emulations for your test. For details, see Network emulations.
For each location that you selected:
Enter the percentage of Vusers you want to distribute to this location. The Vuser distribution must total 100 percent.
Enter the percentage of those Vusers distributed to a location for which you want to assign a network emulation.
The distribution of locations and network emulation will be applied equally to each of the test scripts.
In the following example, we deployed 15 Vusers for a TruClient script and 100 Vusers for a Web HTTP script.
We set the location to Hong Kong for 50 percent of the Vusers, and Virginia for the other 50 percent.
When the test starts in the Hong Kong region, it will launch 7 Vusers for the TruClient script and 50 Vusers for the Web HTTP script.
In Virginia, the test will launch the remaining Vusers: 8 Vusers for the TruClient script and 50 Vusers for the Web HTTP script.
For on-premises locations, you select the on-premises load generators to use for your test.
In the On Premise tab, click + Add from Assets to select the load generators to use for your test, or click * Create to add a new load generator. Sort by any column by clicking on the header name. For details, see Add on-premises load generator assets.
If you selected Enable manual Vuser distribution for on-premises load generators as described in Define general test settings, you can assign scripts and Vusers per load generator. When this setting is enabled, a dropdown box presents two options—Simple and Advanced.
With the Simple assignment method, you indicate which scripts in a load test will run on a specific on-premises load generator.
In the Distribution > On Premise page, the Assigned Scripts pane to the right of the load generator list, shows the scripts assigned to the selected load generator.
Click + Select scripts to assign or unassign scripts. To assign a new script, select it and click OK. To unassign a script, deselect it and click OK.
With the Advanced assignment method, you can also set the number of Vusers per script to run on specific on-premises load generators. After you assign the scripts as described above, return to the right pane and enter the number of the Vusers in the box adjacent to the script name.
Make sure that the number of Vusers that you assign to your script matches the total number of Vusers in your load test. For example, if the script has 500 Vusers, you must manually assign all 500. In addition, the number of Vusers defined in your test must not exceed the maximum number of Vusers defined in your Vuser license.
To see the number of Vusers assigned per script, click + Select scripts and refer to the Vusers column.
- You can only assign scripts of the same type to a load generator. If a script is already assigned to a selected load generator, only scripts of the same type are enabled for assigning. Scripts of other types are disabled.
- Only scripts that are configured with a Location of On-Premise (in the Scripts page) are displayed in the list.
- If you change the location of a script that is already assigned to a load generator, the script is automatically unassigned from the load generator.
When performing load testing, you need to emulate heavy user load on your system. To help accomplish this, you can instruct Vusers to perform a task at exactly the same moment using a rendezvous point. When a Vuser arrives at the rendezvous point, it waits until the configured percentage of Vusers participating in the rendezvous arrive. When the designated number of Vusers arrive, they are released.
You can configure the way LoadRunner Cloud handles rendezvous points included in scripts.
If scripts containing rendezvous points are included in a load test, click the Rendezvous tab and configure the following for each relevant rendezvous point:
Enable or disable the rendezvous point. If you disable it, it is ignored by LoadRunner Cloud when the script runs. Other configuration options described below are not valid for disabled rendezvous points.
Set the percentage of currently running Vusers that must reach the rendezvous point before they can continue with the script.
Set the timeout (in seconds) between Vusers for reaching the rendezvous point. This means that if a Vuser does not reach the rendezvous point within the configured timeout (from when the previous Vuser reached the rendezvous point), all the Vusers that have already reached the rendezvous point are released to continue with the script.
When you select a rendezvous point in the list, all the scripts in which that rendezvous point is included are displayed on the right (under the configuration settings). If the script is disabled, its name is grayed out.
If no rendezvous points are displayed for a script that does contain such points, go to Assets > Scripts and reload the script.
Service level agreements (SLAs) are specific goals that you define for your load test run. After a test run, LoadRunner Cloud compares these goals against performance related data that was gathered and stored during the course of the run, and determines whether the SLA passed or failed. Both the Dashboard and Report show the run's compliance with the SLA.
On the SLA page, you set a percentile, the percentage of transactions expected to complete successfully, by default 90%, and the following goals :
|Percentile TRT (sec)||
The expected transaction response time that it takes the specified percent of transactions to complete (by default 3 seconds). If the percent of completed transaction exceeds the SLA's value, the SLA will be considered "broken".
For example, assume you have a transaction named "Login" in your script, where the 90th percentile TRT (seconds) value is set to 2.50 seconds. If more than 10% of successful transactions that ended during a 5 second window have a response greater than 2.50 seconds, the SLA will be considered "broken". It will show a Failed status. For details about how the percentiles are calculated, see Percentiles.
|Failed TRX (%)||The percent of allowed failed transactions, by default 10%. If the percent of failed transaction exceeds the SLA's value, the SLA will be considered "broken" and will show a Fail status.|
You can set a separate SLA for each of your runs. You can also select multiple SLAs at one time and use bulk actions to apply one or more actions to the set. For details, see Configure SLAs.
You configure one of more SLAs (Service Level Agreements) for each load test:
To configure the SLA:
- In the Load Tests tab, choose a test and open the SLA pane .
Set percentile settings:
Select the percentage of transactions expected to complete successfully. The default value is 90%.
Set the Percentile TRT (sec) (transaction response time) value. This is the expected time it takes the specified percent (from the previous step) of transactions to complete.
For example, if you set your percentile to 90 and the Percentile TRT (sec) value to 2.50 seconds, then if more than 10% of successful transactions that ended during a 5 second interval have a response time higher than 2.50 seconds, an SLA warning is recorded (using the average over time algorithm). For details, see Percentiles.
The default Percentile TRT (sec) value is 3.00 seconds.
- Check Stop to stop the test if the SLA is broken.
Clear the Enable option adjacent to the Percentile TRT (sec) column if you do not want the percentile TRT (seconds) value to be used during the test run.
Note: For tenants created from January 2021 (version 2021.01), SLAs will be disabled by default. You can enable them manually or use the Bulk actions dropdown to enable SLAs for multiple scripts. To change this behavior and have all SLAs enabled by default, open a support ticket.
- Set failed transaction settings:
- Set the Failed TRX (%) value (default 10%). If the number of transaction failures exceeds this value, the SLA is assigned a status of Failed.
- Set the Enable option if you want the Failed TRX (%) value to be used during the test run.
Tip: Select multiple SLAs at one time and use Bulk actions to apply one or more actions to the set.
When enabled, this feature lets you create client side breakdown data to analyze user experience for a specific business process.
Note: You cannot generate single user performance data for:
- Load tests configured for the Iterations run mode.
- Scripts configured to run on on-premises load generators.
Select one or more of the following report types:
|Report type||How to|
Generate a comprehensive report based on the selected script that provides information about how your application performs for a specific business process.
Generate an NV Insights report:
For details, see The NV Insights report.
Generate TRT breakdown data:
For details, see Transaction response time breakdown data.
Generate a WebPage report:
For details, see WebPage Test report data.
Schedule a test to run on a specific date and time.
Caution: The most up-to-date test settings will be used when the test is launched.
To schedule a test:
- From Load Tests page, open the Schedules pane.
Click the Add schedules button.
Note: You can configure up to 50 schedules for a load test.
- From the date picker, select a date to run the test.
- From the time picker, specify a time to run the test.
To apply the schedule, click the toggle to On.
The Schedule On button appears in the cart section indicating that a schedule is set for this load test.
Tip: In the Loads Test page, a test with a set schedule has the icon displayed next to its status.
On the lower part of the page, underneath the list of set schedules, the following two tabs show additional schedule information:
The Runs tab displays a list of previously scheduled runs and their launch status.
Launched. The test was launched successfully. Click the run ID to view results.
Failed to launch. The test was not launched successfully.
Troubleshooting failed launches:
The number of Vusers scheduled in your test exceeds the amount remaining in your license.
A load generator or monitor that is defined in your test is down.
- This tab is only displayed for tests whose license mode is set as VU.
- This tab is not displayed for load tests configured for the Iterations run mode.
The Timeline tab displays the timeline for the selected schedule, as well as for any other schedule in the project that overlaps with the selected schedule.
If the current project uses global licenses (that is, it does not have dedicated licenses assigned to it), then schedules from other global license projects that overlap with the selected schedule are also displayed.
The timeline is colored according to its status, as follows:
OK (green). There are no conflicts and this schedule will run as planned.
Conflicting (orange). This schedule impacts another schedule and will cause it to breach your license count, but this schedule will still run as planned.
Blocked (red). This schedule will breach your license count and may not run.
The Run Preview window lets you see the following details before running your test:
how load generators are allocated (both cloud and on-premises)
how scripts are distributed across load generators
the number of Vusers allocated to each load generator per script
Knowing this information will help you better understand your load test runs and run them more efficiently. For example, if Vusers were distributed unevenly, you will be able to understand why CPU alerts occurred on specific load generator machines.
In addition, the run preview lets you verify the changes that you may have made to a test's settings. For example, if you adjusted the distribution locations or if you selected a different on-premises load generator, the run preview lets you see its impact on the load distribution.
The grid shows the load generator vendors, their locations, the scripts that will be used in the load test, and number of Vusers running on the load generator.
To show the run preview:
- On the Load Tests page, click on a test.
In the right corner of the masthead, click Run Preview. The Run Preview window opens showing your Vuser distribution. If LoadRunner Cloud detects a potential error, you will be alerted when you open the window.
Optionally, expand the Group by dropdown to select another way to display the information—by Vendor, Location, Load generator, or Script.
Before you run the test, click Run preview to review the Vuser distribution, and check the Run Configuration Summary toolbar in the masthead:
- Before running your load test, if you defined any server monitors, make sure that they are up and accessible so that LoadRunner Cloud can monitor the test and display results in the dashboard.
- An on-premises load generator can only be used by one running test at a time.
The number of Vusers defined in your test must not exceed the maximum number of Vusers defined in your Vuser license.
To run the test:
Click Run Test. The Dashboard opens showing the default metrics for your test run in real time.
During the test run, the dashboard's toolbar shows summary information such as the elapsed time, number of running Vusers, and so forth.
- To pause the scheduling during the test run, click the Pause scheduling button on the toolbar. To resume, click . For details, see Pause scheduling during a test run.
- To add Vusers during a test run, click the Change Load button . For details, see Change the Vuser load dynamically.
- To end a test run before its completion, click Stop Test.