Run tests on LRC/LRE from your IDE

LoadRunner Developer integrates with LoadRunner Cloud (LRC) and LoadRunner Enterprise (LRE), providing the ability to scale up your load tests without leaving your IDE.

About scaling your test from your IDE

When running tests using LoadRunner Developer, there is a limit of 50 Vusers. You can scale up your load tests by executing your DevWeb scripts on LRC or LRE, directly from any IDE.

To run this integration, you use the LoadRunner Developer CLI tool, ScaLRD, included in the LoadRunner Developer deployment package. In your IDE, you define a task to run ScaLRD for your DevWeb script, and run the test on LRC or LRE, as required.

Parameters for the tests are configured in the relevant uploader YAML file, for example, the number of Vusers to run.

Prerequisite: To execute the test, you need an LRC or LRE license, as relevant.

Tip: To get help within your IDE on usage, available flags, and examples, run a command in the IDE using the following arguments:

  • ScaLRD --help (for general information on LRC and LRE)
  • ScaLRD lrc--help (for information on LRC)
  • ScaLRD lre--help (for information on LRE)

Back to top

Configure the YAML file

Configure the appropriate configuration file, according to where you want to run your test:

  • To run on LRC, configure lrc_uploader.yml
  • To run on LRE, configure lre_uploader.yml

The settings in the master lrc_uploader.yml or lre_uploader.yml configuration file (located in the same directory as the ScaLRD executable) apply for all scripts that you execute on LRC or LRE.

For local configuration, you can copy the appropriate uploader YAML file to a specific script directory, and customize the file settings for use with that specific script. During the script run, the settings in the local configuration YAML file override the settings in the original (master) file.

The YAML files contain customizable default values for the script run, with explanations for each parameter. These settings include:

tenant or environment settings

These are required values for communication with the LRC or LRE machine. These settings can be defined in the YAML file, or in the command line flags (see Run ScaLRD from your IDE).

  • If using OAuth authentication for connecting to LRC, define the relevant client-id and secret-key.

  • The authenticationMode key for LRE enables you to define the authentication type used to connect to LRE: basic, or accessKey for SSO authentication.

    • If basic is defined, enter the user name and password in the basic section.

    • If accessKey is defined, enter the client ID and client secret in the accessKey section.

Note: The Password key can be deleted from the file, and instead entered during runtime.

We recommend that you do not add sensitive information like passwords in plain text—see Mask and encrypt data.

workspace settings, for LRE only Define the folder structure to use in LRE for the performance test, by defining names for the following parameters: testPlan, testSetFolder, and testSet. If the names are not defined, then the default name, AutoTest, is used at runtime, for example, AutoTestPlan.
hosts/controller, for LRE only Define the load generators and Controller machine to use for the test. For more information, see Define hosts and Controller for LRE tests.
sendEmail, for LRC only When set to true, the results report for the script run is emailed at the end of the run to the logged-in user. To email to additional recipients, enter one or more email addresses for the emailAddresses property.
location, for LRC only Define whether the test is run on LRC cloud locations (default) or on-premises load generators, and then define the specific locations/machines to use. For more information, see Define distribution parameters for LRC tests.
encryption For encrypted data, define the key location. For details, see Mask and encrypt data.
Scenario and general settings For example, the number of Vusers to run, ramp up time, duration, and logging.
waitForRunTimeout

Define the number of seconds for ScaLRD to wait while checking the status of the test run. After the time elapses, ScaLRD exits, but the test continues to run on LRC/LRE.

The default is 10 seconds. You can use -1 to wait continuously until the run is complete.

You can later check status again. For details, see Get test run results.

Script exclude This is a list of regular expressions defining files to exclude from the script when it is uploaded to LRC or LRE.

Back to top

Define distribution parameters for LRC tests

For LRC tests, you can define whether your tests should be run from LRC cloud locations, or using on-premises load generators (OPLG). For cloud locations, you can also define the percentage of Vusers that you want to distribute to each location. You define this information in the lrc_uploader.yml file.

To define cloud location distribution for LRC tests:

  1. To obtain a list of the cloud locations available for the test, in your IDE run a command using the following arguments:

    Windows: %DEVWEB_PATH%\ScaLRD.exe lrc info locations <script folder path>
    Linux or macOS: $DEVWEB_PATH/ScaLRD lrc info locations <script folder path>

    Alternatively, if you are working with Visual Studio Code, select Run Task > Get distribution list from LoadRunner Cloud.

    After running the command, a table of LRC locations is displayed in the terminal window:

  2. From the list of available locations (marked as Yes in the Available column), identify the locations that you want to use for the test.
  3. In the lrc_uploader.yml file for the script, make sure the loadTest > location parameter is defined as cloud.
  4. In the cloud section, define the required locations and the distribution using the following format:

    - name: <location name> The name of the location, as provided in the locations table. For example: aws-us-east-1
    percent: <percentage of Vusers>

    An integer defining the percentage of Vusers to run on this location. For example: 40

    The sum of the percent values distributed across all locations must equal 100%.

    For example:

  5. When you run the test again from LoadRunner Developer, any locations and Vuser distributions defined for the test within LRC are overwritten by the lrc_uploader.yml settings.

To define OPLG machines for LRC tests:

  1. To obtain a list of the OPLG machines available for the test, in your IDE run a command using the following arguments:

    Windows: %DEVWEB_PATH%\ScaLRD.exe lrc info loadGenerators <script folder path>
    Linux or macOS: $DEVWEB_PATH/ScaLRD lrc info loadGenerators <script folder path>

    Alternatively, if you are working with Visual Studio Code, select Run Task > Get load generators list from LoadRunner Cloud.

    After running the command, a table of OPLG machines is displayed in the terminal window:

  2. From the list of connected or running machines, identify the ones that you want to use for the test.
  3. In the lrc_uploader.yml file for the script, make sure the loadTest > location parameter is defined as onPremise.
  4. In the onPremise section, define the required machines (each machine name in single quotes, and separated by commas).

    For example:

    Note:  

    • The Vuser distribution is according to an internal algorithm in LRC, so not every machine is necessarily used.

    • If a machine is allocated to another test at runtime, then it is not used for your test.

  5. When you run the test again from LoadRunner Developer, any OPLG machines defined for the test within LRC are overwritten by the lrc_uploader.yml settings.

Back to top

Define hosts and Controller for LRE tests

For LRE performance tests, you can configure the lre_uploader.yml file to instruct LRE on what load generators hosts to use to run your tests. You can also define a specific Controller machine to use for the test.

You can use any combination of the following:

  • Specific (named) on-premises load generator hosts

  • Automatch on-premises load generator hosts, assigned automatically by LRE
  • Cloud-based load generator hosts

    Note: Use of LRE cloud-based load generators is provided as a tech preview feature, and supported for LRP and LRE from version 2022 R1.

For more information on LRE hosts, see the LoadRunner Enterprise Help Center.

If you choose not to define hosts and/or a Controller machine in lre_uploader.yml, the performance test is run from ScaLRD using the following rules in LRE:

  • For a new test, LRE automatically selects the Controller machine and one load generator to run the test.

  • For a test that already exists in LRE, nothing is changed in the existing load generator configuration for the test in LRE.

To define load generators and Controller for LRE tests:

  1. To obtain a list of the LRE load generator hosts that are available to use for the test run, execute the relevant command from the DEVWEB_PATH.

    For a table listing on-prem hosts:

    ScaLRD.exe lre info hosts

    For a table listing templates to define cloud-based hosts (supported from version 2022 R1):

    ScaLRD.exe lre info cloudTemplates

    Note: If one of the listed hosts is not available at the time of the test run (because it has been scheduled for another test), then the test won't run on that load generator.

  2. Open the lre_uploader.yml file (master file, or local version for the script).
  3. In the hosts section, uncomment and update each type area as relevant, to define the load generators to run the test. You can define more than one type:

    Parameter Description
    type: specific

    Add on-prem hosts.

    Define the name of the host in the options: name property, by copying it from the on-prem hosts table. Add additional hosts by repeating the type: specific parameters.

    For example:

    type: automatch

    Instruct LRE to automatically select hosts for the run.

    Define the number of hosts to use in the options: amount property. For example, amount:10.

    You can define an automatch value in addition to specifically named hosts or cloud hosts, up to the limit of available hosts.

    type: cloud

    Add host templates, used by LRE to provision your cloud-based load generators.

    Note: Cloud-based load generators are supported for LRP and LRE from version 2022 R1.

    Define the name of the template in the options: name property, by copying valid templates from the cloud templates table. Add additional templates by repeating the type: cloud parameters.

  4. If you want to define a specific LRE Controller to run the test, uncomment the controller section and specify the machine name.

  5. Save the file.

    When you run the test from LoadRunner Developer, any hosts/Controller defined for the test within LRE are overwritten by your definitions in lre_uploader.yml.

Back to top

Mask and encrypt data

If the lrc_uploader.yml or lre_uploader.yml file for your script includes a password or other sensitive data, you can mask or encrypt the information in the file.

Mask or encrypt the relevant text as described in Encode sensitive data. Then add the encoded strings to the YAML file, using the relevant syntax:

Purpose: Syntax:
Unmask a masked value Unmask(<masked_data>)
Decrypt an encrypted value Decrypt(<encrypted_data>)

For encrypted data, define the key location for decryption during replay. In the encryption area, define the property keyLocation using the format: "folder/keyFile.txt".

Back to top

Run ScaLRD from your IDE

To scale your load tests in LRC or LRE, define a task to run ScaLRD from your IDE.

Note: ScaLRD uses the system environment variable for http_proxy, https_proxy, or no_proxy, if defined.

To run tests with any IDE:

Tip: Review the FAQ before using ScaLRD.

  1. Create your DevWeb script, and test it in LoadRunner Developer.
  2. Configure the required settings in the lrc_uploader.yml or lre_uploader.yml file, as appropriate.
  3. Open the script in your IDE.
  4. Set up a task for ScaLRD to run the test.

    When setting up the task in your IDE, configure the arguments and flags for the command line using the following format.

    Windows LRC %DEVWEB_PATH%\ScaLRD.exe lrc <flags> run <DevWeb script folder path>
    LRE %DEVWEB_PATH%\ScaLRD.exe lre <flags> run <DevWeb script folder path>
    Linux or macOS LRC $DEVWEB_PATH/ScaLRD lrc <flags> run <DevWeb script folder path>
    LRE $DEVWEB_PATH/ScaLRD lre <flags> run <DevWeb script folder path>

    The following flags are available:

    LRC flags:

    -host= The LRC host name with http/https protocol.
    -user= The user name or client ID to use to log into LRC.
    -password= The password or secret key to use to log into LRC.
    -tenantId= The tenant ID of the user.
    -projectId= The ID of the project on which the tests are run.
    -keyLocation= The full path to a key file used to decrypt encrypted strings in the configuration file.
    -report= The relative or root path to the location for the results report. If a path is not specified, then it is saved by default to the script folder.

    LRE flags:

    -host= The LRE host name with http/https protocol.
    -tenantId= The tenant ID of the user (if required for the environment).
    -domain= The domain name of the user.
    -project= The LRE project name.
    -authenticationMode= The authentication type used to connect to LRE.
    -userName= (For basic authentication) The user name to use to log into LRE.
    -password= (For basic authentication) The password to use to log into LRE.
    -clientId= (For SSO authentication) The client ID required to log into LRE.
    -clientSecret= (For SSO authentication) The client secret required to log into LRE.
    -keyLocation= The full path to a key file used to decrypt encrypted strings in the configuration file.
    -testPlan= The script is uploaded to a test plan folder with this name.
    -testSet= The performance test is created under a test set with this name.
    -testSetFolder= The test set is created under a test set folder with this name.
    -report= The relative or root path to the location for the results report. If a path is not specified, then the report is not generated.

    All flags apart from report can be configured in the lrc_uploader.yml or lre_uploader.yml file. If you add them in the command line, the flag definitions override the YAML file definitions.

    For example, you might add to the command line:

    -report=<report location> -user=KimW

    Note: We recommend that you do not add sensitive information like passwords in plain text—see Mask and encrypt data.

  5. Run the ScaLRD task.

    • LRE test: The DevWeb script folder is uploaded to the test plan folder configured in the lre_uploader.yml file (parameter testPlan). If a test plan name is not configured, the script is uploaded by default to AutoTestPlan.

      A performance test is created for the test in the same test set plan folder, using the label <Script Name>_PerformanceTest.

    • LRC test: If this is the first run for the test using ScaLRD, the DevWeb script folder is uploaded to LRC, and the test is given the label Autogenerated_<script name>. If a test with the same name already exists in LRC, the script is updated and used for the test run.

      If relevant, the email addresses for the result report are added in LRC.

  6. The test is executed, using the configuration settings in the relevant uploader configuration file.

    • LRE test: When the test is run, the test instance is assigned to the test set and test set folder that are configured in the lre_uploader.yml file. If these parameters are not defined, then AutoTestSet and AutoTestSetFolder are used by default.

      The test run is given an ID number, for example, Test Id 184.

    • LRC test: When the test is run, any SLA attached to the test is disabled. (The SLA is disabled for the test, not just for the current run.) The test run is given an ID number, for example, #184.

    The ID can be used together with the test name to identify the run in LRC or LRE.

    You can watch the run progress from the LRC or LRE UI. For details, see the LoadRunner Cloud Help Center or the LoadRunner Enterprise Help Center.

  7. When the test is complete, LoadRunner Developer displays the summary report. If email notification is configured, the results are emailed.

    You do not need to wait for the task to complete. You can close it locally in your IDE, or ScaLRD exits after reaching the timeout. In both these cases, the test continues to run on LRC or LRE. You can later return to the test in your IDE to see if the run is complete and to get the results. For details, see Get test run results.

Back to top

Get test run results

You can run a task in your IDE to check the results of a ScaLRD performance test on LRC or LRE. This option enables you to get the test results after ScaLRD exited (before test execution was complete and results were available).

Tip:  

  • If you want ScaLRD to run until the performance test is finished, configure the value -1 in waitForRunTimeout in the YAML file. For details, see Configure the YAML file, above.

  • In Visual Studio Code, you can use the Get test run results from LoadRunner Cloud/LoadRunner Enterprise tasks to get the results.

To get test run results:

  1. Configure the arguments for the command line to get the test run results.

    • For LRC, the following options are available (on Windows OS):

      Purpose Command
      Print a table with all the test run IDs for the test. %DEVWEB_PATH%\Scalrd.exe lrc <flags> getResults <script folder path>

      Get results for the specified run ID.

      The applied login information is taken from the lrc_uploader.yml file located in the same folder as ScaLRD.exe.

      %DEVWEB_PATH%\ScaLRD.exe lrc <flags> getResults <test run ID in LRC>

      Get results for the specified run ID.

      The lrc_uploader.yml file located in the specified script directory overrides the one located with ScaLRD.exe.

      %DEVWEB_PATH%\ScaLRD.exe lrc <flags> getResults <test run ID in LRC> <script folder path>
    • For LRE, the following options are available (on Windows OS):

      Purpose Command
      Get results for the last run for the specified test. %DEVWEB_PATH%\ScaLRD.exe lre <flags> getResults <script folder path>

      Get results for the specified run ID.

      The applied login information is taken from the lre_uploader.yml file located in the same folder as ScaLRD.exe.

      %DEVWEB_PATH%\ScaLRD.exe lre <flags> getResults <test run ID in LRE>

      Get results for the specified run ID.

      The lre_uploader.yml file located in the specified script directory overrides the one located with ScaLRD.exe.

      %DEVWEB_PATH%\ScaLRD.exe lre <flags> getResults <test run ID in LRE> <script folder path>

    Note: On Linux or macOS, use $DEVWEB_PATH/ScaLRD instead of %DEVWEB_PATH%\ScaLRD.exe.

  2. Run the task. The progress pane indicates if the load test is still running, or shows the summary report if it has finished.
  3. If the test is still running, you can return later and run the getResults task again.

Back to top

Example: Run from Visual Studio Code

The following example describes how to set up and run a task to scale your tests from Visual Studio Code.

Tip: A predefined tasks.json template for VS Code is provided for use with ScaLRD. The template is based on running a test on LRC, and is available here: <DevWeb root folder>\examples\VsCodeTemplate

To set up a task for ScaLRD in VS Code:

  1. Create your DevWeb script, and test it in LoadRunner Developer.
  2. Open the script in VS Code.
  3. Either open an existing tasks.json file, or add a new task.

    To add a new task:

    1. Select Terminal > Configure Tasks (in older versions of VS Code, select Tasks > Configure Tasks).  
    2. Select Create tasks.json file from template from the task list.
    3. In the Select a Task Template list, select Others. VS Code creates a tasks.json file as a basic template.
  4. Add the relevant content to the tasks.json file, as follows:

    Property name Value
    task Define your own label to identify the task.
    type Define as shell.
    panel Define as new. Every run will be in a new panel.
    <Platform> You can run the task on Windows, Mac, or Linux OS (windows, osx, linux).
    command Define the path to the ScaLRD executable file, for the specified platform.
    args Add the path to the relevant script folder, and the flags you want to include (the flag definitions override the settings in the lrc_uploader.yml or lre_uploader.yml file). For details, see Run ScaLRD from your IDE.

    Example:

  5. Save the file.

To run the test:

  1. In VS Code, select Terminal > Run Task.
  2. In the Select the task to run list, select your task (the label you defined previously).
  3. In the displayed list of options, select Continue without scanning the task output. The task starts running, and the script is uploaded to LRC or LRE. Progress is displayed in the Terminal pane.

  4. A brief replay summary is displayed, providing information on the run and a transaction summary. It also provides the path to the report, containing the full details and graphs for the test run.

    Example:

Back to top

FAQ

Is a load test run from the LoadRunner Developer integration the same as a test run from the LRC or LRE UI?

Yes—when a load test is executed for the LoadRunner Developer integration using ScaLRD, it is run like any other LRC or LRE test.

How is my load test created in LRC or LRE the first time I run the test?

When you execute your test:

  • In LRC: ScaLRD uploads the script to LRC, and creates a load test using the label Autogenerated_<scriptname>.
  • In LRE: ScaLRD uploads the script to the LRE folder defined in lre_uploader.yml; or, if the folder is not defined, to the AutoTestPlan folder. It also creates a performance test using the label <Script Name>_PerformanceTest.

Note: If you make configuration changes to the test in the LRC or LRE UI, when you execute the test again from the LoadRunner Developer integration, some of your configuration items might be overwritten.

How can I change the script schedule settings?

You can change schedule settings, for example for number of Vusers, duration, and so forth, in the scenario section of the lrc_uploader.yml or lre_uploader.yml configuration file. When you execute the test using ScaLRD, these settings overwrite the settings in the LRC or LRE UI.

How can I add additional assets/groups to my load test?

Whenever you execute a test with ScaLRD, only the uploaded script asset (LRC) or group (LRE) is used for the test. You cannot run multiple assets or groups.

How can I change distribution for my LRC load test?

You can configure distribution across locations for your test in the lrc_uploader.yml file. For details, see Define distribution parameters for LRC tests.

How can I define monitors for my LRC load test?

You can set up monitors for the test in the LRC UI. When you execute the test again using ScaLRD, it uses the latest configuration in the LRC.

How can I change general settings for my LRC load test?

You can change some general settings using the LRC UI. Note that the email report address is overwritten whenever the test is executed with ScaLRD.

How can I edit the SLA setting for my LRC load test?

When you execute the test using ScaLRD, the SLA for the test transactions in LRC is disabled. You can enable it again if you run the test from the LRC UI.

How can I configure a specific load generator for my LRE performance test?

You can configure load generators in the hosts section of the lre_uploader.yml file. For details, see Define hosts and Controller for LRE tests.

Back to top

See also: