SLA rules

This topic describes how to define SLA performance goals for your LoadRunner Developer tests.

About SLA rules

You can define rules for your DevWeb script tests, to determine if the AUT performance levels meet your service level agreement (SLA) goals. Determine the criteria for success or failure by defining rule thresholds that you associate with the script.

Whenever the test is executed, the rules are applied, and a report is created to show the results of the rule calculations. This enables you to determine at which point the application failed to meet minimum performance levels.

Note: SLA rules are supported only for DevWeb tests run locally in LoadRunner Developer.

Back to top

Supported rules

The following SLA rules are supported.

  • Transaction Response Time (type: averageTRT). This rule measures the average response time for the transaction during the test, and compares it with a defined threshold (averageTime, in milliseconds). If the average time is above this value, the SLA fails.

    The rule can be applied to multiple transactions.

  • Failing Transactions to Total Transaction Ratio (type: failureRate). This rule measures the overall failure rate for transactions during the test. The SLA fails if the percentage of failed transactions is greater than a defined percentage (a whole integer between 1-100).

  • Total Passing Transactions (type: passingTransactions). This rule compares the total amount of transactions that passed against a defined threshold amount. The SLA fails if the number is below the threshold.

  • Transaction Rank Percentile (type: TRP). This rule compares defined percentile ranks against the ranges for the transaction duration, for passed transactions. The SLA fails if the target percentage is not reached.

    For a specified transaction for the TRP rule, you define the following arguments:

    Argument Definition Example
    ranges The ranges (starting from zero) are defined using range markers, in milliseconds (ms).

    If the range markers are defined as [2000, 5000, 10000], this delimits the following duration ranges:

    • 0-2000 ms
    • 0-5000 ms
    • 0-10000 ms
    ranks This defines the percentile rank required for each specified range. You must define the same number of ranks as ranges.

    If the ranks are defined as [88, 98, 100], this means the following results are required for the SLA to succeed:

    • 88% of transactions must have a duration of 2000 ms or less.
    • 98% of transactions must have a duration of 5000 ms or less.
    • 100% of transactions must have a duration of 10000 ms or less.

Back to top

Defining SLA rules

Define the criteria for your SLA rules in the sla_rules.yml file.

Add the sla_rule.yml to your script folder, before running the test. If the rules in the file are enabled, then at the end of test execution, the replay summary displays a pass or fail status, and generates a report file.

Two report formats are supported:

  • JUnit: Format for the JUnit unit testing framework, in an XML file.
  • TAP: Format for the TAP unit testing framework, in a TAP file.

To run a test with SLA rules:

  1. Copy the sla_rules.yml file from <DevWeb root>\examples\SLAJUnit to your script folder.

  2. Open the sla_rules.yml file and edit the contents, and add extra rules as required, as relevant for your script. Use one of the examples given below in SLA rules file example as a guide.

    For each transactionName value, enter the name of the transaction as it appears in the script.

    You can enable or disable the whole file, or individual rules.

  3. Define the report settings:

    • The type of report file (reporterType) to generate, JUnit or Tap.

    • The name for the report, in the format <name>.xml for JUnit reports, or <name>.tap for TAP reports.

    The report is generated inside the script folder.

  4. Execute the test. See Run scripts.

  5. At the end of the test, check the SLA rule results (after the replay summary) :

  6. Open the script folder and locate the JUnit or TAP report. Review the report to see more detailed information on the rule calculations.

SLA rules file example

Example of an sla_rules.yml file:

enabled: true                                                   # Allows the user to enable or disable all the rules in a category 
reporterType: JUnit                                             # Allows to change the reporter type, supported reporters: [JUnit, Tap]
reportName: sla_junit_report.xml                                # The report name, which is generated inside the script directory
rules:                                                          # List of the rules which the SLA enforces
  - rule: Transaction Response Time                             # The SLA rule measures the average duration of the given transactions
    type: averageTRT                                            # The transaction type
    enabled: true                                               # Allows the user to enable or disable a specific rule
    arguments:                                                  # The arguments which a given type is expecting (at least one entry)
      - transactionName: name1                                  # The name of the transaction for which we want to calculate the average
        threshold:                                              
        - averageTime: 1600                                     # The average time in milliseconds above which the SLA rule will fail
      - transactionName: name2                                  # The name of the transaction for which we want to calculate the average
        threshold:
        - averageTime: 1650                                     # The average time in milliseconds above which the SLA rule will fail
  - rule: Transaction Response Time
    type: averageTRT 
    enabled: false
    arguments:
      - transactionName: example1
        threshold:
        - averageTime: 100
      - transactionName: example2
        threshold:
        - averageTime: 5000
  - rule: Failing Transactions to Total Transaction Ratio     # The SLA rule measures the percentage of total failed transactions
    type: failureRate                                         # The transaction type
    enabled: false                                            # Allows the user to enable or disable a specific rule
    arguments:                                                # The arguments which a given type is expecting (one entry)
      percentage: 10                                          # The percentage threshold above which the SLA rule will fail
  - rule: Total Passing Transactions                          # The SLA rule validates that the amount of passed transactions reach a minimum threshold.
    type: passingTransactions
    enabled: true
    arguments:
      amount: 25                                              # The minimum amount of passed transactions for the SLA to pass. 
  - rule: Transaction Rank Percentile                       # The SLA rule validates if transaction duration measurements meet percentile rank specified in the arguments.
    type: TRP                                                 
    enabled: true
    arguments:
      - transactionName: name1                                # The name of the transaction for which we want to calculate the transaction percentile rank
        ranges: [2000, 5000, 10000]                           # Defines the duration ranges in milliseconds. For example, 5000 defines the range 0-5000.
        threshold:                                            # The number of threshold ranks must match the number of ranges. 
          ranks: [88, 98, 100]                                # Defines the percentile ranks for the ranges. For example, 98% of transactions
								 are required to have a duration of 0-5000 milliseconds.

Back to top

See also: