This topic describes how to define SLA performance goals for your LoadRunner Developer tests.
About SLA rules
You can define rules for your DevWeb script tests, to determine if the AUT performance levels meet your service level agreement (SLA) goals. Determine the criteria for success or failure by defining rule thresholds that you associate with the script.
Whenever the test is executed, the rules are applied, and a report is created to show the results of the rule calculations. This enables you to determine at which point the application failed to meet minimum performance levels.
Note: SLA rules are supported only for DevWeb tests run locally in LoadRunner Developer.
The following SLA rules are supported.
Transaction Response Time (type: averageTRT). This rule measures the average response time for the transaction during the test, and compares it with a defined threshold (averageTime, in milliseconds). If the average time is above this value, the SLA fails.
The rule can be applied to multiple transactions.
- Failing Transactions to Total Transaction Ratio (type: failureRate). This rule measures the overall failure rate for transactions during the test. The SLA fails if the percentage of failed transactions is greater than a defined percentage (a whole integer between 1-100).
Total Passing Transactions (type: passingTransactions). This rule compares the total amount of transactions that passed against a defined threshold amount. The SLA fails if the number is below the threshold.
Transaction Rank Percentile (type: TRP). This rule compares defined percentile ranks against the ranges for the transaction duration, for passed transactions. The SLA fails if the target percentage is not reached.
For a specified transaction for the TRP rule, you define the following arguments:
Argument Definition Example ranges The ranges (starting from zero) are defined using range markers, in milliseconds (ms).
If the range markers are defined as [2000, 5000, 10000], this delimits the following duration ranges:
- 0-2000 ms
- 0-5000 ms
- 0-10000 ms
ranks This defines the percentile rank required for each specified range. You must define the same number of ranks as ranges.
If the ranks are defined as [88, 98, 100], this means the following results are required for the SLA to succeed:
- 88% of transactions must have a duration of 2000 ms or less.
- 98% of transactions must have a duration of 5000 ms or less.
- 100% of transactions must have a duration of 10000 ms or less.
Defining SLA rules
Define the criteria for your SLA rules in the sla_rules.yml file.
Add the sla_rule.yml to your script folder, before running the test. If the rules in the file are enabled, then at the end of test execution, the replay summary displays a pass or fail status, and generates a report file.
Two report formats are supported:
- JUnit: Format for the JUnit unit testing framework, in an XML file.
- TAP: Format for the TAP unit testing framework, in a TAP file.
To run a test with SLA rules:
- Copy the sla_rules.yml file from <DevWeb root>\examples\SLAJUnit to your script folder.
Open the sla_rules.yml file and edit the contents, and add extra rules as required, as relevant for your script. Use one of the examples given below in SLA rules file example as a guide.
For each transactionName value, enter the name of the transaction as it appears in the script.
You can enable or disable the whole file, or individual rules.
Define the report settings:
- The type of report file (reporterType) to generate, JUnit or Tap.
The name for the report, in the format <name>.xml for JUnit reports, or <name>.tap for TAP reports.
The report is generated inside the script folder.
- Execute the test. See Run scripts.
At the end of the test, check the SLA rule results (after the replay summary) :
- Open the script folder and locate the JUnit or TAP report. Review the report to see more detailed information on the rule calculations.
Example of an sla_rules.yml file:
enabled: true # Allows the user to enable or disable all the rules in a category reporterType: JUnit # Allows to change the reporter type, supported reporters: [JUnit, Tap] reportName: sla_junit_report.xml # The report name, which is generated inside the script directory rules: # List of the rules which the SLA enforces - rule: Transaction Response Time # The SLA rule measures the average duration of the given transactions type: averageTRT # The transaction type enabled: true # Allows the user to enable or disable a specific rule arguments: # The arguments which a given type is expecting (at least one entry) - transactionName: name1 # The name of the transaction for which we want to calculate the average threshold: - averageTime: 1600 # The average time in milliseconds above which the SLA rule will fail - transactionName: name2 # The name of the transaction for which we want to calculate the average threshold: - averageTime: 1650 # The average time in milliseconds above which the SLA rule will fail - rule: Transaction Response Time type: averageTRT enabled: false arguments: - transactionName: example1 threshold: - averageTime: 100 - transactionName: example2 threshold: - averageTime: 5000 - rule: Failing Transactions to Total Transaction Ratio # The SLA rule measures the percentage of total failed transactions type: failureRate # The transaction type enabled: false # Allows the user to enable or disable a specific rule arguments: # The arguments which a given type is expecting (one entry) percentage: 10 # The percentage threshold above which the SLA rule will fail - rule: Total Passing Transactions # The SLA rule validates that the amount of passed transactions reach a minimum threshold. type: passingTransactions enabled: true arguments: amount: 25 # The minimum amount of passed transactions for the SLA to pass. - rule: Transaction Rank Percentile # The SLA rule validates if transaction duration measurements meet percentile rank specified in the arguments. type: TRP enabled: true arguments: - transactionName: name1 # The name of the transaction for which we want to calculate the transaction percentile rank ranges: [2000, 5000, 10000] # Defines the duration ranges in milliseconds. For example, 5000 defines the range 0-5000. threshold: # The number of threshold ranks must match the number of ranges. ranks: [88, 98, 100] # Defines the percentile ranks for the ranges. For example, 98% of transactions are required to have a duration of 0-5000 milliseconds.