Correlate dynamic values

This topic describes the process used to correlate dynamic data in your script.

About correlation

Web pages sent from the server to the browser often contain dynamic data that the client must send within later requests—for example, the session ID, which may contain a timestamp. This data changes every time you access the page.

After recording a business process, the script might contain dynamic values in the server responses. When the script is replayed, these recorded values are sent to the server. However, the web server rejects them because they are not valid in the replay session. Generally, the test fails when this happens.

To enable successful replay, the script must:

  • Locate the dynamic data in the server responses, including headers.
  • Extract and save that dynamic data.
  • Replace the hard-coded dynamic data from the recording session with the data from the current replay session.

This process of locating, extracting, and replacing recorded dynamic values with values valid at replay is called correlation.

The DevWeb engine handles correlation during script generation by applying pre-defined correlation rules, or by performing a record scan on the recorded data.

Back to top

Correlation methods

Two methods are available for automatic correlation for dynamic values: rule-based scans and record-based scans. Both add correlations to the generated code using Extractors and appliers.

By default, the rules and record scans are disabled. You enable them in the generator_config.yml file (or in VuGen, in Recording Options > Code Generation).

Rule-based correlation

The DevWeb engine implements a set of predefined, extendable rules for correlating dynamic values in standard environments. These rules are defined in <correlation_rules>.yml files, in the correlation rules directory.

During code generation, the Offline Script Generator loads all activated rules found in the *.yml correlation rule files, and scans server responses for values that match the correlation rules. When a matching value is found, a step is added to the web request to extract the value and save it in a parameter.

Then, the script code is scanned for the value and matches are replaced with a reference to the parameter.

Use rule-based correlation:

In the correlation section of the <DevWeb root folder>\generator_config.yml file:

  • Enable the rules scan by setting rulesScanEnabled to true.
  • Change defaults for other correlation parameters as relevant, for example, the scan timeout or maximum length for the extracted value displayed in the script.

For details on defining correlation rule files, see Using correlation rules

Record-based correlation

The DevWeb engine provides automatic record-based correlation. When record scan is enabled, then during the code generation stage, the engine automatically scans the server responses for correlations. The search is based on internal engine configuration settings.

When a matching value is found, a step is added to the web request to extract the value and save it in a parameter, in the same way as for rules scan correlation. If the rules and the record scans are both enabled, the rules scan takes priority; this means that if both scans identify the same value, the engine applies the rule-based correlation.

The extractor type added by the record scan may be boundary or regular expression (according to the defined preferred extractor type), or an extractor appropriate for the detected content type, for example, an HTML, JSON, or XPath extractor.

Note: During the record scan, LoadRunner Developer uses predefined settings to filter content and patterns, to help detect the correlation candidates. These configurations are defined in the <DevWeb root folder>\resources\CorrelationEngine\record_scan_config.yml file.

We recommend that you do not make changes to this file. If you do need to make changes, copy the record_scan_config.yml file to a new location and make the changes in the copy. Then change the path in the recordScanConfigDirectory key, as described below.

Use record-based correlation:

  1. In the correlation section of the <DevWeb root folder>\generator_config.yml file:

    • Enable the record scan by setting recordScanEnabled to true.
    • If required, change the preferred extractor type for plain text in the extractorType property: boundary or regexp.
    • If required, update the path in the recordScanConfigDirectory property.
    • Change defaults for other correlation parameters as relevant, for example, the scan timeout or maximum length for the extracted value displayed in the script.
  2. Record the script. Extractors added by the record scan are indicated in the comment:

  3. After code generation, you can use the generation log ..\<Script folder>\codegen to help identify if the candidates found are real correlations.

Back to top

Extractors and appliers

DevWeb engine correlations are based on extractors and appliers. The extractors can be added automatically during code generation based on rule-based or record-based scans; or added manually to the script.

When scanning server responses, the Offline Script Generator adds an extractor for each identified dynamic value. This enables the value to be extracted and saved in a parameter. Then, the script code is scanned for the value, and matches are replaced with a reference to the parameter.

Tip: When the log level (in rts.yml) is set to debug or trace, the extractor substitutions are reported in the log. The log messages document each place in the script that the extractor is applied, and the substitute value used for each instance.

Applier types

The following table lists the applier types available.

The definitions for the supported parameters are provided here: Applier parameters

Applier type Description Supported parameters
simple Replaces the extracted value with a dynamic parameter that was created from the extractor. The replacement can occur in any subsequent requests to the original response (the server response containing the extractor).
  • occurrence (optional)
  • scope (optional)
  • fieldName (optional)
boundary Replaces an extracted value between defined boundaries (left and right), with a dynamic parameter that was created from the extractor. The replacement can occur in any subsequent requests to the original response (the server response containing the extractor).
  • leftBoundary (mandatory)
  • rightBoundary (mandatory)
  • occurrence (optional)
  • scope (optional)
  • caseInsensitive (optional)
  • fieldName (optional)
regexp Replaces an extracted value that follows a regular expression, with a dynamic parameter that was created from the extractor. The replacement can occur in any subsequent requests to the original response (the server response containing the extractor).
  • expression (mandatory)
  • flags (optional)
  • occurrence (optional)
  • scope (optional)
  • groupNumber (optional)
  • fieldName (optional)
none

Used when you want to enable the generation of extractors in the script, but do not want the extracted values to be applied in the script.

This avoids the default behavior: If DevWeb cannot find where to apply the extracted values, then the extractor is not generated.

n/a

Applier parameters

The following gives the definitions for the applier parameters. To see which parameters are supported for each applier type, see Applier types, above.

expression string The regular expression.
flags string The flags for the regular expression.
leftBoundary string The left boundary.
rightBoundary string The right boundary.
caseInsensitive boolean

Search boundaries using non-case sensitive comparison.

Default: true

occurrence string/number

The occurrence of the value to replace:

  • all. All the found occurrences are replaced.
  • first. The first occurrence is replaced.
  • last. The last occurrence is returned.
  • A number defining the position of the value to replace, where 0 defines the first occurrence. So, for example, 2 replaces the third occurrence.

If occurrence is not defined, the first occurrence is replaced.

scope

string

The relevant HTTP part in which the value will be replaced: url, querystring, headers, body, or all

Default: all

fieldName list of strings

A key (or keys) defining the focus within the scope area.

For example, if the value for scope is headers, but you want the replacement to be in a particular header MyHeader, set fieldName to [“MyHeader”].

Back to top

Using correlation rules

Rules-based correlation scans use the rules defined in correlation rule definition files, <correlation_rules>.yml. You create customized correlation rule files, to add and edit rules for your specific DevWeb scripts.

Correlation rule files are, by default, located in the <DevWeb root folder>\CorrelationRules\ folder. This folder includes a predefined correlation rules file for an ASP.NET testing environment, correlation_ASP_NET.yml. You can edit this file, or make copies to use as templates. Edit and extend the rule definitions as needed for your testing environment. You can also refer to the Example file below.

Each correlation rule group includes Extractors and appliers. Within the file you can enable or deactivate each rule group, and enable or deactivate the whole correlation rules file. If rules scan is enabled in generator_config.yml, then during code generation the Offline Script Generator scans all *.yml files in the CorrelationRules folder, and loads all enabled rules.

Tip: Check out the video: Tutorial #7: Correlations.

To use correlation rules:

  1. Define correlation rule files (*.yml) for your testing environment. Save the files in the default <DevWeb root folder>\CorrelationRules\ folder, or in a different location.

  2. Before recording your script, make sure that you have enabled all required correlation files and rule groups in the correlation *.yml files.
  3. Configure correlation parameters in the <DevWeb root folder>\generator_config.yml file, including setting rulesScanEnabled to true, and modifying the rulesDirectory path if you are using a different folder for the correlation rule files. (See Rule-based correlation, above.)
  4. Record the script. When a match is found for a rule, an extractor for the value is added to the script.

Example file

The following gives a example of an annotated <correlation_rules>.yml file:

categories:
  - name: category1                   #The name of the category, not used in code, this is for the user
    enabled: true                     #Allows the user to enable or disable all the rules in a category
    rules:                            #A list of rules
      - name: myRule1                 #The name of the rule, used in the code the user writes to identify the rule. Must be unique across all categories
        type: boundary                #The type of rule. Can be "boundary", "regexp", "json", "xpath"
        enabled: true                 #Allows the user to enable and disable particular rules
        options:                      #The options particular to this rule type
           leftBoundary: abc          #Left boundary of the boundary-based correlation
           rightBoundary: efg         #Right boundary of the boundary-based correlation
           caseInsensitive: false     #Search boundaries using case-insensitive comparison (default: true)
           scope: all                 #Scope define the required HTTP part: headers, body, or all Default: all
        appliers:
          - type: simple              #The type of the applier. One of "simple", "boundary", "regexp" or "none" which prevent apply automatically
            scope: headers            #Scope define the required HTTP part: url, querystring, headers, body, or all Default: all
            options:
              occurrence: first       #Occurrence focus on how many items to apply on all, first, last or valid number
              fieldName: ["cookie"]   #Field Name allow the applier to focus on one or more key entries
      - name: myRule2
        type: regexp
        enabled: true
        options:
           expression: "[0..9]*"      #The regular expression. See https://github.com/google/re2/wiki/Syntax for details
           flags: "im"                #The flags of the regular expression
           groupNumber: 0             #The group number (Default: 1 - for the first group)
           scope: body                #Scope define the required HTTP part: headers, body, or all Default: all
        appliers:
          - type: boundary
            scope: url
            options:
              leftBoundary: le>            #Left boundary of the boundary based correlation
              rightBoundary: </t            #Right boundary of the boundary based correlation
              caseInsensitive: false         #Search boundaries using case-insensitive comparison (default: true)
  - name: category2
    enabled: true
    rules:
      - name: myRule3
        type: json
        enabled: true
        options:
           path: abc                        #The JSONPath. See http://goessner.net/articles/JsonPath/ for details
        appliers:
          - type: regexp
            scope: all
            options:
              expression: "\\w+(.*?)\\w+"      #The regular expression see https://github.com/google/re2/wiki/Syntax for details
              flags: "i"                       #The flags of the regular expression, please note the group number is default 1 no need to define it
              occurrence: first
      - name: anotherRule3                     #Appliers section is optional on this case it will use the Default Applier
        type: xpath
        enabled: true
        options:
          path: //myEntry                      #The XPath, see https://godoc.org/gopkg.in/xmlpath.v2 for details

If no appliers are defined for a rule, the following default applier are used:

appliers:
  - type: simple
    scope: all
    options:
      occurrence: all

Back to top

See also: