Correlation for DevWeb scripts
This topic describes the processes used to correlate dynamic data in your scripts.
Correlation refers to the process of locating, extracting, and replacing recorded dynamic values with values that are valid at replay. For general information about correlations in VuGen, see Correlation overview.
DevWeb handles correlation for dynamic values using two methods: rules-based scan and record-based scan. Both add correlations to the generated code using extractors and appliers. During code generation, when a correlation candidate is discovered, the appropriate extractor is added to the web request step. For details, see Extractors and appliers.
|Rules scan||DevWeb includes a set of predefined, extendable rules for correlating dynamic values in standard environments. You can edit these rules and define new ones. You can also activate or disable the rules for your scripts. During code generation, if rules scan is enabled, then all activated rules are loaded from the correlation rule files.|
|Record scan||When the record scan is enabled, DevWeb automatically scans the server responses for correlations. The search is based on internal engine configuration settings.|
You can also manually search for values to correlate in your script. For details see Search and correlate selection.
Note: The record scan uses predefined settings to filter content and patterns, to help detect the correlation candidates. These configurations are defined in the <DevWeb_data_path>\resources\CorrelationEngine\record_scan_config.yml file.
We recommend that you do not make changes to this file. If you do need to make changes, copy the record_scan_config.yml file to a new location and make the changes in the copy. Then change the path in the recordScanConfigDirectory key, as described below.
By default, the rules and record scans are disabled. For either or both correlation scans to occur, you must enable them in the recording options.
If the rules and the record scans are both enabled, the rules scan takes priority. So if both scans identify the same value, the engine uses the rules scan correlation.
To use correlation rules:
- Open your DevWeb script.
Define or edit the correlation rules that you require for your testing environment in the Correlation rules node of the recording options. Select the rules that you want to include.
Note: The recommended way to edit correlation rules is using the recording options UI. However, you can also manually edit the correlation rule files (*.yml). They are located (by default) in the <DevWeb root folder>\CorrelationRules\ folder.
Enable and configure the required correlation parameters in the Code Generation node of the recording options:
Rules scan Enable to run a rules scan during code generation. Record scan
Enable to run a record scan during code generation.
Select the preferred extractor type for plain text, Boundary or Regular expression.
Show correlation candidates during code generation Enable to display a dialog box with all the scan results during script generation. When this option is selected, you can manually select and approve the discovered correlations before they are applied to the code.
- Generate or regenerate the script.
- If the Correlation Candidates dialog box is displayed during code generation, select the required correlation candidates and click Correlate. For details, see Managing correlation candidates.
- When a correlation match is found, the relevant extractor is added to the web request step.
After script generation, you can view correlation scan information in the Output pane:
In the code, view the comments to see extractors added by the scans:
DevWeb engine correlations are based on extractors and appliers. When the correlations are applied during code generation, extractors are added based on the rules or record scans.
You can define extractors and appliers for your DevWeb scripts by creating your own customized correlation rules. For details, see Correlation rules node.
When scanning server responses, the DevWeb engine adds an extractor for each identified dynamic value. This enables the value to be extracted and saved in a parameter. Then, the script code is scanned for the value, and matches are replaced with a reference to the parameter.
The extractors locate and extract the dynamic data in the responses. They can be based on boundary definitions or regular expressions, or can be used to query for object values inside JSON, XML, and HTML-formatted data. They also include converters for Base64, URL, and HTML encoding and decoding.
The appliers apply the extracted values, so that the saved parameter is used instead of the recorded value in subsequent requests to the server. You use the appliers to refine the way the extracted value is applied in the request.
If no appliers are defined for a rule, a default Simple applier is used.
Tip: When the log level (in DevWeb runtime settings) is set to Debug or Trace, the extractor substitutions are reported in the log. The log messages document each place in the script that the extractor is applied, and the substitute value used for each instance. (This functionality is available from version 2022 R1.)
Note: This feature is provided as tech preview.
When the Correlation Candidates dialog box is displayed during code generation, you can manage the correlations that are applied to your script.
The Correlation Candidates dialog box provides a list of the results from the rules or record scans. It is displayed when you record and generate/regenerate the script, if there are discovered correlation candidates.
For each correlation candidate you can view the method used to discover it—rule or record scan. You can also view the response text to be correlated, and the extractor name that will replace the text in the requests.
To manage correlation candidates:
Prerequisite: To display the Correlation Candidates dialog box, make sure Show correlation candidates during code generation is enabled in the recording options. For details, see Working with correlation rules.
- In the Correlation Candidates dialog box, to change an extractor name, select the appropriate correlation candidate line and type in the new extractor name.
To exclude a correlation candidate from this script, and from all future correlation scans for all scripts, select the candidate line and click the Exclude candidate button. Then click Exclude in the displayed warning message.
The correlation candidate is removed from the list.
Tip: You can remove strings from the excluded list by editing the <DevWeb root folder>\resources\CorrelationEngine\excluded_strings.txt file.
To continue with code generation for the script, select the check boxes for the correlation candidates that you want to apply to the script, and click Correlate.
To continue code generation without applying correlations to the script, click Ignore.
Note: This feature is provided as tech preview.
If your DevWeb script includes snapshots during replay, you can manually select the text you want to correlate, and then use the Correlate Selection option to add an extractor to the first reference to the value.
Tip: You can also generate extractors directly into web requests from the DevWeb Snapshot pane. For details, see Actionable snapshots.
To correlate your selection:
- In the VuGen editor, within a web request in your DevWeb script, select the text to correlate.
Select Correlate Selection from the right-click menu.
Note: The Correlate Selection option is enabled only if the following are true:
- The selected text is in the context of a web request.
- The selected text is not in the first request.
- The selected text must contain at least 4 characters.
VuGen searches backwards to find the first occurrence of the value in a snapshot, and adds a boundary extractor to the web request.
If required, manually change the extractor to a different extractor type.