Share this page

# Integrate with external tests

To integrate with external tests running on Jenkins, a post-build action is introduced for the Application Automation Tools plugin: Upload test result to ALM using field mapping. This plugin reads test reports of external testing frameworks or tools, upload the test result to OpenText Application Quality Management, and then update or create tests, test sets, test instances, and test runs.

Note: The previous external tests integration solution, as described in Import External Tests, is no longer updated. You can still use it if you prefer. However, we recommend you use the solution introduced in this topic.

In this topic:

## High-level steps

Here is the summary of what you should do to use the post-build action for uploading test result from an XML test report to OpenText Application Quality Management:

1. Prepare an XML file that includes the test result. You can manually prepare the XML file or have it generated by a testing tool or framework.
2. Install the plugin, version 6.4 or later. See Application Automation Tools.

3. Create a new Jenkins job to test the plugin configuration. See Create a job to test plugin configuration.

   We recommend you create a new job for the first time you use the plugin. You need to test the plugin configuration before implementing it in your production environment.

4. Add the post-build action Upload test result to ALM using field mapping to the new Jenkins job or an existing job. See Set up the post-build action.

Notes:

- If you want to upload the duration information, such as test execution time, to OpenText Application Quality Management, create a string type user-defined field in OpenText Application Quality Management. This is because the system field Duration allows integers only. It cannot be used to record the duration information (normally a float number) from the XML file.

- The plugin does not support setting date type system fields for entities. For example, "Creation Date" in tests and "Exec date" in test instances. Such date field values are set automatically when the test result is uploaded.

  If you want to upload the date information from the test result XML file, create a date type user-defined field. This filed only supports date and does not support time.

  The date format in the XML file should be yyyy-mm-dd.

Back to top

## Create a job to test plugin configuration

Follow the steps below to create a new job from the scratch. This is recommended if you want to try this feature in your testing environment.

If you want to use it directly in your production environment where there is already a Jenkins job to run tests, skip to Set up the post-build action.

To create a job to test the plugin configuration:

1. Go to the Jenkins Server home page.
2. Click the New Item link.
3. Enter a job name.
4. Select Build a free-style-software project and click OK.

5. In the Project Configuration section scroll down to the Build section.

6. Expand the Add build step drop-down and select Execute shell for non-Windows systems or Execute Windows batch command for Windows systems.

   Write the command to copy the sample test result XML file to the Jenkins job folder.

   The plugin supports searching the test result file in both Jenkins server and agent machines. You can copy the XML file to either location.

   To copy the sample XML file to the Jenkins server, run the following command:

   - Windows:

     Copy code

         copy "<Path_of_copied_sample_XML>\<Report_Name>.xml" <Jenkins_Home_Directory>/jobs/<Jenkins_Job_Name>/builds/%BUILD_ID%

   - Non-Windows:

     Copy code

         cp -f <Path_of_copied_sample_XML>/<Report_Name>.xml <Jenkins_Home_Directory>/jobs/<Jenkins_Job_Name>/builds/${BUILD_ID}

   To copy the sample XML file to the Jenkins agent machine, run the following command:

   - Windows:

     Copy code

         copy "<Path_of_copied_sample_XML>\<Report_Name>.xml" %WORKSPACE%

   - Non-Windows:

     Copy code

         cp -f <Path_of_copied_sample_XML>\<Report_Name>.xml {WORKSPACE}

Back to top

## Set up the post-build action

Follow the steps below to set up the Upload test result to ALM using field mapping post build action.

To set up the post-build action:

1. Open a Jenkins job that you created in Create a job to test plugin configuration or an existing Jenkins job that runs tests automatically.

2. Click Configure. In the Post-build Actions section, expand the Add post-build action drop-down and select Upload test result to ALM using field mapping.

   Specify the following fields.

   Field | Description
   --- | ---
   ALM Server | Select one of the servers that you configured in the Configure the connection to ALM step. See https://github.com/jenkinsci/hpe-application-automation-tools-plugin/blob/latest/doc/ALM_Integration.md.
   ALM Credentials | Select a pair of credentials. If not available, click Add to add credentials.
   Domain | Enter the domain name.
   Project | Enter the project name.
   Client Type | Enter a client type that you can use to access the server. It's required only when API restriction is enabled in the server. Skip this field if there is no API restriction in the server.
   Test Folder | Enter the path of the test folder that will contain the uploaded tests. Don't include the root test folder (Subject) in the path. If any folders in the path do not exist in OpenText Application Quality Management, the plugin will create these folders based on the structure and create an entity under the path. For example, if the test folder is `sampletestfolder\subfolder`, and if neither `sampletestfolder` nor `subfolder` exists, then the plugin will create the `sampletestfolder` folder under `Subject`, the `subfolder` folder under `sampletestfolder`, and a new test under `subfolder`. Test folder supports Jenkins environment variables.
   Test Set Folder | Enter the path of the test set folder. Don't include the root test set folder in the path. The plugin searches the entire test set tree by the test set name defined in the XML file. If none is found, a new test set is created under the path. If any folders in the path do not exist, the plugin will create these folders based on the structure. Test set folder supports Jenkins environment variables.
   Testing Result File | Specify the path to test result XML file in the Ant glob syntax, such as `**/*.xml`. Make sure you do not include any non-report files into this pattern. You can specify multiple patterns of files separated by commas.
   Field Mapping | Map the XML elements and attributes to the OpenText Application Quality Management entities and fields. Field mappings support using Jenkins environment variables. For details, see How to set field mappings.
   Create new test if no match | Whether to create tests in OpenText Application Quality Management if no matching results are found.
   Run Status Mapping | Map run statuses and run step statuses. Prerequisite: To map run statuses, you should map an XML element or attribute to status. To map run step statuses, you should map an XML element or attribute to both stepMessage and stepRegEx. You can map both system run statuses, such as No Run, Passed, Failed, Blocked, and Not Completed, and user-defined statuses. A key represents a status in the XML file. Its value represents the mapped status in OpenText Application Quality Management. You can define the mapping using attributes or conditions. For example, both the following mapping patterns are supported. However, we recommend you use attributes for mapping.
       status:
        Success: "Passed"
        Failed: ">>0" # If status attribute value is greater or equals than 0 in the XML file, then the run will
        be marked as "Failed" in OpenText Application Quality Management.
   - For sample status mapping, see Sample run status mapping. If the status values in the XML file are the same as the status values in OpenText Application Quality Management, you don't need to set run status mapping. You don't need to remove mappings if they are no longer relevant. To make sure the test result can be uploaded successfully, we recommend that you list all possible status mappings. For example, the following mappings can co-exist. As long as at least one of the mappings is valid, the test result can be uploaded.
         Success: "Passed"
         True: "Passed"
         1: "Passed"

### How to set field mappings

The mapping is configured in the YAML format. It should include the following sections with the same order: "testset", "test", and "run". In each section, you should define the values for the following keys:

- root: The root node path of test set, test, or run.

- All mandatory fields of test set, test, and run in your project

  A key represents an entity's field. Its value represents the element or attribute value in the XML file that will be uploaded to the field.

  If you want to upload a value to a system field, the key should be the field name. Field names are not always identical to field labels. You should use field names instead of labels as keys. For details about how to know field names, see REST APIs to get entity field names.

  If you want to upload a value to a user defined field, the key should start with `udf|` followed by the field label. For example, if you want to upload a value to the user-defined field "Duration", the key should be `udf|duration`.

- (Optional) Other fields you want to map.

  Applicable to the run entity: If you want to upload run step statuses, you must map stepMessage and stepRegEx for the run entity. stepMessage represents details of run steps. stepRegEx refers to a regular expression pattern used to extraction specific run steps information.

How to specify values?

There are two types of values: Values that start with `x:` and values that start with `v:`.

`x:` | The value that follows `x:` is the XPath (XML Path Language) of an element or attribute in the XML file. You can define either absolute path (that starts with `/`) or relative path (that starts directly with the XML element name). `x:.` means the XPath is exactly the same as is defined in the previous entity. For example, if `root: "x:."` is defined in the "run" section, and the "run" section follows the "test" section in the field mapping, then it means the run's root is the same as test's root. If the value to upload to a field is an XML element value, what follows `x:` should be the element name. For example, `x:duration`. If the value to upload is the attribute value of an XML element, what follows x: should be `@<attribute_name>`. For example, `x:@length`.
`v:` | The value that follows `v:` is a fixed string value that will be uploaded to a test set, test, or test run. For example, if in the testset section, it has `name: "v:2021-01-29__Build_Verification_Test"`, then the test set name uploaded to OpenText Application Quality Management will be `2021-01-29__Build_Verification_Test`. To specify values for the system field "Type", you should pick the values from Test set, test, and run types and prefix the values with `v:`.

If a value is composed of values from several XML elements or attributes, use `|` to combine all parts of the value together. For example, `x:file|v:_|x:name`.

For sample test result and field mapping, see Sample test result XML and .

### REST APIs to get entity field names

Call the following REST APIs to get the field names of the test set, test, and run entities. The name of a field is the value of the Name attribute in the Field element.

Get field names of test set | http://<server>:<port>/qcbin/rest/domains/<domain_name>/projects/<project_name>/customization/entities/test-set/fields?login-form-required=y
Get field names of test | http://<server>:<port>/qcbin/rest/domains/<domain_name>/projects/<project_name>/customization/entities/test/fields?login-form-required=y
Get field names of run | http://<server>:<port>/qcbin/rest/domains/<domain_name>/projects/<project_name>/customization/entities/run/fields?login-form-required=y

### Test set, test, and run types

Test set type | hp.qc.test-set.default hp.qc.test-set.external
Test type | ALT-SCENARIO ALT-TEST BUSINESS-PROCESS DB-TEST EXTERNAL-TEST FLOW LEANFT-TEST LR-SCENARIO MANUAL QAINSPECT-TEST QTSAP-TESTCASE QUICKTEST_TEST SERVICE-TEST SYSTEM-TEST VAPI-XP-TEST WR-AUTOMATED WR-BATCH XR-TEST
Run type | custom.run.LEANFT-TEST hp.qc.run.ALT-SCENARIO hp.qc.run.ALT-TEST hp.qc.run.BUSINESS-PROCESS hp.qc.run.DB-TEST hp.qc.run.FLOW hp.qc.run.LR-SCENARIO hp.qc.run.MANUAL hp.qc.run.QAINSPECT-TEST hp.qc.run.QTSAP-TESTCASE hp.qc.run.QUICKTEST_TEST hp.qc.run.SERVICE-TEST hp.qc.run.SYSTEM-TEST hp.qc.run.VAPI-XP-TEST hp.qc.run.WR-AUTOMATED hp.qc.run.WR-BATCH hp.qc.run.XR-TEST hp.qc.run.external-test hp.sprinter.run.SPRINTER

### Samples

Sample test result XML

Copy code

    <?xml version='1.0' encoding='UTF-8'?>
        <result>
            <suites>
                <suite>
                    <file>Changes file</file>
                    <name>Changes Test Set 1</name>
                    <duration>2.293</duration>
                    <cases>
                        <case>
                            <duration>8.293</duration>
                            <className>ChangesManagement</className>
                            <testName>test1</testName>
                            <testVersion>4.0</testVersion>
                            <skipped>false</skipped>
                            <failedSince>0</failedSince>
                        </case>
                        <case>
                            <duration>8.293</duration>
                            <className>ChangesManagement</className>
                            <testName>List changes 2</testName>
                            <testVersion>4.0</testVersion>
                            <skipped>false</skipped>
                            <failedSince>0</failedSince>
                        </case>
                    </cases>
                </suite>
             <duration>0.576</duration>
           </suites>
        </result>

Sample field mapping

    testset:
      root: "x:/result/suites/suite"
      name: "x:file|v:_|x:name"
      udf|duration: "x:duration"
      subtype-id: "v:hp.qc.test-set.external"
    test:
      root: "x:cases/case"
      name: "x:testName"
      subtype-id: "v:EXTERNAL-TEST"
      udf|duration: "x:duration"
      udf|test version: "x:testVersion"
    run:
      root: "x:."
      duration: "x:duration"
      status: "x:failedSince"
      stepMessage: "x:@log"
      stepRegEx: "[+-] (\\S+) \\s+ ((?i:Step)\\s*\\d+.*?)\\r*\\n((?:(?!^[+-] \\S+ \\s+ (?i:Step)\\s*\\d+).*$\\r?\\n?)*)"

Sample run status mapping

    status:
      True: "Passed"
      False: "Failed"
      1: "Passed"
      0: "Failed"
      Success: "Passed"
      Failure: "Failed"
      Block: "Blocked"

    stepstatus:
     Passed: Passed
     Failure: Failed
     Success: Passed
     Completed: Passed

Back to top

## Run the job

Run or schedule the job as you would with any standard Jenkins job.

Back to top

## After the job

After the upload is successfully done:

- If an existing test is found by the test name defined in the XML file, and if stepMessage is mapped, the design steps in the test are removed and replaced with what is mapped to stepRegEx. If the stepMessage is not mapped, the design steps of the existing test are kept.
- If no existing test sets are found by the test set names defined in the XML file, new test sets are created.
- If no existing test instances are found by the test names defined in the XML file, new test instances are created.
- If the test results to upload are for the same existing test sets and tests, the plugin only creates test runs in the test instances.

Back to top

## Review the results

To review the results:

1. Open the Jenkins job. In the Build History section, click a build ID.

2. In the build details page, click Console Output to view the log of the test result upload.

   To quickly locate the upload-related log, search the log by "Start login to ALM server".

3. Log in to OpenText Application Quality Management to view the tests, test sets, test instances, and test runs that were created.

Note: The console output is important to understand the details of how the plugin works, especially when the job fails at the first time. It helps you to find and fix the issues.

Back to top

See also:

- Jenkins integrations
- Integrations

https://admhelp.microfocus.com Application Quality Management 8942
Tips to improve your search results... OpenText™ Application Quality Management Help Center OpenText™ Application Quality Management Help Center OpenText™ Application Quality Management Help 26.1 All View Local Help Center View Online Help Center Version: