Upload External Test Result to ALM

Micro Focus ALM Help Center

Software version: For Micro Focus Application Automation Tools Plugin 6.4 and later

Software release date: January 2021

Document publication date: February 2022

We introduced a post-build action for the Micro Focus Application Automation Tools plugin (short for plugin): Upload test result to ALM using field mapping. This post-build action allows you to upload test results from an XML file to ALM. The test results can come from tests running in any frameworks or tools, not limited to JUnit, NUnit, or TestNG. After the upload, the plugin will update or create tests, test sets, test instances, and test runs of corresponding types in ALM.

In this topic:

## High-level steps

Here is the summary of what you should do to use the post-build action for uploading test result from an XML test report to ALM:

1. Prepare an XML file that includes the test result. You can manually prepare the XML file or have it generated by a testing tool or framework.
2. Install the plugin, version 6.4 or later. See https://plugins.jenkins.io/hp-application-automation-tools-plugin/.

3. Create a new Jenkins job to test the plugin configuration. See Create a job to test plugin configuration.

   We recommend you create a new job for the first time you use the plugin. You need to test the plugin configuration before implementing it in your production environment.

4. Add the post-build action Upload test result to ALM using field mapping to the new Jenkins job or an existing job. See Set up the post-build action.

Notes:

- If you want to upload the duration information, such as test execution time, to ALM, create a string type user-defined field in ALM. This is because the ALM system field Duration allows integers only. It cannot be used to record the duration information (normally a float number) from the XML file.

- The plugin does not support setting date type system fields for ALM entities. For example, "Creation Date" in tests and "Exec date" in test instances. Such date field values are set automatically by ALM when the test result is uploaded.

  If you want to upload the date information from the test result XML file to ALM, create a date type user-defined field. This filed only supports date and does not support time.

  The date format in the XML file should be yyyy-mm-dd.

## Create a job to test plugin configuration

Follow the steps below to create a new job from the scratch. This is recommended if you want to try this feature in your testing environment.

If you want to use it directly in your production environment where there is already a Jenkins job to run tests, skip to Set up the post-build action.

1. Go to the Jenkins Server home page.
2. Click the New Item link.
3. Enter a job name.
4. Select Build a free-style-software project and click OK.

5. In the Project Configuration section scroll down to the Build section.

6. Expand the Add build step drop-down and select Execute shell for non-Windows systems or Execute Windows batch command for Windows systems.

   Write the command to copy the sample test result XML file to the Jenkins job folder.

   The plugin supports searching the test result file in both Jenkins server and slave machines. You can copy the XML file to either location.

   - To copy the sample XML file to Jenkins server

     Non-Windows:

     `cp -f <Path_of_copied_sample_XML>/<Report_Name>.xml <Jenkins_Home_Directory>/jobs/<Jenkins_Job_Name>/builds/${BUILD_ID}`

     Windows:

     `copy "<Path_of_copied_sample_XML>\<Report_Name>.xml" <Jenkins_Home_Directory>/jobs/<Jenkins_Job_Name>/builds/%BUILD_ID%`

   - To copy the sample XML file to your Jenkins slave machine

     Non-Windows:

     `cp -f <Path_of_copied_sample_XML>\<Report_Name>.xml {WORKSPACE}`

     Windows:

     `copy "<Path_of_copied_sample_XML>\<Report_Name>.xml" %WORKSPACE%`

## Set up the post-build action

1. Open a Jenkins job that you created in Create a job to test plugin configuration or an existing Jenkins job that runs tests automatically.

2. Click Configure. In the Post-build Actions section, expand the Add post-build action drop-down and select Upload test result to ALM using field mapping.

   Specify the following fields.

   Field | Description
   --- | ---
   ALM Server | Select one of the ALM servers that you configured in the Configure the connection to ALM step. See https://github.com/jenkinsci/hpe-application-automation-tools-plugin/blob/latest/doc/ALM_Integration.md.
   ALM Credentials | Select a pair of ALM credentials. If not available, click Add to add credentials.
   Domain | Enter the domain name.
   Project | Enter the project name.
   Client Type | Enter a client type that you can use to access the ALM server. It's required only when API restriction is enabled in the ALM server. Skip this field if there is no API restriction in the ALM server.
   Test Folder | Enter the path of the test folder that will contain the uploaded tests. Don't include the root test folder (Subject) in the path. If any folders in the path do not exist in ALM, the plugin will create these folders based on the structure and create an entity under the path. For example, if the test folder is `sampletestfolder\subfolder`, and if neither `sampletestfolder` nor `subfolder` exists in ALM, then the plugin will create the `sampletestfolder` folder under `Subject`, the `subfolder` folder under `sampletestfolder`, and a new test under `subfolder`. Test folder supports Jenkins environment variables.
   Test Set Folder | Enter the path of the test set folder. Don't include the root test set folder in the path. The plugin searches the entire test set tree by the test set name defined in the XML file. If none is found, a new test set is created under the path. If any folders in the path do not exist in ALM, the plugin will create these folders based on the structure. Test set folder supports Jenkins environment variables.
   Testing Result File | Specify the path to test result XML file in the Ant glob syntax, such as `**/*.xml`. Make sure you do not include any non-report files into this pattern. You can specify multiple patterns of files separated by commas.
   Field Mapping | Map the XML elements and attributes to the ALM entities and fields. Field mappings support using Jenkins environment variables. For details, see How to set field mappings.
   - Run Status Mapping | Map test status values to ALM run status values. Prerequisite: You should map an XML element or attribute to the ALM status field. You can map both ALM system run statuses, such as No Run, Passed, Failed, Blocked, and Not Completed, and user-defined statuses. For sample run status mapping, see Sample run status mapping. Note: If the status values in the XML file are the same as the ALM status values, you don't need to set run status mapping. You don't need to remove mappings if they are no longer relevant. To make sure the test result can be uploaded successfully, we recommend that you list all possible status mappings. For example, the following mappings can co-exist. As long as at least one of the mappings is valid, the test result can be uploaded.
         Success: "Passed"
         True: "Passed"
         1: "Passed"

### How to set field mappings

The mapping is configured in the YAML format. It should include the following sections with the same order: "testset", "test", and "run". In each section, you should define the values for the following keys:

- root: The root node path of test set, test, or run.

- All mandatory fields of test set, test, and run in your ALM project

  A key represents an ALM entity's field. Its value represents the element or attribute value in the XML file that will be uploaded to the ALM field.

  If you want to upload a value to a system field, the key should be the field name. ALM field names are not always identical to field labels. You should use ALM field names instead of labels as keys. For details about how to know field names, see REST APIs to get ALM entity field names.

  If you want to upload a value to a user defined field, the key should start with `udf|` followed by the field label. For example, if you want to upload a value to the user-defined field "Duration", the key should be `udf|duration`.

- (Optional) Other ALM fields you want to map

How to specify values?

There are two types of values: Values that start with `x:` and values that start with `v:`.

`x:` | The value that follows `x:` is the XPath (XML Path Language) of an element or attribute in the XML file. You can define either absolute path (that starts with `/`) or relative path (that starts directly with the XML element name). `x:.` means the XPath is exactly the same as is defined in the previous entity. For example, if `root: "x:."` is defined in the "run" section, and the "run" section follows the "test" section in the field mapping, then it means the run's root is the same as test's root. If the value to upload to an ALM field is an XML element value, what follows `x:` should be the element name. For example, `x:duration`. If the value to upload is the attribute value of an XML element, what follows x: should be `@<attribute_name>`. For example, `x:@length`.
`v:` | The value that follows `v:` is a fixed string value that will be uploaded to a test set, test, or test run. For example, if in the testset section, it has `name: "v:2021-01-29__Build_Verification_Test"`, then the test set name uploaded to ALM will be `2021-01-29__Build_Verification_Test`. To specify values for the ALM system field "Type", you should pick the values from Test set, test, and run types and prefix the values with `v:`.

If a value is composed of values from several XML elements or attributes, use `|` to combine all parts of the value together. For example, `x:file|v:_|x:name`.

For sample test result and field mapping, see Sample test result XML and Sample field mapping.

### REST APIs to get ALM entity field names

Call the following REST APIs to get the field names of the test set, test, and run entities. The name of a field is the value of the Name attribute in the Field element.

Get field names of test set | http://<alm_server>:<port>/qcbin/rest/domains/<domain_name>/projects/<project_name>/customization/entities/test-set/fields?login-form-required=y
Get field names of test | http://<alm_server>:<port>/qcbin/rest/domains/<domain_name>/projects/<project_name>/customization/entities/test/fields?login-form-required=y
Get field names of run | http://<alm_server>:<port>/qcbin/rest/domains/<domain_name>/projects/<project_name>/customization/entities/run/fields?login-form-required=y

### Test set, test, and run types

Test set type | hp.qc.test-set.default hp.qc.test-set.external
Test type | ALT-SCENARIO ALT-TEST BUSINESS-PROCESS DB-TEST EXTERNAL-TEST FLOW LEANFT-TEST LR-SCENARIO MANUAL QAINSPECT-TEST QTSAP-TESTCASE QUICKTEST_TEST SERVICE-TEST SYSTEM-TEST VAPI-XP-TEST WR-AUTOMATED WR-BATCH XR-TEST
Run type | custom.run.LEANFT-TEST hp.qc.run.ALT-SCENARIO hp.qc.run.ALT-TEST hp.qc.run.BUSINESS-PROCESS hp.qc.run.DB-TEST hp.qc.run.FLOW hp.qc.run.LR-SCENARIO hp.qc.run.MANUAL hp.qc.run.QAINSPECT-TEST hp.qc.run.QTSAP-TESTCASE hp.qc.run.QUICKTEST_TEST hp.qc.run.SERVICE-TEST hp.qc.run.SYSTEM-TEST hp.qc.run.VAPI-XP-TEST hp.qc.run.WR-AUTOMATED hp.qc.run.WR-BATCH hp.qc.run.XR-TEST hp.qc.run.external-test hp.sprinter.run.SPRINTER

### Samples

Sample test result XML

    <?xml version='1.0' encoding='UTF-8'?>
        <result>
            <suites>
                <suite>
                    <file>Changes file</file>
                    <name>Changes Test Set 1</name>
                    <duration>2.293</duration>
                    <cases>
                        <case>
                            <duration>8.293</duration>
                            <className>ChangesManagement</className>
                            <testName>test1</testName>
                            <testVersion>4.0</testVersion>
                            <skipped>false</skipped>
                            <failedSince>0</failedSince>
                        </case>
                        <case>
                            <duration>8.293</duration>
                            <className>ChangesManagement</className>
                            <testName>List changes 2</testName>
                            <testVersion>4.0</testVersion>
                            <skipped>false</skipped>
                            <failedSince>0</failedSince>
                        </case>
                    </cases>
                </suite>
                <duration>0.576</duration>
        </result>

Sample field mapping

    testset:
      root: "x:/result/suites/suite"
      name: "x:file|v:_|x:name"
      udf|duration: "x:duration"
      subtype-id: "v:hp.qc.test-set.external"
    test:
      root: "x:cases/case"
      name: "x:testName"
      subtype-id: "v:EXTERNAL-TEST"
      udf|duration: "x:duration"
      udf|test version: "x:testVersion"
    run:
      root: "x:."
      duration: "x:duration"
      status: "x:failedSince"

Sample run status mapping

    status:
      True: "Passed"
      False: "Failed"
      1: "Passed"
      0: "Failed"
      Success: "Passed"
      Failure: "Failed"
      Block: "Blocked"

## Run the job

Run or schedule the job as you would with any standard Jenkins job.

## After the upload is successfully done

- If no existing test sets are found by the test set names defined in the XML file, new test sets are created.
- If no existing test instances are found by the test names defined in the XML file, new test instances are created.
- If the test results to upload are for the same existing test sets and tests, the plugin only creates test runs in the test instances.

## Review the results

1. Open the Jenkins job. In the Build History section, click a build ID.

2. In the build details page, click Console Output to view the log of the test result upload.

   To quickly locate the upload-related log, search the log by "Start login to ALM server".

3. Log in ALM to view the tests, test sets, test instances, and test runs that were created within ALM.

Note: The console output is important to understand the details of how the plugin works, especially when the job fails at the first time. It helps you to find and fix the issues.

© 2002 - 2022 Micro Focus

Send documentation feedback