UFT One and Jenkins

Run UFT One tests as part of your Jenkins continuous integration process.

Set up your Jenkins server with the Application Automation Tools plugin

Set up your Jenkins server to run UFT One tests.

Do the following:

  1. Install Jenkins and the Application Automation Tools plugin. For details, see the Jenkins plugin on GitHub.

  2. From the Jenkins Server home page, click New Job, or select an existing job.

    If you select to create a new job, enter the job name.

  3. Select Build a free-style software project, and click OK.

Continue with one of the following:

Back to top

Set up a job to run local UFT One tests

Set up a job to run one or more local UFT One tests.

Select execution nodes for your job

Select one or more execution node for the test runs in this job.

Do the following:

  1. In the General section of your Jenkins job, select This project is parameterized.

  2. Select Add Parameter > Node, and enter or define the following Node options:

    Name Define a name for your node definitions.
    Default nodes

    Select one or more default node to use when the job is triggered by an outside process.

    Possible nodes

    Select one or more node to use when the job is built manually.

    Build options

    Select one of the following options for node selection when triggering a build:

    • Run next build only if build succeeds

    • Run next build only if build succeeds or is unstable

    • Run next build regardless of build result

    • Allow multi-node selection for concurrent builds. Required for jobs configured with the Execute concurrent builds if necessary.

      When configured, the build is executed on all selected nodes in parallel. Ensure that all selected nodes are online before you build this job.

    • Disallow multi-node selection when triggering build manually. Select this option to enable users to select a build node when they trigger a build.

    Node eligibility

    Select how to handle nodes that are offline or temporarily offline.

    • All Nodes. Trigger the job on all selected nodes, regardless of their online/offline state.
    • Ignore Offline Nodes. Trigger the job only on nodes that are online and have an available UFT One instance to run the test.
    • Ignore Temp Offline Nodes. Trigger the job on all selected nodes, except those currently marked as offline.
    Description Define a description for your node configuration.

Note: If you configure failure scenarios, the node configured here must match the one selected in the failure scenario.

For more details, see Configure failure scenarios for your test runs.

Tip: The UFT One Jenkins plug-in supports a variety of methods for defining nodes for your job. This topic describes one of the most commonly used and recommenced procedures.

For more details, see Create an execution node on the Jenkins plugin GitHub page.

Add a build step to run UFT One tests

  1. Scroll down to the Build section, click Add build step, and select Execute Micro Focus tests from file system.
  2. In the Tests box, enter a test with its full absolute path, or a folder or MTB containing one or more tests.

    To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.

    Tip: Alternatively, indicate a batch file that contains a list of tests, along with their parameters. For details, see Use a batch file to specify multiple tests.

  3. Optional settings:

    Results directory Specify the location in which to save the test's results. The result files are named such that they do not overwrite any existing results in that location.
    UFT One parallel running mode

    Select to run your test on multiple environments.

    For more details, see Run tests on multiple environments in parallel.

    Timeout

    Indicate a timeout, in seconds, after which the job will fail.

    Required to configure Jenkins to load UFT One run results for any tests that had completed for this build step, even if a test failed and prevented other tests from running.

    Run Mode

    Define a UFT One run mode for your tests:

    1. Select UFT One-specific settings.
    2. From the Run Mode dropdown list, select Fast or Normal.

    For more details, see Test Runs Pane (Options Dialog Box > GUI Testing Tab).

    Configure failure scenarios

    Configure the actions that UFT One performs in case your test, or a test in your test set, fails.

    For details, see Configure failure scenarios for your test runs.

    Caution: We do not recommend configuring failure scenarios together with parallel test runs. Doing so may cause UFT One to behave unexpectedly.

  4. Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.
  5. Add a post-build action to define settings for the UFT One test results.

    In the Post-build Actions section, click Add post-build action, and select Publish Micro Focus test results.

    Select one of the following archiving options:

    Archive test reports for failed tests Only save test result reports for failed tests.
    Always archive test reports Always save test result reports.
    Do not archive test reports Never save the test results.
  6. Run or schedule the job as you would with any standard Jenkins job.

    When test is completed, continue with Review test run results.

Back to top

Security precaution

Although any sensitive information is encrypted and encoded, Micro Focus encourages you to routinely remove unnecessary files generated by your Jenkins jobs, which are not removed by the Micro Focus Jenkins Application Automation Tools plugin.

These are the files located in the <Jenkins installation folder>\workspace\<job name> folders.

By not implementing the file removal you may expose your system to increased security risks. You understand and agree to assume all associated risks and hold Micro Focus harmless for the same.

It remains at all times the Customer’s sole responsibility to assess its own regulatory and business requirements. Micro Focus does not represent or warrant that its products comply with any specific legal or regulatory standards applicable to Customer in conducting Customer's business.

Back to top

Use a batch file to specify multiple tests

Configure the Jenkins job to trigger multiple tests by specifying the tests in a batch file, together with their parameters.

Specify multiple tests, or even the same test several times, each time with different parameters. When specifying the path to your tests, Jenkins environment variables are supported for path names.

Give your batch file an .mtbx extension, and use the following sample syntax:

<Mtbx>
 <Test name="test1" path="c:\tests\APITest1">
 <Parameter name="A" value="abc" type="string"/>
 ….
 </Test>
 <Test name="test2" path="${WORKSPACE}\test2">
  <Parameter name="p1" value="123" type="int"/>
  <Parameter name="p4" value="123.4" type="float"/>
  ….
 </Test>
</Mtbx>

Continue with your build configuration

When you're done defining your batch file, go back to the main build configuration steps.

Back to top

Run tests on multiple environments in parallel

Configure Jenkins to run a series of sequential web or mobile tests, on multiple environments in parallel.

When configured, each test is run sequentially. While running each test, multiple environments are tested in parallel.

Do the following:

  1. In the Execute Micro Focus tests from file system build step, select the UFT One parallel running mode option.

  2. Define one or more environments for each execution set of tests using the UI configuration dialogs.

Note: To run tests on mobile devices, you must have a UFT Mobile connection configured.

For details, see Integrate mobile tests and Jenkins.

Define your parallel tests

In the Tests box, enter a test with its full absolute path, or a folder or MTB containing one or more tests.

  • To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.

  • Alternatively, indicate a batch file that contains a list of tests and parameters. For details, see Use a batch file to specify multiple tests.

Define your parallel environments

Do the following:

  1. Select Mobile or Web.

  2. Click Environment wizard to select a browser or device.

  3. Click + Environment to add a new environment for your execution set. UFT One supports up to four parallel test runs.

    If you configure more than four environments, each subsequent environment will wait in queue.

  4. Click + EXECUTION SET to add a new set of tests to run, with their own environments defined.

Continue with your build configuration

When you're done defining your parallel test runs, go back to the main build configuration steps.

Back to top

Configure failure scenarios for your test runs

Configure the actions that UFT One performs in case your test, or specific tests in your test set, fail.

Do the following:

  1. In the Execute Micro Focus tests from file system build step, click the UFT One-Specific Settings button.

  2. Select the On failure option, and then select one of the following:

    Option Description Jenkins plugin version 6.3 or earlier
    Rerun the entire set of tests.

    Select this option to rerun all the tests run by this build job if any of the tests failed.

    Option name is Of any of the build's tests
    Rerun specific tests in the build.

    Select this option to define failure scenarios for one or more specific test only. This enables you to rerun specific failed tests in your test set without having to rerun them all. For details, see Configure failure scenarios for one or more specific tests.

    Option name is Of a specific test in the build
    Rerun only failed tests. Select this option to rerun any and all failed tests. Option is not available.
  3. Select a node to handle rerunning the test or tests.

    Note: Ensure that the node selected here matches the node configured in the General area of the build job configuration. This ensures that the test paths used for the tests run by this job are correct.

    For details, see Select execution nodes for your job.

  4. In the Reruns field, define the number of times you want to rerun the test or tests.

  5. In the Cleanup test field, define an optional test to run as a cleanup after the entire test set has been run and a failure has occurred. The test defined is run before each new test rerun, to create the correct environment for the test run.

Configure failure scenarios for one or more specific tests

If you selected the option to rerun only specific failed tests, configure the tests to rerun as follows:

  1. The list of tests displays all tests included in this build job. Select the check boxes for each of the tests you want to rerun if it fails.

  2. Do one of the following:

    Define failure scenarios per test

    For each selected test, define the number of times to rerun the test, and any cleanup test to run before rerunning.

    Copy values to all selected tests

    Copy the main failure scenario defined to all selected tests below.

    Next to the main Reruns field, click the Copy/Paste button.

    To clear all of the per-test definitions, click Clear.

Continue with your build configuration

When you're done defining your failure scenarios, go back to the main build configuration steps.

Back to top

Review test run results

To view your UFT One test results, do the following:

  1. Click the UFT One Report icon in the left toolbar. The UFT One report page opens with a table listing the test runs (test name, status, link to the report, etc.)

  2. Click on a link to open the desired report:

    • HTML Report: The browser opens the HTML report.

      If you ran multiple tests in parallel, your report includes results from all tests run. For more details, see Sample ParallelRunner response and run results.

    • Run Results Report: The download of the zip file begins. Extract its contents. The extracted zip file contains the result files, viewable in the Run Results Viewer.

Note:  

  • If you are viewing the log for this build job, you will see a number of timeout messages displayed, including PerScenarioTimeout and FileSystemTestRunner.

    The PerScenarioTimeout can be ignored as it is only relevant for LoadRunner Professional tests.

  • You can also add a post-build step to publish a JUnit test result report. This converts the UFT One .xml results file into a JUnit report.

Back to top

Run functional test sets from ALM

Use Jenkins to run server-side, unattended functional tests from test sets in ALM. ALM test sets enable you to organize test execution in the ALM Test Lab module according to specific testing goals.

Note: Parallel testing is not supported for running tests from ALM.

For details, see the ALM Help Center.

Set up a Jenkins job to trigger a test set run from ALM

Do the following:

  1. In the Jenkins job, scroll down to the Build section, click Add build step, and select Execute functional tests from Micro Focus ALM.

  2. Select one of the configured ALM servers and enter the server credentials, project, and domain.

    Depending on the authentication type required by your ALM server, credentials can be a username and password or an API key for SSO authentication.

    To use SSO authentication, select the SSO enabled and enter the Client ID and API key secret obtained from your ALM site administrator.

    Note: If you are using the ALM scheduler, it will run under the Jenkins agent user.

    For example, if Jenkins is running as a System user, the scheduler will run the tests as a System user. This will not affect test execution.

    For more details, see Defining ALM credentials on the Jenkins server.

  3. Add the test set folders, or the specific test sets that you want to include, using the ALM path.

    • To add a specific test, add the test name after the test set path.

    • To add multiple entries, click the down arrow on the right of the field, and enter each item on a separate line.

      For example:

      Root\testfolder1\testset_a

      Root\testfolder1\testset_b

      Root\testlab_folder

      Root\testlab_folder\testset_a\test-name

  4. Optionally, add test parameters to use for the tests.

    In the Test sets area, use the following syntax:

    <test or test set path> "<parameter name>":"<string value>", "<parameter name>": <number value>

    Where:

    • <test or test set path> is the path to the test or test set.

      If you specify a test, the specified parameters are used for that test only.

      If you specify a test set, the specified parameters are used for all of the tests in that test set. When the test set runs, each test uses the parameters it requires and ignores any others.

    • <parameter name> is the name of your test parameter

    • <string value> is a string value for your parameter, in quotes

    • <number value> is a number value for your parameter, without quotes

    For example:

    TestSet\ "Country":"US", "Count": 3

    TestSet2\ "Country":"UK"

    TestFolder\Test1 "Country":"China", "Count": 4

    Tip: Specify values for all test parameters required by your tests, or make sure that the parameters have default values defined in UFT One or ALM.

  5. Optionally, indicate a timeout, in seconds, after which the job will fail.

  6. Optionally, set up a filter for your ALM test set, instructing Jenkins to run only part of the test in the test set.

    Select Filter ALM test sets, and filter the tests to run by name or status or both: 

    • In Run tests with names containing, specify a string to find within the test names.

    • In Run tests with status, specify the statuses of the tests you want to run. For example, if you don't want to rerun tests that already passed, don't select the Passed status.
  7. Click Advanced to indicate a Run mode, such as local, remote, or planned host.

    Note: If you specify a remote host mode, also specify a host name. This must be a machine with a valid UFT One installation.

    You can configure the UFT One machine so that Jenkins can run tests without anyone manually logging in to the machine and starting UFT One. For details, see Running UFT One and UFT One tests in a remote session.

  8. Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.

  9. In the Post-build Actions section, click Add post-build action, and select Publish Micro Focus test result.
  10. Run or schedule the job as you would with any standard Jenkins job.

Review the results from ALM

Do the following to view test run results from ALM:

  1. From the dashboard, click on the job.
  2. Click the Console link to view the ALM information.
  3. Copy the ALM link to your Internet Explorer browser and view the Test Set results from within ALM.

Back to top

Defining ALM credentials on the Jenkins server

This section explains where to define and use the ALM credentials that Jenkins uses to connect to ALM when running UFT One tests stored on ALM.

Jenkins plugin version ALM credentials
6.3 and earlier

Enter the credentials in each job that runs UFT One tests from ALM.

6.4 and later

Define the ALM credentials globally in the Jenkins configuration page (Manage Jenkins > Configure System, Application Lifecycle Management section).

You can define credentials for multiple ALM servers and for multiple users on the same ALM server.

Define either usernames and passwords or client IDs and API key secrets, depending on the ALM server requirements.

Then, in a job that runs UFT One tests from ALM, select from the configured ALM usernames or ALM Client IDs as necessary.

If you defined ALM credentials within your jobs, and subsequently upgraded to Jenkins plugin version 6.4 or later: 

  • You cannot edit the credentials within the jobs but Jenkins continues to use them when running the job.

  • You can automatically move ALM credentials defined in existing Jenkins jobs to the Jenkins global configuration area:

    Create and run a Jenkins job with the post-build action Migrate ALM Credentials. The first run of this job migrates the ALM credentials from all jobs to the Jenkins configuration page. Subsequent runs do nothing.

Back to top

See also: