Use Jenkins to run local tests

This topic describes how to run OpenText Functional Testing tests stored on your local machine.

In this topic:

Run local OpenText Functional Testing tests

Configure a Jenkins build step to run one or more local OpenText Functional Testing tests.

To configure a Jenkins build step to run OpenText Functional Testing tests

1. In the Project Configuration section of the Jenkins job, scroll down to the Build section, click Add build step, and select Execute OpenText tests from file system for a freestyle project.

   Note: For a pipeline, in the Pipeline Syntax page, select runFromFSBuilder or UFTScenarioLoad from Sample Step drop-down, generate the script, and copy the script to the pipeline script box.

   The UFTScenarioLoad option is obsolete and is supported for backwards compatibility only.

2. In the Tests box, enter a test with its full absolute path, a folder, or MTB containing one or more tests.

   To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.

   Tip: Alternatively, indicate a batch file that contains a list of tests, along with their parameters. For details, see Use a batch file to specify multiple tests.

3. (Optional) Specify test parameters to use in one of the following ways:

   Specify test parameters in the Tests box

   In the Tests box, use the following syntax:

   <test path or test folder path> "<parameter name>":"<string value>", "<parameter name>": <number value>

   Where:

   • <test path or test folder path> is the path to the test or test folder.

     If you specify a test, the specified parameters are used for that test only.

     If you specify a test folder, the specified parameters are used for all of the tests in that test folder. When you run tests in the test folder, each test uses the parameters it requires and ignores any others.

   • <parameter name> is the name of your test parameter

   • <string value> is a string value for your parameter, in quotes

   • <number value> is a number value for your parameter, without quotes

   For example:

   TestFolder Path\ "Country":"US", "Count": 3

   TestFolder Path\Test "Country":"China", "Count": 4

   Specify test parameters in a table

   Supported on OpenText Application Automation Tools plugin version 7.3 and later

   1. Select Specify test parameters.

   2. In the parameter table, specify the test number and the name, value, and type of a parameter.

      The test number indicates the 1-based order of tests defined in the Tests box. When you define the test number, the test path is automatically filled in read-only mode.

      Specify the parameter name as defined in OpenText Functional Testing.

   3. Click + PARAMETER to add more parameters to the table.

   Note:  

   • We recommend specifying test parameters in the table instead of specifying them in the Tests box. The first method, which lets you provide only string or number parameter values, is supported for backward compatibility only.

   • Specify values for all test parameters required by your tests, or make sure that the parameters have default values defined in OpenText Functional Testing.

   • OpenText Application Automation Tools plugin version 7.6 and later: You can define a GUI test parameter of type Password in the table. For information on how to encrypt the values of Password parameters in the test run results, see Encode passwords in test run results.

4. (Optional) Configure the following settings:

   Results directory	Specify the location in which to save the test's results. The result files are named such that they do not overwrite any existing results in that location.
   OpenText Functional Testing parallel running mode	Select to run your test on multiple environments. For more details, see Run tests on multiple environments in parallel.
   Timeout	Indicate a timeout, in seconds, after which the job will fail. Required to configure Jenkins to load OpenText Functional Testing run results for any tests that had completed for this build step, even if a test failed and prevented other tests from running.
   Run Mode	Define an OpenText Functional Testing run mode for your tests: Select OpenText Functional Testing-specific settings. From the Run Mode dropdown list, select Fast or Debug. For more details, see Test Runs Pane (Options Dialog Box > GUI Testing Tab).
   Configure failure scenarios	Configure the actions that OpenText Functional Testing performs in case your test, or a test in your test run, fails. For details, see Configure failure scenarios for your test runs.
   Output encoding (Plugin version 7.4 and later)	Select the encoding character set to use for Jenkins Console Output text. The default value is UTF-8.

   Caution: We do not recommend configuring failure scenarios together with parallel test runs. Doing so may cause OpenText Functional Testing to behave unexpectedly.

5. Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.

6. Add a post-build action to define settings for the OpenText Functional Testing test results.

   In the Post-build Actions section, click Add post-build action, and select Publish OpenText test results for the freestyle project.

   Note: For a pipeline, in the Pipeline Syntax page, select publishMicroFocusTestResults from Sample Step drop-down, generate the script, and copy the script to the pipeline script box.

   Select one of the following archiving options:

   Archive test reports for failed tests	Only save test result reports for failed tests.
   Always archive test reports	Always save test result reports.
   Do not archive test reports	Never save the test results.

7. (For a pipeline) To display results of all executed tests in case you abort the pipeline in the middle of the run, insert a try-finally block into the script in the pipeline script box. See the following example:

   Example:

                           try {
     runFromFSBuilder fsTests: 
     '''C:\\JenkinsTestsFromFS\\TestToFail''',
     uftSettingsModel:[
       cleanupTest: '', fsTestType: 'Rerun the entire set of tests', 
   numberOfReruns: '3', onCheckFailedTest: 'true'
       ] 
     } finally {
     publishMicroFocusTestResults 'ALWAYS_ARCHIVE_TEST_REPORT'
   }

8. Run or schedule the job as you would with any standard Jenkins job.

   When test is completed, continue with Review test run results.

Back to top

Use a batch file to specify multiple tests

Configure the Jenkins job to trigger multiple tests by specifying the tests in a batch file, together with their parameters.

Specify multiple tests, or even the same test several times, each time with different parameters. When specifying the path to your tests, Jenkins environment variables are supported for path names.

Give your batch file an .mtbx extension, and use the following sample syntax:

<Mtbx>
 <Test name="test1" path="c:\tests\APITest1">
 <Parameter name="A" value="abc" type="string"/>
 ….
 </Test>
 <Test name="test2" path="${WORKSPACE}\test2">
  <Parameter name="p1" value="123" type="int"/>
  <Parameter name="p4" value="123.4" type="float"/>
  ….
 </Test>
</Mtbx>

Pass parameters from one test to another in a batch test run

Supported on OpenText Application Automation Tools plugin version 7.6 and later

You can pass an output parameter from one test as an input parameter in another test.

To pass parameters from one test to another:

1. In OpenText Functional Testing, define the output parameters to use as inputs for subsequent tests in each test.

2. In each <Parameter> element in the batch file, you can leave the value empty and include a source attribute specifying the name of the output parameter whose value should be used.

   Example:

   <Test name="test2" path="c:\tests\GUITest1">
   <Parameter name="inputParam1" value="" type="string" source="outParam1"/>
   …
   </Test>

Continue with your build configuration

When you're done defining your batch file, go back to the main build configuration steps.

Back to top

Exclude parameter details from test run results

Supported on OpenText Application Automation Tools plugin version 7.6 and later

By default, test parameter details are displayed in the Console Output and Test Result page after each test run. To prevent exposing sensitive data, you can define a job-level parameter to control the build process and exclude the parameter data from the test run results.

To exclude parameter data from the run results:

1. In the General area of the build job configuration, select This project is parameterized and add the following parameter using the Add Parameter drop-down button.

   • Name: UFT_PRINT_TEST_PARAMS

   • Type: Boolean

   • Values: true (include the parameters); false (exclude the parameters)

2. Launch your build with the defined parameter. For more information, see relevant Jenkins documentation.

Back to top

Run tests on multiple environments in parallel

Configure Jenkins to run a series of web or mobile tests on multiple environments in parallel.

When configured, each test is run sequentially. While running each test, multiple environments are tested in parallel.

To run tests in parallel:

1. In the Execute OpenText tests from file system build step, select the UFT parallel running mode option.

2. In the Tests box, enter a test with its full absolute path, or a folder or MTB containing one or more tests.

   To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.

   Alternatively, indicate a batch file that contains a list of tests and parameters. For details, see Use a batch file to specify multiple tests.

3. Define one or more environments for each execution set of tests.

   1. Select Mobile or Web.

   2. Click Environment wizard to select a browser or device.

   3. Click + Environment to add a new environment for your execution set. OpenText Functional Testing supports up to four parallel test runs.

      If you configure more than four environments, each subsequent environment will wait in queue.

   4. Click + EXECUTION SET to add a new set of tests to run, with their own environments defined.

   Note: To run tests on mobile devices, you must have a Digital Lab connection configured. For details, see Integrate with Digital Lab and Jenkins.

Continue with your build configuration

When you're done defining your parallel test runs, go back to the main build configuration steps.

Back to top

Configure failure scenarios for your test runs

Configure the actions that OpenText Functional Testing performs in case your test, or specific tests in your test run, fail.

To configure failure scenarios:

1. In the Execute OpenText tests from file system build step, click the UFT-Specific Settings button.

2. Select the On failure option, and then select one of the following, depending on the version of your OpenText Application Automation Tools plugin:

   Option	Description	Plugin version 6.3 or earlier
   Rerun the entire set of tests.	Select this option to rerun all the tests run by this build job if any of the tests failed.	Option name is Of any of the build's tests
   Rerun specific tests in the build.	Select this option to define failure scenarios for one or more specific test only. This enables you to rerun specific failed tests in your test run without having to rerun them all. For details, see Configure failure scenarios for one or more specific tests.	Option name is Of a specific test in the build
   Rerun only failed tests.	Select this option to rerun any and all failed tests.	Option is not available.

3. Select a node to handle rerunning the test or tests.

   Note: Ensure that the node selected here matches the node configured in the General area of the build job configuration. This ensures that the test paths used for the tests run by this job are correct.

   For details, see Use Jenkins to run local tests.

4. In the Reruns field, define the number of times you want to rerun the test or tests.

5. In the Cleanup test field, define an optional test to run as a cleanup after the entire test run has finished and a failure has occurred. The test defined is run before each new test rerun, to create the correct environment for the test run.

Configure failure scenarios for one or more specific tests

If you selected the option to rerun only specific failed tests, configure the tests to rerun as follows:

1. The list of tests displays all tests included in this build job. Select the checkboxes for each of the tests you want to rerun if it fails.

2. Do one of the following:

   Define failure scenarios per test	For each selected test, define the number of times to rerun the test, and any cleanup test to run before rerunning.
   Copy values to all selected tests	Copy the main failure scenario defined to all selected tests below. Next to the main Reruns field, click the Copy/Paste button. To clear all of the per-test definitions, click Clear.

Continue with your build configuration

When you're done defining your failure scenarios, go back to the main build configuration steps.

Back to top

Run tests as a specified Windows user

Supported on OpenText Application Automation Tools plugin version 7.7 and later, and relevant for GUI, API, and parallel testing

You can run OpenText Functional Testing tests as a Windows user that is different from the currently logged-in Windows user. To do that, perform the following steps:

To run tests as a specified Windows user

1. In the General area of the build job configuration, select This project is parameterized and add the following parameters using the Add Parameter drop-down button.

   Parameter	Type	Description
   UFT_RUN_AS_USER_NAME	String	The username for running the Jenkins job. Note: If the authentication fails, specify the value in the UPN format username@DNS_domain_name.
   UFT_RUN_AS_USER_ENCODED_PASSWORD	Password	The encoded password for running the Jenkins job. Encode the password with the Password Encoder tool installed with OpenText Functional Testing.
   UFT_RUN_AS_USER_PASSWORD	Password	The real password for running the Jenkins job. Add either UFT_RUN_AS_USER_ENCODED_PASSWORD or this parameter.

2. Launch your build with the defined parameters.

Back to top

Review test run results

To view your OpenText Functional Testing test results:

1. Click the UFT Report icon in the left toolbar. The UFT Report page opens with a table listing the test runs (test name, timestamp, status, link to the report)

   Note: You can specify the timestamp format for each test run on the Jenkins configuration page (Manage Jenkins > Configure System, Run Result (UFT Report) timestamp format string section).

2. Click on a link to open the desired report:

   • HTML Report: The browser opens the HTML report.

     If you ran multiple tests in parallel, your report includes results from all tests run. For more details, see Sample ParallelRunner response and run results.

     Issue: The HTML report does not open.

     Possible cause: Starting with Jenkins version 1.641 (or 1.625.3), Jenkins introduced the Content-Security-Policy header. This blocks certain links, such as the links to the HTML reports.

     Workarounds:

     • View the reports without using the link, from one of the following locations:

       On the execution node, in your Jenkins results folder under the relevant build ID.

       On the controller Jenkins machine in the Jenkins builds folder. Go to the %jenkins% \jobs\<job_name>\builds\<job_count>\archive\UFTReport folder. For example, C:\Program Files (x86)\Jenkins\jobs\Uft-htmlReport\builds\80\archive\UFTReport\GuiTest2\run_results.html.

       In the Results directory that you specified in your location that you specified in your job's configuration.

     • Set the Jenkins system property to allow browser links:

       Click Manage Jenkins > Script Console. In the Script Console, enter the following line and press Run.

       System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "")

   • Run Results Report: The download of the zip file begins. Extract its contents. The extracted zip file contains the result files, viewable in the Run Results Viewer.

     View the results in the Run Results Viewer:

     1. Open the Run Results Viewer and select File > Open.
     2. In the Open dialog box, select Results XML file and navigate to the Results.xml file in the folder that you extracted.
     3. Click Open and view the results.

Back to top

Release licenses when a job is aborted

When you abort a job before it completes the test run, OpenText Functional Testing licenses may remain in use.

To check whether the license was released:

1. On AutoPass License Server, click the Usage Reports menu.

2. Under UFT Enterprise Concurrent User, click the History tab.

3. In the table on the bottom of the page, check the CLIENT INFO column and find the record about your license.

   • If you can find the license record there, your license was successfully released.

   • If your license is on the In Use tab, your license remains in use.

To release a license when a run is aborted:

OpenText Application Automation Tools plugin version 7.6 and earlier:

If the OpenText Functional Testing license was not immediately released back to the license pool when the job was aborted, it will be released automatically after about 10 minutes.

OpenText Application Automation Tools plugin version 7.7 and later:

OpenText Functional Testing releases the license when a job is aborted, if the Jenkins server provides OpenText Functional Testing with enough time to do so.

The wait time is controlled by the SoftKillWaitSeconds property (default: 5s). If you find your OpenText Functional Testing license is still in use after the Jenkins job ended, do the following:

In the jenkins.xml file, add the -DSoftKillWaitSeconds=value statement before -jar. Set the property to a value larger than 5 seconds.

Example:

Back to top

See also:

• Integrate with Digital Lab and Jenkins
• Run web tests from Jenkins on cloud browsers
• Continuous integration
• Known issues: Running tests in parallel