Use Jenkins to run local tests

This topic describes how to run OpenText Functional Testing tests stored on your local machine.

In this topic:

Run local tests

Configure a Jenkins build step to run one or more local OpenText Functional Testing tests.

To configure a Jenkins build step to run tests

1. In the Project Configuration section of the Jenkins job, scroll down to the Build section, click Add build step, and select Execute OpenText functional tests from file system for a freestyle project.

   Note: For a pipeline, in the Pipeline Syntax page, select runFromFSBuilder or UFTScenarioLoad from Sample Step drop-down, generate the script, and copy the script to the pipeline script box.

   The UFTScenarioLoad option is obsolete and is supported for backwards compatibility only.

2. In the Tests box, enter a test with its full absolute path, a folder, or MTB containing one or more tests.

   To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.

   Tip: Alternatively, indicate a batch file that contains a list of tests, along with their parameters. For details, see Use a batch file to specify multiple tests.

3. (Optional) Specify test parameters in the Tests box or in a table, as follows:

   Specify test parameters in the Tests box

   In the Tests box, use the following syntax:

   <test path or test folder path> "<parameter name>":"<string value>", "<parameter name>": <number value>

   Where:

   • <test path or test folder path> is the path to the test or test folder.

     If you specify a test, the specified parameters are used for that test only.

     If you specify a test folder, the specified parameters are used for all of the tests in that test folder. When you run tests in the test folder, each test uses the parameters it requires and ignores any others.

   • <parameter name> is the name of your test parameter

   • <string value> is a string value for your parameter, in quotes

   • <number value> is a number value for your parameter, without quotes

   For example:

   TestFolder Path\ "Country":"US", "Count": 3

   TestFolder Path\Test "Country":"China", "Count": 4

   Specify test parameters in a table

   1. Select Specify test parameters.

   2. In the parameter table, specify the test number and the name, value, and type of a parameter.

      The test number indicates the 1-based order of tests defined in the Tests box. When you define the test number, the test path is automatically filled in read-only mode.

      Specify the parameter name as defined in OpenText Functional Testing.

   3. Click + PARAMETER to add more parameters to the table.

   Note:  

   • We recommend specifying test parameters in the table instead of specifying them in the Tests box. The first method, which lets you provide only string or number parameter values, is supported for backward compatibility only.

   • Specify values for all test parameters required by your tests, or make sure that the parameters have default values defined in the test.

   • You can define a GUI test parameter of type Password in the table. For information on how to encrypt the values of Password parameters in the test run results, see Encode passwords in test run results.

4. (Optional) Configure the following settings:

   Option	Description
   Results directory	Specify the location in which to save the test's results. The result files are named such that they do not overwrite any existing results in that location.
   Functional Testing parallel running mode	Select to run your test on multiple environments. For more details, see Run tests on multiple environments in parallel.
   Timeout	Indicate a timeout, in seconds, after which the job will fail. Required to configure Jenkins to load run results for any tests that had completed for this build step, even if a test failed and prevented other tests from running.
   Run Mode	Define an OpenText Functional Testing run mode for your tests: Select OpenText Functional Testing-specific settings. From the Run Mode dropdown list, select Fast or Debug. For more details, see Test Runs Pane (Options Dialog Box > GUI Testing Tab).
   Export run results to PDF	Select to specify that after this job runs tests, a PDF of the run results is generated and exported to Jenkins. When you run tests from Jenkins, the Export run results to PDF setting overwrites this option in the OpenText Functional Testing Options dialog box: Automatically export run results when the run session ends. For more details exporting run results to PDF, see Run Sessions Pane (Options Dialog Box > General Tab). Note: Supported on OpenText Application Automation Tools plugin version 25.2 or later.
   Configure failure scenarios	Configure the actions to perform if your test, or a test in your test run, fails. For details, see Configure failure scenarios for your test runs.
   Output encoding	Select the encoding character set to use for Jenkins Console Output text. The default value is UTF-8.

   Caution: We do not recommend configuring failure scenarios together with parallel test runs. Doing so may cause OpenText Functional Testing to behave unexpectedly.

5. Click Apply to save your changes and continue with more build steps. Click Save when you are finished adding build steps.

6. Add a post-build action to define settings for the test results.

   In the Post-build Actions section, click Add post-build action, and select Publish OpenText test results for the freestyle project.

   Note: For a pipeline, in the Pipeline Syntax page, select publishMicroFocusTestResults from Sample Step drop-down, generate the script, and copy the script to the pipeline script box.

   Select one of the following archiving options:

   Option	Description
   Archive test reports for failed tests	Only save test result reports for failed tests.
   Always archive test reports	Always save test result reports.
   Do not archive test reports	Never save the test results.

7. (For a pipeline) To display results of all executed tests in case you abort the pipeline in the middle of the run, insert a try-finally block into the script in the pipeline script box. See the following example:

   Example:

                           try {
     runFromFSBuilder fsTests: 
     '''C:\\JenkinsTestsFromFS\\TestToFail''',
     uftSettingsModel:[
       cleanupTest: '', fsTestType: 'Rerun the entire set of tests', 
   numberOfReruns: '3', onCheckFailedTest: 'true'
       ] 
     } finally {
     publishMicroFocusTestResults 'ALWAYS_ARCHIVE_TEST_REPORT'
   }

8. Run or schedule the job as you would with any standard Jenkins job.

   When test is completed, continue with Review test run results.

Back to top

Use a batch file to specify multiple tests

Configure the Jenkins job to trigger multiple tests by specifying the tests in a batch file, together with their parameters.

Specify multiple tests, or even the same test several times, each time with different parameters. When specifying the path to your tests, Jenkins environment variables are supported for path names.

Give your batch file an .mtbx extension, and use the following sample syntax:

<Mtbx>
 <Test name="test1" path="c:\tests\APITest1">
 <Parameter name="A" value="abc" type="string"/>
 ….
 </Test>
 <Test name="test2" path="${WORKSPACE}\test2">
  <Parameter name="p1" value="123" type="int"/>
  <Parameter name="p4" value="123.4" type="float"/>
  ….
 </Test>
</Mtbx>

Pass parameters from one test to another in a batch test run

Supported on OpenText Application Automation Tools plugin version 7.6 and later

You can pass an output parameter from one test as an input parameter in another test.

To pass parameters from one test to another:

1. In OpenText Functional Testing, define the output parameters to use as inputs for subsequent tests in each test.

2. In each <Parameter> element in the batch file, you can leave the value empty and include a source attribute specifying the name of the output parameter whose value should be used.

   Example:

   <Test name="test2" path="c:\tests\GUITest1">
   <Parameter name="inputParam1" value="" type="string" source="outParam1"/>
   …
   </Test>

Continue with your build configuration

When you're done defining your batch file, go back to the main build configuration steps.

Back to top

Exclude parameter details from test run results

By default, test parameter details are displayed in the Console Output and Test Result page after each test run. To prevent exposing sensitive data, you can define a job-level parameter to control the build process and exclude the parameter data from the test run results.

To exclude parameter data from the run results:

1. In the General area of the build job configuration, select This project is parameterized and add the following parameter using the Add Parameter drop-down button.

   • Name: UFT_PRINT_TEST_PARAMS

   • Type: Boolean

   • Values: true (include the parameters); false (exclude the parameters)

2. Launch your build with the defined parameter. For more information, see relevant Jenkins documentation.

Back to top

Run tests on multiple environments in parallel

Configure Jenkins to run a series of web or mobile tests on multiple environments in parallel.

When configured, each test is run sequentially. While running each test, multiple environments are tested in parallel.

To run tests in parallel:

1. In the Execute OpenText functional tests from file system build step, select the Functional Testing parallel running mode option.

2. In the Tests box, enter a test with its full absolute path, or a folder or MTB containing one or more tests.

   To specify multiple entries, click the down arrow on the right of the field and enter each test path on a separate line. Make sure that the paths are accessible from all machines in the local network.

   Alternatively, indicate a batch file that contains a list of tests and parameters. For details, see Use a batch file to specify multiple tests.

3. Define one or more environments for each execution set of tests.

   1. Select Mobile or Web.

   2. Click Environment wizard to select a browser or device.

   3. Click + Environment to add a new environment for your execution set. OpenText Functional Testing supports up to four parallel test runs.

      If you configure more than four environments, each subsequent environment will wait in queue.

   4. Click + EXECUTION SET to add a new set of tests to run, with their own environments defined.

   Note: To run tests on mobile devices, you must have a connection to OpenText Functional Testing Lab configured. For details, see Integrate with OpenText Functional Testing Lab and Jenkins.

Continue with your build configuration

When you're done defining your parallel test runs, go back to the main build configuration steps.

Back to top

Configure failure scenarios for your test runs

Configure the actions to perform in case your test, or specific tests in your test run, fail.

To configure failure scenarios:

1. In the Execute OpenText functional tests from file system build step, click OpenText™ Functional Testing-Specific Settings.

2. Select the On failure option, and then select one of the following, depending on the version of your OpenText Application Automation Tools plugin:

   Option	Description	Plugin version 6.3 or earlier
   Rerun the entire set of tests.	Select this option to rerun all the tests run by this build job if any of the tests failed.	Option name is Of any of the build's tests
   Rerun specific tests in the build.	Select this option to define failure scenarios for one or more specific test only. This enables you to rerun specific failed tests in your test run without having to rerun them all. For details, see Configure failure scenarios for one or more specific tests.	Option name is Of a specific test in the build
   Rerun only failed tests.	Select this option to rerun any and all failed tests.	Option is not available.

3. Select a node to handle rerunning the test or tests.

   Note: Ensure that the node selected here matches the node configured in the General area of the build job configuration. This ensures that the test paths used for the tests run by this job are correct.

   For details, see Use Jenkins to run local tests.

4. In the Reruns field, define the number of times you want to rerun the test or tests.

5. In the Cleanup test field, define an optional test to run as a cleanup after the entire test run has finished and a failure has occurred. The test defined is run before each new test rerun, to create the correct environment for the test run.

Configure failure scenarios for one or more specific tests

If you selected the option to rerun only specific failed tests, configure the tests to rerun as follows:

1. The list of tests displays all tests included in this build job. Select the checkboxes for each of the tests you want to rerun if it fails.

2. Do one of the following:

   Option	Description
   Define failure scenarios per test	For each selected test, define the number of times to rerun the test, and any cleanup test to run before rerunning.
   Copy values to all selected tests	Copy the main failure scenario defined to all selected tests below. Next to the main Reruns field, click the Copy/Paste button. To clear all of the per-test definitions, click Clear.

Continue with your build configuration

When you're done defining your failure scenarios, go back to the main build configuration steps.

Back to top

Run tests as a specified Windows user

Relevant for GUI, API, and parallel testing.

You can run tests as a Windows user that is different from the currently logged-in Windows user. To do that, perform the following steps:

To run tests as a specified Windows user

1. In the General area of the build job configuration, select This project is parameterized and add the following parameters using the Add Parameter drop-down button.

   Parameter	Type	Description
   UFT_RUN_AS_USER_NAME	String	The username for running the Jenkins job. Note: If the authentication fails, specify the value in the UPN format username@DNS_domain_name.
   UFT_RUN_AS_USER_ENCODED_PASSWORD	Password	The encoded password for running the Jenkins job. Encode the password with the Password Encoder tool installed with OpenText Functional Testing.
   UFT_RUN_AS_USER_PASSWORD	Password	The real password for running the Jenkins job. Add either UFT_RUN_AS_USER_ENCODED_PASSWORD or this parameter.

2. Launch your build with the defined parameters.

Run tests using an open instance of OpenText Functional Testing

Supported on OpenText Application Automation Tools plugin version 25.2 or later.

To run tests using the open OpenText Functional Testing visible instance, if it exists, instead of opening a new instance, perform the following steps:

1. In the General area of the build job configuration, select This project is parameterized and use the Add Parameter list to add a Boolean parameter named LEAVE_UFT_OPEN_IF_VISIBLE.

2. When launching your build, enable the defined parameter by selecting the checkbox. OpenText Functional Testing remains open at the end of the run as well.

   Note: If you start OpenText Functional Testing manually before running the pipelines, it is best to pre-select all the add-ins required for this run.

Back to top

Review test run results

To view your OpenText Functional Testing test results:

1. Click the Functional Testing Report icon in the left toolbar. The Functional Testing Report page opens with a table listing the test runs (test name, timestamp, status, links to the report folder and archive (zip file))

   Note: You can specify the timestamp format for each test run on the Jenkins configuration page (Manage Jenkins > System, Run Result (OpenText Functional Testing Report) timestamp format string section).

2. Click Open to open the folder containing the reports, or click Download to download a zip file with this folder's content.

   Select the report you want to open.

   Report type	Description
   HTML Report	Open the run_results.html file from the report folder, or extract it from the downloaded archive. If you ran multiple tests in parallel, your report includes results from all tests run. For more details, see Sample ParallelRunner response and run results. The Content-Security-Policy header in Jenkins might block the links to HTML reports. Workarounds: View the reports without using the link, from one of the following locations: On the execution node, in your Jenkins results folder under the relevant build ID. On the controller Jenkins machine in the Jenkins builds folder. Go to the %jenkins% \jobs\<job_name>\builds\<job_count>\archive\UFTReport folder. For example, C:\Program Files (x86)\Jenkins\jobs\Uft-htmlReport\builds\80\archive\UFTReport\GuiTest2\run_results.html. In the Results directory that you specified in your location that you specified in your job's configuration. Set the Jenkins system property to allow browser links: Click Manage Jenkins > Script Console. In the Script Console, enter the following line and press Run. System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "")
   Run Results Report	Extract the run_results.xml file from the downloaded archive and view it in the Run Results Viewer. To view the results in the Run Results Viewer: Open the Run Results Viewer and select File > Open. In the Open dialog box, select Results XML file and navigate to the Results.xml file in the folder that you extracted. Click Open and view the results.
   PDF Report	Open the run_results.pdf file from the report folder, or extract it from the downloaded archive. Tip: To configure your runs to generate PDF reports, see Export run results to PDF

Back to top

Release licenses when a job is aborted

When you abort a job before it completes the test run, OpenText Functional Testing licenses may remain in use.

To check whether the license was released:

1. On AutoPass License Server, click the Usage Reports menu.

2. Under UFT Enterprise Concurrent User, click the History tab.

3. In the table on the bottom of the page, check the CLIENT INFO column and find the record about your license.

   • If you can find the license record there, your license was successfully released.

   • If your license is on the In Use tab, your license remains in use.

To release a license when a run is aborted:

OpenText Functional Testing releases the license when a job is aborted, if the Jenkins server provides OpenText Functional Testing with enough time to do so.

The wait time is controlled by the SoftKillWaitSeconds property (default: 5s). If you find your OpenText Functional Testing license is still in use after the Jenkins job ended, do the following:

In the jenkins.xml file, add the -DSoftKillWaitSeconds=value statement before -jar. Set the property to a value larger than 5 seconds.

Example:

Back to top

See also:

• Integrate with OpenText Functional Testing Lab and Jenkins
• Run web tests from Jenkins on cloud browsers
• Continuous integration servers
• Known issues: Running tests in parallel