Automate Gherkin tests

Create automated tests from your ALM Octane Gherkin tests, add them to an automation project, then run them in your CI server builds.

In this topic:

Gherkin automation process

The following components are involved in automating Gherkin tests:

Component What do I do here?
ALM Octane
  • Create Gherkin tests
  • Download test scripts to be uploaded to the IDE
  • View automated test run results and analyze them
Your IDE Add automation for the test, through a .feature file included in a JUnit Runner automation project.
CI system Run the test as a part of a build/pipeline run

There are several different flows, depending on the stage of development and your role.

Admin Flow

This is a one-time flow for admins:

Developer Customization Flow

This is a one-time flow for developers:

Single Test Automation Flow

This is a single test automation flow that can be configured for each test run:

Back to top

Prerequisites

Back to top

Create and download the Gherkin test

To work with a Gherkin test in your IDE, you must create the test.

To create and download the test:

  1. In ALM Octane, if necessary create and add scenarios to your Gherkin test. For details, see Create Gherkin tests.

  2. In the Scripts tab of the test, click the Download script button.

    The script is downloaded as a .feature file.

    Note: ALM Octane uses the Test ID tag to map the automation results. DO NOT DELETE the Test ID tag!

  3. Add the .feature file to your automation project.

Back to top

Configure octane-cucumber-jvm in your project

To generate a Cucumber-based test results file that the CI server sends to ALM Octane, add the octane-cucumber-jvm library to your automation project.

The octane-cucumber-jvm library is supported for use on Java environments only.

Back to top

Configure octane-jbehave-gherkin-reporter in your project

The JBehave Gherkin reporter enables uploading JBehave tests back into ALM Octane.

You use the reporter in your JBehave tests as described below. When running the tests, the reporter outputs files with the results.

The ALM Octane plugin for CI reads these files and uploads the results back to ALM Octane (see Install and configure the ALM Octane CI plugin on your CI server). You can then see the results in your Gherkin or BDD Scenario test in ALM Octane.

The octane-jbehave-gherkin-reporter library is supported for use on Java environments only.

To enable reporting to ALM Octane:

  1. In your project, add this dependency in your POM file:

    <dependencies>
    <dependency>
         <groupId>com.microfocus.adm.almoctane.jbehave</groupId>
         <artifactId>octane-jbehave-gherkin-reporter</artifactId>
         <version>1.0.0</version>
    </dependency>
</dependencies>

  2. Add the OctaneGherkinReporter to your running configuration as a reporter, using the withReports method of StoryReporterBuilder. For example:

    public class OctaneStoriesRunner extends JUnitStories {
    private final Class<? extends Embeddable> embeddableClass = this.getClass();
    
    @Override
    public Configuration configuration() {
        String resultDir = "gherkin-results";
        return new MostUsefulConfiguration()
            .useStoryParser(new GherkinStoryParser())
            .useStoryLoader(new LoadFromClasspath(embeddableClass))
            .useStoryReporterBuilder(
                new StoryReporterBuilder()
                    .withReporters(new OctaneGherkinReporter(embeddableClass.getClassLoader(), resultDir))
            );
    }
}

    The OctaneGherkinReporter constructor accepts two input parameters:

    • classLoader - your embedder class loader.

    • resultDir - (optional) a path to write the reports. If no path is provided, OctaneGherkinReporter will write the reports to target/jbehave/octane.

Back to top

Create a build job in your CI

In your build job configuration, add a build step that runs the test.

If required by your CI, add a post-build step to copy the test results to the job folder. For example, in Jenkins, add an ALM Octane Cucumber test reporter step from the actions section.

    Note:

  • If you add an ALM Octane Cucumber test reporter step, the CI server sends only the Gherkin test results to Octane. Test results from other automated tests are not sent as part of the build.

  • The CI Server must be configured with sufficient memory to inject large Cucumber test results. For example, for Jenkins, the minimum recommended memory is 2 GB.

The post-build action instructs the CI to copy the test results XML files from the path specified in the post-build actions to the job build folder. From this build folder, the results are copied to ALM Octane.

During the test run in the CI server, the OctaneCucumber runner generates a run results report called <test name>_OctaneGherkinResults.xml in a dedicated folder called gherkin_results.

Note: If you are using the octane-cucumber-jvm library version 12.53.19 and earlier, the file name of the results is called OctaneGherkinResults.xml.

If the post-build Cucumber test report step fails, the CI server fails the entire build.

Back to top

Add the build job to a pipeline

Ensure that your build project or build job that runs the test is included in an ALM Octane pipeline. When you run the pipeline, your CI generates the test results. The Application Automation Tools plugin reads the results file and sends the details to ALM Octane.

For details on creating and working with pipelines, see Create and configure pipelines.

Back to top

Run the pipeline

To send the results to ALM Octane, you must run the corresponding pipeline.

To run the pipeline:

  1. In ALM Octane or in the CI server, run the pipeline.

    ALM Octane links the test results to the original Gherkin tests. ALM Octane uses the Test ID tags found at the top of the .feature file and the test script in ALM Octane ALM Octane assigns the test using these criteria:

    • If the test already exists in ALM Octane and the Test ID tag is included in the .feature file, ALM Octane does the mapping by the Test ID tag.

    • If the Test ID tag is not included in the .feature file, ALM Octane does the mapping by the .feature script path.

    • If the test does not exist in ALM Octane, the system creates a new test in ALM Octane and maps the results to it.

  2. Analyze the results in ALM Octane. 

Back to top

Troubleshooting

This section contains troubleshooting suggestions for issues relating to running automated Gherkin tests.

I ran the automated test but I cannot see it in ALM Octane

  1. Check your code. Make sure that you are using right syntax: Cucumber does not always indicate syntax errors. Run your test locally.

  2. If you were able to run the test locally, check that an XML was generated.

    Check that your POM file is defined properly and includes the octane-cucumber-jvm library along with the correct version and make sure that OctaneCucumber is defined as the runner in your test class. For details, see Configure octane-cucumber-jvm in your project. If both POM file and test class have been updated properly and an XML is not generated, contact your ALM Octane support representative.

  3. If the test runs locally, check that the test ran in your CI tool. Make sure the Application Automation Tools plugin is installed and configured on the CI server. For details, see Install and configure the ALM Octane CI plugin on your CI server.

  4. Check the job's console to make sure the post build action found the XML. You can see it in the job console log:

    Micro Focus Octane Cucumber test reporter: Got result file content
    Micro Focus Octane Cucumber test reporter: Result file copied to …

  5. Check the plugin logs. They should display: 

    Successfully pushed test results of build [<build name>#<build number>]

  6. Check the server logs for errors. XSD validation errors are common.

I ran the test x times, but instead of seeing x runs in ALM Octane under a single test, I see X tests with one run each

Check that the test script in the .feature file includes the test ID (TID) generated by ALM Octane. The test ID is required for mapping automation results. If the TID is missing, download the test script and implement it again.

Back to top

Next steps: