Create and run BDD scenarios

You can create and run BDD (behavior-driven development) tests using Gherkin-syntax scenarios. You create specifications to describe the scenarios that you want to test, manage the automation effort, and track run results.

In this topic:

Create and run scenarios

Use the Backlog module to add scenarios to BDD specifications.

To create and run scenarios:

1. Create, edit, and delete scenarios inside the feature context.

   When you create a script, tags for each scenario are generated, so the script can be later identified when results are injected from the CI.

   For details, see Create BDD specifications.

2. After you save the script, its scenarios are automatically populated in the BDD Scenarios tab, and you can run them manually. For details, see Run manual and Gherkin tests.

   Note:  

   • By default, the BDD Scenarios tab has a filter applied to show Is in latest version, so that you only see scenarios that are currently relevant to your script. You can modify the filter as required.

   • If you remove a scenario from a script, its old run results are still saved. If you delete a scenario in the BDD Scenarios pane, its run results are also deleted, but the scenario is not deleted from the spec. The spec is an independent entity that can be edited as needed.

3. When a scenario is ready for automation, change its status in the Automation status field. If a spec is partially automated, this means that some of its scenarios are automated and some are manual.

4. Assign an owner who is responsible to automate the scenario.

   The owner sees the automation task in My Work, automates the test, the test runs in the CI, and results are pushed to the scenario's runs.

   For details, see Automate Gherkin tests.

5. Open the scenario's Runs tab and see results of its automated runs.

   Scenarios appear within the Tests modules in the same way as any other type of test.

Within a scenario's Details tab, you can click Go to BDD Spec and edit the script where the scenario is defined.

If you changed the spec and now are not aligned with the code in the SCM repository, the spec's Code alignment field shows that you need to align with the SCM.

You can accept or reject the external code to determine which option is active. For details, see Prepare Gherkin tests for automation.

Back to top

Automated scenario injection

You can inject your .feature file using the CI, and automatically create specs and scenarios.

Follow the flow described in Automate Gherkin tests, but note the following mandatory steps to enable the use of a BDD spec model.

To use a BDD spec model:

1. After you set up a pipeline, open the pipeline's Topology tab and access the configuration of each relevant job.

2. By default, new content is injected as a BDD specification, as defined by the BDD_INJECT_SPECS_TYPE_BY_DEFAULT space parameter. This means that a separate run is created for each scenario in the injected new feature file. You can override the parameter setting by selecting Inject new BDD content as Gherkin test/BDD Specification in a pipeline step.

   In spaces where pipelines were already created: By default, each job is set to Inject new BDD content as Gherkin test. This means that new tests are injected so that one run is created for all the scenarios in a script. To work with BDD granularity as described above, select Inject new BDD content as a BDD Specification. A separate run is created for each scenario in the injected new feature file. This setting impacts all of the jobs that are nested below the current job.

3. Within the feature file, each scenario must have a unique name. If you try to inject a scenario that does not have a name or if you have two scenarios with the same name, an alert warns you of the problem.

Back to top

Automation status on the BDD scenario

Automation status is defined on BDD scenarios as follows:

• Scenarios that are created start as Not Automated. They can be manually set as Ready for automation. After an automated run for a scenario is received, the status becomes Automated.

• Scenarios that are created as a result of run injection are created as Automated from the start.

There is also an automation status field on the BDD spec, which serves as the summary of its scenarios.

Back to top

Inject BDD test results

The bdd2octane tool enables you to inject test results from all common BDD frameworks using your CI.

You provide the tool with a path to your feature files, and a path to the report files produced by your BDD execution tool.

The tool takes the information from the JUnit XML report generated by your BDD framework, together with the feature file in Gherkin syntax, and sends the result using the Application Automation Tools plugin. The tool is configured as a build step before OpenText Core Software Delivery Platform Cucumber test reporter, and after the step that runs the tests.

You can inject BDD test results using an executable .jar file, or using a Maven plugin. This solution is open-sourced, to enable you to add custom frameworks. For details, refer to the bdd2octane tool.

The utility is extensible to any BDD framework using a JUnit report.

Back to top

Inject BDD test results from SpecFlow

You can inject BDD test results from SpecFlow, using a report generation template file. For details, see OpenText Core Software Delivery Platform BDD Automation with SpecFlow.

In this process, you add a .cshtml template file to your test solution. This template is then used by SpecFlow when it generates the text report xml.

The generated reports are injected as Gherkin automated runs. The first run of the pipeline generates a Gherkin automated run entity that is linked to the existing Gherkin test. If the Gherkin test does not exist, it is created. The following runs of the automated test are generated under the Previous Runs tab of the initial Gherkin automated run.

In the Pipeline section, the automated test results can be seen inside each run of the build in the Tests tab.

Back to top

How test results are counted

Test results are counted differently in Jenkins.

For example, suppose you have a Gherkin feature file run by Jenkins and injected as a BDD spec. This feature file has one normal scenario, another outline scenario with two examples, and one scenario tagged as excluded.

When you look at the run result in Jenkins, it shows three test runs, because the normal scenario is counted once, the outline scenario (with two examples) is counted twice, and the excluded scenario does not run and therefore is not counted.

When the same feature file is injected as a BDD specification, two tests are counted: the normal scenario is counted once, and the outline scenario with two examples is counted once. A scenario outline is treated as a single BDD scenario entity, no matter how many iterations it has.

Back to top

Generate example-level reports

You can set up Dashboard widgets to report on runs of examples in a scenario outline. This means that each example is counted separately as a run. This is available for the following widgets.

Widget	Runs reported
Summary graphs	Test run history (manual) Last test runs Backlog's last test runs
Traceability graph	Backlog's last test runs Test’s last test runs

To enable example-level reporting:

• When configuring a supported widget select the Count iterations as separate runs check box.

The status of an iteration is updated automatically based on the status you assign to the steps in that iteration when running the test. If you do not assign a status to the steps, the iteration status remains as Planned.

Version compatibility

• Runs created in versions earlier than 16.1.200 have an iteration count of 1.

• Planned runs created in versions earlier than 23.4.24 have an iteration count of 1.

Back to top

Next steps:

• Create BDD specifications
• Run manual and Gherkin tests
• Automate Gherkin tests