Synchronization and mapping scripts

This topic describes the batch scripts that allow you to automate your mappings and synchronizations.

In this topic:

CopySyncSets script

The CopySyncSets script allow you to automate the creation of type-specific project pairs for mapped types. Use this script to create a copy of the syncsets spanning the mapped types and introduce new project pairs. This script helps you avoid mapping errors that are common with manual mapping.

Before you run this script:

1. Create a property bag called copysyncsets.property.bag.txt in the usermaps folder.

2. Make sure you have a connection with at least one project mapped pair. The mapped pair may include several types, where each type has its own properties, such as calculated values, sync criteria, and server filters.

3. Stop the Connect service. If you do not stop the connection, it will be automatically suspended during the running of the script.

To copy the syncsets, use this script in one of the following modes:

• Multiple targets to a single source project or workspace.

• Multiple source projects to a single target project or workspace.

• Multiple source projects mapped to multiple target projects.

Multiple targets to a single source

This mode copies multiple target projects to a single source project/workspace. For this mode, add the following keys to the property bag:

Multiple source projects to a single target

This mode copies multiple source projects to a single target project/workspace. For this mode, add the following keys to the property bag:

Multiple source projects mapped to multiple target projects

This mode combines the capabilities of the two other modes, by allowing you to copy multiple source projects to multiple target projects. For this mode, add the following keys to the property bag:

Running this script creates new syncsets. In these syncsets, all the types in the connection's template projects that were selected, are mapped to the newly targeted projects within the connection. They include all of the sync criteria, calculated values, and server filters.

After running the scripts, restart the Connect service, review the mappings, and make sure that the script created them as required.

You can run this script as often as required in a sequence, for different datasets.

Back to top

DataSourceUsers and UserMaps scripts

The DataSourceUsers and UserMaps scripts can be used together to automate the process of storing mapped user representations from different systems into the database.

• DataSourceUsers: Extracts all users directly from the data sources.
• UserMaps: Pairs and imports users into the Connect database.

It is useful when the user data is accessible from a common authentication store such as Active Directory or LDAP.

To automatically create user maps, see Automatic user mapping.

Back to top

MatchByName script

Different products use different models for representing releases and sprints. For example:

• In Jira, sprints are independent of releases.
• In ALM Octane, sprints are children of releases.
• In QC/ALM, release cycles are children of releases, which are contained in release folders.

• In Azure DevOps, there is no explicit concept of a sprint. The closest are iterations, which are described as tree structures.

The different models make it impractical to synchronize release and sprint entities in one product with their equivalents in the other product.

Instead, we recommend that users manually create releases and sprints (or their equivalents) in the synchronized projects. The release and sprint names in both projects should be identical, including their letter cases.

The MatchByName scripts build tables of matching IDs for the identical releases and sprints in the synchronized projects.

With the tables of matching IDs, synchronizations can be done for field values in items, such as stories and defects that reference releases and sprints.

The following scripts are available for mapping data types between selected pairs of products:

• matchByNameAlmQcOctane

• matchByNameAzureDevOpsAlmQc

• matchByNameJiraAlmQc

• matchByNameJiraOctane

The following tables describe how the scripts map the data types between pairs of synchronized products:

When run, the script explores the Connect database as follows:

1. Discovers data sources based on the products specified in the script.
2. Discovers the connections that use the data sources.
3. Discovers the mapped source and target projects in the connections.
4. Between the source and target projects, creates tables of matching IDs for identical pairs of releases and sprints (or their equivalents).

You can choose to run the scripts according to the following schedules:

• Nightly, as part of the Purge script, so that the Connect database can catch up with any newly-added releases or sprints (or their equivalents).
• On demand by an administrator, based on advice from stakeholders, when new releases or sprints are added to any of the synchronized projects.

Back to top

CustomSync

The CustomSync script reads cross references from the Connect database, and synchronizes them between the source and target endpoints. This script allows you to perform actions not available from the user interface, such as synchronizing multi-value sets to strings, and single or multi-value lists to similar lists, without having to perform value mapping.

The utility uses a structured property bag which describes the syntax for mapping target properties to master properties. It is supported for the following endpoints: ALM/QC, ALM Octane, Jira, Azure DevOps, Rally, VersionOne, GitLab, and ServiceNow.

Use the following syntax for the CustomSync script's property bag:

Argument list

Argument	Meaning	Remarks
MasterType, TargetType	The type names	The JSON type names—not the label or display names.
TargetFieldName, MasterFieldName	The field names	The JSON field names—not the label or display names.
:SERVER	Generates Jira- specific JQL ignoring the projectkey from the Connect database syncset.	This optional property only applies to Jira endpoints.
;MASTER|;TARGET|;BOTH	The direction of synchronization: MASTER- a uni-directional synchronization from the target to master TARGET- a uni-directional synchronization from the master to target BOTH- a bi-directional synchronization.	For the BOTH option: The synchronization runs slower, since history queries need to be executed for each endpoint artifact. Only supported for ALM/QC, ALM Octane, Jira, Azure DevOps, and ServiceNow.
+connectionname, -connectionname	The connections to include (+) or exclude (-) for this script.	The terms +connectionname and -connectionname are mutually exclusive; you cannot use them in the same property bag. Use +connectionname to specify all the connections you intend to include, or -connectionname to specify all the connections you want to exclude.
{datasourcename}.defaultuser	The default user representation for user mapping for your data source.	If you do not have a user map for the connection, add this entry. For example, for a Jira source, you could enter MyDataSource.defaultuser=Tom Jones

Property list

This section lists the optional properties.

Argument	Meaning	Remarks
allusers	Include all users	When set to true, the default value, all users are considered, When set to false, only users associated with participating projects in the connections/datasources defined in the CustomSync property bag will be considered.
alwaysauthenticate	Send basic authentication credentials	When set to true, basic authentication credentials are sent with each REST request, even those that are not required by the Jira data source.
defaultlinktype	Specify a link type	If present, uses the specified linktype name when creating Jira issue links, for example, defaultlinktype=Blocks.
resturls, rawdata, and restheaders	Optional properties that generate files with REST URLs, raw data, or REST headers.	These are optional properties primarily for debugging and analysis purposes. If you use resturls, an output file is generated. In the next run, it will be archived into a .zip file and backed up in the AppData/backup folder. Note: If you have a nightly purge scheduled, it deletes all files older than 10 days from the backup folder.
roles	Use Jira roles	For Jira endpoints only: When set to true, only users with Jira roles will be considered. Roles can only be set to true if the allusers property is set to false.
sqldml	SQL query logging	When set to true, this property logs each SQL query executed against the Connect database in preparation for the synchronization.
statistics	Return statistics	When set to true, this property returns a list of REST commands, and the number of times each of these commands was executed during the synchronization. It also provides the total time spent in sending and receiving data for the command.
usewatermarks	Set a watermark	When set to true, this property instructs the synchronization to set a watermark for the last run. This helps optimize the queries in the next run, as the synchronization is only done for the projects that changed. When set to false, the default value, the next synchronization will run against all projects in the syncsets.

Follow these guidelines:

• All contents of the property bag are case-sensitive.

• The parameters of the property mappings must be in the order shown above.

• Use a sequence of MasterFieldName parameters to synchronize a single target property to multiple master properties, such as MasterFieldName1,MasterFieldName2,MasterFieldName3

• Use a sequence of TargetFieldName parameters to synchronize a single master property to multiple target properties, such as TargetFieldName1,TargetFieldName2,TargetFieldName3

• Synchronizing multiple master properties to multiple target properties, such as TargetFieldName1,TargetFieldName2=MasterFieldName1,MasterFieldName2, is not supported.

• This script synchronizes matched cross references per connection per syncset – source project, source type, target project, and target type. It does not create new artifacts at either endpoint, nor does it delete artifacts at either endpoint.

• Properties specified in the custom sync property bag must not be included in the core synchronization.

• The CustomSync script does not use watermarks. Therefore for each run, it needs to query all artifact properties, types, and projects at each endpoint.

• You can schedule the CustomSync script to run independently. If you have a scheduled purge using the Purge scripts, run this script as part of the purge, by adding it to the Purge script's .jar execution.

• We recommend contacting the Support team at MFI-DL-Connect-OnBoarding@opentext.com when initially preparing this script.

Run the script

Run the CustomSync script in one of the following ways:

• In the usermaps folder, add a shared property bag called customsync.property.bag.txt. Run the script, using the above syntax.

• In the usermaps folder, specify individual, uniquely named property bags. Invoke the CustomSync script sync by passing the property bag name as the second parameter to the .jar in the script's bat file. (The first parameter is always the path to the <Connect_installdir> folder.)

Example

The following is a sample property bag for an ALM Octane Jira synchronization, with a connection named Octane_JIRA_Demo. It synchronizes a multi-selectable Jira list to and from an ALM Octane multi-selectable custom list. It synchronizes:

• an ALM Octane defect ID to a Jira custom string

• a Jira key to an ALM Octane custom string

You can add more connections using the syntax shown above, <my_connection_name>=+connectionname

Back to top

See also:

• Batch utility scripts

• Connection scripts

• Data source scripts

• Maintenance scripts