Batch utility scripts
The batch utility scripts allow you to use the Micro Focus Connect capabilities more efficiently.
General run notes
The following notes apply to all batch scripts:
Each batch file should be run as Administrator.
The complete syntax for a batch script is mfc<script_name>[.bat|.sh], depending on your operating system. For example, to run the Purge script on Windows, use mfcPurge.bat in your syntax. For simplicity, this documentation will only refer to the script name, without the mfc prefix and without the extension.
If the script contains a placeholder value, for example, ConnectionName, you must edit it in a text editor and replace the placeholder value with your own value.
The scripts are located in the utilities folder under the Micro Focus Connect installation.
The scripts modify the Micro Focus Connect database.
For most of the scripts, you should stop the Micro Focus Connect server, run the appropriate utility, and then restart the Micro Focus Connect server. For details, see Connection scripts.
The following scripts can run without stopping the Micro Focus Connect server: DataSourceUsers, QueryTemplates, RunOneIteration, StartConnection, and Audit.
The ChangeDataSource script changes the master or target data source of an existing connection.
The original and replacement data sources must represent the same tool. Meaning, you can replace Jira for Jira and ALM Octane for ALM Octane. The two data sources must also target the same endpoint URL.
The script includes three variables: ConnectionName, OldDataSourceName, NewDataSourceName
Replace the values with the exact, case-sensitive names of the entries, as displayed in the Micro Focus Connect UI.
The ClearWaterMarks script resets the watermarks for all active connections.
Resetting watermarks forces a full synchronization by each connection on the first iteration after the server restarts.
You can add parameters to the script to achieve the following results:
Removes from the Micro Focus Connect database all cross references whose source or target value is null.
Removes all matched pair cross references, where either the source or the target value is in the Micro Focus Connect database but not in the corresponding endpoint product.
The script targets the following endpoints only: ALM Octane, ALM/QC, Jira, Azure DevOps, Version One, Rally, ServiceNow
Clears all the watermarks for all connections.
The CopyXrefs script copies the cross references from a common project connection to a matching type-specific connection. For details on common projects and type-specific projects, see Add projects and rules to a connection.
The script includes two variables: SOURCE_CONNECTION and TARGET_CONNECTION.
Replace the values with the exact, case-sensitive names of the corresponding connection names, as displayed in the Micro Focus Connect UI.
The DataSourceUsers and UserMaps scripts can be used together to automate the process of storing mapped user representations from different systems into Micro Focus Connect.
It is useful when the user data is accessible from a common authentication store such as Active Directory or LDAP.
For details, see Automatically create user maps.
The DataSourceExtracts script extracts a data dump of any type, any subset of fields, for any project across seven products: QC/ALM, ALM Octane, Jira, Azure DevOps, Rally, VersionOne and ServiceNow.
The command syntax is:
java -jar mfcDataSourceExtracts.jar "almqc" "..\pathTo\almqc.property.bag.txt"
|"almqc"||Can be one of the following: almqc, octane, jira, azdo, rally, versionone or servicenow|
|"..\pathTo\almqc.property.bag.txt"||the path to the property bag for the product described by the first parameter.|
You can run the DataSourceExtracts script on any machine that has the java jre installed, not just on the Micro Focus Connect server.
The utility receives as input a property bag that describes the connectivity and credentials for the target product.
The parameters in the property bag are a set of key-value pairs in the form of
Key and value guidelines
- All the keys must be lowercase.
- Spaces, tabs or special characters are not supported in the key.
- Do not use quotation marks for the values.
- Value strings are case sensitive, and must match the corresponding data elements in the targeted product.
Property bag structure per product
Following are structures of each property bag with the minimum required parameters:
sharedspace=Default Shared Space
Tip: You can specify clientid and clientsecret instead of the username and password
Note: Rally servers may be configured to require an apikey in addition to username/password:
project=My Connect Project
Note: versionone servers may be configured to require an access token instead of username/password:
project=<project name> (to include all projects, remove the 'project' parameter)
If your endpoint system requires proxy servers, you need to specify their keys in the property bag.
To specify proxy server keys, add the following:
For an http proxy server, add the following with the relevant values:
Only include http.proxyUser and http.proxyPassword for proxy servers that require authentication.
For an https secure proxy server, add the following with the relevant values:
Only include https.proxyUser and https.proxyPassword for proxy servers that require authentication.
For details, see Proxy setup.
Following are parameters that you can append to the command:
An array of field names. When specified, restricts the output data table to the set of specified fields. If not specified, all the fields of the type are returned. The parameter also directs the engine to optimize the query and fetch all the data in a single round trip.
Queries and writes out the list of all fields for the type specified in the property bag.
Splits the output row based on the specified characters. If not specified, the default is |
Directs the engine to append the field/property/column name to each data value in the output
Appends the rest http send/receive headers for each executed rest url command in the resturl log.
A request to the server to limit the items returned according to the query string.
Example: If you are working with an ALM/QC data source, add the following row to extract only requirement folders:
Produces a separate log of each rest url response in json format.
Directs the extractor to produce a log of the rest queries run in sequence.
Specific to Rally. Directs the engine to return child project artifacts in addition to the selected project
Specific to Rally. Directs the engine to return parent project artifacts in addition to the selected project.
Queries and writes out the list of all types supported on a server.
The Fix script cleans up any corrupt or stale data in the database that is the consequence of an upgrade from earlier versions of Micro Focus Connect.
This script typically will not be used, except immediately after an upgrade.
Different products use different models for representing releases and sprints. For example:
- In Jira, sprints are independent of releases.
- In ALM Octane, sprints are children of releases.
- In QC/ALM, release cycles are children of releases, which are contained in release folders.
In Azure DevOps, there is no explicit concept of a sprint. The closest are iterations, which are described as tree structures.
The different models make it impractical to synchronize release and sprint entities in one product with their equivalents in the other product.
Instead, we recommend that users manually create releases and sprints (or their equivalents) in the synchronized projects. The release and sprint names in both projects should be identical, including their letter cases.
The MatchByName scripts build tables of matching IDs for the identical releases and sprints in the synchronized projects.
With the tables of matching IDs, Micro Focus Connect can synchronize field values in items, such as stories and defects, that reference releases and sprints.
The following scripts are available for mapping data types between selected pairs of products:
The following tables describe how the scripts map the data types between pairs of synchronized products:
|Azure DevOps data types||QC/ALM data types|
|Jira data types||ALM Octane data types|
|Jira data types||QC/ALM data types|
When run, the script explores the Micro Focus Connect database as follows:
- Discovers data sources based on the products specified in the script.
- Discovers the connections that use the data sources.
- Discovers the mapped source and target projects in the connections.
- Between the source and target projects, creates tables of matching IDs for identical pairs of releases and sprints (or their equivalents).
You can choose to run the scripts according to the following schedules:
- Nightly, as part of the Purge script, so that the Micro Focus Connect database can catch up with any newly-added releases or sprints (or their equivalents).
- On demand by the Micro Focus Connect administrator, based on advice from stakeholders, when new releases or sprints are added to any of the synchronized projects.
The following scripts allow you to work with existing connections.
If the script contains a placeholder value, for example, ConnectionName, you must edit it in a text editor and replace the placeholder value with your own value.
Run connections in sequence—not simultaneously or in parallel.
This capability helps you from exceeding your network bandwidth, and/or rate limiting or overload on the part of the source and target endpoint systems.
|DeleteConnection||ConnectionName The name of the connection that you want to delete (case-sensitive), as displayed in Micro Focus Connect.||
Deletes the specified connection. Replace the placeholder variable, ConnectionName.
Caution: After a connection is deleted, all synchronization cross-references associated with this connection will also be removed.
Merges two connections into a single connection. You can merge two connections that have the identical master and target data sources.
The script merges the types, projects, synchronization criteria, calculated values and cross references. The new connection will be created in a disabled state.
After your verify that the resultant connection was created correctly, delete the old connection and enable the new one.
|StartAllConnections||-||Starts all connections.|
|mfcStopAllConnections||-||Stops all connections.|
|StartConnection||ConnectionName The name of the connection that you want to start (case-sensitive), as displayed in Micro Focus Connect.||
Starts the specified connection.
|StopConnection||ConnectionName The name of the connection that you want to stop (case-sensitive), as displayed in Micro Focus Connect.||
Stops the specified connection.
The Purge script performs the following actions:
- Deletes all iteration data from the Micro Focus Connect database.
- Removes old logging and audit data from the database and log files.
- Purges the dead space from the file system.
Set the script to run as a nightly, scheduled batch process, preferably at a time that causes the least inconvenience to users. If run every night, the script should require no more than 15 minutes of Micro Focus Connect server downtime. For details, see Purge files on Windows as a nightly task.
Stopping and restarting the Micro Focus Connect server clears out the process runtime memory cache.
Tip: If you would like the nightly purge to automatically generate a nightly audit report, add the Audit command to the purge script. For details, see Audit.
You can use the Purge script as a general purpose utility to perform additional tasks that you need to run on a nightly basis. These include:
Data folder backups
Automatic clearing of watermarks across all connections
Extracting users from endpoint systems and importing them into the Micro Focus Connect database user maps
Stopping all connections, then starting and running them, one connection, one iteration at a time
Stopping all connections, then starting each connection, one at a time
Running custom scripts outside the context of Micro Focus Connect
Additional purge scripts
The following purge scripts are also available:
|PurgeDisabledProjects||Removes all disabled projects across all connections from the Micro Focus Connect database.|
|PurgeDisabledTypes||Removes all disabled types across all connections from the Micro Focus Connect database.|
|PurgePreviousPasswords||Removes all previous passwords across all authenticator users from the Micro Focus Connect database.|
Purge script effects on resources
When the Purge script runs, it extracts the audits stored in the Micro Focus Connect database since the last purge run. It also produces a .zip backup of the data folder, containing the most recent snapshot since the last purge. If SMTP email is configured, it also emails a .zip attachment of the audit extraction, to the addressees that were configured.
The size of the audit file and its content are directly affected by the success or failure of the Micro Focus Connect configuration. The more errors recorded in the audits, the larger the audit file. In general, we recommend increasing your disk space beyond the minimum required size since Micro Focus Connect is a resource intensive service. For details, see Windows installation and upgrade.
The purpose of the audit file productions is to review and investigate errors and determine how to solve them. If the Micro Focus Connect synchronization errors are not relevant for your environment, you may turn off the audit file.
To turn off the auditing:
Open the mfcpurge.bat file, and rem out the line that calls audit.jar. Save the file.
Run the Purge script or add it to your automated tasks.
Set up a nightly job to delete all older audit and data .zip backups, for example seven or more days old. For details, see Purge files on Windows as a nightly task.
The Audit script generates a de-normalized flat .txt file representation of the audit UI log entries using the pipe character, |, as the column separator.
The file is given a timestamp and saved in the Connect/AppData/backup folder. The file can be imported into a spreadsheet or an SQL database for backup, querying, reporting, or general processing.
Note: The generated text file provides more details than the Audit tab. If you purge the database at regular intervals, the Audit tab will only show the data since the last purge. Older audit data is exported into a file by the Purge scripts, and saved in the backup folder.
You can produce a subset of the audit rows by specifying the connections by their name. Use the following syntax:
If you use the default and do not specify connection names, you receive the rows from all of the connections.
You can produce a subset of the audit columns by specifying the column names. Use the following syntax:
If you use the default, and do not specify column names, you get all of the columns.
The following is an alphabetical list of the available columns : action, actionvalue, associatedid, endtime, fieldname, fieldvalue, itemactionid, itemid, iterationid, starttime, stacktrace, sourceproject, sourcetype, syncguid, targetproject, and targettype.
You can produce a subset of the audit rows using the datetime=yyyy-mm-dd parameter. You can specify the data after a specific date. For example
datetime=2022-01-03 returns rows that were added after January 3, 2022.
If you use the default, and do not specify a date and time, you get all of the data.
The following examples show common usages of this utility.
This example produces the default report, including all connections with all of the column names:
"%JRE_PATH%\java.exe" -jar "%UTILITIES_PATH%\mfcAudit.jar" "%CONNECT_PATH%" "|" "%BACKUP_PATH%" “connectionnames=” “columnnames=” "datetime="
This example produces a report for the connection named ‘CONN’ with a subset of the columns:
"%JRE_PATH%\java.exe" -jar "%UTILITIES_PATH%\mfcAudit.jar" "%CONNECT_PATH%" "|" "%BACKUP_PATH%" “connectionnames=CONN” "columnnames=starttime,sourceproject,itemid,fieldname,fieldvalue,actionvalue" "datetime="
The QueryTemplates script targets the running Micro Focus Connect server.
This script does not stop or restart the server, as it does not run direct queries against the Micro Focus Connect database.
This script is a template to query types, properties, and property lists from the data sources.
It is useful when discovering case sensitive property names and/or enum names to set up synchronization criteria and calculated values.
This script is useful for debugging issues with mappings, and is often used by support engineers when working one-on-one with the customer.
Use the Rename script to rename either a connection or data source.
In the script, provide values for the following variables:
|ARTIFACT||"connection" or "datasource"|
|OLD_NAME||The current name of the connection or data source that you want to rename.|
|NEW_NAME||The new name for the connection or data source.|
Avoid using white spaces. Use underscores instead.
Avoid using Windows special characters: / \ : * < > |
When run, the script stops the Micro Focus Connect server, changes the name, and restart the server.
The ResetPassword script resets the password of the Micro Focus Connect Administrator user, replacing it with the string changeme.
This script is especially useful when a customer is locked out of Micro Focus Connect.
A subsequent login through the UI requires changeme to be the specified password, and Micro Focus Connect immediately asks the user to provide a new password.