Planning guidelines

This topic provides guidelines on how to plan a deployment of Connect in your organization.

Roles and responsibilities

A proper installation requires network proxy and certificate configuration, and the provisioning of access credentials. These items need to be updated regularly, as certificates and credentials typically have expiration dates.

An enterprise environment requires the usage of back-office utilities such as Windows/Linux shell scripts and the scheduling of regular automated maintenance tasks, in the format of Windows Scheduled Tasks or Linux cron jobs.

Some of these automated tasks are critical for a stable operation, such a s a regular database purge. Others enable special synchronization scenarios that are otherwise not feasible, or which require customer-specific scripting.

Due to the above, smooth operation of the synchronizations require the following skills typical for administering common backend systems:

Server OS skills, such as shell scripting, CPU, monitoring and management for memory and disk space, and backup and restore processes.
Networking skills, such as proxies, certificates, and firewalls.
Knowledge of the operational parameters, deployed configurations, and add-ons of the endpoint products to be synchronized via Connect.

Use case considerations

This section describes the synchronization use cases and what you need to consider when planning a use case.

After you understand your needs, fill in a worksheet in which you specify your endpoints and fields. For an interactive worksheet in Microsoft Word format, download the Worksheet.

Types to synchronize

When planning a Connect deployment, you need to first determine your use case.

You should have answers to these questions:

Question	Description
What type of users are working in each product?	Who are the users? Are they developers, testers, product owners, or users with other roles?
What are the roles of those users?	Are they responsible for the creation or update of the entities, such as the backlog items, issues, requirements, and bugs?
What are the roles of the products?	What system is used to record and store entities, such as stories, sprints, defects, and tests?
What type of information needs to be shared between the users of both products?	Determine what information needs to be shared. This product is not intended to be a full replication tool. Its intended use is to synchronize the most relevant information to the parties involved. This means that not all types and fields will need to be synchronized. We recommend starting with a basic use case and expand it as needed.
Which users will create which items?	Determine which users will be creating items. For example, if using ALM/QC with Azure DevOps, one scenario would be for a user of Azure DevOps to create backlog items that are sent to ALM/QC as different requirement types. Then, users of ALM/QC can update the fields on those backlog items/requirements in Azure DevOps. ALM/QC users may create defects that are synced to Azure DevOps for resolution by the development team.
Which subsets of data need to be synchronized?	Generally, there is a subset of types that need to be shared between roles or products. Maybe you are synchronizing backlog items to requirements and defects to bugs, but tasks may not be relevant to the other team.
What are the permissions?	Will the users of the other product be allowed to update the field information and synchronize it back?
Are there relationships or links between types?	These relationships may be required or optional. For example: In ALM Octane, sprints can only exist within a release and cannot be moved between releases. Therefore, you cannot create a sprint in ALM Octane unless you have created or designated a release for it to exist within. This is a data model type of relationship. Relationship fields. This can apply to a planned sprint field in a user story. Although the sprint field may look like a regular field, in most products, it is a relationship or link field. Therefore you need to indicate the sprint item type in order to set the sprint field on a story. There are many features in that can help you configure this. For details, see Relationship mapping.
What is the direction for the synchronization?	The direction that you set at the type level in your connection dictates which product can create that item type. In most cases, the direction should be unidirectional. If you find that your use case calls for bi-directional creation of items for many types, it is often an indication of a process or governance problem that needs to be addressed. Bi-directional updates of certain fields such as Status, Author, and Comment, are common use cases.
Do you already have information synchronized between the two products?	A common scenario for organizations new to Connect is the move from another synchronization technology, or the manual entering of artifacts into two products. If this is the case, you can indicate that these items already exist in both products and should not be newly created. By following these steps, you can avoid duplication of data, allowing Connect to pick up where you left off with the other synchronization tool. For best results, validate the configuration in a test environment.

In summary, avoid synchronizing all fields and types. Synchronizing all fields and types does not add any value and may even have a negative impact and complicate the configuration. Clarity about the synchronization use cases and actual business needs is essential. If the use case is not clearly defined, do not set up a connection.

Fields to synchronize

After you have determined the types that need to be synchronized and understand the relationships between the types, you need to consider which fields need to be synchronized. Setting the direction on the field level indicates which product can update that field on that item. In general, bi-directional field updates are more common than bi-directional type creation.

Choosing the field types

Fields are of different data types and planning a synchronization requires knowledge of the data types. Not all fields are compatible with one another and the data type of some fields is not always easily apparent. For example:

A Date field might internally be of type Date, or of type Date/Time with the time portion not displayed, but also not set to zero. Another example is a String type with a text value resembling that of a date, but the system does not automatically treat it as a date.
Product add-ons may add additional field data types or display fields in a different manner in the product's user interface, compared with how these fields are stored internally. This is especially common with both single-selection and multi-select lists. The field will always be presented as it is stored internally by the product, and not by the modifications made to the user interface by the add-on.
Not all field data types are supported. Refer to the connector's Readme file for a list of non-supported data types.
For some products (for example Atlassian Jira), add-ons might add custom field data types with a custom storage approach. Synchronizing fields of such data types may not be possible and will often require some experimentation. Refer to the connector's Readme file and the Jira SAFe plugin.

Choosing the fields

The choice of which fields to map from one product to the other is driven by your requirements, not the tool. However, the products themselves will not allow you to map incompatible types. If you configure the connection to do so, the products will return an error. For example, if you map a String field to a Relationship field, it will fail.

The following table lists your primary considerations:

Field	Consideration
Read-only fields	You cannot map a read-only ID to a read-only ID. For example, when synchronizing ALM Octane with ServiceNow, you may want to map the item IDs from both products. Since IDs are system generated fields, Connect cannot override those fields. The same applies to other system generated fields, such as Created time and Modified time. To achieve the use case of seeing the corresponding ID from both products, create a custom field in both products as a text string fields. Then, perform a unidirectional mapping of the ID from each system to the custom ID field on the other side. A message informs you of read-only fields and that the direction of synchronization will be changed. The information is reported by the endpoint product.
Required fields	All required fields on a particular side, master or target, must be mapped when items of that type are being created. Consider the following: Default Values. Some required fields are assigned a default value if no other value is set. If you are satisfied with the endpoint product setting the default for you, you can safely ignore the required field warning message. No corresponding field. If you have a required field that does not have a default value, and there is no corresponding field in the originating product, you can use the Calculated Values feature to set it upon creation. Make sure to determine if the required field is a regular field or a relationship field, as the two require different approaches in their configurations. For details, see Define calculated values. Missing field values. In a case where Severity is a required field in your ALM/QC defects and you are synchronizing them to another product. If no defects are being created in ALM/QC, then you will receive a warning that you are missing required fields. In this case, you can safely ignore the warning for that type, since the defects are not being synchronized.
Enum fields	When mapping enum fields, always map all enums on each side within the value mapping view, even if there are duplicate mappings on one side. Ideally, create matched enum pairs on both sides. If you have duplicate values on one side, the bolded value (the lowest in the list) will be the default value when updating an item TO that product. If that field is a one-way update FROM that product, the bolded value will not be relevant.

Field

Consideration

Read-only fields

You cannot map a read-only ID to a read-only ID. For example, when synchronizing ALM Octane with ServiceNow, you may want to map the item IDs from both products. Since IDs are system generated fields, Connect cannot override those fields. The same applies to other system generated fields, such as Created time and Modified time. To achieve the use case of seeing the corresponding ID from both products, create a custom field in both products as a text string fields. Then, perform a unidirectional mapping of the ID from each system to the custom ID field on the other side.

A message informs you of read-only fields and that the direction of synchronization will be changed. The information is reported by the endpoint product.

Required fields

All required fields on a particular side, master or target, must be mapped when items of that type are being created. Consider the following:

Default Values. Some required fields are assigned a default value if no other value is set. If you are satisfied with the endpoint product setting the default for you, you can safely ignore the required field warning message.
No corresponding field. If you have a required field that does not have a default value, and there is no corresponding field in the originating product, you can use the Calculated Values feature to set it upon creation. Make sure to determine if the required field is a regular field or a relationship field, as the two require different approaches in their configurations. For details, see Define calculated values.
Missing field values. In a case where Severity is a required field in your ALM/QC defects and you are synchronizing them to another product. If no defects are being created in ALM/QC, then you will receive a warning that you are missing required fields. In this case, you can safely ignore the warning for that type, since the defects are not being synchronized.

Enum fields

When mapping enum fields, always map all enums on each side within the value mapping view, even if there are duplicate mappings on one side. Ideally, create matched enum pairs on both sides. If you have duplicate values on one side, the bolded value (the lowest in the list) will be the default value when updating an item TO that product. If that field is a one-way update FROM that product, the bolded value will not be relevant.

Workflows

Another aspect of the planning process is understanding all workflows and workflow rules in your endpoint products. Synchronizations generally cannot violate any workflow rules that the products have in place. However, there may be exceptions. Some products do not enforce the workflow when the API is being used, which is the way Connect communicates with the products. Some products allow you to have role-based privileges override the workflow restrictions. You can take advantage of this capability using the data source user account/role. In cases where the workflows are respected, you will have to account for this in your connection setup. Take time to determine the products through which the workflow the workflow can be ignored as much as possible.

Users

User fields in Connect, refer to fields in your connections that have user values. For example, if you are synchronizing a field on defects called Assigned To, it is likely synchronizing a user value. User values are different than other fields as users must exist in the product to set the field to that value. There are several things to consider when planning the synchronization of user fields:

Consideration	Description
User Representation	Different products represent users in different ways. For example, ALM/QC represents users by a username. ALM Octane can represent a user by their full name ‘Gary Bear’, a username ‘GBear’, or by their email address ‘gary.bear@mycompany.com’. Jira represents its users by their full name. Make sure you know how the products represent users if you intend to synchronize user fields.
User Maps	If you are synchronizing between ALM/QC and Jira and you want to synchronize user fields, you will need a user map for Gary Bear that shows that he is known as ‘Gary Bear’ in Jira, but as ‘GBear’ in ALM/QC. For details, see User maps and user matching and Batch utility scripts.
Field setting	Not all user fields are set the same way in all products. For example, the Assigned to field is commonly set by a user of the tool whereas Created by or Modified by are set automatically and cannot be set manually.
Impersonation	Some products allow impersonation which means that if an item was created in Product A by Gary Bear and synced with Product B, it will set Gary Bear as the creator in Product B. However, if the product does not allow impersonation, Product B will show the Created by user as the data source account.

Prepare for the deployment

Before you deploy the product, consider the following:

Consideration	Description
Credentials	The credentials include everything needed for authentication, for example, user names, passwords, tokens, and API keys. Some data sources require user names and passwords while others require tokens. Refer to the Readme for the applicable connectors. Follow these guidelines: Data source credentials must use native authentication of the system with which you are synchronizing. In other words, they cannot be SSO authenticated, as those types of authentication do not use the native API of the system itself. Use dedicated service account credentials for data sources. Use English language and locale settings.
Access type	Full read/write access is typically required. For details, refer to the Readme for your connector. Note: Different products require different setups. For example, there are specific guidelines for the use of Jira boards and screens.
Proxy and certificate information	Confirm whether the products you are connecting are reached via a proxy or require certificates. Extra steps will need to be taken to set up this configuration.
Sandbox instances and project	When trying out the product for the first time, it is a good idea to set up a separate test server to validate different items, such as data sources, connections, configurations, user mapping strategies, synchronize criteria, and calculated values. Then export the connection and import it to the production server. It is important to understand how Connect works, how it synchronizes, when it synchronizes, and the implications of doing so, before you apply your configuration to production data.
Sample projects and default values	Some connectors require sample projects, boards, or workspaces for the data source configuration. When a project is designated as a sample, it instructs it uses that project as a template for your synchronization. Through this template, synchronization discovers the types, fields, and list values that will be available for configuring a connection using this data source. When you set up a connection, the wizard shows the sample's types, fields, values, and users as options of items to synchronize. The metadata schema of the sample project must match the metadata of the project being synchronized. If you intend to synchronize with different templates, each template will require a different data source. For details, see Create a data source. For example, if you choose an Azure DevOps project that uses a CMMi workflow as your data source sample, you will see those types, fields, and values as options when setting up your connection. You should not use that data source for your connection if you also intend to synchronize projects that use a Scrum workflow, since the types, fields, and values, would be different. This case would require another data source.
Custom types and fields	Prepare this information before completing the worksheet: Document value maps for enumerated value fields Indicate which fields, are relationship fields along with the field type if known. An example of a relationship field may be the name or ID of a related item, or a link to that item. Use the mfcQueryTemplates utility to collect this information from your product. For details, see Batch utility scripts. Indicate which fields, are custom fields. Indicate which types, are custom types. Outline any custom data models, for example, a custom type being synchronized is a sub-type of another type. Outline any custom workflows or rules.

Worksheet completion

Once you have thoroughly reviewed the above guidelines, use the interactive Worksheet for each entity type you intend to synchronize, and for every unique project-to-project mapping.

Obtain the following information before completing each worksheet:

Document value maps for enumerated value fields
Indicate which fields are relationship fields along with the field type if known, for example, the name of a related item, a link to a related item, or an ID of a related item. The mfcQueryTemplates utility can help you collect this information from your product. For details, see Batch utility scripts.
Indicate which fields are custom fields.
Indicate which types are custom types.
Outline any custom data models. For example, if a custom type that is being synchronized is a sub-type of another type, indicate that in your worksheet.
Outline any custom workflows or rules.

System resource considerations

Connect is a network-processor-memory-intensive service. Each connection increases the amount of processing performed by the server. This includes reading and writing data to and from endpoint systems such as ALM/QC, ALM Octane, Jira, Azure DevOps, ServiceNow, Rally, and VersionOne. It also includes caching artifacts and metadata in the machine's memory, running algorithms to determine what has changed at an endpoint and how to present those changes to the opposite endpoint. The server must perform and process these operations in parallel depending upon the number of running connections.

The Connect user interface only allows you to run connections in parallel. Connections can be configured to have a specified time in minutes, with a minimum of 1 minute. As a result, the more connections actively running at a given time, the larger the load on the Connect instance. This load affects the memory, processors, disk operations, network bandwidth, and threads. While it is possible to separate connection iterations by a specific sleep time, it is very difficult to determine how many connections will be running in parallel at any given time.

The actual size or cost of any individual connection itself cannot be accurately predetermined. It depends upon some or all of the following factors:

the quality of the network
the number of other connections contending for the processor and threads
the number or projects being synchronized at each endpoint
the number of types being synchronized within those projects at each endpoint

Note: While connections are processed in parallel (when run from the user interface), projects and types in a connection are always processed in sequential order.
the raw number of artifacts in each syncset, for example each source project, source type, target project, and target type
the quantity of data changing at these endpoints

The threshold is 10 connections running in parallel. This is typically the saturation point for the recommended infrastructure per instance, after which you would need to run a second instance of Connect.