Data model validation

Data model validation enables you to compare responses between a virtual service and a real service.

Overview

While features such as message logging and the simulation report enable you to see the behavior of a virtual service, these features are somewhat limited if you are not sure that the virtual service is still up to date with the real service. Also, there may be situations when the client application still fails, despite having recently learned messages.

In both such cases, you may need to see not only the virtual service responses, but also the real service responses so you can compare them to see where the problems are.

Since there may be lots of messages involved, being able to filter such messages based on similarity between the virtual service and the real service responses is also useful.

Caution: Since the real service receives the same requests as the virtual service, unexpected side effects may occur. The client application may re-send data from previous responses. The chances are that such virtual service response data may leak into the real service and the real service may not handle such data properly.

Note: This feature is similar to hybrid simulation. For hybrid simulation, the simulation process may at some point stop producing the response. The original request is forwarded to the real service and its response is used instead and is sent back to client application.

For data model validation, it is the simulated response which is sent back to client application. The request is also forwarded to the real service and both responses are logged for further comparison.

Enabling logging

In order to compare responses, you must Enable message logging. You can enable or disable logging of the real service responses using the Log Real Service Responses switch.

Note:

Enabling this feature has a negative impact on virtual service performance as the messages need to be persisted locally. This may affect performance testing scenarios in particular.
Some protocols require the virtual service to be restarted as SV needs to start listening on real service endpoints. In such cases, a restart link is displayed.
Some protocols do not support this feature and for such protocols, the Log Real Service Responses switch is disabled.

Filtering messages

For details on message logging, refer to Message Logging. Messages with both real and simulated responses have a specific icon.

When viewing service logs, you can use the filter pane on the left to show only relevant messages. For our needs, the Simulation Response Quality metric is the most important.

The quality calculation is based on both headers and the payload of the message. Internally, the message format is ignored and only real values (such as the one shown in the data model) are significant.

Deciding the acceptable quality depends on the transport protocol and your particular scenario. For example, the client application may be sensitive to a particular header, so even quality close to 100% may cause a problem.

In other cases, a much lower quality percentage may still be acceptable.

Note:

It is always reasonable to identify the very first response causing problems as later responses may just be just due to the first response.
For multi-responses, the quality is calculated for each one independently. The overall quality is a weighted average of all of them. In case of different response counts, the overall quality is set to zero.

Comparing messages

Using the top drop-down menus, you can select what to see on each side so you can compare a request to a response, or a simulated response to a real response. For multi-responses, you can even compare the responses between themselves.

If you have configured your Diff Viewer, you can compare the message headers, message bodies or both using this tool. This enables you to see precise differences, which is useful in the case of complex messages.

Importing messages

For details, see Import logged messages to the Data Model.

Note: Since the SV simulator prefers responses with track positions over responses without track positions, you may need to assign a proper track position to newly imported messages.