Troubleshooting timeslots

This section provides information for troubleshooting issues related to autostart timeslots.

Test failed to run with reserved timeslot

Problem Description

You reserved an autostart timeslot for running your Performance Test and the test failed to run.

Troubleshooting

Open the Timeslots module (calendar view) and select the timeslot you reserved. You can find your timeslot based on start time, name or ID.

Possible Cause Troubleshooting

If you did not find the timeslot that you reserved for autostart: Your timeslot may have been deleted.

Check the event log for the PC_timeslotDeleted event. The event log lists the user who deleted it and the time when it was deleted. If you cannot find the event, consult your project or lab administrator.

If you found the timeslot that you reserved for autostart: You can track the events related to the timeslot in the Event Log tab.

On the My Performance Center navigation bar, select Timeslots and click the Grid View button.

To drill down further, examine the run itself. Take the run ID and locate it in the Test Runs page. Continue with the run troubleshooting sections below.

If you found more than one timeslot similar to the one you reserved for autostart (multiple timeslots connected to the same performance test that you reserved, and consecutive in time): Your timeslot has been split, probably due to an error.

To analyze what caused the split, select the earliest timeslot—the timeslot that has exactly the start time you reserved. The performance test may have run in either one of the split parts; examine the parts and look for the run ID (manually navigate to the Test Runs page, and search for the run based on ID). If necessary, continue with the run troubleshooting sections below.

To track the timeslot splits, look for the PC_timeslotSplitInto and the PC_timeslotSplitFrom events in the original timeslot’s Event Log (on the My Performance Center navigation bar, select Timeslots and click the Grid View button).

Back to top

Run does not exist in the timeslot

Problem Description

The run was not started.

Troubleshooting

  • If the timeslot is not marked as autostart: The timeslot was not scheduled for autostart and therefore no run took place. To schedule an autostart timeslot, make sure the Automatically option is checked when reserving the timeslot.

  • If the timeslot is marked as autostart: The timeslot was scheduled for running, but it did not start. This may be due to timeslot invalidity (for example, a host allocation problem). A timeslot may become invalid due to changes in the lab configuration.

Back to top

Run does not exist and timeslot is valid

Although the timeslot is currently valid, it was probably not valid at the timeslot’s start time. Check the timeslot event log for the following messages: PC_timeslotNotAutostartedDueToInvalidity and PC_timeslotNotAutostartedDueToLackOfTime. The system stops trying to run an autostart timeslot according to the autostart options in the project settings.

This means that if the timeslot becomes valid after a certain time, the test won’t run, even though the timeslot is still open.

It’s also possible that the ALM server was down during the timeslot, and therefore the autostart did not take place.

Note: To check when the timeslot became invalid, look for the PC_timeslotValidityChangedWhenOpen event in the timeslot event log.

Back to top

Run does not exist and timeslot is not valid

In ALM, open the timeslot by double-clicking on it, and check the failure reason. A timeslot may become invalid due to the following reasons:

  • Host allocation. The system was not able to allocate all the resources that were requested by the timeslot. This can be caused by changes in host properties, host status, assignment of host to pools, or assignment of pools to a project.

    In general, the system tries to reorganize host allocation to allow a timeslot to stay valid. This reorganization is subject to timeslot and host priorities, and may not succeed in keeping the timeslot valid if there are not enough available hosts.

    Note that the allocation problem might have been resolved, but during the timeslot period there was a problem.

  • Invalid performance test. Someone changed the performance test and therefore the test became invalid. For example, there may be an error in the allocation of load generators to groups. This may also be caused by a change in the scripts connected to the test. Open the test (you may use the link to the test) and check for errors in the design page.

  • License issue. Someone has changed either the project limits or the license. Consult your administrator.

Note: To check when the timeslot became invalid, open ALM and look for the change in the validity field at the timeslot's History tab (Grid view only), or look for the PC_timeslotValidityChangedWhenOpen event in the timeslot event log.

If the host became non-operational during the timeslot, check the following:

In Performance Center Administration, open Maintenance > Hosts and locate the host in the grid. Click the Host and check the Event Logs. Track the host’s state by examining the PC_hostChangeStateSuccess events. Host status changes are denoted by PC_hostChangeStatusSuccess events.

Alternatively, in My Performance Center, select Hosts.

The Recovery service periodically attempts to recover a non-operational host. Look for events of type PC_ResourceRecoveryHostChangeStatus and PC_ResourceRecovery_FailToRecoverHost to track them.

The recovery service tries to recover each host up to ten times once the host becomes non-operational. This value can be configured from the Resource Recovery task configuration. You can manually change the host status to be operational.

Back to top

Run exists in the timeslot

Problem Description

The system tried to run the test and it did not complete running.

Troubleshooting

Track the run’s progress in the Event Log tab. The following sections describe the troubleshooting according to the run state in the Event Log tab.

Run State Troubleshooting
Running/Stopping/Creating Analysis Data

The run is currently active. Check the Run Screen by clicking the Show online button of the test from the Running column under Start > Runs.

To abort the run immediately, in the Timeslots module, click Abort Timeslot and click OK to abort the selected timeslot and free all used resources. Alternatively, click the Stop Run button in the Test Runs view.
Pending Creating Analysis Data

  • Refresh the screen by clicking the Refresh icon in the Runs module in Performance Center Administration.

  • Make sure there is an available and operational Data Processor host in your project's pool.

  • To analyze a result, a timeslot must be opened for 30 minutes with a Data Processor host. Make sure there are no scheduled timeslots that may prevent the system from reserving the 30 minutes timeslot for analysis.

  • Examine the Data Processor Queue by clicking the toolbar icon in the Runs page. Alternatively, on the My Performance Center navigation bar, select Hosts > Maintenance > Data Processor Queue. If the queue is overloaded, the run may have to wait in the queue until a Data Processor host is available.

  • Note that a run state changes to Failed Creating Analysis Data after 48 hours (configurable via general settings in the lab).
Failed Creating Analysis Data

  1. Analyze the results again.

  2. Make sure there is enough disk space on the Data Processor host. The required free disk size should be at least twice the size of the extracted content of RawResults.zip.

  3. Try manually analyzing the results. To do this, download the RawResults.zip file from the run's Results tab and open the results in the Analysis tool.

  4. If none of the above steps work, verify the system health. For details, see Maintain system health.

Before Collating Results

The run finished successfully. To view its results, first collate the run data, and then analyze it. Use the Collate and Analyze buttons in the Test Runs view.
Before Creating Analysis Data

The run finished successfully. To view its results, first analyze it. Use the Analyze button in the Test Runs view.
Finished

The run has successfully finished, and its results were collated and analyzed. You can access the HTML and SLA reports from the Results tab, which appears at the bottom of the Test Runs view.
Cancelled

The run was canceled. There were no Vuser actions and therefore there are no results to gather. Run the test again.

Run Failure

Examine the Run’s Event Log items to track the run progress. Common reasons for run failure include:

  • One of the allocated hosts – a Controller or a Load Generator – was not responding. In this case, if the host was reserved as an automatch, the system automatically tries to locate a replacement host. This issue would appear in the Event Log with the PC_initRun_ReplaceController and the PC_initRun_ReplaceLoadGenerators events. The system retries to replace the hosts up to three times. If all the hosts are unavailable or not responsive, the run will fail and the timeslot will be split. If enough hosts become available again, the split timeslot may autostart the run later, according to the autostart settings.

  • One of the scripts attached to the performance test was corrupted or deleted. In this case, the run will fail, the timeslot will split, and the system will not retry running the test. For more details about split timeslots, see Your timeslot has been split, probably due to an error.

For additional steps, see Run Failure State - Additional Steps below.
Failed Collating Results

Examine the Run Event Log (Failed Collating Results state) items to track the run progress.

For details on troubleshooting error messages, see Failed Collating Results State - Troubleshooting below.

Run Failure State - Additional Steps

  • Examine the hosts to see why they became non-operational.

  • Check the following section for troubleshooting the event log error events that start with PC_runLTFailed, PC_stopRunFailure, or PC_runLT_Critical.

    Error Message Troubleshooting

    PC_runLT_Critical: Run test failed. Reason: 'Invalid URI: The URI is empty.'

    Ask the lab administrator to reconfigure the host as follows:

    • Select the PC servers from Lab Project > Lab Settings > PC Servers module.

    • Click Reconfigure Server.
    PC_runLTFailed_CommunicationError: 'Could not connect to http://<Host address>:8731/LTOP/LoadTestingService/LoadTestingService.
    TCP error code 10061: No connection could be made because the target machine actively refused it <host ip>:8731. '

    Connect to the problematic host machine and check that the service Performance Center Load Testing Service is running. You may need to restart this service.
    PC_runLTWarning: Run test encountered the following warning: 'Error: Process "lr_bridge.exe" was not created on remote host "<hostname>".
    Reason: communication error. Make sure the Agent process or service is running on the remote machine. [MsgId: MERR-29987]'

    Connect to the problematic host machine and check that the service Performance Center Agent Service is running. You may need to restart this service.

    PC_runLTFailed_ScriptDownloading: ‘Failed to download scripts to Controller’

    1. Go to the Test Plan module and select the test’s scripts. Check if the script appears in the Preview tab.

    2. If you get an error stating that the script wasn’t uploaded correctly, delete the problematic scripts, upload them again, and fix the link to the scripts in the Design Load Test window.

    3. Try to run the test again.

    The problem can also happen in a version control enabled project:

    • Examine the scripts defined in the test you’re trying to run.

    • If a script was uploaded using VuGen to a version control enabled project, it is not immediately checked-in, so a checked-in version of it does not yet exist. This will cause tests using that script to fail. To resolve this issue, check-in the scripts.

    PC_runLTFailed_CommunicationError or PC_runLTFailedQC: 'SSO token validation failed on the host <Host name>'

    1. Go to Hosts module and select the host that failed SSO token validation.

    2. Click Maintenance Reconfigure Host and start the run again.

    3. If the same message re-appears, ask your lab administrator to do the same as above for your PC server from Lab Project > Lab Settings > PC Servers. Click Reconfigure Server for each of the PC servers.

    PC_stopRunFailure_CommunicationError: Failed to stop test. Reason: 'There was no endpoint listening at net.pipe://localhost/CntrlService/Service that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. The pipe endpoint 'net.pipe://localhost/CntrlService/Service' could not be found on your local machine. '

    The Controller machine was down when a user or the system issued a request for the run to stop.

    Run Recover Results and continue with collate and analyze. If there are no problems with the result files, the analysis report will be generated. There may be some missing data for the time period when the controller was down until the performance test was stopped.

    PC_runLTFailed_LaunchCntrl: Run test failed. Reason: Failed to launch Controller.

    1. Go to Hosts module and select the controller that failed.

    2. Click Maintenance > Reconfigure Host and start the run again.

    3. If the test fails to run again with the same message, open a Remote Desktop Connection to the Controller machine and try to run the Controller.

    4. If a message appears about "License security violation" do the following on the controller machine:

      • Run PC_Upgrade.exe from the <Performance Center Host installation dir>\bin directory.

      • Go to Hosts module and select the controller that failed.

      • Click Maintenance > Reconfigure Host.

      • Start the run again.

    Run test failed. Reason: Invalid run logic for script [xxx].

    Script validation failed due to an inconsistency between runtime settings run logic and actual script actions. Open the script in VuGen and fix the inconsistency.

  • If none of the above steps work, verify the system health. For details, see Maintain system health.

Failed Collating Results State - Troubleshooting

Error Message Troubleshooting

PC_collateResultsError: Failed to collate results. Reason: 'Failed to collate results files from the following load generators: <load generator name>'

  • Check that the service Performance Center Agent Service is running on the problematic Load Generators. You may need to restart this service.

  • Try to collate results again by clicking Collate Results in the Test Runs view.

PC_collateResultsError: Failed to collate results. Reason: 'LTOP service failure, unable to recover collate'

Click Collate Results again since it seems that Performance Center Load Testing Service was restarted.

PC_collateResultsError: Failed to collate results. Reason: 'Failed to get collate information'

 Go to Hosts and check if the Controller is there. If it doesn’t exist in the list, then the original host was deleted. In this case, you won’t be able to collate the results. Contact your administrator.

PC_collateResultsError: Result collation critical error. Reason: 'There was an error reading from the pipe: The pipe has been ended. (109, 0x6d).'

Run Collate Results again.

PC_collate_NDMHostNotFound: Network delay monitor source host '<Network delay monitor source machine name>' was not found in hosts list, and will be skipped in collate process.

The source machine in the Network Delay Monitor should appear in the Hosts module. If the source machine was removed from the server or is not added as a host, the collate process will not collect the NDM data from that machine.

The final Analysis report will not contain the NDM data.

If none of the Event log error messages provide a solution, verify the system health. For details, see Maintain system health.

Back to top