Troubleshooting and Limitations for RDP

This section describes troubleshooting information for RDP scripts using the Agent for Microsoft Terminal Server.

Tip: For general VuGen troubleshooting and limitations, see Troubleshooting and Limitations for VuGen.

  • Clipboard sharing supports only short simple textual content.

  • When recording with RDP Agent, applications which were developed using CBuilder may not record properly.

  • RDP does not support 32-bit color depth. If recording uses this color depth, VuGen automatically switches to a lower color depth and a "[RDP Analyzer Warning (790: 418)] 32-bit color depth is not supported, switch to lower one". warning log item appears in the Recording Window.

  • For Windows 8, we recommend using the Windows key to switch between the Desktop and Startup screen. This reduces the number of generated mouse calls and simplifies debugging.

  • When working on a 64-bit Windows 8.1 machine, you must manually install VcDist_x32 for Visual Studio 2012 before installing the RDP agent.

  • Replay fails on rdp_sync_object_mouse_click/double_click steps.

    Workaround 1: Modify RDPAgentCodeGen.cfg file

    The RDPAgentCodeGen.cfg file can configure VuGen to automatically create an rdp_sync_on_image and rdp_mouse_click step the next time the script is generated for each rdp_sync_object_mouse_click/double_click steps which occur within a given window. To do this, you specify the name of the window, update a variable which counts the total number of windows for which this process occurs, and regenerate the script.

    1. Open the RDPAgentCodeGen.cfg file in the Script Directory > data folder.

    2. Open the Step Navigator and double-click the step that failed.

    3. Copy the name of the window.

    4. In the RDPAgentCodeGen.cfg file, increase the value of NumberOfTitles by 1.

    5. Add a line as follows:

      WindowTitleX=<name of window>

      where X is the new value of NumberOfTitles.

    6. Regenerate the script.

      Note: The RDPAgentCodeGen.cfg file can be used to automatically produce rdp_sync_on_image and rdp_mouse_click steps in a similar way for rdp_sync_object_mouse_click/double_click steps which are specified in different ways as well. Steps can be targeted based on the class attribute of the control. For more information, contact Software Support.

    Workaround 2: Manually Insert a New Step

    Manually insert an rdp_sync_on_image and rdp_mouse_click step for each step that fails. This method is less desirable, since steps added in this way will be lost if the script is regenerated.

  • Connecting to a Windows 10 or Windows Server 2016 server

    When using the RDP protocol to record a connection to a Windows 10 or Windows Server 2016 server, the following error may appear on the RDP client: "An authentication error occurred."

    Resolution

    Perform the following procedure on the server:

    1. Open the Local Group Policy Editor by typing "gpedit.msc" into either a Run prompt or the Start menu.
    2. In the Local Group Policy Editor, select Local Computer Policy > Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Security.

    3. Enable "Require use of specific security layer for remote (RDP) connections" and set the security layer to "SSL (TLS 1.0)." If you are testing a non-SSL connection, set the security layer to Negotiate.

  • Failed to connect to RDP server on Windows 10 server

    When using the RDP protocol to record a connection to a Windows 10 server, the following error may appear on the RDP client: "An authentication error occurred."

    Resolution

    Perform the following procedure on the server:

    1. Open the Local Group Policy Editor by typing "gpedit.msc" into either a Run prompt or the Start menu.

    2. In the Local Group Policy Editor, select Local Computer Policy > Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Security.

    3. Enable "Require use of specific security layer for remote (RDP) connections" and set the security layer to RDP.

    Note: Alternatively, open HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp in the Registry Editor, and set the Value data for SecurityLayer to 1.

Back to top