This technical note identifies the symptoms you may see when the number of connections that can be made to Reflection for Secure IT running under Windows Server 2003 reaches the operating system's limit. Relevant error messages and event ids are also listed.
There are a limited number of connections that can be made to a Reflection for Secure IT server running under Windows Server 2003, and this limit may be less than you would expect. The limit is due to desktop heap allocation.
Note: This problem is resolved under Windows Server 2008.
On servers with multiple simultaneous ssh sessions, the memory available for Windows desktop heaps can become exhausted, and processes that require these resources will not be able to start. These include non-interactive processes such as services, scheduled tasks, and scripts.
The Reflection for Secure IT server starts a child process for every sftp session, scp transfer, terminal session, and exec request. Every session with one or more active child process uses desktop heap, the non-interactive Windows resource.
The following approaches can help resolve desktop heap issues:
Windows Server 2008 manages desktop heap dynamically. Testing at Attachmate confirms that desktop heap allocation is not likely to be the limiting factor in the number of connections that can be made to the Reflection for Secure IT server when running under Windows Server 2008.
To avoid exhausting the memory available for desktop heaps, you can use the "Maximum number of connections" setting (on the General pane) to limit the number of possible connections. By default “Maximum number of connections” is set to 60.
If you are running scripts using the Reflection for Secure IT ssh, sftp, or scp command line utilities, you may be able to resolve desktop heap issues by enabling connection reuse.
Connection reuse enables additional Secure Shell sessions to be added to an already established Secure Shell connection. When connection reuse is enabled, the server uses only one desktop for multiple connections from the same client provided that the host name, user name, and port are the same. Connection reuse is enabled by default for all Secure Shell connections made using the Reflection user interface, but is not enabled by default for command line connections.
To enable connection reuse with any of these command line utilities, use one of the following techniques.
ssh "-o connectionReuse=yes" myuser@myhost
sftp "-o connectionReuse=yes" myuser@myhost
It is also possible to increase your system's capacity for non-interactive desktop heaps by decreasing the size of each heap. For details, refer to the Microsoft Knowledge Base.
Your users may experience different symptoms as a result of exhausted desktop heap resources.
Users will connect, authenticate, and immediately be disconnected. Often users will report that they cannot connect to the server. The problem occurs after authentication; a client debug log will show the connection, successful authentication, successful pty request, successful shell request, and then a disconnect.
Some users will report they cannot make additional connections. For example, they may have three working sessions and not be able to establish a fourth session. Or, they may report that after disconnecting an existing connection they can reconnect. However, once reconnected they can no longer make additional connections.
There are a variety of messages that may be generated when desktop heap resource limits are encountered, depending on the client application being used to connect to the server. However, "disconnect" or "unable to connect" messages are not specific to desktop heap issues only. Looking at the server event viewer, server debug logs, and client debug logs may be necessary to identify the exact problem. It is best practice to use all three information sources for troubleshooting.
Reflection for Secure IT Client for Windows, sample error message:
Reflection Secure Shell Error
Connection closed to host.
Reflection FTP Client, sample error message:
A Secure Shell error occurred while trying to make a connection. (10054) An existing connection was forcibly closed by the remote host.
In the SSH command line utility, you may see the following:
C:\Documents and Settings\Administrator>ssh administrator@host
Connection closed to host.
Console app errorlevel set to 128.
From the Reflection for Secure IT UNIX Client and some Windows command line utilities, users may see "authentication successful" or "connection closed" and no Windows prompt displayed.
Connections to the Reflection for Secure IT server running on Windows Server 2003 may not result in any error messages; the client connection may just hang after entering a password.
In addition to the error messages above, client and server debug logs are likely to include exit status 128 or exit code 128. See the example in Review Server and Client Debug Logs.
It is important to gather a detailed description of the user's experience and the product behavior when troubleshooting desktop heap issues. To assist you with troubleshooting, you may be asked to provide a detailed problem description, debug logs, event ids, and screen shots.
If your users experience the symptoms described above and see one of the error messages displayed, use the following tools to help troubleshoot the problem.
The following EventID may display in the Event Viewer:
EventID: 244 - Failed to create a desktop due to desktop heap exhaustion
Note that the absence of this event is not conclusive since the server might not be configured correctly to record this event.
Try having several users disconnect. The same several users should be able to reconnect, but an additional user should not be able to connect.
The following server log sample suggests a desktop heap issue. Notice the user's profile is loaded and cmd.exe is being executed and at that point, the process is terminated.
000000013989 2008-01-03 17:12:36.019 4852 SK-RSSW7\ADMINISTRATOR:[Trace] .\SessionContext.cpp:SessionContext::LoadProfileIfNotYetLoaded(1732) User's profile successfully loaded
000000013990 2008-01-03 17:12:36.019 4852 SK-RSSW7\ADMINISTRATOR:[Trace] .\SessionContext.cpp:SessionContext::Impersonate(471) Starts....
000000013991 2008-01-03 17:12:36.050 4852 SK-RSSW7\ADMINISTRATOR:[Info] .\sshd.cpp:WindowsSessionServer::HandleRequest(4363) Session server 84: Executing command 'toterm cmd:"C:\WINDOWS\System32\cmd.exe"' in working directory 'C:\Documents and Settings\Administrator'.
000000013992 2008-01-03 17:12:36.128 3880 SK-RSSW7\ADMINISTRATOR:[Trace] .\sshd.cpp:WindowsServerManager::OnKeepAlive(7412) Sending keep-alive packet to client
000000013993 2008-01-03 17:12:36.144 4852 SK-RSSW7\ADMINISTRATOR:[Trace] .\SessionContext.cpp:SessionContext::SetPassword(385) Starts....
000000013994 2008-01-03 17:12:36.144 4852 Administrator:[Trace] .\sshd.cpp:WindowsSessionServer::HandleRequest(4443) End.
000000013995 2008-01-03 17:12:36.175 4852 SK-RSSW7\ADMINISTRATOR:[Trace] .\sshd.cpp:WindowsSessionServer::HandleProcessExit(4671) Session server 84: Process terminated with exit code 128, reporting exit code to remote
You can also use Microsoft’s Desktop Heap Monitor to help troubleshoot desktop heap issues on. Windows Server 2003. Download version 8.1 of the utility from Microsoft: