I’ve spent a lot of time to debug and find a solution for this issue so I’m posting this warning to save others some time.
If you deploy your solutions on a Linux server and you experience many network related errors like the following:
Signalling channel lost, removing ports: [3433] java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:723)
...
SocketAcceptor failure for socket: 205c81[SSL_NULL_WITH_NULL_NULL: Socket[addr=/10.10.15.50,port=3371,localport=1099]] java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:723)
...
java.rmi.ConnectIOException: Exception creating connection to: 10.10.15.77; nested exception is: java.io.InterruptedIOException: SocketPool is closed Signalling channel lost when reading pings or client export notifies, removing ports: [3389] java.net.SocketException: No route to host
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
...
and some other variants that I don’t have handy at this time: check the level of entropy of your server!
Entropy is the randomness collected by an operating system or application for use in cryptography or other uses that require random data.
Servoy needs a good amount of entropy because it’s used to generate UUIDs, UUIDs are heavily used by Servoy in many situations and are especially used to uniquely identify client sessions (thus the connection resets); Servoy relies on /dev/random to get the needed random numbers and on some linux distribution when entropy is not available you run into problems because Servoy cannot generate UUIDs.
Unfortunately /dev/random uses keyboard input, mouse input and IDE interrupts to generate high quality random numbers, this is fine for a desktop or laptop machine but not a production server cause usually there is no keyboard or mouse attached and good servers use SCSI disks, not IDE.
To check the entropy level use this:
cat /proc/sys/kernel/random/entropy_avail
If the number you get is less than 100-200 you could run into problems.
Fortunately the solution is pretty easy: use /dev/urandom instead of /dev/random, this one doesn’t stop if there is no available entropy because it generates pseudo-random numbers (low quality randomness).
Don’t worry about security cause Java relies on it only to generate UUIDs and not for encryption or other security reasons.
To use /dev/urandom instead of /dev/random edit your servoy_server.sh file and add “-Djava.security.egd=file:/dev/urandom” at the beginning of the file like this:
#!/bin/sh
while true
do
java -Djava.security.egd=file:/dev/urandom -Djava.awt.headless=true -Xmx256m -Xms64m ... ...
Keep in mind that is very very difficult to diagnose entropy problems because when you connect to your server using the console or SSH to investigate the issue all your keyboard input is used to generate entropy and so your investigation mitigates the issue and sometimes prevents you from understanding the real cause and you end up blaming the network like I did.
So, if you or your customer are experiencing poor Servoy performance, strange network problems with smart clients, if you see the above errors in the log or if you see that Servoy server is eating too much cpu cycles (not 100% sure if the last one is related but it happened in my case) beside checking the network for badly configured NAT routers and switches and beside lowering the ping timeout setting in the admin pages also check the entropy.
I don’t know exactly in which distributions all of this happens but I know for sure that it happens in Fedora and I suspect that it also happens in Suse cause I had problems with that as well long time ago. Ubuntu (and so Debian) seems to not have this issue but I’m keeping an eye open just to be sure. According to Sebster, it could also happen in FreeBSD so I would keep an eye open even with his cousin OpenBSD.
If you want to know more on entropy have a look at these links:
If this post saved your butt I expect a beer at next Servoy World