Occasional Server Bogging

Classic Servoy
1

Hi Servoy Crew.

We have a SaaS server in place and occasionally, in the last 2 months, the server has crawled to almost a standstill… if I check the app server, I see Java taking up most of the CPU and the CPU is running at 100%

If I restart the app server, things go back to normal.

We have an app server and a separate database server running on the Amazon Cloud. Both are running Linux CentOS 5.

I’m wondering how we might be able to better determine what might be happening or what are some things we can do to limit this occurrence.

Sometimes I wonder if our server is configured optimally. Other times, I wonder if we have some sort of bug in our code that is causing it.

Also, what experiences have any of you had with updating solutions while the server has clients in session. Does this open the door to issues?

Any ideas or information would be greatly appreciated.

Current stack and system info is below:

Current time: Mon Feb 22 16:30:38 CST 2010

Server Information
Servoy version 4.1.5 -build 687
Repository version 35

Uptime:  13 minutes 39 seconds

JDK Information
java.vm.name=Java HotSpot(TM) Client VM
java.version=1.6.0_02-b05
java.vm.info=mixed mode, sharing
java.vm.vendor=Sun Microsystems Inc.

Operating System Information
os.name=Linux
os.version=2.6.16-xenU
os.arch=i386

System Information
Heap memory: allocated=119980K, used=79501K, max=260160K
None Heap memory: allocated=63616K, used=56092K, max=184320K

Thread: RMI TCP Connection(72)-96.2.101.220, state: RUNNABLE, total cpu time: 0.0ms, total user time: 0.0ms
  java.net.SocketInputStream.socketRead0(Native Method)
  java.net.SocketInputStream.read(SocketInputStream.java:129)
  com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:113)
  com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:160)
  com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:188)
  com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1994)
  com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2411)
  com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2916)
  com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1631)
  com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1723)
  com.mysql.jdbc.Connection.execSQL(Connection.java:3283)
  com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1332)
  com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1458)
  sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  java.lang.reflect.Method.invoke(Method.java:597)
  com.servoy.j2db.persistence.datasource.Ze.invoke(Ze.java:5)
  $Proxy1.executeQuery(Unknown Source)
  org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:179)
  org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:179)
  org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:179)
  com.servoy.j2db.dataprocessing.SQLEngine.performSelect(SQLEngine.java:487)
  com.servoy.j2db.dataprocessing.SQLEngine.performQuery(SQLEngine.java:414)
  com.servoy.j2db.dataprocessing.SQLEngine.performCustomQuery(SQLEngine.java:362)
  sun.reflect.GeneratedMethodAccessor247.invoke(Unknown Source)
  sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  java.lang.reflect.Method.invoke(Method.java:597)
2

if this happens again (high CPU etc…) please look, if the allocated RAM == max RAM
if that happens, you are hitting the max memory: 256MB, from what I can see. 256Mb is not much, IMHO…

3

Thanks Harjo,

We’re trying some new tweaks and we’ll see how it goes. We upped our max memory and lowered our ping delay.

If it comes back… we’ll come back with more details.