Servoy server blocks

udrescu_bogdan · April 8, 2011, 4:37pm

Hello,

Our online Servoy server blocks at least once per week. No client can be open, usually the servoy-admin page do not respond (only once it responded, today, but I couldn’t see the active clients page). I usually have to kill it using kill -9. And then start it again.
The PostgreSQL database server respond OK when Servoy is blocked. Usually I kill it as well and start it over. But today I didn’t, I restart only Servoy server and then it worked OK.

Any idea why Servoy might block?

Greetings,
Bogdan.

ngervasi · April 8, 2011, 5:10pm

Anything useful in the server.log and servoy_log.txt? Anything useful in the system log? Which Lnux/UNIX are you using?

Harjo · April 8, 2011, 5:36pm

The only time I have seen Servoy server blocking is when the server is out of memory.
so what are your memory settings for the server??

udrescu_bogdan · April 8, 2011, 10:21pm

Hello,

It blocked again. I was asleep and had to wake to restart it. I don’t know if I should be or :cry:

Thanks for the tip. I checked the memory, and it might be from there. Servoy is installed on a VPS with 1038 Mb RAM. When it’s online almost all of that memory is used. Only after first start almost 200Mb are free, then if I use the app for couple of minutes the free memory goes down to ~20Mb. If it’s not running there are ~350Mb free.
I also just increased the -Xmx256m and -Xms64m to 512m and 256m.
So, shall we get another 1Gb or RAM there, and to increase the Xmx to 1Gb?
And also we get some exceptions in the log file related to memory. But I don’t know if it refer to the disk space or to heap memory. We have few Gb free on that server so the disk space shouldn’t be a problem.

2011-04-08 16:00:24,142 ERROR [http-213.154.231.180-80-1] com.servoy.j2db.util.Debug - Throwable
com.servoy.j2db.persistence.RepositoryException: org.postgresql.util.PSQLException: ERROR: could not extend relation base/16384/16490: No space left on device
	at com.servoy.j2db.server.Zc.Za.Za(Za.java:793)
	at com.servoy.j2db.server.Ze.Zl.importRootObject(Zl.java:70)
	at com.servoy.j2db.server.Ze.Zeb.Za(Zeb.java:429)
	at com.servoy.j2db.server.Ze.Zeb.importFromJarFile(Zeb.java:459)
	at com.servoy.j2db.server.servlets.ConfigServlet.Zj(ConfigServlet.java:293)
	at com.servoy.j2db.server.servlets.ConfigServlet.service(ConfigServlet.java:138)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.postgresql.util.PSQLException: ERROR: could not extend relation base/16384/16490: No space left on device
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1525)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:354)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:308)
	at sun.reflect.GeneratedMethodAccessor214.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.servoy.j2db.Za.Za.Zf.invoke(Zf.java:15)
	at $Proxy0.executeUpdate(Unknown Source)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:105)
	at com.servoy.j2db.server.Zc.Za.Za(Za.java:1410)
	at com.servoy.j2db.server.Zc.Za.Za(Za.java:219)
	... 19 more

Besides these exceptions, we also get others that I don’t understand. They don’t get through my code, so I’m quite looking at a blank screen, so it would be nice if you can see what might be wrong. Thanks! These are some of them:

2011-03-31 09:54:20,948 ERROR [http-213.154.231.180-80-10] com.servoy.j2db.util.Debug - Throwable
java.lang.RuntimeException: Exception executing calculation: balance_amount of table contacts, error: org.mozilla.javascript.WrappedException: Wrapped java.lang.RuntimeException: com.servoy.j2db.persistence.RepositoryException: Client is not registered with the server. (getContactBalance#2700)
    at com.servoy.j2db.scripting.TableScope.getCalculationValue(TableScope.java:197)
    at com.servoy.j2db.scripting.TableScope.getCalculationValue(TableScope.java:214)
    at com.servoy.j2db.dataprocessing.FoundSet.getCalculationValue(FoundSet.java:2517)
    at com.servoy.j2db.dataprocessing.Record.getValue(Record.java:181)
    at com.servoy.j2db.dataprocessing.Record.validateStoredCalculations(Record.java:139)
    at com.servoy.j2db.dataprocessing.EditRecordList.testIfRecordIsChanged(EditRecordList.java:811)
    at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:829)
    at com.servoy.j2db.dataprocessing.EditRecordList.stopEditing(EditRecordList.java:262)
    at com.servoy.j2db.dataprocessing.Record.stopEditing(Record.java:309)
    at com.servoy.j2db.dataprocessing.JSDatabaseManager.recalculateRecord(JSDatabaseManager.java:770)
    at com.servoy.j2db.dataprocessing.JSDatabaseManager.js_recalculate(JSDatabaseManager.java:750)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
    at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
    at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
    at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
    at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
    at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
    at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
    at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
    at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
    at com.servoy.j2db.FormController.executeFunction(FormController.java:3970)
    at com.servoy.j2db.FormController.executeFunction(FormController.java:3851)
    at com.servoy.j2db.plugins.ClientPluginAccessProvider$MethodExecutor.run(ClientPluginAccessProvider.java:528)
    at com.servoy.j2db.server.headlessclient.SessionClient.invokeAndWait(SessionClient.java:1159)
    at com.servoy.j2db.plugins.ClientPluginAccessProvider.executeMethod(ClientPluginAccessProvider.java:402)
    at com.servoy.extensions.plugins.rest_ws.servlets.RestWSServlet.wsService(Unknown Source)
    at com.servoy.extensions.plugins.rest_ws.servlets.RestWSServlet.doPost(Unknown Source)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at com.servoy.j2db.server.servlets.WebServicesServlet.service(WebServicesServlet.java:37)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
    at java.lang.Thread.run(Thread.java:662) 

2011-04-06 08:00:16,940 ERROR [TaskExecuter[1]] com.servoy.j2db.dataprocessing.Zb - Error flushing message buffer to client 86B77525-E284-4EEA-8922-335C953A385A
java.rmi.ConnectIOException: Exception creating connection to: 192.168.1.6; nested exception is: 
	java.io.InterruptedIOException: The socket pool for a client 192.168.1.6:3056 did not recieve a callback socket from the client within 60000ms, pool still empty
	at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:614)
	at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
	at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
	at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:178)
	at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:132)
	at $Proxy16.isAlive(Unknown Source)
	at com.servoy.j2db.dataprocessing.ClientProxy.Zc(ClientProxy.java:143)
	at com.servoy.j2db.dataprocessing.Zo.run(Zo.java:9)
	at com.servoy.j2db.util.TaskExecuter.run(TaskExecuter.java:82)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.InterruptedIOException: The socket pool for a client 192.168.1.6:3056 did not recieve a callback socket from the client within 60000ms, pool still empty
	at com.servoy.j2db.rmi.SocketPool.getSocket(SocketPool.java:49)
	at com.servoy.j2db.rmi.ServerTwoWaySocketFactory.createSocket(ServerTwoWaySocketFactory.java:337)
	at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
	... 10 more
2011-04-06 08:16:38,875 ERROR [ClientExportNotifyListner[57]] com.servoy.j2db.util.Debug - Signalling channel lost when reading pings or client export notifies, removing ports: [3056]
java.net.SocketException: Connection timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:129)
	at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
	at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)
	at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
	at com.servoy.j2db.rmi.compressing.CompressingInputStream.readIn(CompressingInputStream.java:99)
	at com.servoy.j2db.rmi.compressing.CompressingInputStream.read(CompressingInputStream.java:85)
	at java.io.DataInputStream.readInt(DataInputStream.java:370)
	at com.servoy.j2db.rmi.ServerTwoWaySocketFactory$1.run(ServerTwoWaySocketFactory.java:106)


2011-04-08 13:16:38,789 ERROR [http-213.154.231.180-80-8] org.apache.wicket.RequestCycle - org.apache.wicket.WicketRuntimeException: After 1 minute the Pagemap null is still locked by: Thread[http-213.154.231.180-80-1,5,main], giving up trying to get the page for path: 4:1
org.apache.wicket.protocol.http.request.InvalidUrlException: org.apache.wicket.WicketRuntimeException: After 1 minute the Pagemap null is still locked by: Thread[http-213.154.231.180-80-1,5,main], giving up trying to get the page for path: 4:1
	at org.apache.wicket.protocol.http.WebRequestCycleProcessor.resolve(WebRequestCycleProcessor.java:262)
	at com.servoy.j2db.server.headlessclient.WebClientsApplication$5.resolve(WebClientsApplication.java:431)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1310)
	at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1428)
	at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
	at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:479)
	at com.servoy.j2db.server.servlets.Zl.doGet(Zl.java:5)
	at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.wicket.WicketRuntimeException: After 1 minute the Pagemap null is still locked by: Thread[http-213.154.231.180-80-1,5,main], giving up trying to get the page for path: 4:1
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at com.servoy.j2db.dataprocessing.FoundSetManager.getRelatedFoundSet(FoundSetManager.java:459)
	at com.servoy.j2db.dataprocessing.FoundSet.getRelatedFoundSet(FoundSet.java:4584)
	at com.servoy.j2db.dataprocessing.Record.getRelatedFoundSet(Record.java:658)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.showFoundSet(WebTabPanel.java:652)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.setRecord(WebTabPanel.java:605)
	at com.servoy.j2db.dataprocessing.DataAdapterList.setRecord(DataAdapterList.java:377)
	at com.servoy.j2db.server.headlessclient.dataui.WebDataRenderer.refreshRecord(WebDataRenderer.java:303)
	at com.servoy.j2db.FormController.refreshAllPartRenderers(FormController.java:2175)
	at com.servoy.j2db.FormController.valueChanged(FormController.java:2250)
	at com.servoy.j2db.FormController.tableChanged(FormController.java:2204)
	at com.servoy.j2db.dataprocessing.TableAndListEventDelegate.fireTableModelEvent(TableAndListEventDelegate.java:75)
	at com.servoy.j2db.dataprocessing.TableAndListEventDelegate$1.run(TableAndListEventDelegate.java:168)
	at com.servoy.j2db.dataprocessing.TableAndListEventDelegate.fireTableAndListEvent(TableAndListEventDelegate.java:175)
	at com.servoy.j2db.dataprocessing.SwingFoundSet.fireFoundSetEvent(SwingFoundSet.java:156)
	at com.servoy.j2db.dataprocessing.FoundSet.completeFire(FoundSet.java:3954)
	at com.servoy.j2db.dataprocessing.FireCollector.done(FireCollector.java:39)
	at com.servoy.j2db.dataprocessing.RowManager.fireRowNotifyChanges(RowManager.java:1210)
	at com.servoy.j2db.dataprocessing.RowManager.access$0(RowManager.java:1197)
	at com.servoy.j2db.dataprocessing.RowManager$1.run(RowManager.java:211)
	at com.servoy.j2db.server.headlessclient.WebClient.invokeLater(WebClient.java:512)
	at com.servoy.j2db.dataprocessing.RowManager.getRowBasedonPKFromEntireColumnArray(RowManager.java:216)
	at com.servoy.j2db.dataprocessing.RelatedFoundSet.<init>(RelatedFoundSet.java:94)
	at com.servoy.j2db.dataprocessing.SwingRelatedFoundSet.<init>(SwingRelatedFoundSet.java:44)
	at com.servoy.j2db.dataprocessing.SwingFoundSetFactory.createRelatedFoundSet(SwingFoundSetFactory.java:37)
	at com.servoy.j2db.dataprocessing.RelatedFoundSet.createRelatedFoundSets(RelatedFoundSet.java:334)
	at com.servoy.j2db.dataprocessing.FoundSetManager.getRelatedFoundSet(FoundSetManager.java:535)
	at com.servoy.j2db.dataprocessing.FoundSet.getRelatedFoundSet(FoundSet.java:4584)
	at com.servoy.j2db.dataprocessing.Record.getRelatedFoundSet(Record.java:658)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.showFoundSet(WebTabPanel.java:652)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.setCurrentForm(WebTabPanel.java:410)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.notifyVisible(WebTabPanel.java:554)
	at com.servoy.j2db.dataprocessing.DataAdapterList.notifyVisible(DataAdapterList.java:499)
	at com.servoy.j2db.server.headlessclient.dataui.WebDataRenderer.notifyVisible(WebDataRenderer.java:290)
	at com.servoy.j2db.FormController.notifyVisible(FormController.java:3076)
	at com.servoy.j2db.FormManager.showFormInMainPanel(FormManager.java:700)
	at com.servoy.j2db.server.headlessclient.WebFormManager.showFormInDialog(WebFormManager.java:274)
	at com.servoy.j2db.scripting.JSApplication.js_showFormInDialog(JSApplication.java:1855)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
	at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
	at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
	at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
	at com.servoy.j2db.FormController.executeFunction(FormController.java:3970)
	at com.servoy.j2db.FormController.executeFunction(FormController.java:3858)
	at com.servoy.j2db.FormController.executeFunction(FormController.java:3780)
	at com.servoy.j2db.FormController$ScriptExecuter.executeFunction(FormController.java:3635)
	at com.servoy.j2db.ui.BaseEventExecutor.fireEventCommand(BaseEventExecutor.java:270)
	at com.servoy.j2db.ui.BaseEventExecutor.fireDoubleclickCommand(BaseEventExecutor.java:222)
	at com.servoy.j2db.server.headlessclient.dataui.WebEventExecutor.onEvent(WebEventExecutor.java:357)
	at com.servoy.j2db.server.headlessclient.dataui.WebEventExecutor$3.onEvent(WebEventExecutor.java:222)
	at org.apache.wicket.ajax.AjaxEventBehavior.respond(AjaxEventBehavior.java:177)
	at org.apache.wicket.ajax.AbstractDefaultAjaxBehavior.onRequest(AbstractDefaultAjaxBehavior.java:286)
	at org.apache.wicket.request.target.component.listener.BehaviorRequestTarget.processEvents(BehaviorRequestTarget.java:119)
	at org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:92)
	at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1250)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
	... 20 more
2011-04-08 13:16:38,790 ERROR [http-213.154.231.180-80-8] com.servoy.j2db.util.Debug - Error rendering the page null
org.apache.wicket.protocol.http.request.InvalidUrlException: org.apache.wicket.WicketRuntimeException: After 1 minute the Pagemap null is still locked by: Thread[http-213.154.231.180-80-1,5,main], giving up trying to get the page for path: 4:1
	at org.apache.wicket.protocol.http.WebRequestCycleProcessor.resolve(WebRequestCycleProcessor.java:262)
	at com.servoy.j2db.server.headlessclient.WebClientsApplication$5.resolve(WebClientsApplication.java:431)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1310)
	at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1428)
	at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
	at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:479)
	at com.servoy.j2db.server.servlets.Zl.doGet(Zl.java:5)
	at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.wicket.WicketRuntimeException: After 1 minute the Pagemap null is still locked by: Thread[http-213.154.231.180-80-1,5,main], giving up trying to get the page for path: 4:1
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:485)
	at com.servoy.j2db.dataprocessing.FoundSetManager.getRelatedFoundSet(FoundSetManager.java:459)
	at com.servoy.j2db.dataprocessing.FoundSet.getRelatedFoundSet(FoundSet.java:4584)
	at com.servoy.j2db.dataprocessing.Record.getRelatedFoundSet(Record.java:658)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.showFoundSet(WebTabPanel.java:652)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.setRecord(WebTabPanel.java:605)
	at com.servoy.j2db.dataprocessing.DataAdapterList.setRecord(DataAdapterList.java:377)
	at com.servoy.j2db.server.headlessclient.dataui.WebDataRenderer.refreshRecord(WebDataRenderer.java:303)
	at com.servoy.j2db.FormController.refreshAllPartRenderers(FormController.java:2175)
	at com.servoy.j2db.FormController.valueChanged(FormController.java:2250)
	at com.servoy.j2db.FormController.tableChanged(FormController.java:2204)
	at com.servoy.j2db.dataprocessing.TableAndListEventDelegate.fireTableModelEvent(TableAndListEventDelegate.java:75)
	at com.servoy.j2db.dataprocessing.TableAndListEventDelegate$1.run(TableAndListEventDelegate.java:168)
	at com.servoy.j2db.dataprocessing.TableAndListEventDelegate.fireTableAndListEvent(TableAndListEventDelegate.java:175)
	at com.servoy.j2db.dataprocessing.SwingFoundSet.fireFoundSetEvent(SwingFoundSet.java:156)
	at com.servoy.j2db.dataprocessing.FoundSet.completeFire(FoundSet.java:3954)
	at com.servoy.j2db.dataprocessing.FireCollector.done(FireCollector.java:39)
	at com.servoy.j2db.dataprocessing.RowManager.fireRowNotifyChanges(RowManager.java:1210)
	at com.servoy.j2db.dataprocessing.RowManager.access$0(RowManager.java:1197)
	at com.servoy.j2db.dataprocessing.RowManager$1.run(RowManager.java:211)
	at com.servoy.j2db.server.headlessclient.WebClient.invokeLater(WebClient.java:512)
	at com.servoy.j2db.dataprocessing.RowManager.getRowBasedonPKFromEntireColumnArray(RowManager.java:216)
	at com.servoy.j2db.dataprocessing.RelatedFoundSet.<init>(RelatedFoundSet.java:94)
	at com.servoy.j2db.dataprocessing.SwingRelatedFoundSet.<init>(SwingRelatedFoundSet.java:44)
	at com.servoy.j2db.dataprocessing.SwingFoundSetFactory.createRelatedFoundSet(SwingFoundSetFactory.java:37)
	at com.servoy.j2db.dataprocessing.RelatedFoundSet.createRelatedFoundSets(RelatedFoundSet.java:334)
	at com.servoy.j2db.dataprocessing.FoundSetManager.getRelatedFoundSet(FoundSetManager.java:535)
	at com.servoy.j2db.dataprocessing.FoundSet.getRelatedFoundSet(FoundSet.java:4584)
	at com.servoy.j2db.dataprocessing.Record.getRelatedFoundSet(Record.java:658)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.showFoundSet(WebTabPanel.java:652)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.setCurrentForm(WebTabPanel.java:410)
	at com.servoy.j2db.server.headlessclient.dataui.WebTabPanel.notifyVisible(WebTabPanel.java:554)
	at com.servoy.j2db.dataprocessing.DataAdapterList.notifyVisible(DataAdapterList.java:499)
	at com.servoy.j2db.server.headlessclient.dataui.WebDataRenderer.notifyVisible(WebDataRenderer.java:290)
	at com.servoy.j2db.FormController.notifyVisible(FormController.java:3076)
	at com.servoy.j2db.FormManager.showFormInMainPanel(FormManager.java:700)
	at com.servoy.j2db.server.headlessclient.WebFormManager.showFormInDialog(WebFormManager.java:274)
	at com.servoy.j2db.scripting.JSApplication.js_showFormInDialog(JSApplication.java:1855)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
	at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
	at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
	at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
	at com.servoy.j2db.FormController.executeFunction(FormController.java:3970)
	at com.servoy.j2db.FormController.executeFunction(FormController.java:3858)
	at com.servoy.j2db.FormController.executeFunction(FormController.java:3780)
	at com.servoy.j2db.FormController$ScriptExecuter.executeFunction(FormController.java:3635)
	at com.servoy.j2db.ui.BaseEventExecutor.fireEventCommand(BaseEventExecutor.java:270)
	at com.servoy.j2db.ui.BaseEventExecutor.fireDoubleclickCommand(BaseEventExecutor.java:222)
	at com.servoy.j2db.server.headlessclient.dataui.WebEventExecutor.onEvent(WebEventExecutor.java:357)
	at com.servoy.j2db.server.headlessclient.dataui.WebEventExecutor$3.onEvent(WebEventExecutor.java:222)
	at org.apache.wicket.ajax.AjaxEventBehavior.respond(AjaxEventBehavior.java:177)
	at org.apache.wicket.ajax.AbstractDefaultAjaxBehavior.onRequest(AbstractDefaultAjaxBehavior.java:286)
	at org.apache.wicket.request.target.component.listener.BehaviorRequestTarget.processEvents(BehaviorRequestTarget.java:119)
	at org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(AbstractRequestCycleProcessor.java:92)
	at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1250)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
	... 20 more

java.lang.NullPointerException
	at com.servoy.j2db.FlattenedSolution.getStyleForForm(FlattenedSolution.java:1327)
	at com.servoy.j2db.FlattenedSolution.getStyleForForm(FlattenedSolution.java:1303)
	at com.servoy.j2db.component.ComponentFactory.getOverriddenStyleName(ComponentFactory.java:520)
	at com.servoy.j2db.server.headlessclient.dataui.TemplateGenerator.getFormHTMLAndCSS(TemplateGenerator.java:294)
	at com.servoy.j2db.server.headlessclient.ServoyResourceStreamLocator$1.locate(ServoyResourceStreamLocator.java:135)
	at org.apache.wicket.util.resource.locator.ResourceStreamLocator.locate(ResourceStreamLocator.java:122)
	at com.servoy.j2db.server.headlessclient.ServoyResourceStreamLocator.locate(ServoyResourceStreamLocator.java:215)
	at org.apache.wicket.markup.DefaultMarkupResourceStreamProvider.getMarkupResourceStream(DefaultMarkupResourceStreamProvider.java:82)
	at org.apache.wicket.markup.MarkupCache.getMarkup(MarkupCache.java:305)
	at org.apache.wicket.markup.MarkupCache.hasAssociatedMarkup(MarkupCache.java:245)
	at org.apache.wicket.MarkupContainer.hasAssociatedMarkup(MarkupContainer.java:1796)
	at org.apache.wicket.Component.findParentWithAssociatedMarkup(Component.java:1367)
	at org.apache.wicket.MarkupFragmentFinder.find(MarkupFragmentFinder.java:50)
	at org.apache.wicket.Component.locateMarkupStream(Component.java:3883)
	at org.apache.wicket.Component.renderComponent(Component.java:2617)
	at org.apache.wicket.ajax.AjaxRequestTarget.respondComponent(AjaxRequestTarget.java:853)
	at org.apache.wicket.ajax.AjaxRequestTarget.respondComponents(AjaxRequestTarget.java:682)
	at org.apache.wicket.ajax.AjaxRequestTarget.respond(AjaxRequestTarget.java:592)
	at org.apache.wicket.request.AbstractRequestCycleProcessor.respond(AbstractRequestCycleProcessor.java:105)
	at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1258)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
	at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1428)
	at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
	at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:479)
	at com.servoy.j2db.server.servlets.Zl.doGet(Zl.java:5)
	at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
	at java.lang.Thread.run(Thread.java:662)

2011-04-08 23:20:05,081 ERROR [http-213.154.231.180-80-4] com.servoy.j2db.util.Debug - No solution and form found from MySolution/myForm
java.lang.NullPointerException
	at com.servoy.j2db.FlattenedSolution.getStyleForForm(FlattenedSolution.java:1327)
	at com.servoy.j2db.FlattenedSolution.getStyleForForm(FlattenedSolution.java:1303)
	at com.servoy.j2db.component.ComponentFactory.getOverriddenStyleName(ComponentFactory.java:520)
	at com.servoy.j2db.server.headlessclient.dataui.TemplateGenerator.getFormHTMLAndCSS(TemplateGenerator.java:294)
	at com.servoy.j2db.server.headlessclient.ServoyResourceStreamLocator$1.locate(ServoyResourceStreamLocator.java:135)
	at org.apache.wicket.util.resource.locator.ResourceStreamLocator.locate(ResourceStreamLocator.java:122)
	at com.servoy.j2db.server.headlessclient.ServoyResourceStreamLocator.locate(ServoyResourceStreamLocator.java:215)
	at org.apache.wicket.markup.DefaultMarkupResourceStreamProvider.getMarkupResourceStream(DefaultMarkupResourceStreamProvider.java:82)
	at org.apache.wicket.markup.MarkupCache.getMarkup(MarkupCache.java:305)
	at org.apache.wicket.markup.MarkupCache.hasAssociatedMarkup(MarkupCache.java:245)
	at org.apache.wicket.MarkupContainer.hasAssociatedMarkup(MarkupContainer.java:1796)
	at org.apache.wicket.Component.findParentWithAssociatedMarkup(Component.java:1367)
	at org.apache.wicket.MarkupFragmentFinder.find(MarkupFragmentFinder.java:50)
	at org.apache.wicket.Component.locateMarkupStream(Component.java:3883)
	at org.apache.wicket.Component.renderComponent(Component.java:2617)
	at org.apache.wicket.ajax.AjaxRequestTarget.respondComponent(AjaxRequestTarget.java:853)
	at org.apache.wicket.ajax.AjaxRequestTarget.respondComponents(AjaxRequestTarget.java:682)
	at org.apache.wicket.ajax.AjaxRequestTarget.respond(AjaxRequestTarget.java:592)
	at org.apache.wicket.request.AbstractRequestCycleProcessor.respond(AbstractRequestCycleProcessor.java:105)
	at org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:1258)
	at org.apache.wicket.RequestCycle.step(RequestCycle.java:1329)
	at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1428)
	at org.apache.wicket.RequestCycle.request(RequestCycle.java:545)
	at org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:479)
	at com.servoy.j2db.server.servlets.Zl.doGet(Zl.java:5)
	at org.apache.wicket.protocol.http.WicketServlet.doGet(WicketServlet.java:138)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:567)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
	at java.lang.Thread.run(Thread.java:662)

ngervasi · April 8, 2011, 10:35pm

Hmmmm… how many clients are connecting to that server? Are they smart or web clients? If they are webclients: how big is the solution they are using?
The memory usage depends on that. I’m running 80 smart clients with 760MB dedicated to servoy server and I’ve never saw an usage above 560MB.

About the errors you reported: the first one comes from Postgres, it looks like it has no table space left, the second is a mix of smart clients connection errors, and errors I’ve never seen before but that should not have any relation with the server freezing.

First thing I would check the Postgres problem.

udrescu_bogdan · April 9, 2011, 7:15am

There are no more the 10 - 15 concurrent clients. One of them is a batch process which runs some functions on Cron from 5 to 5 minutes, to send mails or to set set some data automatically. We also have a Rest web service which use in general 2 clients. And then the end-users which prefer the WebClient because it’s faster, but some of them also use the SmartClient.

What do you mean by that Postgresql problem. There is a table in the database which is full and can not be written in it? Why this thing might appear?

I found another thing in server.log. Please take a look at the log time. It started this night at 2:47 (except for the first exceptions which don’t have the time), just 2 hours after I resetarted and it goes till this morning when I restarted again!

Exception in thread "pool-4-thread-2" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2760)
	at java.util.Arrays.copyOf(Arrays.java:2734)
	at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
	at java.util.ArrayList.add(ArrayList.java:351)
	at com.servoy.j2db.dataprocessing.RowManager.addCalculationDependency(RowManager.java:1653)
	at com.servoy.j2db.dataprocessing.Record.manageCalcDependency(Record.java:847)
	at com.servoy.j2db.dataprocessing.Record.getValue(Record.java:194)
	at com.servoy.j2db.dataprocessing.Record.validateStoredCalculations(Record.java:139)
	at com.servoy.j2db.dataprocessing.EditRecordList.testIfRecordIsChanged(EditRecordList.java:811)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:829)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopEditing(EditRecordList.java:262)
	at com.servoy.j2db.dataprocessing.Record.stopEditing(Record.java:309)
	at com.servoy.j2db.dataprocessing.JSDatabaseManager.recalculateRecord(JSDatabaseManager.java:770)
	at com.servoy.j2db.dataprocessing.JSDatabaseManager.js_recalculate(JSDatabaseManager.java:742)
	at sun.reflect.GeneratedMethodAccessor217.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
	at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
	at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
	at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$MethodExecutor.run(ClientPluginAccessProvider.java:502)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeAndWait(SessionClient.java:1159)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeLater(SessionClient.java:1149)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$1.run(ClientPluginAccessProvider.java:430)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
Exception in thread "pool-4-thread-3" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2760)
	at java.util.Arrays.copyOf(Arrays.java:2734)
	at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
	at java.util.ArrayList.add(ArrayList.java:351)
	at com.servoy.j2db.dataprocessing.RowManager.addCalculationDependency(RowManager.java:1653)
	at com.servoy.j2db.dataprocessing.Record.manageCalcDependency(Record.java:847)
	at com.servoy.j2db.dataprocessing.Record.getValue(Record.java:194)
	at com.servoy.j2db.dataprocessing.Record.validateStoredCalculations(Record.java:139)
	at com.servoy.j2db.dataprocessing.EditRecordList.testIfRecordIsChanged(EditRecordList.java:811)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:829)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopEditing(EditRecordList.java:262)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:837)
	at com.servoy.j2db.dataprocessing.EditRecordList.hasEditedRecords(EditRecordList.java:132)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopIfEditing(EditRecordList.java:156)
	at com.servoy.j2db.dataprocessing.FoundSet.js_loadRecords(FoundSet.java:950)
	at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
	at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
	at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
	at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$MethodExecutor.run(ClientPluginAccessProvider.java:502)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeAndWait(SessionClient.java:1159)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeLater(SessionClient.java:1149)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$1.run(ClientPluginAccessProvider.java:430)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
Exception in thread "pool-4-thread-4" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2760)
	at java.util.Arrays.copyOf(Arrays.java:2734)
	at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
	at java.util.ArrayList.add(ArrayList.java:351)
	at com.servoy.j2db.dataprocessing.RowManager.addCalculationDependency(RowManager.java:1653)
	at com.servoy.j2db.dataprocessing.Record.manageCalcDependency(Record.java:847)
	at com.servoy.j2db.dataprocessing.Record.getValue(Record.java:194)
	at com.servoy.j2db.dataprocessing.Record.validateStoredCalculations(Record.java:139)
	at com.servoy.j2db.dataprocessing.EditRecordList.testIfRecordIsChanged(EditRecordList.java:811)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:829)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopEditing(EditRecordList.java:262)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:837)
	at com.servoy.j2db.dataprocessing.EditRecordList.hasEditedRecords(EditRecordList.java:132)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopIfEditing(EditRecordList.java:156)
	at com.servoy.j2db.dataprocessing.FoundSet.js_loadRecords(FoundSet.java:950)
	at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
	at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
	at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
	at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$MethodExecutor.run(ClientPluginAccessProvider.java:502)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeAndWait(SessionClient.java:1159)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeLater(SessionClient.java:1149)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$1.run(ClientPluginAccessProvider.java:430)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
Exception in thread "pool-4-thread-5" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:2760)
	at java.util.Arrays.copyOf(Arrays.java:2734)
	at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
	at java.util.ArrayList.add(ArrayList.java:351)
	at com.servoy.j2db.dataprocessing.RowManager.addCalculationDependency(RowManager.java:1653)
	at com.servoy.j2db.dataprocessing.Record.manageCalcDependency(Record.java:847)
	at com.servoy.j2db.dataprocessing.Record.getValue(Record.java:194)
	at com.servoy.j2db.dataprocessing.Record.validateStoredCalculations(Record.java:139)
	at com.servoy.j2db.dataprocessing.EditRecordList.testIfRecordIsChanged(EditRecordList.java:811)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:829)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopEditing(EditRecordList.java:262)
	at com.servoy.j2db.dataprocessing.EditRecordList.removeUnChangedRecords(EditRecordList.java:837)
	at com.servoy.j2db.dataprocessing.EditRecordList.hasEditedRecords(EditRecordList.java:132)
	at com.servoy.j2db.dataprocessing.EditRecordList.stopIfEditing(EditRecordList.java:156)
	at com.servoy.j2db.dataprocessing.FoundSet.js_loadRecords(FoundSet.java:950)
	at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:179)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:353)
	at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3666)
	at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2680)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:166)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:387)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3127)
	at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:165)
	at com.servoy.j2db.scripting.ScriptEngine.executeFunction(ScriptEngine.java:458)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$MethodExecutor.run(ClientPluginAccessProvider.java:502)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeAndWait(SessionClient.java:1159)
	at com.servoy.j2db.server.headlessclient.SessionClient.invokeLater(SessionClient.java:1149)
	at com.servoy.j2db.plugins.ClientPluginAccessProvider$1.run(ClientPluginAccessProvider.java:430)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
Exception in thread "pool-4-thread-7" java.lang.OutOfMemoryError: Java heap space
....
Apr 9, 2011 2:47:37 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet servoy_web_services threw exception
java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-4-thread-35" java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 2:47:44 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet servoy_web_services threw exception
java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 2:47:55 AM org.apache.catalina.connector.CoyoteAdapter service
SEVERE: An exception or error occurred in the container during the request processing
java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 2:48:04 AM org.apache.coyote.http11.Http11Processor process
SEVERE: Error finishing response
java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 2:48:04 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet servoy_web_services threw exception
java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-4-thread-36" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-4-thread-37" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-4-thread-38" java.lang.OutOfMemoryError: Java heap space
....
Exception in thread "pool-4-thread-58" java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 4:39:41 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet default threw exception
java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 4:39:50 AM org.apache.catalina.connector.CoyoteAdapter service
SEVERE: An exception or error occurred in the container during the request processing
java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 4:39:59 AM org.apache.coyote.http11.Http11Processor process
SEVERE: Error finishing response
java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-4-thread-60" java.lang.OutOfMemoryError: Java heap space
Exception in thread "pool-4-thread-61" java.lang.OutOfMemoryError: Java heap space
....
Exception in thread "pool-4-thread-108" java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 8:22:34 AM org.apache.catalina.connector.CoyoteAdapter service
SEVERE: An exception or error occurred in the container during the request processing
java.lang.OutOfMemoryError: Java heap space
Apr 9, 2011 8:22:44 AM org.apache.coyote.http11.Http11Processor process
SEVERE: Error finishing response
java.lang.OutOfMemoryError: Java heap space
....
Apr 9, 2011 8:54:14 AM org.apache.catalina.session.StandardManager doLoad
SEVERE: IOException while loading persisted sessions: java.io.EOFException
java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2280)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2749)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:779)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:279)
	at org.apache.catalina.util.CustomObjectInputStream.<init>(CustomObjectInputStream.java:58)
	at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:362)
	at org.apache.catalina.session.StandardManager.load(StandardManager.java:321)
	at org.apache.catalina.session.StandardManager.start(StandardManager.java:637)
	at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:438)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4358)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:987)
	at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:909)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:495)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
	at org.apache.catalina.core.StandardService.start(StandardService.java:516)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.servoy.j2db.server.WebServer.startWebServer(WebServer.java:89)
	at com.servoy.j2db.server.ApplicationServer.startWebServer(ApplicationServer.java:630)
	at com.servoy.j2db.server.ApplicationServer.main(ApplicationServer.java:600)
Apr 9, 2011 8:54:14 AM org.apache.catalina.session.StandardManager start
SEVERE: Exception loading sessions from persistent storage
java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2280)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2749)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:779)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:279)
	at org.apache.catalina.util.CustomObjectInputStream.<init>(CustomObjectInputStream.java:58)
	at org.apache.catalina.session.StandardManager.doLoad(StandardManager.java:362)
	at org.apache.catalina.session.StandardManager.load(StandardManager.java:321)
	at org.apache.catalina.session.StandardManager.start(StandardManager.java:637)
	at org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:438)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4358)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
	at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:987)
	at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:909)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:495)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
	at org.apache.catalina.core.StandardService.start(StandardService.java:516)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.servoy.j2db.server.WebServer.startWebServer(WebServer.java:89)
	at com.servoy.j2db.server.ApplicationServer.startWebServer(ApplicationServer.java:630)
	at com.servoy.j2db.server.ApplicationServer.main(ApplicationServer.java:600)

udrescu_bogdan · April 9, 2011, 11:17am

We just increased the memory on that VPS to 4Gb and also increase the Xmx to 1024m and this is our memory status after one SmartClient login:

             total       used       free     shared    buffers     cached
Mem:          4110       1292       2818          0         15        456
-/+ buffers/cache:        821       3289
Swap:          478          0        478

On this server only Servoy runs and Postgresql. No other application. Just Servoy!

jcompagner · April 11, 2011, 8:44am

looking at all these dumps you should be looking into 2 problems yes

first this should be fixed:

org.postgresql.util.PSQLException: ERROR: could not extend relation base/16384/16490: No space left on device

that really seems that the db doesn’t have enough space to write to, where even it tries to write to (maybe another partition?)

also the out of memory heap should be fixed by increasing the memory of the server.

udrescu_bogdan · April 11, 2011, 10:26am

Hello,

When the PostgreSQL error appeared we had 1Gb free on the HDD. Now we have 3 Gb free.

About the memory, shall we get more that 4Gb? Shall I also set the -Xmx argument to more then 1Gb or 2Gb, or in case we get 8Gb or RAM to set -Xmx to 4Gb?

The problem started to appears after we imported 10.000 users together with all related data in our database and we also import them in servoy_repository for authentication. But we have only 10-15 concurrent users.

But also in the past the server blocked with an average of once per week. But now it does almost each hour sometimes, and when we are lucky, especially during the night, it runs all night or blocks at most at interval of couple of hours.

If you have also other suggestions, they are most welcome!

Thanks,
Bogdan.

ngervasi · April 11, 2011, 10:34am

For that amount of users 512mb assigned to servoy should be more than enough. I would check the batch processor and the webservice. Can you try to turn them off for some time to see if the problem persists? Can you check the code they are running? Maybe there’s something wrong in a big loop.

udrescu_bogdan · April 11, 2011, 11:26am

I have found some pieces of code in this weekend in those batch processes that loads all data and processes. I removed them, but the situation doesn’t get too much better, so what I stated in the last post about the blocks is still valid.

So what shall I do is to investigate my code and see if somewhere data is overloaded. Related with this the approach that we use is this: We have companies in our database, each company have users. When the user login he has access only to the data of the company. I do two things:

At startup I call databaseManager.addTableFilterParam and add filter for all tables with the company_id. But this slowdown the SmartClient very much, so I actually think to remove it.
All forms and all sql requests are based on Relations using the company_id so I don’t receive all data but only the data for that company_id. All tables in my database have the company_id set. I have some questions regarding this:

How Servoy manage Relations?
It filters the data after it receive it from the database or it creates the sql query so it will receive exact the filtered data?
Would be better to create in all tables an index on the company_id? Would this speed up the loading of the data?

About the memory, while writing this post, I increase the Xmx from 512 to 2048, while we have 4Gb available, and for 15 minutes, we’ve started about 3 SmartClients and 3 WebClients and also the WebService and Batch process, and they behaves even better then before. Let’s see how much time it will be stable . But still this doesn’t solve the base problem.

jcompagner · April 11, 2011, 12:00pm

smart clients don’t really use any memory on the server.
Its really the serverside processes that do that, like the webclient or the batchprocessors (or other headless clients)

What does the admin home page now say ? how much is in use and what is the max?

udrescu_bogdan · April 11, 2011, 1:24pm

This is the memory on server now:

Heap memory: allocated=1667840K, used=1281812K, max=1864192K

In between I restarted once, as it went really slow. I’m checking the WebService code and other headless clients I have.

What about the questions I’ve asked in the previous post? About Relations and indexing also on the company_id column together with the id column in each table?

jcompagner · April 11, 2011, 1:37pm

that you use that much is very weird, you really should look what you do in 1 of the clients that run on the server itself (web/headless/batch)
because you seem to load and hold on to enormous amount of data, Are you running reports?

if you use table/foundset filters on company_id or have relations based on that company_id and you set that company_id in a global or something then you only get that data the filtering is done in the database
So yes if you use that as a where param everywhere then it is a good column to index.

udrescu_bogdan · April 13, 2011, 8:27am

I start digging in my code to see were data can be overloaded. It’s quite boring and until now I didn’t figure out almost anything.

I do sometimes call this:

var myFoundset = databaseManager.getFoundSet("myServer", "myTable");
myFoundset.loadRecords(id);

Can this be something that loads more then the data for that “id” that I put in the “loadRecords” call?

Also, I think I will have more questions then this, as I search in the code.

Thanks!

jcompagner · April 13, 2011, 8:30am

that id is just a single pk value?
that will only load 1 record in memory

udrescu_bogdan · April 13, 2011, 9:59am

yes, it’s the PK value, thanks!

udrescu_bogdan · April 13, 2011, 11:33am

another question: if I have a form I use only for processing stuffs and I use only the methods there, but that form has a dataSource set. I don’t have any relations set on it because it’s not added in any tabbedPane. I only call a method, let’s say:

function processStuf(tableID) {
    foundset.loadRecords(tableID);
    // do other calculations
}

I’m doing this because of the leak of classes. I don’t like to write all code in a single form, and I have different functionality in different forms, for object-oriented reasons.

But, BEFORE calling the form with that method, any other data from that table will be loaded in the foundset of the form? And if so, I think all data from the table will be loaded! I just found in Performance Data on server a query which selects all data from a table with large amount on information and the Type of the query was FoundSet, … or something like this! I don’t remember because I cleared the data to have another try, but I can’t reproduce right now.

jcompagner · April 13, 2011, 1:03pm

we never load all data of a table, we only load in the first 200 PK’s and some of the first records around the selected index.

If you show a form by default we will load the initial foundset it has.
You can control that by setting the OnShowAllRecords command of that form so that you override what it loads.

udrescu_bogdan · April 13, 2011, 1:42pm

I found also in Performance Data queries like this with type “Load foundset”, but not only for this table, but for all tables in database:

select contact_id from contacts order by contact_id asc limit ?

And for some others queries like this one, the “order by” was also on other field, not only on the PK field.

Can this be a problem? From where can they be? I don’t query this directly.

Topic		Replies	Views
Server stops Classic Servoy	41	26178	January 16, 2012
Server ??? WTH? Classic Servoy	3	2090	January 12, 2010
Application server keeps freezing Classic Servoy	6	5460	November 15, 2017
Servoy performance and/or resources issues Classic Servoy	5	4409	July 9, 2014
Servoy Server performance problem Questions	3	2867	February 5, 2020

Servoy server blocks

Related topics