OutOfMemory-Exception with Heap memory

Questions and Answers on installation, deployment, management, locking, tranasactions of Servoy Application Server

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Mon Dec 12, 2022 10:57 pm

but you should not move to 2022.09 if you don't plan to move to .12 after that very soon
because 09 is already end of live and will not get any more updates

If you don't plan to update often (so 06 to 09 to 12) then it is way better to just install the lts, so move to 2022.03.x LTS
That will get the most important fixes, the next release of that will have fixes if you would get websocket disconnected and clients are not removed, it a certain scenario.
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8822
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby darren » Tue Dec 13, 2022 12:19 am

I understand your reasoning on moving to new versions. I shouldn't say we're not willing to move to 2022.12, it is just unfortunate that doing so requires this other big change point (Java 17), when there is this other large problem (OOM Java heap space) we're trying to solve. We could move to 2022.03 LTS right now, but from earlier posts in this thread, it appears as though we'll still have the OOM problem. So perhaps we'll wait until the new 2022.03 LTS with the fixes is released.

Are you able to comment on my other questions:

You say there are multiple things that could be causing the java.lang.OutOfMemoryError: Java heap space error. Do they all eventually lead to "websocket disconnected" clients? which in turn cause the Java heap space error? If so, I can monitor our UAT environment and see if there are any "websocket disconnected" clients.

I'm not convinced that it is programming errors in our solution code, only because we weren't encountering the error with a 2020.06 .war file... unless there are changes within Servoy between 2020.06 and 2022.09 that would 'expose' such solution code and cause the error. Perhaps that's what you're referring to with the fixes in 2022.12?

Additionally, I've uploaded the relevant logging from our servoy application log file in the lead up to the OOM - Java heap space error, in case there are some clues. There are certainly some log messages that I haven't seen before. Perhaps you could take a quick look.
Thanks again.

servoy-application-log.txt
(1.28 MiB) Downloaded 47 times
Last edited by darren on Sat Nov 11, 2023 4:18 pm, edited 2 times in total.
darren
 
Posts: 19
Joined: Mon Nov 01, 2021 11:10 pm

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Tue Dec 13, 2022 11:00 am

there can be hunderds of causes that will end up in OOM..
i have seen many, like big report generation, huge files loaded in to memory and then that intermediate result of "websocket disconnected" can also be stuff like databasetransactions in deadlock blocking stuff, never ending loops in code, code deadlocks (this last thing is something servoy must fix), websocket/browser connections errors that results in a very weird state at the server (this is a fix that is in .12 and in the next 03.LTS) and then in the end you OOM out because the clients are never shutdown.

there are some very weird exceptions in that log:

2022-12-09 11:47:46,018 ERROR [Executor,uuid:F252A3E:2, clientid: 3ED40EB9-9CF4-4D29-8DAA-EF03F92E5C65] org.sablo.websocket.utils.JSONUtils - unsupported value type:null for value: QBSelect(SELECT columns FROM db:/qfi/v_deal_progress where XXXX)
at org.sablo.websocket.utils.JSONUtils$FullValueToJSONConverter.toJSONValue(JSONUtils.java:479) ~[sablo_2022.9.2.3805.jar:?]
at org.sablo.websocket.utils.JSONUtils$FullValueToJSONConverter.toJSONValue(JSONUtils.java:1) ~[sablo_2022.9.2.3805.jar:?]
at org.sablo.websocket.utils.JSONUtils.writeData(JSONUtils.java:101) ~[sablo_2022.9.2.3805.jar:?]
at org.sablo.websocket.BaseWindow.sendMessageInternal(BaseWindow.java:611) ~[sablo_2022.9.2.3805.jar:?]
at org.sablo.websocket.BaseWindow.sendAsyncMessage(BaseWindow.java:509) ~[sablo_2022.9.2.3805.jar:?]

for some reason it tries to push of a query to the client.. i have really no idea how that is possible. that should never happen in the components and services we ship

the rest really seems to be a lot of users that are pressing a button, changing data, showing form in a dialog, so really interacting with the server, but then just close the browser or tab, a lot of exceptions that the websocket is disconnected in the middle of calls..
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8822
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby darren » Wed Dec 14, 2022 1:41 am

The QBSelect exception is something we've had for awhile and tried to work through with Servoy support, but never got anywhere and since it the form was working correctly, we chose to ignore it. Your comment on it today prompted us to revisit the exception, and we found two ways to avoid the exception... there's something weird going on there. I'll get around to writing it up one day.

I'm guessing the websocket disconnect exceptions at the end of the file were causes by the imminent OOM.

We've decided to go ahead and deploy the problem 2022.09 .war file sometime soon and try to capture the required memory dump. Thanks.
darren
 
Posts: 19
Joined: Mon Nov 01, 2021 11:10 pm

Previous

Return to Servoy Server

Who is online

Users browsing this forum: No registered users and 1 guest

cron