OutOfMemory-Exception with Heap memory

Questions and Answers on installation, deployment, management, locking, tranasactions of Servoy Application Server

OutOfMemory-Exception with Heap memory

Postby rph » Tue Jan 11, 2022 4:27 pm

Hi

We've got memory issues with one of our application.

It's a NG application, that is used from public users (no authentication needed). Usually there are about 10 to 50 clients running. An average session runs about 5 minutes and then often the session remains another 10 minutes because the user doesn't exit the application. So the Servoy "plugin.svyIdle"-Plugins will kick out the user after 10 minutes of inactivity. This all works very well, except, that the usage of "heap memory" increases until we get "OutOfMemoryError: Java heap space". Even if we remove all clients, the memory usage remains. The only solution then is to restart the Tomcat-server.

I don't know what leaks. My assumptions:
- InMemory-Foundset (we use just two quit small InMemory-Foundsets with less then 50 Records!)
- Scheduler-Plugin
- ???

Our Environment:
- Windows-Server 2019 (Windows 10)
- OpenJDK 64-Bit 14.0.2
- Servoy 2020.03 (Built 3562)
- max. JavaHeap space configured 12GB!!! (we lose about 2 GB per day, so the server must be rebooted after at least one week)

I will add a dump from the servoy admin page.

Any ideas?
Thanks in advance for any suggestions
Roland
Attachments
dump.txt
dump
(54.8 KiB) Downloaded 114 times
Last edited by rph on Tue Jan 11, 2022 4:37 pm, edited 1 time in total.
rph
 
Posts: 75
Joined: Wed Aug 10, 2011 11:44 am
Location: Cham, Switzerland

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Tue Jan 11, 2022 4:35 pm

to understand this you need to make a memory dump (with yourkit or another profile tool)
and it would be nice if the dump could be made after a few days (when it did grow) and at that time almost had no clients then we really can see what keeps hanging around
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8829
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby rph » Wed Jan 12, 2022 10:55 am

Hi Johan

Thanks for your reply. I'll do that and send the dump in the next few days, as soon as I got one.
rph
 
Posts: 75
Joined: Wed Aug 10, 2011 11:44 am
Location: Cham, Switzerland

Re: OutOfMemory-Exception with Heap memory

Postby rph » Wed Jan 19, 2022 9:53 am

Hi Johan

I'll send you a PM with the link to download the memory dump.
rph
 
Posts: 75
Joined: Wed Aug 10, 2011 11:44 am
Location: Cham, Switzerland

Re: OutOfMemory-Exception with Heap memory

Postby rph » Fri Jan 21, 2022 9:47 am

Just for the sake of completeness: as Johan confirmed the reason is an issue with the InMemory database (HSQLDB) in Servoy versions prior 2021.03 LTS. The memory is not cleaned up propperly at the end of the session.

So the solution to our problem would be:
a) update to Servoy 2021.03 LTS
OR
b) replace InMemory foundset with datasets
rph
 
Posts: 75
Joined: Wed Aug 10, 2011 11:44 am
Location: Cham, Switzerland

Re: OutOfMemory-Exception with Heap memory

Postby marazzi1539352432 » Tue Feb 08, 2022 12:58 pm

Hi, everybody for us same problem but the update to Servoy 2021.03 and beyond 2021.09 it does not work.
I can know what command you ran to dump the memory from linux?

I added these parameters when launching the tomcat application but it does not create the dump file for me
Environment='CATALINA_OPTS=-Xms16G -Xmx24G -server -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/dump_memory'
marazzi1539352432
 
Posts: 7
Joined: Fri Oct 12, 2018 3:53 pm

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Tue Feb 08, 2022 1:43 pm

what do you mean it doesn't not work (.12.2 or the latest 03 LTS)?
You mean you upgraded it and you still have out of memory after a few days? so the memory slowly keeps on growing and you get an out of mem exception in the log ?
those flags (XX) seems fine (if tomcat process can write in that dir)

Else you need to manually trigger a memory dump with the various tools like yourkit or jmap
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8829
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby marazzi1539352432 » Tue Feb 08, 2022 3:24 pm

yes now we have several server with this version
Servoy version 2021.12.1 -releaseNumber 3723 the memory continues to grow up to the classic out of memory.

the same server are VM cloned with Servoy version 2020.6.0 -releaseNumber 3582 has a regular and constant memory.

if the parameters I enter to get the dump go veins it could then be that tomcat cannot write to that folder in fact I could try to change path.
if I don't put any path parameter where does the dump write by default?
thanks a lot this problem is consuming us :)
marazzi1539352432
 
Posts: 7
Joined: Fri Oct 12, 2018 3:53 pm

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Tue Feb 08, 2022 3:35 pm

if nothing is specified i think it just starts to dump it in the "user.dir" so thats the start dir of the application.

ah but you are using a 12.1 and i guess you have profiling enabled on the admin page? there was a memory leak fixed in 12.2: https://wiki.servoy.com/display/DOCS/2021.12
upgrade to 12.2 or make sure you disable in production all your profiling (client and database)
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8829
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby marazzi1539352432 » Tue Feb 08, 2022 3:54 pm

I don't remember ever activating a profiles in servoy admin ... this is this page here attached. I don't think there is anything active
Attachments
Cattura.JPG
Cattura.JPG (115.41 KiB) Viewed 4793 times
marazzi1539352432
 
Posts: 7
Joined: Fri Oct 12, 2018 3:53 pm

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Tue Feb 08, 2022 5:28 pm

not profiles but profiling, so the 2 latest items in the list "Database performance" and "Client performance"
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8829
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby marazzi1539352432 » Tue Feb 08, 2022 6:01 pm

Are you talking about these parameters? This is our actual configuration. Should we change this to "disabled"?
Screenshot_1.png
Screenshot_1.png (27.25 KiB) Viewed 4762 times

Screenshot_2.png
Screenshot_2.png (21.89 KiB) Viewed 4762 times
marazzi1539352432
 
Posts: 7
Joined: Fri Oct 12, 2018 3:53 pm

Re: OutOfMemory-Exception with Heap memory

Postby jcompagner » Tue Feb 08, 2022 6:21 pm

yes or update to 2021.12.2
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8829
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: OutOfMemory-Exception with Heap memory

Postby marazzi1539352432 » Tue Feb 08, 2022 6:23 pm

We'll try and let you know if it will solve the problem.
marazzi1539352432
 
Posts: 7
Joined: Fri Oct 12, 2018 3:53 pm

Re: OutOfMemory-Exception with Heap memory

Postby rmidata » Wed Feb 09, 2022 5:22 pm

Hello
We are having the same issue with Servoy 2021.12.2 with the WAR and tomcat 9.x and with the MariaDB 10.x database engine.
The memory is not being freed and it causes the application server to crash, this happens after a few days and we have to restart the server.
We are following this thread of conversation, we await news.
Greetings from Argentina.
Roberto Diez.
Interdata Software.
Santa Fe, Argentina.
User avatar
rmidata
 
Posts: 42
Joined: Thu Jul 09, 2009 10:26 pm
Location: Santa Fe, Argentina.

Next

Return to Servoy Server

Who is online

Users browsing this forum: No registered users and 10 guests