Page 1 of 3

OutOfMemory-Exception with Heap memory

PostPosted: Tue Jan 11, 2022 4:27 pm
by rph
Hi

We've got memory issues with one of our application.

It's a NG application, that is used from public users (no authentication needed). Usually there are about 10 to 50 clients running. An average session runs about 5 minutes and then often the session remains another 10 minutes because the user doesn't exit the application. So the Servoy "plugin.svyIdle"-Plugins will kick out the user after 10 minutes of inactivity. This all works very well, except, that the usage of "heap memory" increases until we get "OutOfMemoryError: Java heap space". Even if we remove all clients, the memory usage remains. The only solution then is to restart the Tomcat-server.

I don't know what leaks. My assumptions:
- InMemory-Foundset (we use just two quit small InMemory-Foundsets with less then 50 Records!)
- Scheduler-Plugin
- ???

Our Environment:
- Windows-Server 2019 (Windows 10)
- OpenJDK 64-Bit 14.0.2
- Servoy 2020.03 (Built 3562)
- max. JavaHeap space configured 12GB!!! (we lose about 2 GB per day, so the server must be rebooted after at least one week)

I will add a dump from the servoy admin page.

Any ideas?
Thanks in advance for any suggestions
Roland

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Jan 11, 2022 4:35 pm
by jcompagner
to understand this you need to make a memory dump (with yourkit or another profile tool)
and it would be nice if the dump could be made after a few days (when it did grow) and at that time almost had no clients then we really can see what keeps hanging around

Re: OutOfMemory-Exception with Heap memory

PostPosted: Wed Jan 12, 2022 10:55 am
by rph
Hi Johan

Thanks for your reply. I'll do that and send the dump in the next few days, as soon as I got one.

Re: OutOfMemory-Exception with Heap memory

PostPosted: Wed Jan 19, 2022 9:53 am
by rph
Hi Johan

I'll send you a PM with the link to download the memory dump.

Re: OutOfMemory-Exception with Heap memory

PostPosted: Fri Jan 21, 2022 9:47 am
by rph
Just for the sake of completeness: as Johan confirmed the reason is an issue with the InMemory database (HSQLDB) in Servoy versions prior 2021.03 LTS. The memory is not cleaned up propperly at the end of the session.

So the solution to our problem would be:
a) update to Servoy 2021.03 LTS
OR
b) replace InMemory foundset with datasets

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 12:58 pm
by marazzi1539352432
Hi, everybody for us same problem but the update to Servoy 2021.03 and beyond 2021.09 it does not work.
I can know what command you ran to dump the memory from linux?

I added these parameters when launching the tomcat application but it does not create the dump file for me
Environment='CATALINA_OPTS=-Xms16G -Xmx24G -server -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/dump_memory'

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 1:43 pm
by jcompagner
what do you mean it doesn't not work (.12.2 or the latest 03 LTS)?
You mean you upgraded it and you still have out of memory after a few days? so the memory slowly keeps on growing and you get an out of mem exception in the log ?
those flags (XX) seems fine (if tomcat process can write in that dir)

Else you need to manually trigger a memory dump with the various tools like yourkit or jmap

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 3:24 pm
by marazzi1539352432
yes now we have several server with this version
Servoy version 2021.12.1 -releaseNumber 3723 the memory continues to grow up to the classic out of memory.

the same server are VM cloned with Servoy version 2020.6.0 -releaseNumber 3582 has a regular and constant memory.

if the parameters I enter to get the dump go veins it could then be that tomcat cannot write to that folder in fact I could try to change path.
if I don't put any path parameter where does the dump write by default?
thanks a lot this problem is consuming us :)

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 3:35 pm
by jcompagner
if nothing is specified i think it just starts to dump it in the "user.dir" so thats the start dir of the application.

ah but you are using a 12.1 and i guess you have profiling enabled on the admin page? there was a memory leak fixed in 12.2: https://wiki.servoy.com/display/DOCS/2021.12
upgrade to 12.2 or make sure you disable in production all your profiling (client and database)

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 3:54 pm
by marazzi1539352432
I don't remember ever activating a profiles in servoy admin ... this is this page here attached. I don't think there is anything active

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 5:28 pm
by jcompagner
not profiles but profiling, so the 2 latest items in the list "Database performance" and "Client performance"

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 6:01 pm
by marazzi1539352432
Are you talking about these parameters? This is our actual configuration. Should we change this to "disabled"?
Screenshot_1.png
Screenshot_1.png (27.25 KiB) Viewed 4930 times

Screenshot_2.png
Screenshot_2.png (21.89 KiB) Viewed 4930 times

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 6:21 pm
by jcompagner
yes or update to 2021.12.2

Re: OutOfMemory-Exception with Heap memory

PostPosted: Tue Feb 08, 2022 6:23 pm
by marazzi1539352432
We'll try and let you know if it will solve the problem.

Re: OutOfMemory-Exception with Heap memory

PostPosted: Wed Feb 09, 2022 5:22 pm
by rmidata
Hello
We are having the same issue with Servoy 2021.12.2 with the WAR and tomcat 9.x and with the MariaDB 10.x database engine.
The memory is not being freed and it causes the application server to crash, this happens after a few days and we have to restart the server.
We are following this thread of conversation, we await news.
Greetings from Argentina.