How much data is a variable allowed to hold in Servoy?

[bobpee]

How much data is a variable allowed to hold in Servoy currently? It looks like Servoy (version 4.0 upwards) is somehow restricting the amount of data that can be stored or queried into a variable for further processing.

I can not get databaseManager.getDataSetByQuery to return data which is or consumes more than 512MB ram memory - Servoy throws "Out of Memory" error when the data involved is more than 512MB. Is there another way to query a huge db, without such a constraint, while making use of available hardware resources?

[rgansevles]

Bob,

There is no limit of memory usage per variable, memory is managed over the entire jvm.
You can tune memory settings on the admin page.

Rob

[bobpee]

Rob,

I've been to the admin page and precisely to the place I suppose you're referring to (see snapshot). The question which setting(s) has to be altered or changed to increase the amount of hardware memory that Servoy can use? Is it
servoy.initialClientHeap, servoy.maxClientHeap or servoy.vmClientArgs, and how? servoy.maxClientHeap has for instance a default of 64MB. I would like to make use of say 2G of available memory.

Thanks for the feedback.

[jcompagner]

normally for a average solution i would say that servoy.initialClientHeap=32 and servoy.maxClientHeap=256 (but 128 should also be more then enough for most)
The client args can be anything, not directly related to memory, but we have now there a default setting that makes the behavior of our caches much better.

If you really need to process on the clients that much you can set the maxClientHeap to 1000 or something like that.
But are you really sure you want to do that? These kind of things are way better done on the server, because all that data has to go to the client that that is a lot.
So if you want to do bulk processing why not have an batchprocessor client on the server that does that for you?

Problem is with that max heap is that it will not always result in a way better behaving application or better behaving OS. Because the application takes so much that it also needs to manage an the os cant use that anymore for other things.

[bobpee]

Hi Johan,

Thanks very much for the insight. We have tried running the client by setting the maxClientHeap=1000 on the admin-page, but Servoy( vers 4.1b6 and earlier 4.1 betas) wouldn't
go beyond the java heap memory settings found in

<Servoy installation folder>\developer\servoy.ini :

launcher.XXMaxPermSize 256M
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx256m
-XX:MaxPermSize=256M


We're able to increase the default setting of 256mb in the above .ini file, but not beyond 512mb, which would have been good for us.

The fact is that we have enough ram memory on our test server, leftover (after the normal consumption that Servoy will need to run) for the task we're dealing with now. We ain't able to make use of this leftover mem. b'cos it appears impossible to go beyond 512mb.

Why do we need so much memory? First of all, the extra memory settings that we need now may not be that much when we get our final solution working.
We need that much memory b'cos we need to run preliminary queries to prepare our data for our final solution. Our final solution, I suppose will have the 'normal' settings that ships with Servoy as default.

Having said that, some of our preliminary queries return datasets which contain millions of recs. per query.
These datasets will then have to be wholly further processed to maintain data integrity. This is where we do get "OutOfMemoryError" errors.

FYI, our queries in this case are not necessarily run in batches, we run them one at a time, at least for now.

We need more insights!

[ngervasi]

Hi Johan,

Thanks very much for the insight. We have tried running the client by setting the maxClientHeap=1000 on the admin-page, but Servoy( vers 4.1b6 and earlier 4.1 betas) wouldn't
go beyond the java heap memory settings found in

<Servoy installation folder>\developer\servoy.ini :

launcher.XXMaxPermSize 256M
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx256m
-XX:MaxPermSize=256M

Wait, you are mixing up things.
This setting relates to Servoy Developer not Servoy Client, you are not running the client from Eclipse don't you?
You should change the setting in the admin page and assign something like 1000MB to servoy.maxClientHeap, then in your client machine you need to delete the downloaded client (delete java cache, delete the .servoy folder in your home folder) and then you should start Servoy Server (not Servoy Developer) and redownload a new client, if you don't redownload a new client you won't use the new settings.

[bobpee]

Hi Nicola,

"Client" as I've used earlier refers to a "Servoy Smart Client" fired from within a Servoy Developer in Eclipse. So, I have been referring to settings as related to Developer. My Developer is incidentally running an Appl. Server which makes it impossible to fire a Smart Client from outside of it , get "Starting of webstart clients not possible from developer (running an application server)" error an attempt is made.
Have emptied my java cache too, but to no avail. I still get an "OutOfMemoryError" error when the Java heap memory rises to about 508mb.

Thanks for the attention!

[ngervasi]

Hi Nicola,

"Client" as I've used earlier refers to a "Servoy Smart Client" fired from within a Servoy Developer in Eclipse.

Don't do this, shutdown developer, go to the application_server folder and start Servoy server, then connect using smart client.

[bobpee]

I do still get "Java heap space" errors from time to time, although,, a couple of queries which did run earlier do run now Will give a more detail observation in the course of time. Will be increasing the maxClientHeap to see if that helps.

Thanks!

[jcompagner]

which 256M are you altering?
Dont touch the permsize variant let that one be what it is..

If you want to create more space for your developer or your client that you run inside the developer then only alter this setting:

-Xmx256m

in the ini file.

You should be able to up that one to 1000M but not much higher (on 32bit systems)

[bobpee]

Hi Johan,
Thanks very much for the tip. I am beginning to realize some positive results, starting my smart-client outside Developer .
The bottom line is, if we're able to gain a little bit more memory power (I still working on that now), our problem will be solved. Will keep you updated.
I appreciate the support!