saveData internals - what's going on?

[fdoddridge]

I have a question about what is going on in the saveData method... does Servoy build a bunch of inserts and updates for all data in the solution and execute them when it is called, or is it specific to the form that the controller is in?

If it is the first, where it is basically a global update method, does it go through any conditional logic to see if the data needs updating before executing the update, or does it just save everything?

The reason I ask is that we have a specific form that is taking quite a while to execute saveData. It is nested about 5 levels deep, but I'm not sure if that is the real issue...

This form contains some relations to thumbnail binary image data in a separate database, and the slowness seems to be proportional to the number of thumbnails that this form contains. The frustrating thing though is that the thumbnails are not changing, nor do I need them to update - the relations are there only to pull the data out.

Here's another interesting thing: If I remove the relations to the thumbnail image data, and replace it with queries that assign the thumbnails to global variables and make the global variables the data provider, I get the same slowness results when calling saveData!

I wish I had access to the Servoy source or that I could step into the savedata method in the debugger... Is there anything I should be aware of that could potentially slow down saveData?

[Jan Blok]

saveData saves all outstanding changes
Tip: check the admin page "DB performance data" to see if it is necessary to index the foreign key used by the relations which load the thumbnails

[idoctor]

Jan,

I finally figured this one out. Servoy has a memory leak that is specific to forms that have buttons with media dataproviders. If one is displaying such a form and runs a method with saveData () the program hangs and will eventually give a Java out of memory error message. It is more obvious if you place about five such buttons on the same form. In our case if the button's dataproviders were null or when the buttons were replaced with fields associated with the media dataproviders the error went away.

John McCann
Servoy Developer
Version R2 2.2-build 328
Java version 1.4.2_08-b03 (Windows 2003)

[jcompagner]

i looked at the code and we are cleaning up the images that buttons are given.

How big are those images? (in dimensions and size on disk)

Can you give me an example where i can see this happening?

[idoctor]

Johan,

The image are 120x90 pixel thumbnails generated by Servoy and then stored prior to display. So they are very small.

John McCann

[gdurniak]

we just upgraded from Servoy 3 to 6.1, and found that saving large duplicated related found sets ( e.g. 14,000 Records ) now locks up the Smart Client:

related = fs[relatedFs];
for( i=1;i<=related.getSize();i++)
{
relatedOriginal = related.getRecord(i);
relatedDub = dup[relatedFs].getRecord(dup[relatedFs].newRecord(false,false));
databaseManager.copyMatchingFields( relatedOriginal, relatedDub);

// To fix this, I added another saveData inside the FOR Loop, to save the records in smaller batches:
if ( i%500 == 0 ) {
databaseManager.saveData();
}

}

databaseManager.saveData(); // used to be controller.savedata

greg

[jcompagner]

are you saying that if you save per 500 records it works fine?
Because yes that is a lot of data if you don't save it now and then. What happens if you give the smart client more memory?

[gdurniak]

This did work in Servoy 3 using controller.savedata

I made it work in Servoy 6.1 by saving more often ( every 500 records ). Otherwise, the Smart Client just hangs

I did increase the maximum heap to 2056, but it made no difference

as the original post asked, what does "savedata" actually do ? does servoy store up a bunch of SQL queries, then fire them off all at once ? or does it just mark the end of a transaction ?

greg

[jcompagner]

controller or databaseManager.saveData() is the same thing.
we just relocated it because you really don't save just that form/controllers data but everything.

I would definitely expect that you need to save more often when you create 14K records over many related foundsets.
It could be that we now do have to use more memory to keep all the tracking (for example calculations)

saveData() doesn't have directly anything to do with a database transaction. SaveData just sends all new or updated records that are in memory to the database (as insert/update statements yes)

So as long as you don't call saveData() all record that you create or update are all kept in memory.

[gdurniak]

It seems saveData is pretty important then.

I now see why no records were added when Smart Client locked up, even though it had already "created" the 14,000 records in memory

Are there any cases when Servoy would run "saveData" automatically, e.g. the end of a Method ?


> SaveData just sends all new or updated records that are in memory to the database (as insert/update statements yes)

So as long as you don't call saveData() all record that you create or update are all kept in memory.

[lwjwillemsen]

Strange that autoSave is never mentioned in this thread...

[gdurniak]

autoSave works fine: "as the user navigates the client session. Specific actions like clicking in a form's area, navigating to a different form, clicking a button, etc. all trigger a save event"

but using saveData in Methods in the background can be problematic. I had no clue, until it finally failed

greg