I have a table called Media where I used to store all my media contents.
The system was working fast until I only used to store pictures (<500Kb).
Yesterday, I loaded a 32MB wmv file. After that, even though I did not place the media field on the form the table loads slowly (takes around 15 secs) when that record is accessed first time.
I searched for that record (the media field is not placed) and the same problem.
I also tried to access some other text field values in the media table using relations from other table and it still slow.
So, I don’t know why it loads slow even though the field is not accessed. I have planned to put heavy files in the table. But I’m worried because of this issue
My plugin cannot intercept the data-load calls made within Servoy.
Sorry for the confusion.
On the topic, I can only assume that Servoy loads all media data for some (initially) reasonable asumptions - such as the notion that all media is of type ‘image’ and hence should be loaded bc the view expects it.
Secondly, record-caching is not trivial, and I guess that this is another primary reason that full record data is loaded.
I’m sure they’ll look at a change in this behviour - maybe even a configurable option.
I too have noticed the delay - and such delays when manipulating the data made me decide to write a data stream plugin
Somehow even if we don’t ask for it the database starts streaming the data for a record even if we don’t open a stream.
If you are storing large blobs, please store them in a seperate table. Where you point to with a relation.
Then the blob wil only loaded when you load the relation.
ahmad:
…After that, even though I did not place the media field on the form the table loads slowly (takes around 15 secs) when that record is accessed first time…
We did include the blob/media column in the select , it seems the drivers do implement the loading different, some do retrieve the blob data at once some do only when the data is required from the jdb result.
In Servoy 2.2rc3 we not longer select the column so no driver should request the data from the db any longer, which does speed up the record retrieve.
jcompagner:
If you are storing large blobs, please store them in a seperate table. Where you point to with a relation.
Then the blob wil only loaded when you load the relation.
Yes.. I’m stroing all the media is a seperate table called media.
For eg: I have a gallery table where the inforamation about the media is stored. The actual media is stored in the media table.
I have a field called extension in media table. When the user browses the gallery record, using gallery_id_to_media relation I’m first checking what the extension is in a calculation field. If the extension is jpg or gif etc.. then I include the media content in the calculation if not show a different icon..
Jan Blok:
In Servoy 2.2rc3 we not longer select the column so no driver should request the data from the db any longer, which does speed up the record retrieve.
Somehow even if we don’t ask for it the database starts streaming the data for a record even if we don’t open a stream.
If you are storing large blobs, please store them in a seperate table. Where you point to with a relation.
Then the blob wil only loaded when you load the relation.
Which databases do not stream all the data in a record, but only what Servoy requests?
My company builds mostly image related Servoy solutions. One solution contains 50.000 records. When the blob columns were in the same table as the rest of the data, the client would get very quick disconnected from the server when browsing through the records, as it runs out of memory.
I moved the blob data to another table and displayed a comping image from a relation on the form. It would still run out of memory, but not as fast as in the previous solution.
Now, the form displays only a thumbnail image(this was the only solving of the ‘out of memory’ problem), but the customer is not satisfied with this, since he wants to see the image in more detail.
Does Servoy cache also the related blob data? I understood it gets it only at request. But when you go to the next record and so on, does the memory get cleaned by the previous media data cached in it?
Also, when you show a form, I see that a select statement is performed, and that statement selects all the columns in the table. Are you sure that caching all the columns is a driver/database related problem?
If you can’t find a simple solution, I’m sure I could provide you with a bean that would load the full image, by streaming, on request.
I haven’t thought thru the detail, but I am basically thinking that the user might click a button in the bean to load the detail image.
The view would be provided by my plugin and the data would be written to a temporary location on the client hdd.
I’m not sure that this would satisfy your need, but if you request the data this way (ie. not as part of the form itself), then you should not suffer out-of-memory conditions.
Yes, i meant that the image would be displayed on the form, because the bean is placed directly on the form. However, the data-load would be entirely under your control. I think it is possible for me to do this, but the trigger to load the data becomes the issue. I suggested a button; it may be some other programmatic event (???) such as “onRecordLoadComplete()” - if such an event exists in the servoy api.
The point is that the image data are not actually part of the servoy form.
The data exist in a temporary location following the request to stream the data out.
The remaining issue then becomes how much memory is utilized by java.awt.Image to display the data we have just streamd out.
It is clearer now; it can be triggered onRecordSelection for example.
Just one question: what happens with the memory when you go to the next record? Are you using Image.flush(), which “flushes all resources being used by this Image object”? If so, when?
I assume you can also make flush() as a public method, but when do you suggest it may be called?
I will have to check what my bean implementation does (or did, rather).
Initially, for the MediaManager plugin, I was using a bean on the form but for the intended purposes of playback, import, export, comparison etc…the best option was to run with separate popped-up ui’s. Hence, the MediaManager plugin functions do not use a bean on a form (as originally I had intended).
However, in your case, this bean-in-the-form is precisely what is called for. So, in this case the behaviour would be different because the instance of the bean is the same across the form (irrespective of the record on which your user is resting)…so, traversing from one record to another would automatically invoke a “flush” style of function.
That way there would never be a risk of showing a previous record image in the next record’s form view.
So, no caching would occur and the off-screen buffer would be purged immediately…I think