Problems loading files .pdf of .indd (>1MB) in to MySQL

Classic Servoy
1

Hi there,

I’m a newby on Servoy but learning by doing, this brings me to where I’m now. At this point I have the following problem;
My Servoy app has to store larger files, one PDF (> 1MB) and one Adobe InDesign docs (>3 MB) per record. I’ve created al the appropiate fields and a method to import the two docs and put some extra info concerning these two files in other fields.

When I run the method all the ‘normal’ fields are filled with data but after 10-15 seconds I got an Error message: ‘Error saving 1 records’ (the details button is disabled).

The method I use is:

var success = bestelling_to_files.newRecord(true);

var bestelling_fileID = klant_id + '_' + id + '_' + utils.dateFormat(timestamp_besteld,'yyyyMMddHHmmss');
var globalpath = '/Users/Shared/MyMenucards/bestellingen/';
var filepathPDF = globalpath + 'pdf/' + forms.bestelling.klant_id + '/' + bestelling_fileID + '.pdf';
var filepathINDD = globalpath + 'indd_files/' + forms.bestelling.klant_id + '/' + bestelling_fileID + '.indd';

if(filepathPDF)
{
bestelling_to_files.pdf_file = plugins.file.readFile(filepathPDF)
bestelling_to_files.filepath_pdf = filepathPDF
bestelling_to_files.pdf_name = bestelling_fileID + '.pdf'
bestelling_to_files.pdf_size = plugins.file.getFileSize(filepathPDF);
}

if(filepathINDD)
{
bestelling_to_files.indd_file = application.readFile(filepathINDD)
bestelling_to_files.filepath_indd = filepathINDD
bestelling_to_files.indd_name = bestelling_fileID + '.indd'
bestelling_to_files.indd_size = plugins.file.getFileSize(filepathINDD)
}

The back end db is MySQL on Mac OSX.

Anyone familiar with this problem, or is it just me? :(

2

RMKortman:
The back end db is MySQL on Mac OSX.

Anyone familiar with this problem, or is it just me? :(

Maybe you’re trying to upload a packet too big for the backend, take a look at MySQL my.cnf file, look for this option and set it big enough for the kind of files you’re going to upload:

max_allowed_packet = 5M

Hope it helps.

3

Cybersack R1.2.9.1 will be released on Monday 5th of June.
It supports data streaming directly from the UI.
This will allow you to push whatever you like to the server and not suffer any timeout or memory issues when inserting media to your database records. You can even store data on the server filesystem if you prefer.

I will post the download details to the Plugin and Beans list as soon as I have pushed it up to the server.

4

Sounds great

5

Problem is likely the max packet size of MySQL. Check these postings:

http://forum.servoy.com/viewtopic.php?p=30641

Best,

Rich Coulombre

6

Hi Rich.

I don’t think the problem has anything to do with the database configuration.
I believe the issue is either a socket failure during a read-from or a write-to the database.
The database has allocated sufficient storage capacity for the media records.
The data had been reported (by user dpward) to be present.
The media cells are

MySQL MEDIUMBLOB (maximum size: 16,277,215 bytes).

So, that fact and given that the user reports an error after a 10-15 second interval suggest to me that it is the data transport that’s failing, not the database itself.

Now, the max-packet-size setting is important for the data transfer between servoy server and the database (and ultimately in or out of the client), and could impact any potential writes if packets greater than the specified size are sent across the socket binding the server to the database.

Now, quoting MySql.org

the maximum number of bytes in a packet for the client

is what max_packet_size is all about. I can’t imagine that the servoy server would even attempt to send a packet of that size. So, maybe it has something to do with a timeout, or even a disk-full ?

7

FYI I had the same problem a few months ago… uploading a blob over 1 meg to MySQL caused an error. Increasing the packet size in MySQL fixed it. A simple edit of the .cnf file should tell you right away if it works or not.

Regarding whether Servoy server would send it if it was not OK… Servoy Server really does not know how we have configured our backend database. It will dutifully send a SQL statement and the blob image to the backend database, and assume that we have configured things properly for the user submission. It is up to us as DBA’s to make sure the server is properly configured for the types of data that our users submit.

Cheers,

Rich