Text file import - why doesn't this code work

I looked around the forum but haven’t figured this out. When the user selects a menu item, the browser’s select/upload dialog comes up. The code appears to run, and I get my error message, “Could not find the header information. Please check file type and try again.” But I cannot trace it (it wont stop at the break), and it won’t accept application.output(), either. What’s missing?

/**
 * @properties={typeid:24,uuid:"92F7C151-14B9-4F13-A01A-8FF8BB311D00"}
 */
function browse_import() {
	//for the web you have to give a callback function that has a JSFile array as its first argument (also works in smart), only multi select and the title are used in the webclient, others are ignored
	plugins.file.showFileOpenDialog(null, null, false, myCallbackMethod, 'Select the import file')
} 

/**
 * @param {Array} callbackArray
 * 
 * @properties={typeid:24,uuid:"BBDBB0B0-5B76-415C-975D-9A6DD0EEFFED"}
 */
function myCallbackMethod(callbackArray) {
	var tempText = "";
	var headerText = "";
	var importFileText = "";
	var position = 0;
	var endOfRec = String.fromCharCode(251);
	var fieldSep = String.fromCharCode(252);
	
	// Read content from a text file selected from the file open dialog.
	importFileText = callbackArray[0].getBytes();
	application.output("Test of output:");
	if(importFileText) {		
		application.output(importFileText);
		position = importFileText.indexOf(endOfRec,0);
		if(position > 0) {
			headerText = importFileText.substr(0, endOfRec);
			application.output(headerText);
		} else {
			tempText = "Could not find the header information. Please check file type and try again. ";
			globals.UTIL_alertDlg(tempText,"Check import file type...",null);
		}
	} else {
		tempText = "Could not extract any text from this file (cannot import)."
		globals.UTIL_alertDlg(tempText,"Cannot read file...",null);
	} // (importFileText)
}

This code is in a form.js file.

Thank you,
Don

Think I got it to work…

importFileText = plugins.file.readTXTFile(callbackArray[0]);

…is this the correct way to do it? What happens if the text file happens to be 40-50 MB in size, is that still okay?

Thanks,
Don

Hi Don,

The plugin will load the whole file into RAM first before you get it so with large files you want to read it line by line.
I tend to use the following code:

var _oFile = plugins.file.showFileOpenDialog(1);

if (!_oFile) {
    return;
}

// Use BufferedReader so we don't have to read the whole file into memory
// And we make sure it reads it as UTF8
var _oFR = new Packages.java.io.FileInputStream(_oFile),
    _oIR = new Packages.java.io.InputStreamReader(_oFR, "UTF8"),
    _oBR = new Packages.java.io.BufferedReader(_oIR),
    _sLine = "dummy",
    _nReadLine = 0;

// using a transaction will speed things up when inserting data (on PostgreSQL for example)
databaseManager.startTransaction(); 

try {
    while (_sLine) {
        _sLine = _oBR.readLine();
        _nReadLine++;

        if (_sLine) {


            // -----------
            // Put your code to process the data here


            // -----------
                                
            if (_nReadLine % 50 == 0) {
                // save every 50 records
                if (!databaseManager.saveData()) {
                    throw "Can't save data!";
                }
            }
        }
    }
    if (!databaseManager.saveData()) {
        throw "Can't save data!";
    }

    if ( !databaseManager.commitTransaction() ) {
        throw "Can't commit the data!";
    }

} catch (_oErr) {
    application.output("ERROR: " + _oFile.getName() + " at row " + _nReadLine, LOGGINGLEVEL.ERROR);
    application.output("ERROR: " + _oErr, LOGGINGLEVEL.ERROR);
    databaseManager.rollbackEditedRecords();
    databaseManager.rollbackTransaction();
} finally {
    //do NOT forget this close! to prevent memory leaks
    _oBR.close();
    // garbage collection (for good measure)
    _oFR = null;
    _oIR = null;
    _oBR = null;
}

Hope this helps.

Hi Robert,

Am I correct that this code is set up for Smart Client? I would like to be able to use it with the web client. For web client, would it need to be transferred to disk on the server, and in this case is it in memory already?

Thank you,
Don

Hi Don,

djlapin:
Am I correct that this code is set up for Smart Client? I would like to be able to use it with the web client. For web client, would it need to be transferred to disk on the server, and in this case is it in memory already?

The code works locally yes, so it works with smartclient and headless/webclient (on the server only).
When you upload a file with webclient then it is indeed already in memory or in cache (temp dir) (see viewtopic.php?t=16160)