Hi,
i want to import data from an xml file and im not beign able to import the data.
Im trying to use the XMLreader plugin but im not getting how to use the nodes it returns.
Can anyone give me some explanation of how to do this??
thanks in advance
I’d use the JavaScript XML implementation (see XML and XMLList under JS Lib): more powerfull.
Just read the file using the file plugin, create an XML object form the contents of the file (assuming it’s valid XML) and process it to your liking.
To create an XMl object from a string: var x = new XML(‘’)
Note that if your XML file starts with something like “<?xml version="1.0" encoding="UTF-8"?>”, you need to first strip it off, because the JavaSCript XMl implementation doesn’t consider that valid XML.
Paul
pbakker:
Note that if your XML file starts with something like “<?xml version="1.0" encoding="UTF-8"?>”, you need to first strip it off, because the JavaSCript XMl implementation doesn’t consider that valid XML.
which can be easily done using replace like this.
var $xml = myXmlString;
$xml = $xml.replace(/<\/?[^>]+(>|$)/g, function (strMatch, p1){return (/xml version/.test(strMatch)) ? '' : strMatch})
I too had to import data from an xmL file recently. I tried to use the XML plugin (i did not know the Javascript XML implementation) but it is confusing and isn’t flexible enough (by the way: why having a XML plugin in Servoy when JavaScript can be used instead?). Anyway, if you want to see an example, I posted some code here http://www.servoy.com/forum/viewtopic.php?f=4&t=14501. The original file was a list of ISO regional and country codes.
The XML plugin was there before the XML implementation in JavaScript was there.
Paul
Paul - any chance we could get some 'How To’s on reading and writing XML using both methods (Plugin and Java) - in addition to the Wiki refrences?
If you google for “e4x”, which is the official name for the JavaScript XML implementation, you’ll find quite a bit of information.
Paul
pbakker:
If you google for “e4x”, which is the official name for the JavaScript XML implementation, you’ll find quite a bit of information.Paul
Cheers M8y
I’ve been trying to get the content of the attached zipfile imported as XML with E4X.
I imported this as textfile (utf-8 encodeing) and then tried to get rid of the xml-header.
But in some way I just can’t seem to get it working…
Probably I’m missing a small step, but what… (it shouldn’t be so hard )
Any advise?
sales_order.zip (403 Bytes)
Hi Marc,
first when you create files with UTF-8, always make sure your editor doesn’t output UTF BOM, because most of the time Java doesn’t like it…
UTF Byte Order Mark are the 3 first bytes inserted in UTF-8 files so that they are recognized as such (EF BB BF). You can open you file with an HexaDecimal editor and you will see that they are present in your xml file (although invisible).
Most text/xml editors will offer an option to strip these bytes, unfortunately NotePad on Windows doesn’t, so use a real editor
That being said, as you have guessed, you need to strip the xml declaration from your file.
If you do so, you shouldn’t have any problem using it with E4X in Servoy…
I just tried it, using a simple test method like this:
function testE4X() {
var file = plugins.file.showFileOpenDialog(1, null, false, ['XML files', 'xml'], null, 'Open XML File');
if (file) {
var txt = plugins.file.readTXTFile(file);
if (txt) {
txt = utils.stringTrim(txt.substring(txt.indexOf('\n')+1));
var xml = new XML(txt);
application.output(xml.salesorder);
}
}
}
Works fine here!
Hi Patrick,
As I said… we were close to the key: the trim function was the actual key to get it working…
We use kind of the same code you posted, except that we don’t use ‘\n’ to strip the header, as it might be a file consisting of just 1 line…
Thnkx for your help, much appreciated!
mboegem:
pbakker:
Note that if your XML file starts with something like “<?xml version="1.0" encoding="UTF-8"?>”, you need to first strip it off, because the JavaSCript XMl implementation doesn’t consider that valid XML.which can be easily done using replace like this.
var $xml = myXmlString;
$xml = $xml.replace(/</?[^>]+(>|$)/g, function (strMatch, p1){return (/xml version/.test(strMatch)) ? ‘’ : strMatch})
Hi Marc,
Did you come encounter any slowdown if the string is big(e.g. length = 3586241)?
I don’t think I ran into numbers of this size, so I can’t really tell.
It wouldn’t surprise me, as this function will scan the complete XML for the matching pattern.
Maybe you can look for a way that it will stop scanning as soon as it finds the first match.
I feel a slight breeze. Must be Monday.
Hi guys,
I am using xml files to backup data in our application. Basically I backup each table into an xml file and restoring them by reading the xml files and inserting data into the tables. It works fine for everything except for our documents table which usually contains data > 50Mb. When I try to restore data, I read the xml file belong to each table into an string by using it2be file plugin and then I make an xml object of that string using:
var newXml = new XML(myXmlFileString);
When the program wants to make an object of a string of that size (50MB) it throws a java heap stack memory exception! I was wondering if there is any size limitation for the xml object? If so, how can I deal with this problem?
Cheers,
Siavash
50MB is pretty big. You can try playing around with the maxClientHeap setting in Servoy Admin.
Note that Servoy’s XML is a DOM based API. The advantage is that you can randomly access anything in the xml object. The disadvantage is that you need to read the entire xml document in memory first.
When using xml to backup and restore data, typically a streaming xml api implementation is used: http://en.wikipedia.org/wiki/StAX. There are a number of Java implementations out there. With a little work you could roll one of them up into a Servoy plugin.
But I would think it would be much easier to use your SQL vendor’s backup and restore tools for this kind of job (no logic needed in the archive or restore process). If you need to automate from within Servoy there is always application.executeProgram(). Trigger via the headless client plugin or batch processor so it runs on the server.
Thanks for your reply david. Changing maxClientHeap size is not really a good option because the table data might of any size and in some point later I’ll face the memory issue again when the data size gets bigger and bigger. As for the direct SQL backup/restore, I can’t use that method as I need to read the content of the backup when I’m restoring the data (the backup might be an old backup and to restore that backup I need to manipulate the data to match the current structure of the database and restore it that way) and even if I can manipulate it, I might face this memory issue again.
writing a plugin is a good option but I’m not sure how much work is it for me to do it so I was thinking of checking the file size during the backup procedure and don’t let it get bigger than an acceptable amount of size (let’s say 500K which is easy to load in RAM) and basically instead of storing the whole table data in one xml file, I break it down to a few, this way reading the files are not going to be an issue (Hopefully!). I’m going to read some documents about xml streaming API’s implemented in java to see if it is going to be easy for me to write a servoy plugin using one of those implementations.
Example code that might help:
function xmlStreamTest() {
importPackage(javax.xml.stream)
importPackage(java.io)
// input
var xml = <xml>
<fruit>
<title>orange</title>
<color>orangish</color>
<taste>tangy</taste>
</fruit>
<fruit>
<title>apple</title>
<color>red</color>
<taste>refreshing</taste>
</fruit>
<fruit>
<title>banana</title>
<color>yellow</color>
<taste>energizing</taste>
</fruit>
</xml>
// output
var jsonFruit = []
// pseudo file
var tempFile = plugins.file.createTempFile('xmlFile', '.xml');
plugins.file.writeTXTFile(tempFile, xml);
// initialize XMLStreamReader
var factory = XMLInputFactory.newInstance()
var reader = factory.createXMLStreamReader('testFile', new FileInputStream(tempFile.getAbsolutePath()))
// process xml and transform to json object
var fruitItem = new Object
while ( reader.hasNext() ) {
reader.next();
try {
if ( reader.isStartElement() ) {
var element = reader.getLocalName()
switch (element) {
case "title":
fruitItem["title"] = reader.getElementText()
break;
case "color":
fruitItem["color"] = reader.getElementText()
break;
case "taste":
fruitItem["taste"] = reader.getElementText()
break;
}
}
else if ( reader.isEndElement() && reader.getLocalName() == "fruit" ) {
jsonFruit.push(fruitItem)
fruitItem = new Object
}
} catch (e) {
application.output(e.message)
}
}
}
I’m not sure but doing Java like this in Servoy…probably need to explicitly close stuff down when you’re done. A point for the Java gurus to weigh in on.
Yes, as-is your code will leave the stream opened, and lock the file, so you want to add a finally clause with a reader.close();
Thanks David, I have temporarily solved the issue by braking down the xml files into smaller size within the code, so when I read them in the memory they are not going to be big. But I’ll give your solution a try soon.