Parsing JSON

Questions, tips and tricks and techniques for scripting in Servoy

Parsing JSON

Postby antonio » Sat Mar 23, 2013 4:25 pm

I've been playing with the OAuth plugin on servoyforge to integrate with Xero. The plugin works nicely, but I'm struggling with the JSON that gets returned
With
Code: Select all
function OAuthXeroGET(client_id, secret) {
   try
   {
      plugins.oauth // initialise
      var oauth_api = OAuthAPI.XERO
      var resource_url = "https://api.xero.com/api.xro/2.0/Organisation"
      application.output(resource_url)
      var method = Methods.GET;
      var request = plugins.oauth.createOAuthRequest(oauth_api, client_id, secret, method);
      request.setResourceUrl(resource_url);
      request.execute();
      var response = request.getResponse();
      var json = response.getAttributeValue("Organisations")
      application.output(json)
      var vName =  json[0]['LegalName']
      var vPhone = json[0]['Phones'][0]['PhoneNumber']   
   }
   catch(e)
   {
      application.output("OAuth error " + e)
   }
}


I get this which looks like a JSON array,
Code: Select all
[{"Phones":[{"PhoneCountryCode":"","PhoneAreaCode":"","PhoneNumber":"","PhoneType":"DDI"},{"PhoneCountryCode":"","PhoneAreaCode":"","PhoneNumber":"","PhoneType":"FAX"},{"PhoneCountryCode":"","PhoneAreaCode":"","PhoneNumber":"","PhoneType":"MOBILE"},{"PhoneCountryCode":"","PhoneAreaCode":"","PhoneNumber":"","PhoneType":"DEFAULT"}],"TaxNumber":"53003086616","Addresses":[{"PostalCode":"1234","Region":"","AddressLine4":"","AddressType":"POBOX","AttentionTo":"","Country":"Mozambique","AddressLine1":"23 Main Street","City":"Marineville","AddressLine2":"","AddressLine3":""},{"PostalCode":"","Region":"","AddressLine4":"","AddressType":"STREET","AttentionTo":"","Country":"","AddressLine1":"","City":"","AddressLine2":"","AddressLine3":""}],"OrganisationType":"COMPANY","OrganisationStatus":"ACTIVE","RegistrationNumber":"11111111138","OrganisationEntityType":"COMPANY","PeriodLockDate":"/Date(1214740800000+1200)/","BaseCurrency":"AUD","LineOfBusiness":"","Name":"Demo Company (AU)","LegalName":"Demo Company (AU)","FinancialYearEndDay":30,"PaysTax":true,"Timezone":"AUSEASTERNSTANDARDTIME","IsDemoCompany":true,"CountryCode":"AU","ShortCode":"dB62e","CreatedDateUTC":"/Date(1363381036907+1300)/","Version":"AU","FinancialYearEndMonth":6}]

And I can read it with the JSON editor http://braincast.nl/samples/jsoneditor/
but I'm unable to extract anything in Servoy such as

var vName = json[0]['LegalName']

Java class "org.json.JSONArray" has no public instance field or method named "0".

I've tried several things including changing it to a JSON object which is also readable with the JSON editor, yet still gives an error in Servoy.

var jsonobj = json.getJSONObject(0)
var vName = jsonobj[0]['LegalName']

Java class "org.json.JSONObject" has no public instance field or method named "0".

Can anyone help me with the correct syntax?
Tony
Servoy 8 - 2022.03 LTS
antonio
 
Posts: 638
Joined: Sun Apr 02, 2006 2:14 am
Location: Australia

Re: Parsing JSON

Postby david » Sat Mar 23, 2013 6:41 pm

You need to parse the return string to a json object before proceeding. ie: var json = JSON.parse(response).
David Workman, Kabootit

Image
Everything you need to build great apps with Servoy
User avatar
david
 
Posts: 1727
Joined: Thu Apr 24, 2003 4:18 pm
Location: Washington, D.C.

Re: Parsing JSON

Postby antonio » Sat Mar 23, 2013 11:36 pm

david wrote:You need to parse the return string to a json object before proceeding. ie: var json = JSON.parse(response).

Is that a Servoy 6 feature? In 5.2.16 get
ReferenceError: "JSON" is not defined
and I can't find JSON in the Solution Explorer. The JSONTools plugin only has convertXMLtoJSONObject and convertXMLtoJSONString

Instead this worked
Code: Select all
var json = eval('(' + response + ')')

though I understand eval is potentially less secure than JSON as it can execute malicious script. I'm working with a trusted source (me!) so I can live with that.
Tony
Servoy 8 - 2022.03 LTS
antonio
 
Posts: 638
Joined: Sun Apr 02, 2006 2:14 am
Location: Australia

Re: Parsing JSON

Postby david » Sun Mar 24, 2013 12:51 am

Ah yea, 6 and above only. The eval approach is fine -- it was the old way of doing it before JSON was implemented natively in browsers (and Mozilla's Rhino project that Servoy uses).

You could also use plugins.serialize.fromJSON(string).
David Workman, Kabootit

Image
Everything you need to build great apps with Servoy
User avatar
david
 
Posts: 1727
Joined: Thu Apr 24, 2003 4:18 pm
Location: Washington, D.C.

Re: Parsing JSON

Postby ptalbot » Sun Mar 24, 2013 2:09 am

Or plugins.VelocityReport.fromJSON(string) that is working in 5.x as well...
Patrick Talbot
Freelance - Open Source - Servoy Valued Professional
https://www.servoyforge.net
Velocity rules! If you don't use it, you don't know what you're missing!
User avatar
ptalbot
 
Posts: 1654
Joined: Wed Mar 11, 2009 5:13 am
Location: Montreal, QC

Re: Parsing JSON

Postby mboegem » Sun Mar 24, 2013 9:32 am

ptalbot wrote:Or plugins.VelocityReport.fromJSON(string) that is working in 5.x as well...

and very good :D
Marc Boegem
Solutiative / JBS Group, Partner
• Servoy Certified Developer
• Servoy Valued Professional
• Freelance Developer

Image

Partner of Tower - The most powerful Git client for Mac and Windows
User avatar
mboegem
 
Posts: 1743
Joined: Sun Oct 14, 2007 1:34 pm
Location: Amsterdam

Re: Parsing JSON

Postby simonfroggatt76 » Wed Jul 24, 2013 12:16 pm

Hi Guys,

I'm using to use the oauth plugin for connecting to Xero....I've copied and pasted the code from the post above replacing with my details.
The problem I'm getting is that the line:
var request = plugins.oauth.createOAuthRequest(oauth_api, client_id, secret, method);
is returning null for the request variable...??

I've got it working with php where this requires you to give the path of the private and public keys that you need to create. (But not done this for servoy ??)
Am I missing something obvious here?

I've tried it on 5.2, 6 and 7.

Thanks in advance
Simon
simonfroggatt76
 
Posts: 2
Joined: Wed Jul 24, 2013 12:12 pm

Re: Parsing JSON

Postby antonio » Wed Jul 24, 2013 1:52 pm

Hi Simon,

Are you passing in something useful for secret? Instructions are here for generating it
http://developer.xero.com/code-samples/libraries/java/ and it should look like

Code: Select all
var secret = "MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALu6AsBb+xLiNaRA\n"+
         "4k0pCzm+52/jv1+vMTLn8HHN51bxzeYuZp1nSBTt3rfISalDJkKoT2pgJIPHD7t5\n"+
<snip>
         "/jRe7a1YEVGf+OOgw6W7dvzc6aDd15xZt0BjZgQaaqdkylZrPQWBlbk6hi9Bl4rv\n"+
         "KAoJypVxuHnLsg==";


This code works for me

Code: Select all
function OAuthXeroGET(secret, vEndpoint, vAttributeName) {
   try
   {
      plugins.oauth // initialise
      var oauth_api = OAuthAPI.XERO
      var client_id = "CVUBEGYVHxxxxxxxxJZS0XFJACF"; // from Xero https://api.xero.com/Application/List
      var resource_url = 'https://api.xero.com/api.xro/2.0/' + vEndpoint;
      var method = Methods.GET;
      var request = plugins.oauth.createOAuthRequest(oauth_api, client_id, secret, method);
      request.setResourceUrl(resource_url);
      request.execute();
      var response = request.getResponse();
      if (!response)
         return null
      var vAttribute = response.getAttributeValue(vAttributeName)
      var json = eval('(' + vAttribute + ')')
      return json
   }
   catch(e)
   {
      application.output("OAuth error " + e)
   }
}
Tony
Servoy 8 - 2022.03 LTS
antonio
 
Posts: 638
Joined: Sun Apr 02, 2006 2:14 am
Location: Australia

Re: Parsing JSON

Postby sbutler » Mon Jul 29, 2013 6:24 pm

You really shouldn't be using eval, especially with a value returned from a webservice. Image if the value returned was "foundset.deleteAllRecords()". :) You should parse it by using the json2.js examples here: http://json.org/ under JavaScript. Just takes a few modifications to get it into Servoy.
Scott Butler
iTech Professionals, Inc.
SAN Partner

Servoy Consulting & Development
Servoy University- Training Videos
Servoy Components- Plugins, Beans, and Web Components
Servoy Guy- Tips & Resources
ServoyForge- Open Source Components
User avatar
sbutler
Servoy Expert
 
Posts: 759
Joined: Sun Jan 08, 2006 7:15 am
Location: Cincinnati, OH

Re: Parsing JSON

Postby m.vanklink » Tue Jul 30, 2013 4:34 pm

json2.js also uses eval, however it checks beforehand whether it is safe to call it.

When we expect an object would it be save to do the following simple test, or could it then still be possible to insert malicious code?

Code: Select all
if (_jsonstring && _jsonstring[0] == '{' && _jsonstring[_jsonstring.length-1] == '}') {
  _obj = eval('('+_jsonstring+')')
}
Michel van Klink
Vision Development
m.vanklink
 
Posts: 70
Joined: Thu Feb 23, 2012 9:15 am
Location: The Netherlands

Re: Parsing JSON

Postby sbutler » Tue Jul 30, 2013 4:48 pm

The json2.js version is safe. It checks what its about to eval and only allows for certain characters to be used in the eval.
Why not just use that? I've used it for MANY webservices with no problems. Here is a sample...

Code: Select all
function JSON_get() {
   
//http://www.JSON.org/json2.js

if (!this.JSON) {
    this.JSON = {};
}
else{
   return this.JSON //don't rebuild it
}

(function () {

    function f(n) {
        // Format integers to have at least two digits.
        return n < 10 ? '0' + n : n;
    }

    if (typeof Date.prototype.toJSON !== 'function') {

        Date.prototype.toJSON = function (key) {

            return isFinite(this.valueOf()) ?
                   this.getUTCFullYear()   + '-' +
                 f(this.getUTCMonth() + 1) + '-' +
                 f(this.getUTCDate())      + 'T' +
                 f(this.getUTCHours())     + ':' +
                 f(this.getUTCMinutes())   + ':' +
                 f(this.getUTCSeconds())   + 'Z' : null;
        };

        String.prototype.toJSON =
        Number.prototype.toJSON =
        Boolean.prototype.toJSON = function (key) {
            return this.valueOf();
        };
    }

    var cx = /[\u0000\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g,
        escapable = /[\\\"\x00-\x1f\x7f-\x9f\u00ad\u0600-\u0604\u070f\u17b4\u17b5\u200c-\u200f\u2028-\u202f\u2060-\u206f\ufeff\ufff0-\uffff]/g,
        gap,
        indent,
        meta = {    // table of character substitutions
            '\b': '\\b',
            '\t': '\\t',
            '\n': '\\n',
            '\f': '\\f',
            '\r': '\\r',
            '"' : '\\"',
            '\\': '\\\\'
        },
        rep;


    function quote(string) {

// If the string contains no control characters, no quote characters, and no
// backslash characters, then we can safely slap some quotes around it.
// Otherwise we must also replace the offending characters with safe escape
// sequences.

        escapable.lastIndex = 0;
        return escapable.test(string) ?
            '"' + string.replace(escapable, function (a) {
                var c = meta[a];
                return typeof c === 'string' ? c :
                    '\\u' + ('0000' + a.charCodeAt(0).toString(16)).slice(-4);
            }) + '"' :
            '"' + string + '"';
    }


    function str(key, holder) {

// Produce a string from holder[key].

        var i,          // The loop counter.
            k,          // The member key.
            v,          // The member value.
            length,
            mind = gap,
            partial,
            value = holder[key];

// If the value has a toJSON method, call it to obtain a replacement value.

        if (value && typeof value === 'object' &&
                typeof value.toJSON === 'function') {
            value = value.toJSON(key);
        }

// If we were called with a replacer function, then call the replacer to
// obtain a replacement value.

        if (typeof rep === 'function') {
            value = rep.call(holder, key, value);
        }

// What happens next depends on the value's type.

        switch (typeof value) {
        case 'string':
            return quote(value);

        case 'number':

// JSON numbers must be finite. Encode non-finite numbers as null.

            return isFinite(value) ? String(value) : 'null';

        case 'boolean':
        case 'null':

// If the value is a boolean or null, convert it to a string. Note:
// typeof null does not produce 'null'. The case is included here in
// the remote chance that this gets fixed someday.

            return String(value);

// If the type is 'object', we might be dealing with an object or an array or
// null.

        case 'object':

// Due to a specification blunder in ECMAScript, typeof null is 'object',
// so watch out for that case.

            if (!value) {
                return 'null';
            }

// Make an array to hold the partial results of stringifying this object value.

            gap += indent;
            partial = [];

// Is the value an array?

            if (Object.prototype.toString.apply(value) === '[object Array]') {

// The value is an array. Stringify every element. Use null as a placeholder
// for non-JSON values.

                length = value.length;
                for (i = 0; i < length; i += 1) {
                    partial[i] = str(i, value) || 'null';
                }

// Join all of the elements together, separated with commas, and wrap them in
// brackets.

                v = partial.length === 0 ? '[]' :
                    gap ? '[\n' + gap +
                            partial.join(',\n' + gap) + '\n' +
                                mind + ']' :
                          '[' + partial.join(',') + ']';
                gap = mind;
                return v;
            }

// If the replacer is an array, use it to select the members to be stringified.

            if (rep && typeof rep === 'object') {
                length = rep.length;
                for (i = 0; i < length; i += 1) {
                    k = rep[i];
                    if (typeof k === 'string') {
                        v = str(k, value);
                        if (v) {
                            partial.push(quote(k) + (gap ? ': ' : ':') + v);
                        }
                    }
                }
            } else {

// Otherwise, iterate through all of the keys in the object.

                for (k in value) {
                    if (Object.hasOwnProperty.call(value, k)) {
                        v = str(k, value);
                        if (v) {
                            partial.push(quote(k) + (gap ? ': ' : ':') + v);
                        }
                    }
                }
            }

// Join all of the member texts together, separated with commas,
// and wrap them in braces.

            v = partial.length === 0 ? '{}' :
                gap ? '{\n' + gap + partial.join(',\n' + gap) + '\n' +
                        mind + '}' : '{' + partial.join(',') + '}';
            gap = mind;
            return v;
        }
    }

// If the JSON object does not yet have a stringify method, give it one.

    if (typeof JSON.stringify !== 'function') {
        JSON.stringify = function (value, replacer, space) {

// The stringify method takes a value and an optional replacer, and an optional
// space parameter, and returns a JSON text. The replacer can be a function
// that can replace values, or an array of strings that will select the keys.
// A default replacer method can be provided. Use of the space parameter can
// produce text that is more easily readable.

            var i;
            gap = '';
            indent = '';

// If the space parameter is a number, make an indent string containing that
// many spaces.

            if (typeof space === 'number') {
                for (i = 0; i < space; i += 1) {
                    indent += ' ';
                }

// If the space parameter is a string, it will be used as the indent string.

            } else if (typeof space === 'string') {
                indent = space;
            }

// If there is a replacer, it must be a function or an array.
// Otherwise, throw an error.

            rep = replacer;
            if (replacer && typeof replacer !== 'function' &&
                    (typeof replacer !== 'object' ||
                     typeof replacer.length !== 'number')) {
                throw new Error('JSON.stringify');
            }

// Make a fake root object containing our value under the key of ''.
// Return the result of stringifying the value.

            return str('', {'': value});
        };
    }


// If the JSON object does not yet have a parse method, give it one.

    if (typeof JSON.parse !== 'function') {
        JSON.parse = function (text, reviver) {

// The parse method takes a text and an optional reviver function, and returns
// a JavaScript value if the text is a valid JSON text.

            var j;

            function walk(holder, key) {

// The walk method is used to recursively walk the resulting structure so
// that modifications can be made.

                var k, v, value = holder[key];
                if (value && typeof value === 'object') {
                    for (k in value) {
                        if (Object.hasOwnProperty.call(value, k)) {
                            v = walk(value, k);
                            if (v !== undefined) {
                                value[k] = v;
                            } else {
                                delete value[k];
                            }
                        }
                    }
                }
                return reviver.call(holder, key, value);
            }


// Parsing happens in four stages. In the first stage, we replace certain
// Unicode characters with escape sequences. JavaScript handles many characters
// incorrectly, either silently deleting them, or treating them as line endings.

            cx.lastIndex = 0;
            if (cx.test(text)) {
                text = text.replace(cx, function (a) {
                    return '\\u' +
                        ('0000' + a.charCodeAt(0).toString(16)).slice(-4);
                });
            }

// In the second stage, we run the text against regular expressions that look
// for non-JSON patterns. We are especially concerned with '()' and 'new'
// because they can cause invocation, and '=' because it can cause mutation.
// But just to be safe, we want to reject all unexpected forms.

// We split the second stage into 4 regexp operations in order to work around
// crippling inefficiencies in IE's and Safari's regexp engines. First we
// replace the JSON backslash pairs with '@' (a non-JSON character). Second, we
// replace all simple value tokens with ']' characters. Third, we delete all
// open brackets that follow a colon or comma or that begin the text. Finally,
// we look to see that the remaining characters are only whitespace or ']' or
// ',' or ':' or '{' or '}'. If that is so, then the text is safe for eval.

            if (/^[\],:{}\s]*$/.
test(text.replace(/\\(?:["\\\/bfnrt]|u[0-9a-fA-F]{4})/g, '@').
replace(/"[^"\\\n\r]*"|true|false|null|-?\d+(?:\.\d*)?(?:[eE][+\-]?\d+)?/g, ']').
replace(/(?:^|:|,)(?:\s*\[)+/g, ''))) {

// In the third stage we use the eval function to compile the text into a
// JavaScript structure. The '{' operator is subject to a syntactic ambiguity
// in JavaScript: it can begin a block or an object literal. We wrap the text
// in parens to eliminate the ambiguity.

                j = eval('(' + text + ')');

// In the optional fourth stage, we recursively walk the new structure, passing
// each name/value pair to a reviver function for possible transformation.

                return typeof reviver === 'function' ?
                    walk({'': j}, '') : j;
            }

// If the text is not JSON parseable, then a SyntaxError is thrown.

            throw new SyntaxError('JSON.parse');
        };
    }
}());

return JSON


}

/**
* @properties={typeid:24,uuid:"8BF4B5D9-DB9B-4EA3-B311-191366714FFE"}
*/
function JSON_stringify(_value) {
   return JSON_get().stringify(_value)
}

/**
* @properties={typeid:24,uuid:"BABC831B-B09B-4B52-8C37-03EE25C9AAF2"}
*/
function JSON_parse(_value) {
   return JSON_get().parse(_value)
}


Then later to use it:
Code: Select all
var json = JSON_parse(jsonString)


I've used it since Servoy 4 so you could updated it to 6 and put in its own scope, but it works.
Scott Butler
iTech Professionals, Inc.
SAN Partner

Servoy Consulting & Development
Servoy University- Training Videos
Servoy Components- Plugins, Beans, and Web Components
Servoy Guy- Tips & Resources
ServoyForge- Open Source Components
User avatar
sbutler
Servoy Expert
 
Posts: 759
Joined: Sun Jan 08, 2006 7:15 am
Location: Cincinnati, OH

Re: Parsing JSON

Postby simonfroggatt76 » Mon Sep 02, 2013 4:26 pm

Hi,

Sorry I never replied...I'm been off working on a different project and only dropped back onto this today.

Perfect! Thank you so much, I've tried it and that works exactly as I hoped it would.

Thanks
Simon

antonio wrote:Hi Simon,

Are you passing in something useful for secret? Instructions are here for generating it
http://developer.xero.com/code-samples/libraries/java/ and it should look like

Code: Select all
var secret = "MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALu6AsBb+xLiNaRA\n"+
         "4k0pCzm+52/jv1+vMTLn8HHN51bxzeYuZp1nSBTt3rfISalDJkKoT2pgJIPHD7t5\n"+
<snip>
         "/jRe7a1YEVGf+OOgw6W7dvzc6aDd15xZt0BjZgQaaqdkylZrPQWBlbk6hi9Bl4rv\n"+
         "KAoJypVxuHnLsg==";


This code works for me

Code: Select all
function OAuthXeroGET(secret, vEndpoint, vAttributeName) {
   try
   {
      plugins.oauth // initialise
      var oauth_api = OAuthAPI.XERO
      var client_id = "CVUBEGYVHxxxxxxxxJZS0XFJACF"; // from Xero https://api.xero.com/Application/List
      var resource_url = 'https://api.xero.com/api.xro/2.0/' + vEndpoint;
      var method = Methods.GET;
      var request = plugins.oauth.createOAuthRequest(oauth_api, client_id, secret, method);
      request.setResourceUrl(resource_url);
      request.execute();
      var response = request.getResponse();
      if (!response)
         return null
      var vAttribute = response.getAttributeValue(vAttributeName)
      var json = eval('(' + vAttribute + ')')
      return json
   }
   catch(e)
   {
      application.output("OAuth error " + e)
   }
}
simonfroggatt76
 
Posts: 2
Joined: Wed Jul 24, 2013 12:12 pm

Re: Parsing JSON

Postby d.pearce1417196993 » Wed Dec 24, 2014 5:43 pm

SImon,

I know this is a year later. Are you still integrating with XERO?

I am looking to do this, but it appears that the example you have appears depracted.

Its all a bit over my head, but I created an application (Public) in XERO and got a list of URLs and keys, but thats about as far as I can get!

Do you have any latest code snippets for initial login and perhaps then pasring one account and invoice to XERO. Would be really helpful.

Happy Xmas

David
d.pearce1417196993
 
Posts: 22
Joined: Fri Nov 28, 2014 7:49 pm


Return to Methods

Who is online

Users browsing this forum: No registered users and 11 guests