plugins.VelocityReport.fromJSON fails on UTF-8 file

Questions and answers on developing, deploying and using plugins and JavaBeans

plugins.VelocityReport.fromJSON fails on UTF-8 file

Postby Bernd.N » Thu Feb 15, 2018 2:22 pm

We created an interface where we get a JSON file from SAP SuccessFactors EmployeeCentral when employee data got changed there.

The first lines of such a JSON look like this:
Code: Select all
{
  "S136" : {
    "e_all_in_contract": null,
    "e_cost_center": "10 3800",
...

We read the JSON into an object variable with
Code: Select all
oEC_EIM_persons = plugins.VelocityReport.fromJSON( plugins.file.readTXTFile( jsFile.getAbsolutePath() ));


Everything works fine when the JSON is ANSI coded.
However when I switch the JSON to UTF-8 coded, Servoy prompts this strange error:
JAVASCRIPT ERROR
net.stuff.plugin.velocityreport.org.json.JSONException: Expected a ':' after a key at 3 [character 4 line 1]


Does someone know why?
I also tried to give plugins.file.readTXTFile() the second argument "UTF-8" as codepage, but that did not change anything.
Bernd Korthaus
LinkedIn
Servoy 7.4.9 SC postgreSQL 9.4.11 Windows 10 Pro
User avatar
Bernd.N
 
Posts: 497
Joined: Mon Oct 21, 2013 5:57 pm
Location: Langenhorn, North Friesland, Germany

Solution

Postby Bernd.N » Fri Feb 16, 2018 8:40 am

I found the reason here:
https://en.wikipedia.org/wiki/UTF-8#Byte_order_mark

There are UTF-8 files with and without a BOM (Byte Order Mark) at the beginning.
That BOM is the sequence EF BB BF.
When saving my JSON file as "UTF-8 without BOM" (can be done easily with Notepad++ in menu 'Coding / Convert to UTF-8 without BOM'), everything works well.

So to be able to process also "UTF-8 with BOM", VelocityReport.fromJSON() could check if the BOM is there, and then ignore or strip it before processing.
As a workaround, one has to make sure to process always UTF-8 without BOM.

I created a ticket on Servoyforge.
Bernd Korthaus
LinkedIn
Servoy 7.4.9 SC postgreSQL 9.4.11 Windows 10 Pro
User avatar
Bernd.N
 
Posts: 497
Joined: Mon Oct 21, 2013 5:57 pm
Location: Langenhorn, North Friesland, Germany

Re: plugins.VelocityReport.fromJSON fails on UTF-8 file

Postby Bernd.N » Fri Feb 16, 2018 8:56 am

To make everything work fine in the end, it is also important to add the charset parameter on the call of plugins.file.readTXTFile() :

oEC_EIM_persons = plugins.VelocityReport.fromJSON( plugins.file.readTXTFile( jsFile.getAbsolutePath(), "UTF-8" ));

Same is true when using plugins.file.writeTXTFile() in this context.
Bernd Korthaus
LinkedIn
Servoy 7.4.9 SC postgreSQL 9.4.11 Windows 10 Pro
User avatar
Bernd.N
 
Posts: 497
Joined: Mon Oct 21, 2013 5:57 pm
Location: Langenhorn, North Friesland, Germany


Return to Plugins and Beans

Who is online

Users browsing this forum: No registered users and 4 guests

cron