I have this problem in my Servoy solutions unsolved for a long time and now I got to solve this problem since my clients keep asking for this.
Envoriment:
Winxp Pro, Servoy R2 2.2.1rc2-build 331 & Java version 1.5.0_04-b05
DB: mysql 5
Problem:
I have a html area field in Servoy
When I change the keyboard input to Chinese PRC or Chinese (Taiwan) and type in the html area field - I could only see squares but not the actual text. But if I type Arabic, Servoy displays perfectly
I placed a text area field on the layout with the same field and checked the html code. Servoy converted all I typed (shows as sqaures) to numbers like #1610; #1587; #1588 etc.. which I assume that they are unicode equivalents.
I wrote a php script to read & display the Chinese mysql data, it displayed perfectly on the browser with the UTF-8 encoding
But I’m very sad that why Servoy cannot display it
Pleaseeeeee help me because the only main reason for which we moved from Filemaker to Servoy was the lag of chinese support in FMP6 but later we took great advantage of the Servoy’s power. I’m sure Servoy is able to display the chinese except that I donno how to do
I did a lot of tests and have seen the following problems. This may give you more idea to provide me some help
I created a form and placed the chinese field
I installed Arial Unicode MS in my PC
I attached Arial Unicode MS to the chinese field
TEST 1: ======
my db server is in Sybase
if I copy & paste the chinese text into the field which has the text area property, the text is shown perfectly
when I changed the display type to html area and pasted the text into the field, the chinese data again displayed correctly
when I see the html code for the above field, Servoy converted all the unicode characters to html equivalents
TEST 2: ======
I changed the db server to mysql (version 5)
the collation for whole table is utf8_general_ci
when the field display is in html area then it displays correctly - but when display type is changed to text area then it displays the chinese characters until I go to the layout mode or restart the solution - after that the whole chinese text changes to “wS” (please see pic) which does not mean anything
TEST 3: ======
I copied the chinese text and inserted into mysql directly using phpmyadmin
The text was inserted perfectly and displayed correctly
Now when I opened Servoy I cannot see the Chinese correctly instead it showed some characters
I copied the characters to notepad and saved as html, then after changing the encoding to UTF-8 the text was displayed correctly on the browser (please see pic)
Problems:
? Obvisously I need to make the TEST 2 to work properly.. how?
? Why Servoy converts unicode characters to html equivalents when we display as html area because how to search for those characters if there are numbers
? Why I cannot see the chinese in the default locale under Edit->Preferences?
? sometimes Servoy displays the Chinese characters without even any fonts attached to that. But sometimes it shows squares. When I attach the Arial Unicode MS then it displays correctly
This might be a long post. But I wanted to mention everything I know. I could have a proper sleep only after solving this chinese problem
Yes. If I use mysql in Servoy it does not work but when I use phpmyadmin I could insert the chinese text without any problem. Is this something related to driver problem?
Servoy works 100% in UNICODE which is supposed to support any characterset. Servoy does not do any character translation/mapping whatsoever. Your post also indicates it works fine on Sybase ASA which is a solid database with solid database drivers and if anything doesn’t work on it we have support contracts in place with Sybase to find out what could be wrong. You indicate that it only doesn’t work on Mysql so my first thought would be to contact them, from what I see there must be a problem with their database or their JDBC driver. Also: if it works fine on ASA: I recommend using ASA!
1 Why Servoy converts unicode characters to html equivalents when we display as html area because how to search for those characters if there are numbers
2 Why I cannot see the chinese in the default locale under Edit->Preferences?
3 Sometimes Servoy displays the Chinese characters without even any fonts attached to that. But sometimes it shows squares. When I attach the Arial Unicode MS then it displays correctly
I think it has to, when following the html spec on a non chinese PC/OS
We list all the locales found on your PC/OS, so likly not installed
For chinese you need indeed a font capable of displaying, how it’s possible it works somethimes without is strange
jaleman:
Servoy works 100% in UNICODE which is supposed to support any characterset. Servoy does not do any character translation/mapping whatsoever. Your post also indicates it works fine on Sybase ASA which is a solid database with solid database drivers and if anything doesn’t work on it we have support contracts in place with Sybase to find out what could be wrong. You indicate that it only doesn’t work on Mysql so my first thought would be to contact them, from what I see there must be a problem with their database or their JDBC driver. Also: if it works fine on ASA: I recommend using ASA!
OK I stay with Sybase for the moment and do the further testing. But please try to help me with the following
You said “Servoy does not do any character translation/mapping whatsoever”
When we type Chinese into the text area field Servoy does no translation. But when we type the Chinese into the html area fields it converts into numbers..
If this is the case how do I search for the actual character on the web or within Servoy? Could you please clarify me
ahmad:
When we type Chinese into the text area field Servoy does no translation. But when we type the Chinese into the html area fields it converts into numbers..
Servoy is not doing that but Java Swing lib. is, as said in the previous post “I think it has to, when following the html spec on a non chinese PC/OS”, I have no explanation for this nor can we change this.
If this is a big problem we can ask SUN why this happens.
Jan Blok:
Servoy is not doing that but Java Swing lib. is, as said in the previous post “I think it has to, when following the html spec on a non chinese PC/OS”, I have no explanation for this nor can we change this.
If this is a big problem we can ask SUN why this happens.
Hi Jan,
Thanks a lot for your prompt reply. I understand the situation. But one thing I don’t understand is how come it does not change when we use the textarea where as it changes for the html field (might because of the html object).
If this is a display problem on a non chinese machines, can’t it be solved by installing the correct fonts. I’m now using the english machine only where I can see the Chinese characters without any problem if I attach the Arial Unicode MS font to the field
This is not a big problem for me unless I could search that in Servoy & php. Now in Servoy I could not search the chinese text when I use the html area (and text area for the search) because internally it has only numbers
Sorry for keep bugging you on this because this is the decision factor for our company’s business. If you can help us to get things right we would very much grateful
In mean time (Servoy 2.2.2) we will add a helper function on Utils script node:
var dot = utils.getUnicodeCharacter(9679);//returns a big dot
so at least you can convert the “& # 9679 ;” things to normal chars again before saving after an edit
Jan Blok:
In mean time (Servoy 2.2.2) we will add a helper function on Utils script node:
var dot = utils.getUnicodeCharacter(9679);//returns a big dot
so at least you can convert the “& # 9679 ;” things to normal chars again before saving after an edit
Thanks Jan. I need another help
In my Chinese website, people submits some Chinese data in the enquiry forms (like name, message, address etc..)
In the php I wrote a function to convert all the Chinese characters to utf8 numbers
So my data is stored some thing like
“some english data &# 32879; &# 32097;&# 38651;&# 35441;english data”
Now I have to bring this data into Servoy,
I use plugins.http.getPageData(phpURL) to get the data into Servoy field some how
The data is imported into Servoy field as shown above. The field is not formatted as html area but it’s a text field
So how can I loop through this data and convert to the actual character so that it displays perfcetly in the text field?
On what basis I have to split the text to loop and get the actual character because there is also some english data mixed. On the website I only convert the Chinese charactres to UTF8 numbers and not the ASCII characters
There are several ways to loop/search trough text, the easy way is likely doing a regexp. split on the string with the expression like “&#[0-9]*;”
and check each part and call the function getUnicodeCharacter if a part is a char representation, and combine all the parts back as a string
