How to handle special characters in HTML?

[swingman]

Hi, I have the following setup:

Servoy 2.1 with a mySQL database holding text for web pages.
PHP to create XHTML webpages.

The users type text into fields in Servoy, and can see a rough preview (in an HTMLarea) of what the page will look like. It all works fine until they type quotes, &, accented characters etc. The Servoy HTML preview gets confused displaying the characters as square boxes. The same happens in the webpage created with PHP.

I tried using for quotes. Neither Servoy or PHP likes it, displaying the and instead of the quotes...

What is the correct way of encoding the characters?
- using Javascript in Servoy?
- using PHP?

[Jan Blok]

replace special chars wtih html code,like:
'<' - "& l t ;" (unable to post without spaces)
'>' - "& g t ;" (unable to post without spaces)
etc.

[swingman]

Hi Jan,

I was hoping there was some Servoy built-in function I hadn't spotted to deal with this...

I will need to write a function which translates all special characters... I will start writing somthing based this code I just found on

http://developer.irt.org/script/243.htm

[IT2Be]

I would be able to include such a thing in my tools plugin. Let me know and I will do somewhere over the weekend...

[Jan Aleman]

try:

Code: Select all

encodeURIComponent(htmlcode)

eg:

Code: Select all

var htmlcode = '<>'
var encodedHtml = encodeURIComponent(htmlcode)

note that this encodes to UTF-8 escape sequences. It should work in all browsers though.

[swingman]

try:

Code: Select all

encodeURIComponent(htmlcode)

eg:

Code: Select all

var htmlcode = '<>'
var encodedHtml = encodeURIComponent(htmlcode)

note that this encodes to UTF-8 escape sequences. It should work in all browsers though.

I tried, this but the HTMLArea just displays these codes raw! Lots of %20s instead of spaces etc...

To reiterate:

The problem I'm trying to solve is as follows:

1. User types text into normal text area.
2. There is a calculated field which adds basic CSS styling to the text. The result is displayed in an HTML area.
3. The text is stored in an mySQL database which will be accessed by PHP to build XHTML pages.

I'm sure anyone who tries to manage a website content from Servoy will come across the very same problem.

Marcel, I like your idea of writing it into your plug-in...

Here are a couple of screenshots to give you an idea:

[Jan Aleman]

I see in your examples that you are using high ascii characters. Is your backend database set to the correct character set? If you use Sybase by default it uses UTF encoding which supports any characterset. I think default mysql databases don't.

[automazione]

Hi swingman,

Probably mine is not a stylish solution, but I had a similar problem and I solved it replacing the "strange high ASCII" characters like or ΄ with plain low ASCII " or ' before storing the text area field:

htmltext_field = htmltext_field.replace('', '"')
htmltext_field = htmltext_field.replace('΄', "'")

etc....

hope it helps...

[swingman]

I see in your examples that you are using high ascii characters. Is your backend database set to the correct character set? If you use Sybase by default it uses UTF encoding which supports any characterset. I think default mysql databases don't.

Didn't think of that! I will have a look at my mySQL database....

[swingman]

Hi Jan,

You are right! The version of mySQL at my webhost is prior to 4.1, so it does not support UTF-8, so it simply strips out all our high ascii characters.

I have come up with the following work-around:

- OnRecordSave I encode all text fields with the following global function, called encodeText:

Code: Select all

var s = arguments[0];
var i = 0;
while (i < s.length)
{
   var ccode = s.charCodeAt(i);
   if(ccode >= 160) {
      var entity = '&#' + ccode + ';';
      s = utils.stringReplace(s,s[i],entity);
      i += entity.length;
   } else {
      i += 1;
   }
}

return s;


OnRecordEdit I decode, using a global function called decodeText, so my user can see what they have typed:

Code: Select all

var s = arguments[0];
var startIndex = s.indexOf('&#',0);
while (startIndex > -1)
{
   var endIndex = s.indexOf(';',startIndex);
   if(endIndex > -1) {
      var entity = s.substring(startIndex,endIndex + 1);
      s = utils.stringReplace(s,entity,String.fromCharCode(utils.stringToNumber(entity)));
   }   
   startIndex = s.indexOf('&#',0);
}
return s;


My next question: Is there any way of looping through all editable text fields on a form?
:-)

Otherwise I will have to write OnRecordEdit and OnRecordSave handlers for every form :-(

[Jan Aleman]

probably the easiest is to adjust your method to read the name of the element that triggered it using Application.getMethodTriggeredElementName.
Name the elements that need to have validation before you use this.

Alternatively:
Put some identifiable in the elementname that tells you it must be validated, then in your onrecordsave and edit loop through all elements and validate the ones that contain your specific criteria for validation.

[swingman]

One small Problem:
My encode function encodes the old value of the last field edited...

How do I ensure I get the actual field contents?

[swingman]

Looks like adding

application.updateUI();

before I encode the fields sort this out.