Problem saving data

Classic Servoy
1

When copying multiple records from sybase to postgreSQL I get this error:

org.postgresql.util.PSQLException: ERROR: invalid byte sequence for encoding "UTF8": 0x00 Hint: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".

It happens when I issue the databaseManager.saveData()

The source table contains a TEXT-field that sometimes contains ‘strange’ characters as a result of a conversion, this conversion was done by a Servoy solution.

Has anyone seen this problem?

2

Michel,

The contents of the sybase field seems to be stored as a binary data (what is the column type in the db?).
When strings are stored in bytes, they are always converted using an encoding and when the bytes are converted to strings again, the same encoding must be used.

If you copy the bytes from one binary column to another and the second column expects another encoding than the one that was used, i can imagine this error.

How were both columns created?

Rob

3

Hi Rob,

Both fields were created by Servoy, source and destination are TEXT(5000), columntype in posgreSQL: character varying(5000), columntype in sybase: varchar(5000).

Source database url: jdbc:sybase:Tds:localhost:2638?ServiceName=user_data&CHARSET=utf8
Destination database url: jdbc:postgresql://localhost:5432/user_data

Source table was filled by a Servoy solution that copied data from a foxpro table to the sybase database. No problems occurred during this copy. The string fields containing the 0x00 characters were already present in the foxpro table.

4

Michel,

When the sybase column is types as varchar, we get the string from the driver, not the bytes, so the bytes should already be converted by the sybase driver.

Do you use the latest postgres driver?
We updated the driver in Servoy 5.2.3, the new driver size is 502694.

Rob

5

Yes we are using Servoy 5.2.4. I checked the filesize of the postgresql.jar it is 502694 bytes.

So you mean the driver of sybase did not generate correct utf8?

6

Michel,

I think the data is not correctly encoded in UTF8 in the sybase table, but it is being interpreted as UTF8.
Do you have an idea what the encoding could be?

Rob

7

No, I don’t. We have always provide the UTF8 encoding in the sybase connection URL and we are using the sybase database shipped with servoy.

Connection URL of the foxpro database (thas served as the source for the sybase database) is jdbc:dbf:/c:<some_path>?charSet=ISO8859_1 using the com.hxtt.sql.dbf.DBFDriver