problem with character set after upgrading from version 1.92 to version 2+ or 3+

More
1 year 5 months ago #189186 by ictfinance
Hi,

I have an old limesurvey instance running on Windows, with IIS as web server, and connected to a MS SQL Server 2008R2 for hosting the database. We host surveys in English, French and Dutch. LS version is 1.92. Everything is displayed correctly.
I need to transfer the surveys from this old server to a new one, running on Ubuntu, with Apache and MySQL, with LS version 3.18.0
I plan to export surveys as .lsa archive. So I upgraded a copy of my current instance to version 2.06. But the text containing characters like é, ç, à, è are incorrectly displayed. When I look into the database, I see that text is saved as ASCII characters. So for example the following sentence "Liens entre les soins à l'hôpital et à domicile (y compris continuité des soins)" is saved as "Liens entre les soins à l’hôpital et à domicile (y compris continuité des soins)" .
The problem does not appear on version 1.92, but is present for versions 2+ and 3+. I was advised to upgrade the instance to the latest LS version: 3.18.0, but problem remains.
Any idea about a way I can tell LS to appropriately convert text from database to UTF8 ?
Kind regards,

Pierre
The topic has been locked.
More
1 year 5 months ago #189188 by c_schmitz
LimeSurvey is only using UTF-8. In this particular case it seems that the database does not properly store UTF-8.

Can you install the latest LimeSurvey version in a different directory pointing to a new empty database on the same server and check if you have the same problem there?

Best regards

Carsten Schmitz
LimeSurvey project leader
The topic has been locked.
More
1 year 5 months ago #189192 by ictfinance
I think I understand the problem: I edited the text of a survey on the old 1.92 LS server. The wysiwyg editor (ckeditor) uses HTML entities for é, à, ç, è,... i.e. é , à , ç , è ...
On versions 2+ and 3+ the wysiwyg editor does not use HTML entities, and text is correctly saved in the MS SQL database.
I made the following test: I updated the values directly in the database for the "end text" field, using HTML entities:
<h3 style="text-align: center">
	&nbsp;</h3>
<h3 style="text-align: center">
	&nbsp;</h3>
<h3 style="text-align: center">
	Vos r&eacute;ponses ont &eacute;t&eacute; transmises.&nbsp;Nous vous remercions d&#39;avoir r&eacute;pondu &agrave; ce questionnaire.</h3>

If I display this text in LS, it is OK. If I edit the field in LS, I see that HTML entities were converted:
<h3 style="text-align: center"> </h3>
 
<h3 style="text-align: center"> </h3>
 
<h3 style="text-align: center">Vos réponses ont été transmises. Nous vous remercions d'avoir répondu à ce questionnaire.</h3>

Now if I save this field my é, à, ç and à characters are correctly saved.


So I suppose I need to find a way to convert text directly in the database.
The topic has been locked.
More
1 year 5 months ago #189204 by ictfinance
The database used by version 1.92 uses columns of type "text", whereas the database upgrade process converts these columns to nvarchar, which allows storing unicode.
So after database migration LS stores ç à é and è characters as unicode.
But I need a way to convert the stored data to unicode.
The topic has been locked.

Start now!

Just create your account and start using Limesurvey today.

Register now