Error Saving Results

More
10 months 2 weeks ago #190982 by egisupport
We currently have a large survey running with many thousands of students.

A percentage of these fail or apparently fail as the administrator (not me) gets hundreds of emails saying that a database update failed.

Upon checking it appears that multiple users are using the same token and updating at different times. After one user completes the survey further updates fail and an email is sent.

This is all and good except we are dealing with a a foreign set of schools with over 100K students operating in Arabic. So I cannot be certain that my assumption is the correct one. and I cannot ask the survey respondents what they are seeing.

The code in question is in em_manager_helper.php in the function _UpdateValuesInDatabase()

However I am not 100% certain how it works

This appears to be where the email message is triggered

if (isset($_SESSION[$this->sessid]) && $this->surveyOptions)
{
$query .= $_SESSION[$this->sessid];

//If the responses already have been submitted once they are marked as completed already, so they shouldn't be changed.
$oSurveyResponse = SurveyDynamic::model($this->sid)->findByAttributes(]);
$result = true;
if ($oSurveyResponse->submitdate == null || Survey::model()->findByPk($this->sid)->alloweditaftercompletion == 'Y') {
$result = !Yii::app()->db->createCommand($query)->query();
//$result = !dbExecuteAssoc($query);
}

if ($result)
{
// TODO: This kills the session if adminemail is defined, so the queries below won't work.
$message = submitfailed('', $query); // TODO - report SQL error?


As $result is set to true and not changed I am wondering if the email is sent in error? at this point I cannot see how the code checks to see if the response has already been submitted.

The more I look at this code the more confused I get. Like why is it in the Expression Manager?


Can anyone provide some insight as to what is possibly happening

Please Log in to join the conversation.

LimeSurvey Partners
More
10 months 2 weeks ago #190990 by DenisChenu
Replied by DenisChenu on topic Error Saving Results

egisupport wrote: Upon checking it appears that multiple users are using the same token and updating at different times. After one user completes the survey further updates fail and an email is sent.

With your current settings : it's the needed behaviour (but error sent too admin is bad (i think i report this issue)).

Then : Why multiple user with same token ? If you need this (something like a Survey password) : and need multiple response : your survey need to have
1. Enable token-based response persistence to off (each users create a new response without reloading another user response)
2. Allow multiple responses or update responses with one token to on : this allow multiple response with only one token.

If something other is needed : please explain why you have this ? You can try reloadAnyResponse plugin : there are a system to disable multiple access to the same response line.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development . I don't answer to private message.

Please Log in to join the conversation.

More
10 months 1 week ago - 10 months 1 week ago #191001 by jelo
Replied by jelo on topic Error Saving Results

egisupport wrote: The more I look at this code the more confused I get. Like why is it in the Expression Manager?

What is in the Expression Manager? Explain what confuses you.


But first:
What exact version of LimeSurvey is used?
Are you sure that one TOKEN is used more than once at the same time?
If yes, how did you came to that conclusion?
Are many respondents in the same building/network (e.g. schools, companies) when answering the survey?

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users
Last edit: 10 months 1 week ago by holch. Reason: Fixed the quote

Please Log in to join the conversation.

More
10 months 1 week ago #191048 by egisupport
Replied by egisupport on topic Error Saving Results
Hi Denis

We need token based persistence so we cannot turn it off. Its a long survey involving school children which may need to be stopped and started at will.

We cannot have multiple responses for one token. Each token is assigned to a particular student we cannot have the same token with multiple students. One of the reasons this is required is for security, if our database is hacked we cannot release the name and personal details of students. Therefore the token is the only way to identify the student.

I am not sure that plugin adds any value in this use case.

Also this survey has started and cannot be stopped.

Lastly the reason why an email is sent to admin is so admin can manually enter the results. it appears to be a last effort catch all where limesurvey has failed for whatever reason so leaves it to the admin to deal with. In this case I do not really think its a bug. if it wasn't happening we would not be aware of the issue

Please Log in to join the conversation.

More
10 months 1 week ago #191051 by egisupport
Replied by egisupport on topic Error Saving Results
Hi Jelo

The version is 3.15.1 but the code in question has not changed it is identical in the current build.

The emails being sent on failure contain the token. when I check the database I can see the emails are sent after the survey has been marked as completed. However I cannot see from the code where this is flagged.

Many of the respondents are in classrooms so they are sitting side by side at desks as one does in school, however there are a large number of schools and classrooms involved. So they are spread out across an entire country. ~100K students involved.

I have just noticed that the forum software has removed some of the code when I copied and pasted it so the original code is...

github.com/LimeSurvey/LimeSurvey/blob/e6...ger_helper.php#L5580

I am pretty certain this is a simple case of students copying other students tokens however due to a number of different reasons I do not want to state that as fact until I am confident the issue is not software based.

To add to the complexity the survey is behind a load balancing proxy server, and due to the massive size of the session variable it is nigh on impossible save the sessions in the DB. It must be file based and that makes debugging even harder.

We are looking into ways to improve session handling but we cannot make changes in the middle of what you must realise is a very large and sensitive survey.

The reason I came onto this forum was to try and see if there were some logical steps I could take to rule in or out my assumptions.

Please Log in to join the conversation.

More
10 months 1 week ago #191059 by DenisChenu
Replied by DenisChenu on topic Error Saving Results

egisupport wrote: We need token based persistence so we cannot turn it off. Its a long survey involving school children which may need to be stopped and started at will.

We cannot have multiple responses for one token. Each token is assigned to a particular student we cannot have the same token with multiple students. One of the reasons this is required is for security, if our database is hacked we cannot release the name and personal details of students. Therefore the token is the only way to identify the student.

Then your issue was this one

egisupport wrote: Upon checking it appears that multiple users are using the same token and updating at different times. After one user completes the survey further updates fail and an email is sent.


Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development . I don't answer to private message.

Please Log in to join the conversation.

More
10 months 1 week ago #191069 by jelo
Replied by jelo on topic Error Saving Results

egisupport wrote: I am pretty certain this is a simple case of students copying other students tokens however due to a number of different reasons I do not want to state that as fact until I am confident the issue is not software based.

I still don't get how every student gets the URL including the token to enter the survey.
How do you distribute the URL to every student? Is there a token for every student?


A filebased session (you're right about the high size of sessiondata) in combination with a proxy load balancer is a nice blackbox. I wonder how that will work at all when it comes to sharing the sessionfiles.

Since you get no complete feedback from the respondents it's hard to sort out things.

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users
The following user(s) said Thank You: DenisChenu

Please Log in to join the conversation.

More
10 months 1 week ago #191126 by egisupport
Replied by egisupport on topic Error Saving Results
Hi Denis. It would seem that way. I am operating in a complete void. It is quite difficult. I have placed a little extra logging into the system to try and prove that the problem is token re-use. Thanks for your efforts in trying to help me get a handle on this. it is appreciated.

Please Log in to join the conversation.

More
10 months 1 week ago #191127 by egisupport
Replied by egisupport on topic Error Saving Results
Hi Jelo

It is my understanding that the students are manually entering the tokens. Thus the chance of copying or even being provided with the wrong token increases.

As mentioned to Dennis I have added some logging to try and identify the issue but as it is a number of schools I am expecting to get little information If I am really lucky I might get unique forwarded IP addresses might even get the students internal IP addresses

But that's just wishful thinking.

Apart from that I am struggling to come up with a scenario where I can determine what the cause is.

Thanks for your efforts.

Please Log in to join the conversation.

More
10 months 1 week ago #191130 by DenisChenu
Replied by DenisChenu on topic Error Saving Results
gitlab.com/SondagesPro/coreAndTools/reloadAnyResponse disable multiple access of a survey with same token (if Enable token-based response persistence is ON).
Then : i think it's a potential solution

Alternative solution : Enable token-based response persistence to OFF

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development . I don't answer to private message.

Please Log in to join the conversation.

More
10 months 1 week ago #191193 by egisupport
Replied by egisupport on topic Error Saving Results
We need token based persistence so I cannot turn that off.

Can I ask how "Disable multiple access of a survey with same token (if Enable token-based response persistence is ON)." works?

I have also just found through my logging that this is not the only survey to generate these errors. Clearly other survey administrators have not been reporting the issue to us. However this survey is the busiest at the moment so its not surprising it's generating the most errors.

Please Log in to join the conversation.

More
10 months 1 week ago #191200 by DenisChenu
Replied by DenisChenu on topic Error Saving Results

egisupport wrote: Can I ask how "Disable multiple access of a survey with same token (if Enable token-based response persistence is ON)." works?

You can easily test …

For the way it work : see code …

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development . I don't answer to private message.

Please Log in to join the conversation.

More
10 months 1 week ago #191260 by egisupport
Replied by egisupport on topic Error Saving Results
Thanks. If I get a chance I'll run up a fresh copy of LS3 in a dev environment and test it to try and get a handle on what it does.

Please Log in to join the conversation.

More
1 month 1 week ago #204008 by egisupport
Replied by egisupport on topic Error Saving Results
Just thought seen as I was here I'd post an update on this. It appears this problem occurs if someone starts a survey in one browser, completes it in another browser and then goes back to the first browser and tries to complete it once again.

Odd behaviour I know.

The problem is exacerbated by a our load balancer causing two session files being created because each browser gets a different session due to being unrecognised.

However that should not make a difference once the survey has been closed the second session should be invalidated when the user attempts to load the page. As it is LS tries to put the data into the DB and its only then that it fails and sends a message to the administrator instead of simply advising the user that this survey has already been done.

Please Log in to join the conversation.

More
4 weeks 2 days ago #204510 by DenisChenu
Replied by DenisChenu on topic Error Saving Results

egisupport wrote: Just thought seen as I was here I'd post an update on this. It appears this problem occurs if someone starts a survey in one browser, completes it in another browser and then goes back to the first browser and tries to complete it once again.

Yes : multiple access is always a pain …


It's the reason why relmoadAnyResponse plugin add a control before reload a response (by token too)
gitlab.com/SondagesPro/coreAndTools/relo...AnyResponse.php#L935

In such case : if browser #1 are not closed, user receive a 409 header when try to open on a second browser.

With a delay.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development . I don't answer to private message.

Please Log in to join the conversation.

Start now!

Just create your account and start using Limesurvey today.

Register now