We're running a very big survey with 173 questions in 25 groups with many conditions. After opening it up to about 25000 potential participants it crashed after about 900 had tried filling it out. It turned out that the session table exploded, each session being about 2 megabytes in size. I've had to truncate the sessions table to get the site running again. I've switched to PHP's own session driver instead of MySQL and looked in the generated session file, it turns out it stores the complete survey (questions, conditions, intros, etc.) in there for each and every user. Is that how it is intended to work? Our table exploded to about 2 gigabytes in size because of this. Is there any way I can switch this off? Because there's no way I can run the survey like this...

Looking at my own database (running 7 small surveys), I see nothing of the kind of storage you report. What table is the complete survey appearing in for each session?

I've put it on another server now. And with file-based storage, and each individual session still takes about 1.8 megabytes. But it won't crash my DB anymore. It was stored in the lime_sessions table.

Still when I look into each session file PHP's native session handler generates it has the entire survey in there. With everything. So I'm not surprised you won't see as much with a small survey (we haven't had trouble up till now with smaller surveys). But the sessions should still be surprisingly big, even for small surveys. As they take up much more space than just the keys + answers should.

Glad you found a solution. Odd, when running a survey response, I still do not see that, but I am probably trying to replicate your issue the wrong way.

I was actually hoping for a misconfiguration on my part, but I found the following in the documentation this morning:

"When using database sessions, make sure then MySQL setting max_allowed_packet is set large enough, some surveys generate over 2 MB of session data."
- www.limesurvey.org/manual/Global_settings#General_description (last bit of "Session lifetime")

So this seems to be intended behaviour. I fear it makes LimeSurvey unfit for large-scale surveys with big questionnaires. Not complaining, I'm very happy with it for the small stuff. But it seems a pity as I think it's unnecessary to make sessions this huge.

Looks like an opportunity to optimize with a plug-in or a re-code.

jschreuder wrote: So this seems to be intended behaviour. I fear it makes LimeSurvey unfit for large-scale surveys with big questionnaires. Not complaining, I'm very happy with it for the small stuff. But it seems a pity as I think it's unnecessary to make sessions this huge.

Sessionconcept of Limesurvey is still a big malus point.
Even when using php session files you will have to keep an eye on the session generation with big surveys with a lot of users in a small timeframe. Each time someone hits the survey a 2MB file is generated in the php session directory. The default session directory is tmp on UNIX machines. With some hosting providers TMP is put on a secured, locked down partition with a size limit. So there might be a barrier hit, when having a lot of session files generated in a small timeframe.
The I/O burden can also be a pity. Having a SSD for the sessionfiles can help since the 2 MB file is directly written when the user hits the first surveypage.

I'm guessing that the original developers figured that the pre-written file would be a performance improvement over generating the survey on the fly for each user's visit to the site. For smaller surveys, they may be right.

For your case, I wonder whether breaking your humongous survey into perhaps three smaller ones might serve your purposes.

Moradzadeh wrote: ..... a re-code.

Yes for some part, this must needed.
But without breaking existing ....

And we must use Cache and not session for some other part ... session must only keep the answers.

DenisChenu wrote:

Moradzadeh wrote: ..... a re-code.

And we must use Cache and not session for some other part ... session must only keep the answers.

Hi Denis, are there any plans on adding caches/session caches to v. 3.0 at all?

bruce78 wrote: ....
Hi Denis, are there any plans on adding caches/session caches to v. 3.0 at all?

Not currently. And remind : we still use $_SESSIOn : it's one of the reason the first pacge can take a long long long time to call.

DenisChenu wrote: Not currently. And remind : we still use $_SESSIOn : it's one of the reason the first pacge can take a long long long time to call.

Interesting... btw, do you think disk sessions are faster/more reliable than db sessions, all things being equal?

bruce78 wrote:

DenisChenu wrote: Not currently. And remind : we still use $_SESSIOn : it's one of the reason the first pacge can take a long long long time to call.

Interesting... btw, do you think disk sessions are faster/more reliable than db sessions, all things being equal?

Depend

If you can set session save path on a SSD or better in RAM ! file session is quickest sure.
If both are in HHD : file or SQL : i think file still quick ? because we use SQL for other than session
=> Maybe alternative soluton is to use another DB for session, for example a SQLite db (saved in RAM/SSD)

DenisChenu wrote: => Maybe alternative soluton is to use another DB for session, for example a SQLite db (saved in RAM/SSD)

That's a neat idea... do you think something like that might reduce my first page load time from 10 seconds to something less?

bruce78 wrote:

DenisChenu wrote: => Maybe alternative soluton is to use another DB for session, for example a SQLite db (saved in RAM/SSD)

That's a neat idea... do you think something like that might reduce my first page load time from 10 seconds to something less?

Not tested, the no idea ....
But think the best solution is to save session in RAM only or SSD. Maybe in a SQLite DB on RAM is the better way.

Denis