I've been running some benchmarks on Limesurvey as my company expects to send 10-50k surveys out at any given time, and we wanted to be prepared for any spikes in traffic. However, the results aren't looking great so far. I've looked around the forums a bit, and while it appears my results are relatively in line with what is expected, I was hoping there was some additional info I could gather here.

LimeSurvey version
4.3.1
LimeSurvey build
200623
PHP version
7.3.14-1~deb10u1
Web server software
nginx/1.14.2
Web server info
HTTP/1.1
Database driver
pgsql
Database driver version
11.7 (Debian 11.7-0+deb10u1)
Database server version
11.6

Frontend Server:
4x Intel Xeon 2ghz - 4gb ram - 40gb SSD with ~1600 iops

Database Server:
4x Intel Xeon 2ghz - 5gb ram - 40gb SSD with ~1600 iops

Benchmark Results (apachebench) of index page:
~180 requests/sec
~1s max load time
~500ms mean load time
~6mbps data transfer

Results of survey start page:
~10 requests/sec (creating new survey sessions appears to be especially heavy)

This is after using Nginx+PHP-fpm, tuning PHP worker processes, opcache management, aggressive proxy caching, file-based sessions, and other measures I could think up. The frontend server would hit 99.9% CPU load and database server in the upper 80%. Memory usage stayed relatively low, as did iops. We could scale both servers up to 16+ cores and get additional headroom during expected peaks in traffic, but the baseline load that Limesurvey can handle seems abnormally low, making it a fragile installation. I'm wondering if there's any experienced community members with larger instances, and what the server performance should look like.

Limesurvey just appears to be very CPU heavy, and it looks like part of this is from session management. Is there any additional configuration that can be done to Limesurvey to make sessions lighter, or reduce the PHP processing overhead? Is this the best that can be done, and just a matter of throwing more hardware at the problem, to scaling servers up?


I appreciate any advice you can provide, and please let me know if any extra info is needed.

champu wrote: Results of survey start page:
~10 requests/sec (creating new survey sessions appears to be especially heavy)

Yes , an issue here.

Did you try to set session in tmpfs ?

LimeSurvey is abusing the session concept. The session file size is high and you need make sure to have a lot of session space when you expect a lot of visitors. When the visitor is in the survey, it's fine. But the first page hitters cause a lot of session creation.

I'm not sure how you redirect the people between your different frontend servers? If the the complexity is big enough (proxies and caching), you might run into issues with LimeSurvey depeding on the Survey-Features used. Token, Save and Resume etc.

To me the IOPS of the SSDs looks low for filebased sessions. Where do you host? I personally would have gone for one VPS with 16 cpu cores and high IOPS SSD and tried to scale vertically instead of going horizontal. 50k invitations via E-Mail can be fine. Let E-Mails send out over a longer timespan to reduce the peaks of one page clicker.

I don't know when you want to start using your setup, but LS4 is not really ready for productive use. I know that the LimeSurvey-Website is giving a different impression, but use LS3 till LS4 has matured.

What are the apachebenchmark commands you have used? Even though ab is not really good at dynamic pages, I would try your measurements on a simple Apache+PHP-FPM setup

Thanks for your replies. I tried the tmpfs idea, but couldn't squeeze any additional performance. I noticed that after 5k requests, I'd already used ~200mb of tmpfs space, so it's not very memory efficient.

Here's an apachebench output with tmpfs
The command I used is very basic: ab -n 50000 -c 100 -H 'Host: <snip>' http://10.16.0.122:80/
I stopped the test early before my tmpfs ran out of space (656M used after ~22k requests). Pre-tmpfs, I noticed the IOPS didn't break more than ~600, and while the SSD I used is guaranteed for 1600, it has a burst of 16k, so that shouldn't have been the bottleneck. It appears that raw CPU power is really the key here, and it looks like we'll have to scale that up by core count, and then get bigger SSD's for larger IOPS guarantees to scale.

I'll try a Limesurvey 3.x series install and see if it does any better. The Nginx install setup is just a straight include+fastcgi_pass, so nothing fancy, and Nginx load doesn't pass ~1% cpu use.

For reference, these are custom E2 instances on Google Cloud using 40gb local SSDs. I scaled up the SSDs to get more IOPS and saw no noticeable improvement.

I'll try a Limesurvey 3.x series install and see if it does any better. The Nginx install setup is just a straight include+fastcgi_pass, so nothing fancy, and Nginx load doesn't pass ~1% cpu use.

I don't think that LS 3.x will be doing much better in terms of performance, but there are still many bugs and hickups in LS 4 that most of us here in the forum do not vurrently recommend it for production use. Use LS 3.x for production, LS 4 only for testing at the moment.

I would go down the same route as Jelo: one VPS/Server, but I would throw a lot of power at it, 8-16GB ram and quick SSDs.

And one other thing: most other survey systems that I have worked with also usually do not send out 50k of emails, but spread them out over hours and days, to make sure that there are no spikes. That is just good survey management.

If you really have various surveys of 10-50k per day, you probably have to get a pretty powerful server, but of course, then budget shouldn't be a big issue.

Thank you for sharing your experience and measurements.

Did you end up finding a solution for better performance?