Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

Random assignment to different experimental conditions

More
3 weeks 1 day ago #274264 by daisybibi
Please help us help you and fill where relevant: 
Your LimeSurvey version: LimeSurvey Community Edition  Version 3.28.76
Own server or LimeSurvey hosting: university server
Survey theme/template:
==================Hello,
I am encountering a problem while setting up an experimental design in LimeSurvey, and after many tests I still cannot obtain the expected behavior. In fact, I want to:
  1. Display a first part (P1) containing several question groups that are common to all respondents. This block includes questions used to check respondent characteristics and exclude or keep them in the sample (e.g., age, AI usage, etc.).
  2. Randomly assign respondents to one of the four experimental conditions: Condition A, B, C or D. Each condition contains four question groups. In the end, I want to have the same number of respondents in each of the four conditions.
  3. Finally, display a third part (P3) containing two question groups that are identical for all respondents.
What I have implemented:
  • A “Internal variables” group placed right after Part 1
  • An equation question called 
    Code:
    randCondition
    with the equation:
    Code:
    rand(1,4)
  • An equation question called 
    Code:
    condition
    with the equation:
    Code:
    if(randCondition == 1, "B", if(randCondition == 2, "C", if(randCondition == 3, "D", "E")))
  • Conditions A, B, C or D for each group of questions with equations such as:
    • Code:
      condition == "B"
    • Code:
      condition == "C"
    • etc.
  • Block M without any condition
  • I also tried using quotas (BlocB, BlocC, BlocD, BlocE), but in my version of LimeSurvey quotas are necessarily blocking (only options are “Terminate survey” or “Allow the respondent to modify the last answers before terminating”), so it is impossible to use them to route respondents.
However, nothing works. Could you please tell me what is causing the issue or suggest another solution?
Thank you in advance. 

Please Log in to join the conversation.

More
3 weeks 1 day ago #274271 by orvil
Hi, 
here is an example that hopefully helps. Plz see the attached example survey:This survey has 8 groups for better explanation:
  1. Group 1 holds a equation question (randABCD) for creating a random value between 1 and 4. It uses the ES {if(is_empty(Q1xRandom4), rand(1, 4), Q1xRandom4)} to keep the random value constant for all groups/questions!
  2. Group 2 holds one question for all participants
  3. Group 3 holds 4 questions, one to show for each random value, using the relevance equation {Q1xRandom4 == 1} and so on for each question
  4. Group 4a holds 2 questions for random value 1, using the relevance equation like above for the group
  5. Group 4b holds 2 questions for random value 2, same here
  6. Group 4c holds 2 questions for random value 3, same here
  7. Group 4d holds 2 questions for random value 4, same here
  8. Group 5 holds again a question for all participants. No relevance equation :)
BTW: To achieve a more even distribution (depending on the size of your sample) consider using e.g. rand(1,4000) for random creation and {(Q1xRandom4 >= 1 && Q1xRandom <=1000)} for relevance equations.

Cheers!
 

Best regards/Beste Grüße,
O. Villani

Please Log in to join the conversation.

More
3 weeks 1 day ago - 3 weeks 1 day ago #274273 by Joffm
Hi,
at first glance everything seems to be fine. So there seems to be some small hidden mistake.
Therefore you'd always send the lss export of your survey resp. the relevant parts


But one thing you should explain. What made you do this?
First you create a random number from 1-4. This means your "randcondition" will have one of these four values.
Now you do a second step to change these four numbers to letters in "condition".
And you use these letters to set the condition on the groups.

Why didn't you just use "randcondition"?
BlockA: randcondition==1
BlockB: randcondition==2
BlockC: randcondition==3
BlockD: randcondition==4

Here a small "skeleton" with your structure.

And the condition on the groups:
 


Unfortunately I can't send a sample. Your version 3.28. is already 3 and a half year old and not supported anymore.
I myself do not have an installation of this version on the machine I am working on at the moment.

Joffm

As see we prefer this construct to generate a random number:
{if(is_empty(self), rand(1,4),self)}
This avoids the behaviour that a random number changes each time there is a click in this group.
This is  usual behaviour. You can reproduce this in Excel as well.
Therefore this formula says: Only, if the random number is still empty it is created, otherwise it keeps the value.

Volunteers are not paid.
Not because they are worthless, but because they are priceless
Last edit: 3 weeks 1 day ago by Joffm.

Please Log in to join the conversation.

More
3 weeks 1 day ago #274274 by orvil
Does self work in version 3.x ? Gave it a try just now, had no success (V3.19). Expression returns 1 constantly. V6 however, replaces self with qcode

Best regards/Beste Grüße,
O. Villani

Please Log in to join the conversation.

More
3 weeks 1 day ago #274275 by Joffm

Does self work in version 3.x ? Gave it a try just now, had no success (V3.19). Expression returns 1 constantly. 

Of course, it does. If it doesn't work, there is an error. But you did not show anything.

V6 however, replaces self with qcode

It does not replace with QCode. In the question summary it is shown what "self" and "that" do. That they expand to all subquestions.

respective the subquestions you select by a filter, like "self.sq_X".
Read the manual about it.

Joffm
 

Volunteers are not paid.
Not because they are worthless, but because they are priceless

Please Log in to join the conversation.

More
3 weeks 16 hours ago #274279 by daisybibi
A very big thank you for taking the time to reply and for the information you provided.
I actually realised that it wasn’t necessary to change those four numbers into letters, so I removed that step. I will change the construct to generate a random number and use the one you suggest.
The structure of my questionnaire is exactly the same as yours, and the conditions are identical as well.
Regarding the version, I am using the one provided by my university, which is indeed not the most recent.
I will look into all of this carefully over the weekend.
Best

Please Log in to join the conversation.

More
3 weeks 16 hours ago #274280 by daisybibi
A very big thank you for taking the time to reply and for the information you provided. Given how detailed and helpful your explanations are, I will take the time to look through your suggestions very carefully over the course of this weekend.
Best

Please Log in to join the conversation.

More
2 weeks 4 days ago #274284 by daisybibi
Hello,
Despite multiple attempts, I am still unable to assign respondents to one of the four random groups. I tried with your suggestions and I also tried creating an equation question called “eligibilite” with the following expression:Codeif(((A1.NAOK == "Y")) and ((A2.NAOK == "A2")) and ((B2.NAOK != "A3")) and ((B3.NAOK == "A1")) and ((B4_SQ008.NAOK != "Y")), 1, 0)Then I added another equation question called “version”:Codeif(Eligible == 1, rand(1,4), 0)I am attaching a simplified .lss file so you can see the structure. Since I will only have around 50 respondents per questionnaire, I would also like to set up quotas so that, once a quota is reached, respondents are redirected to another version.Questionnaire Structure
  • Group A – Screening (3 questions)
  • Group B – Additional screening questions
  • Group C – Demographic questions
Respondents who do not meet the criteria defined in Groups A, B, or C are progressively redirected to Group X, which displays a message informing them that they do not meet the eligibility criteria for the study.Randomization of Experimental GroupsAn internal variable randomly assigns each respondent to one of the four experimental groups (version = 1, 2, 3, or 4). The questions are identical across groups; only the “Condition” block differs.Group 1 (version == 1)Respondents in Group 1 see:
  • Condition CS1
  • Group F
  • Group G
  • Group H
Each group contains 4 questions. Groups F, G, and H are presented in a random order.Group 2 (version == 2)Respondents in Group 2 see:
  • Condition CS3
  • Group I
  • Group J
  • Group K
Each group contains 4 questions. Groups I, J, and K are presented in a random order.Group 3 (version == 3)Respondents in Group 3 see:
  • Condition CA1
  • Group L
  • Group M
  • Group N
Each group contains 4 questions. Groups L, M, and N are presented in a random order.Group 4 (version == 4)Respondents in Group 4 see:
  • Condition CA3
  • Group P
  • Group Q
  • Group R
Each group contains 4 questions. Groups P, Q, and R are presented in a random order.Common Block
  • Group S is presented to all respondents in Groups 1, 2, 3, and 4.
For clarity: the question content is identical across all versions; only the ConditionXXX block varies between groups. Thank you in advance for your help. 

File Attachment:

File Name: limesurvey...6646.lss
File Size:143.8 KB

Please Log in to join the conversation.

More
2 weeks 3 days ago - 2 weeks 3 days ago #274286 by Joffm
Hi,
there are several mistakes.
1. "Stop".
If you want to screen out respondents; so you should use a quote with limit 0 - you want 0 respondents who do not fulfill the requirements.
2. "Eligible".
The equation - as any equation - has to be surrounded by curly brackets.
3. "version".
The equation has to be surrounded by curly brackets
Therefore there is no creation of a random number.


Revise this "blue" and everything is fine so far.

4. This I do not understand.
In "A1" you ask a Yes/No-question, but not mandatory. Why?
Later "CondCS1", "CondCS3" and "CondCA3" are only show if this question "A1" was not answered while "CondCA1" is shown always.
BUT: Question "A1" is part of the condition of "Eligible". meaning if somebody does not answer "A1" he is not eligible - and should be screened out. So this condition "is_empty(A1)"  in "CondCS1", "CondCS3" and "CondCA3"  is obsolet. A respondent leaving empty "A1" will not come to this question.

5. Some other remarks
a. "A2b". Here you expect a number. So it's better to use the correct question type "numerical input" instead of "short text"
b. The question "Eligible" is not hidden. So the respondent only sees a blank rectangle.
c. Why do you split the "age" question? It should be sufficient to ask the real age. Then you screen out the under 18.
d. In my opinion it is not the best look&feel, if questions suddenly open one after another on the same page. The respondent will be confused, maybe annoyed realizing that he still did not come to the end of the page.
The "gold standard" in online surveys is "one question per page".

Now to your other question

I would also like to set up quotas so that, once a quota is reached, respondents are redirected to another version.

To achieve this you have to count the number of versions in the already submitted surveys.
There is no built-in function in LimeSurvey 3.x. This was inplemented in version 5.x.
But there is a plugin "getStatInSurvey"
[url] gitlab.com/SondagesPro/ExportAndStats/getStatInSurvey [/url]
On the other hand I really doubt if the admin of your university will implement it.

The best and easiest way to achieve your goal is to watch the outcome so far and either adapt the creation of the random number (which is a bit simple arithmetic)
like
Group 1 full: {rand(2,4)} trivial
Group 2 full: {round(1.4*rand(1,3))} Only numbers 1,3 or 4 appear
Group 3 full: {floor(1.4*rand(1,3))} Only numbers 1,2 und 4 appear
Group 4 full: {rand(1,3)} trivial

Group 1 and 2 full: {rand(3,4)} trivial
Group 1 and 3 full: {2*rand(1,2)} 
Group 1 und 4 full: {rand(2,3)} trivial
Group 2 and 3 full: {1+ 3*rand(0,1)} 
Group 2 and 4 full: {1+ 2*rand(0,1)} 
Group 3 and 4 full: {rand(1,2)} trivial

or you change the condition.
Let's say: Group C is full, but Group A is a bit weak. Just change the condition of Group A to "version==1 or version==3" and the condition of Group C to "0"

Joffm
 

Volunteers are not paid.
Not because they are worthless, but because they are priceless
Last edit: 2 weeks 3 days ago by Joffm.

Please Log in to join the conversation.

More
2 weeks 3 days ago - 2 weeks 3 days ago #274289 by daisybibi
Hi Joffm,
A very big thank you for your help — so quick and so precise.
Here are my comments in response to the points you raised.

P2 and P3. Indeed, with the brackets it works perfectly. Thank you so much for spotting this error — it really helped me move forward!

P4. It is actually mandatory for respondents to answer all questions. During my testing phase, I often go through the questionnaire many times, so at first I disable the mandatory setting. But once the testing is finished, I activate the mandatory option — otherwise, as you pointed out, the screening logic would no longer make sense.

P5.
a. Good point — I’ve updated it.
b. Good point — I’ve updated it as well.
c. I will update this too.d. You’re right, this is something to take into consideration. However, it would mean that respondents have to click “next page” for every single question, which could become tedious. I’ll see what the feedback looks like during the pretest and will adjust if necessary. 

Regarding the quotas, this is exactly what I was afraid of, since I hadn’t found any workable solution. It’s clear that my academic institution will not provide us a more recent version of LimeSurvey nor they will agree to implement anything — not even a useful plugin.To reduce the drawbacks, I tried one of Orvil’s suggestions: using rand(1,4000) for random creation and {(version >= 1 && version <= 1000)} to try to limit potential imbalance. I’ll see how this behaves during the pretest. If the imbalance between groups is not too large, then that will be perfectly acceptable.
Best regards, 
Last edit: 2 weeks 3 days ago by daisybibi.

Please Log in to join the conversation.

More
2 weeks 3 days ago #274290 by Joffm
Hi,
this suggestion to create a random number between 1 and 4000 will not result in a better distribution.

LimeSurvey uses the "Mersenne Twister" algorithm (see Wikipedia about this) that creates internally a float number between 0 and 1 (the same as Excel) which then is recalculated to the desired interval.

But there are other options.
You can use the SAVEDID to create a cycling number (1,2,3,4,1,2,3,...).
In my Tutorial 4 I showed this and some other ways.
Unfortunately it's in German and in the German section.

If you are interested you may use Google Translate.

Joffm 

Volunteers are not paid.
Not because they are worthless, but because they are priceless

Please Log in to join the conversation.

Moderators: tpartnerholch

Lime-years ahead

Online-surveys for every purse and purpose