Hi, am I missing something… or is this a bug?I’m building a LimeSurvey survey with a question group containing multiple 5‑point array questions. Each question uses the same four answer options (for the attachments here I used the answer labels L0–L3).
Goal: Randomize the order of these five answer options independently for each question.Setting used: Display / Random order / "Yes" enabled for all individual array questions. Issue:

• In preview mode, randomization works per question as intended.
• After activating the survey, randomization is applied once globally → all questions in the group display the same randomized order.
   

Question:Is this a bug? How can I ensure that each question randomizes its answer options independently in the activated survey, just like in preview?
I’ll attach screenshots showing how it’s supposed to work (preview) vs. how it actually behaves (activated).

Thanks for looking into it!

Cheers,
Display

This is not a bug, but a feature.

Generally one uses randomization to avoid special patterns over the whole sample (eg the first item is chosen most often, etc.), but at the same time it can be extremely irritating for respondents if the same sub questions appear in different order in different questions, especially when they are longer item batteries. Thus, the same item battery is randomized once for the respondent an then keeps the order for the rest of its appearances in the in the questionnaire. For the next respondent the dice is rolled again for the first appearance of the item battery, and then this random order stays the same. This is not a bug, but basically the desired outcome.

I think the same randomization is applied to all.later item batteries, eg item 3 is always on position 2,item 2 is always on position 4,etc.

I am not sure, but you could try to use different sub question codes for the questions, not sure if this helps (eg L11, L12,... And then L21, L22... Etc) . Or you could already post the second item battery in a different order, which should then give you a different pattern after the initial randomization.

I personally think for most purposes the default behavior is actually good, but I can understand that for certain experiments you might want to have a different order of the sub question for every appearance in the questionnaire, not just a randomization per respondent.

Search in the forum, I think joffm or tpartner have posted a solution for this already. I am on my phone, so I won't search for you. But this definitely has been answered before. 

I think you can use a different code list for each subquestion to have a different order.

Else:  18440: Semantic differential array subquestions gets shuffled only once per participant  

I don't understand why LimeSurvey team didn't think this issue is important.

Thanks, DenisChenu.
For anyone who might encounter a similar issue: I haven’t tested it, but I think @holch and @DenisChenu suggestion of simply changing the question codes might also work. I’d try that first. In my case, however, I already found a (somewhat odd) solution:

I solved it by manually changing the order of the answer items for each question. Then the randomization occurred for each question individually. In total I have 20 questions, and with 4 images there are 4! = 24 possible order combinations—just enough for me to assign a unique base order in the survey setup for each question. Then, when running the survey, the randomization works as intended: each question shows a unique randomization of the answer options. Also, if I restart the survey, the randomized orders differ again—so everything is properly random.

It’s important to note, though, that if I had more than 24 questions, I’d eventually need to reuse one specific order. In that case, the randomization would produce the exact same sequence for both questions using that order. So this approach only works as long as the total number of questions with the same randomized answer options does not exceed the number of possible permutations (n!).

holch wrote:

This is not a bug, but a feature.

Generally one uses randomization to avoid special patterns over the whole sample (eg the first item is chosen most often, etc.), but at the same time it can be extremely irritating for respondents if the same sub questions appear in different order in different questions, especially when they are longer item batteries. Thus, the same item battery is randomized once for the respondent an then keeps the order for the rest of its appearances in the in the questionnaire. For the next respondent the dice is rolled again for the first appearance of the item battery, and then this random order stays the same. This is not a bug, but basically the desired outcome.

I think the same randomization is applied to all.later item batteries, eg item 3 is always on position 2,item 2 is always on position 4,etc.

I am not sure, but you could try to use different sub question codes for the questions, not sure if this helps (eg L11, L12,... And then L21, L22... Etc) . Or you could already post the second item battery in a different order, which should then give you a different pattern after the initial randomization.

I personally think for most purposes the default behavior is actually good, but I can understand that for certain experiments you might want to have a different order of the sub question for every appearance in the questionnaire, not just a randomization per respondent.
 

Thanks, holch, for your thorough reply.

Interesting. I do understand that in most cases you’d probably want the same randomization of the same answer options across different questions, to avoid confusion and make the answering process easier. And yeah, your intuition is spot on: in my case—where participants need to evaluate each of four images with respect to a statement—I actually want them to cognitively recalibrate for each statement. Otherwise, they might start answering in a similar way for different statements just because they got used to the position of an image.

P.S. I had this answer ready right away, but for some reason I couldn't pass the spam check. I even saved my answer as a .txt file to get back to you later. Well later it is… I hope the reply goes through now! Cheers.