The "unrestricted" category is probably the most overlooked one in practice. I've seen so many stakeholder requests for matrix grids becuase they seem efficient from the research side, but then the data ends up being pretty useless when everyone ranks everything as top priority. The shift away from matrix questions towards pairwise/MaxDiff makes alot of sense once you factor in both mobile usability and forcing actual tradeoffs.
100% agreed, the "unrestricted" category seems to be a completely missed aspect in all the literature I could find on straightlining and matrix grid's flaws. I think Pairwise Comparison's biggest barrier to adoption is just that not enough people know about it, and for MaxDiff (which is at least a bit better known within researcher circles) it has an unnecessary reputation for complexity that just isn't true. Hopefully articles like this put both methods on more people's radars :)
Good Call daniel, interesting read. I have always said that straightlining is not a respondent problem but a survey problem, with the exception of bots. What I would add to the list explicitly is the most common problem in my experience: Poor question design i.e. the questions are poorly articulated, so the user has to think too much and gets tired, or they just don't make sense.
True that bots are an exception. I had included poorly articulated questions under "#2. Fatigue", but I probably could've called that out clearer. Agreed that it is for sure a cause of straightlining!
Great read! I feel like survey design is way way more important than it seems and even simple mistakes can lead to lots of bias and misleading insights.
For sure! Matrix Grid questions are one of those "seems simple" survey methods because everyone has filled one in before as a respondent, but they can lead to some very useless data and disappointment. Hoping this post helps more people find better alternatives :)
Which type of customer satisfaction survey method is useful for measuring the satisfaction level of an agro tourism customer? Is there any sample of such questionear?
The "unrestricted" category is probably the most overlooked one in practice. I've seen so many stakeholder requests for matrix grids becuase they seem efficient from the research side, but then the data ends up being pretty useless when everyone ranks everything as top priority. The shift away from matrix questions towards pairwise/MaxDiff makes alot of sense once you factor in both mobile usability and forcing actual tradeoffs.
100% agreed, the "unrestricted" category seems to be a completely missed aspect in all the literature I could find on straightlining and matrix grid's flaws. I think Pairwise Comparison's biggest barrier to adoption is just that not enough people know about it, and for MaxDiff (which is at least a bit better known within researcher circles) it has an unnecessary reputation for complexity that just isn't true. Hopefully articles like this put both methods on more people's radars :)
Good Call daniel, interesting read. I have always said that straightlining is not a respondent problem but a survey problem, with the exception of bots. What I would add to the list explicitly is the most common problem in my experience: Poor question design i.e. the questions are poorly articulated, so the user has to think too much and gets tired, or they just don't make sense.
Great to see OpinionX making its mark out there!
True that bots are an exception. I had included poorly articulated questions under "#2. Fatigue", but I probably could've called that out clearer. Agreed that it is for sure a cause of straightlining!
Great read! I feel like survey design is way way more important than it seems and even simple mistakes can lead to lots of bias and misleading insights.
For sure! Matrix Grid questions are one of those "seems simple" survey methods because everyone has filled one in before as a respondent, but they can lead to some very useless data and disappointment. Hoping this post helps more people find better alternatives :)
Which type of customer satisfaction survey method is useful for measuring the satisfaction level of an agro tourism customer? Is there any sample of such questionear?