Welcome to Orcmid's Lair, the playground for family connections, pastimes, and scholarly vocation -- the collected professional and recreational work of Dennis E. Hamilton
Stop Lying About Surveys!
I must remember to have that “Lies to Phone Surveyors” T-shirt made up. This isn’t about that though.
Today’s lament is about on-line surveys and questionnaires. I’ve been receiving invitations to several of those quirky things lately.
Here’s the lie I’m bitching about:
I understand the goal is to have responses to the survey, but making up a statement about how short and quick it is and having that be a lie is a demonstration of incompetence if not untrustworthiness. It will also lead me to stop at some point, with a comment that “you are giving me too much work and I stopped/abandoned/cancelled here.” Usually there is no place for such a comment and all the surveyor will receive is a shockingly low response rate. That might be good: fewer responses to evaluate.
Here are some clues for the unclued:
By the way, I stop phone survey responses and other conversations when the short time I am promised is exceeded and there is no clue how much overtime it will take. This is not like writing some complex program for the first time. It is possible to provide grounded estimates on how long a questionnaire will take and to do the right thing when the answer is not one you are willing to give to the respondent up front.
I first noticed this situation while I was working on my since-abandoned M.Sc in IT. It was amazing how many dissertation projects involved performing surveys. There were often these anxious on-line posts to fellow students to respond to the survey in their IT-professional capacities. I never once received an accurate time estimate, and the most suspicious ones were the ones that were unbelievably optimistic (two minutes and “only a few minutes” being the winners here until I ran into “quick” yesterday). Since this is a good opportunity to re-learn that IT is an empirical activity, I was always helpful in pointing out to the surveyors that they should have timed the questionnaire a few times with family and friends or fellow students.
Another feature of that setting was the desire to use on-line survey systems of one kind or another. Bugs, browser and plug-in incompatibilities, and numerous other exciting adventures were to be found in those cases. The biggest take-away is that the use of radio buttons that require one and only one choice are almost always going to be a problem, especially if some selection is mandatory.
The questionnaire that I just completed was notable for asking essay questions. When I ran into that the first time late last night, I stopped and went to bed. Refreshed this morning, I had my thoughts in order and I went back to the questionnaire. I had to start over, of course, and I did that because I wanted to respond. It was an application to participate in an user council and I want to do that. There was also no mechanism for feedback, I haven’t blogged in some time, and, well, here we are.
[update 2006–12–17T23:09Z Hmm. Something was nagging me about the quick survey that wasn’t quick at all. I know what it was. I received an e-mail invitation. It said that all I need to do was complete a quick survey. At the very end of that statement it mentioned that I would hear if I’d been selected (that should have been the moment of truth right there). Then, when we get to the essay questions, it turns out I am making an application. So, apart from being annoyed that I also just wasted a considerable time not ordering on-line IMAX tickets for Happy Feet, a completely-separate but attitude challenging experience, I realize that the whole non-invitation invitation thing and the disguised solicitation of applications for screening is really sucky. Hey, I wonder if I just dooced myself as a volunteer user-council member? So I’ll stop while I’m so pissed off at how my day is going.]
Comments: Post a Comment
|You are navigating Orcmid's Lair.|