In this segment, we'll talk about two phenomena that involve interactive

prompting, to reduce respondent behaviors that are considered undesirable.

In particular, we'll talk about prompting respondents who are caught speeding,

answering unreasonably fast, and respondents who don't answer at all,

item non response.

Turning first to speeding, in one study, we set 300 milliseconds per word

as the threshold below, which we considered a respondent to be speeding.

In other words, in a ten word question, a response time below

three seconds, or 3,000 milliseconds, would be considered speeding.

This has been proposed, that is 300 milliseconds per word,

has been proposed as the time required for adults in the U.S. to read.

Of course, there are exceptions in both directions.

There are adults who eat faster and adults who eat slower.

So to the extent that this is an effect of threshold, it suggests that

interventions of this sort don't have to be tailored to every respondent but

can be designed in the aggregate.

In any case, when respondents answered below this threshold,

they received a prompt such as this one.

You seem to have responded very quickly.

Please be sure you've given the question sufficient thought

to provide an accurate answer.

Do you want to go back and reconsider your answer?

We compared the amount of speeding in a condition,

experimental treatment in which this prompt these prompts were displayed,

when responded to answers below that threshold.

To a control condition, of which there were no prompts.

We did this for seven questions about autobiographical quantities

like how many nights have you spent away from home in the last year?

For which the true value is not known, at least we didn't have access to it.

Also for seven simple arithmetic or probability questions for which,

the true value was known or at least we could determine the correct answers.

And that was useful because it allowed us to assess response accuracy.

So what we want to see is whether, there was any less speeding when respondents

were prompted upon speeding once, then when they were not.

When they were allowed to speed, more or less, with impunity.

As you can see here in the no prompt condition,

speeding was actually quite prevalent, 61, 62% in two of the experiments.

This means speeding at least one time.

But when we prompted respondents, they sped less in all cases, so

a respondent who was prompted for speeding once or potentially more

overall sped less often than their counterparts in the control condition,

who were never prompted for speeding.

So that suggests that we are slowing them down with this intervention.

Does that mean they're giving more thought to their answers,

and actually answering more accurately?

Or are they just sort of biding their time to avoid future prompts like this.

And then, just giving the same kind of superficial treatment to the question.

So in the one experiment with where we could know the true value,

where they were answering simple arithmetic and probability questions.

We could look at whether prompting not only slowed them down, but

increased their response accuracy.

There was actually no effect overall of prompting on response accuracy but

there was an effect for one group, when we broke the groups down by education level.

In particular, respondents who had some college or

an associates degree were more accurate when we prompted them for speeding.

So they slowed down, and

seemed to use the additional time that give more thought to their answers.

This account for about 40% of the respondents.

The respondents with only a high school degree were unaffected by prompting

as where the respondents with a bachelor's degree or more and

this could well be because those are the high school degree

were struggling with these questions, even though they were considered to be

relatively basic measures of numerical literacy and numeracy.

And so, for the respondents with a high school degree or

less, even more time didn't seem to help.

So that's one explanation for why prompting didn't help them,

and the group with a Bachelor's degree or

more may have found these questions so easy that they were able to answer

quite fast, fast enough that in fact we were considering them to be speeding.

When actually, they were giving adequate thought to the questions.

So it's for this group in the middle that, at least for these items,

that the slow down due to prompting seemed to improve response accuracy.

Turning to the other use of interactive prompting that we'll cover here,

when respondents speed, they're in a sense not really answering the question.

They're just providing sort of an answer at random.

In some cases, respondents don't provide any answers, and

this is known as item missing data or just item non response.

It could be that you can reduce item non response by

absolutely requiring a respondent to provide an answer by not allowing them to

proceed to the next question until they enter something, but

that could be annoying if they really don't want to give an answer and

that might lead them to break off and then this whole strategy backfires.

An alternative would be, to provide a don't know or

refuse to answer options, or decline to answer.

Which in a way invites respondents to use those options but

by combining that with an interactive prompt, which says essentially we value

your answer won't you please consider providing an answer.

This could in the end lead to less item nonresponse,

2.3.3 Speeding Interventions

Data Collection: Online, Telephone and Face-to-face

与讲师见面