High again. Qualitative research,
though very meady and interesting cannot tell you
how an entire population would react to a product or project.
The research is not generalizable for a few major reasons.
After this lesson you'll be able to explain and
communicate limitations of qualitative research,
identify the risks and explain how to mitigate them.
You will also be able to describe interviewer and
respondent bias and explain ways to moderate both.
First, respondents are purposefully chosen and not randomly picked.
They are chosen due to certain demographics or characteristics.
They also may be chosen as a result of
snowball sampling where one participant suggests another.
Second, the individual interviews or
focus groups yield a number that is not quantifiable.
Whereas with quantitative research the magic number is usually
400 respondents that have been randomly chosen.
Third, the responses are not quantifiable during a focus group
since a number of respondents and responses from them are too few.
Rather you should summarize them as few,
some, many, most or all.
Fourth, research is heavily dependent on the skill of the researcher.
A bad or sloppy interviewer can bias results.
Fifth, due to the volume of information the analysis can be time consuming.
And lastly, sometimes anonymity can be compromised when presenting findings.
The presence of the researcher during the interview or focus group can
affect the responses of the participant.
Some of these limitations,
like a small number of respondents chosen on purpose,
are limitations we accept, some pose risks.
You can mitigate by developing your skills.
I'd like to focus on those.
Considering all the limitations just listed,
the last one related to the interviewer affecting
the respondent is one area where practice makes perfect.
To get sound findings you need to be aware of how to make
yourself the least intrusive so that the respondent can speak freely.
Think of your friendly counselor.
Does she tell you how to respond?
Does he give you a time to formulate your own words?
Does she interject herself into the scenario or does he remain neutral?
Does she make you feel like you have to please her with
your response or do you feel like
your counselor is accepting of whatever you have to say.
A good qualitative research moderator needs to act like your friendly counselor.
You pretend that you are very interested for
the respondent to teach you about whatever their feedback is about the project.
Three areas that are under your control are: 1) leading up respondent,
2) interviewer bias and 3) participant or respondent bias.
First, one of the risks of
qualitative research is that it is possible to lead the participant.
Leading happens when the moderator gives an idea to
the participants of what they expect in the response.
So if I were to say,
you look really sad today,
how are you feeling,
I would be leading a person in a direction that might not be necessarily how they feel.
Second, interviewer bias happens
as a result of how and where you were doing your interviewing.
The place of the interview alone can cause bias
and the way you phrase a question may reveal how you feel about the situation.
So if you interview someone at their work they might be more
guarded than if you met them at a local restaurant.
So think about how you best respond to questions from friends,
family, colleagues or bosses.
When do you feel the most comfortable?
Most likely you feel the most comfortable with
someone who seems accepting of any comment you make,
someone who doesn't judge you.
As a moderator or interviewer,
you need to think of yourself as the confidant to the person you're interviewing.
You want to create a safe place where they can speak their mind.
That will help mitigate the risk of interviewer bias.
Next let's look at respondent bias.
Respondent bias or participant bias usually happens in
a focus group atmosphere where some participants feel pressured to respond in
a way that the majority of the group is responding or
don't want to enter their opinions with others since they feel it wouldn't be accepted.
For example, I had a group once of AIDS patients.
There was one group that was just an I.V.
drug user group period.
Within that group there were I.V.
drug users, homosexual men and other ethnicities.
It turned out that a couple of the homosexual participants
blamed the people from another culture and vice versa.
It ended up being a very tense situation.
One type of people felt like they had to answer the questions in
a certain way due to the ethnicity or characteristic of the other.
I had to listen very carefully and
interrupt and tell them that all of their opinions are valuable.
I wanted to hear from each and every one of them individually.
Everything worked out in the end.
You can mitigate respondent bias by not letting
people who know each other sit next to each other.
Be aware that some participants may be forceful
and try to get recognition from the group while they speak.
Part of your job is to make sure
that other participants can voice their opinions right away,
especially if another participant is forcefully presenting their bias.
Once you're aware of the limitations and risks of the qualitative research,
part of your job is to communicate this clearly to clients and stakeholders.
You should always be clear from the start with
your client about the strengths and limitations of
qualitative research so that they can make
a good choice as to what kind of research they want to conduct.
I would run through all the limitations I've outlined here in
the proposal for the research as well as in the final report.
Many people want a qualitative research project to be
generalizable to the public and thus want to run with your findings.
They should know upfront how much they can rely on the data you collected.
So for example, when I conducted focus groups about
immunization services I knew that the results would be time sensitive.
The client refined their advertising campaign as a result of the research.
The research showed immunizations are more acceptable by
certain ethnic groups who shied away in the past.
Therefore, it is important to relate this limitation to the client.
The fact that findings are time sensitive can influence their decision making
therefore in this case I needed to alert the client
to this limitation to help them make the best decision.
Here's an example of a time I needed to emphasize
the limitations of qualitative research for the good of the client.
After a series of 15 individual interviews with a local group of
hospital physicians about what they felt about
the hospital atmosphere and their productivity,
the hospital's CEO wanted a presentation to
his corporate board as well as to senior staff.
He felt I had to have all the answers and that
the hospital should be totally revamped after
many staff expressed their disapproval about their jobs.
I had to explain to him that we may need to look at other things besides the interviews
like patient satisfaction or other hospitals in the area and what they offer.
He had no clue about the limitations of qualitative research or
that the remarks of some of the staff did not mean that all the staff felt that way.
That wraps up our lesson on the limitations and risks of qualitative research.
I hope you can see that it is important to understand and accept certain limitations,
mitigate the risks you can and prepare yourself to
communicate these limitations and risks to your clients.
After this lesson you should now be able
to explain and communicate limitations of qualitative research,
identify risks associated with qualitative research and explain how to mitigate them,
describe interviewer and respondent bias and explain ways to mitigate both.
Susan BermanPresident, Impact Research
