Hello and welcome back. Today we'll be talking about one of my favorite methods, interviews. And we'll cover a little bit about both doing interviews with an individual and doing interviews with a group of people, which is also known as focus groups. So why should you try to interview users? It turns out there are many aspects of a person's experience that are difficult to infer directly from their actions. Just observing somebody drink something, sometimes you can't kind of tell if they're liking it, what their motivations are but most of the time you cannot. It also allows you to answer the why questions. Why did the user do something in a particular way? And it allows for reflections on rare activities and events. So for example, I'm not in a house fire every day, but perhaps I was in one at some point. You can interview me about that event, and you don't have to follow me around my entire life just in the hope that at some point, I will be in a house fire and you'll be able to observe my activities and use of technology in that context. So let's go through a quick example of an interview study. So I conducted interviews with families, where parents and children live apart. There were actually two types of family that I was interested in. I was interested in divorced families, where one parent typically lives in a different household from the children. And I interviewed families with a variety of custody arrangement in those families. And I was also interested in work separated families. These are families where there may be a traveling parent. There's lots of different reasons that parent may travel. It could be deployments, a military deployment is a common one in the United States. It could be something like immigration. So one parent may come to a country that they're planning to immigrate to and then bring their family over later. In fact that was the case of my family. My dad came to America and then a year later was able to bring my mom and me over. And I also interviewed the children from both of these families. We focused mainly on the ages between seven and 13. I did and only ask interview questions. I also provided a few kind of structured questions and called those questionnaires and some drawing exercise for the kids because it's sometimes difficult for children to articulate kind of abstract concepts and so asking them draw can be really powerful. So the two drawings here actually, in response of if you could have any magical thing to help you stay connected with your parent, what would you have? And the drawing on top is a robot that carries secret messages and the drawing on the bottom is a magic door that lets dad in at bedtime to read a story. And this doesn't mean that I'm going to, as a designer, or as a computer scientist, go off and build secret-message carrying robots, or magic doors to look dad in but it does actually point to these common issues that children have when they're separated from their parents. So the top one the secret message robot is actually from a divorced family and one of the things it underlines is that kids are kind of very aware of the tension between their parents over their time and affection. And frequently it's try to keep contact with one parent as private from the other parent as possible. And that's why this kid wanted the robot to carry secret messages. And the one on the bottom actually highlights the importance of routine. So there's certain activities that the children are used to doing with a particular parent, and when that routine is disrupted it can be really problematic for the child. So just to give a quick overview of what we found in the study, so one of the things we were really focused on were the goals that each of the parties brought to the picture. And so the ones that are in light gray were actually common for work separated and divorced families. And the one in kind of a darker blue, we only saw in divorced families. So for kids, their main goals are to have fun, no surprises there. They also want attention and support when they want it. So they're very in the moment creatures. If they have to wait until Sunday at 7 PM Skype to tell their mom about their skinned knee, that's not going to work. They're going to forget about it by then. And the other thing that we saw in divorced families, not in military or other work separated families, was that kids are really sensitive to this tension between their parents particularly competition over that time and affection and really try to minimize those tensions. There are also some common goals that kids and parents had. So for example collaborating on daily activity, so both mention that that's when they have the good conversations when they're playing outside or playing board game or when they're doing daily routines like bedtime or getting ready for bed or breakfast. Homework help as a big time that parents actually felt they were very instrumental in the child's life. And found kind of a lot of things to talk about how school went, we're just asking about how school generally generated a very bland answer just like good. And also creative pursuits. So whether that was drawing together or building something out of Legos, those kinds of things. Now in terms of the parents' goals, all the parents we spoke to really kind of wanted to have the information in the power to act in a child's best interest. They wanted maintain a strong emotional connection with the child but that could actually be very difficult when you don't live or you spend a lot of time away from the child. Phone and video chats were not quite variant. So they were not really engaging as ways of interacting the child over distance. And lastly the parents frequently wanted to minimize tension with other parents. The other parent in the family. And this was actually something we found not only in divorced families, where it kind of makes sense. But also is work separated families. Frequently the parent that's at home kind of has to take a greater share of child care responsibilities. They're the ones kind of making sure the routine still happens for the child and the parent that's abroad may want to be really careful about how they disrupts the routines of how much extra obligation they're introducing for the parent who's still at home. So we actually saw this theme across family types. So this is just an example of an interview study and the results from it. So now it's going to how to interview participants. There's generally about four steps to it. So you want to prepare your questions, you want to schedule and arrange meeting, conduct interview, and then analyze. So in terms of preparing your questions, it's very important to actually plan ahead rather than to think okay well, on the spot I'll figure out what I wanted to ask this, and it'll just be kind of like a conversation. So typically I put together something that I call a protocol or a list of questions that I want to ask and follow up questions. We'll talk a little bit more about preparing a protocol in a later video but I just want to make sure that you understand a few things about this. Four things. So one is You really want to have open-ended wording on interview questions. The main strength of interview questions is develop the person tell their story. And so you don't want to ask interview questions that can be answered with the single word that or that ask the participant to select from a list of choices or there a yes or no questions. And this is actually the most common mistake that I see for people who are first developing their set of interview questions. So as you read through it if there is any question that starts with something like do you or in other way can be answered with a yes or no, you want to make sure to rewrite this. So for example instead of asking do you connect with your child over the phone? Ask, how do you connect with your child? Or tell me a story about the last time you connected with your child while you were apart. You also want to make sure to plan the time. So it can be very easy to miscalculate how much time a particular section's going to take. Or in both directions actually sometimes I think a certain set of questions is going to take a very long time. It doesn't actually end up getting them which response from participants and end up ending short. I also note follow-up questions from myself. So generally the interview is kind of like a conversation that is lead by the participant but sometimes I do want to get a specific pieces of information and I may note those for myself in order to get back to them. And the last thing, and the reason I make it big, because I think this is really important, is that anytime you do an interview study you want to Pilot first. You want to actually sit down with somebody who is either somebody in the family, somebody you know, somebody in your lab, somebody in your work group and go through that interview with them asking them to pretend like they're the participant. It's really powerful. It helps you understand which of your questions are confusing. It helps you get a good kind of better metric for how much time each section takes and whether you're going to be done in the time that you promised the participant It's a really valuable experience. And I would also say that if possible, if the people you're interviewing are not in your age range or have some special characteristics, if possible, see if you can find somebody from the appropriate demographic to do the piloting with. So in my case, if I wanted to pilot with children, I usually recruited my little brother, who was at the time much younger than me, so I could sit him down and pretend to be every kid and actually go through the questions with him. Hopefully, you can also find friendly participants who can stand in for the user as you pilot. Now, the second step is scheduling and arranging, and I actually find that in most interview studies, unless you have somebody to help out with this stage, the scheduling and arranging and recruiting take about a forth of the time and the whole study. So it's a fairly significant undertaking. So just make sure that you plan for that in your schedule. You should definitely follow up to confirm the interviews. It takes a lot of time to recruit people. It takes a lot of time to set up and you don't want that to be lost just because somebody forgot the day or because you had written down the wrong location or something along those lines. I also think it's very important to consider the setting and how you present yourself to the user, and we'll talk a little bit about that on the next slide as well. But, the main point that I'm highlighting here is that you should also make sure to be safe. Especially, if you're going and you're doing these interviews alone and you're going to meet the participants somewhere. See if you can instead meet them at a public location like a coffee shop, that's a very reasonable place to do interviews or a public library. Rather than, let's say a participant's home. Or if you're meeting them at a participant's home, perhaps having another person come with you or letting people know where you are and then planning to check in with them as soon as you leave the interview. Just make sure to be safe because it can be dangerous place out there. So let's talk a little bit more about this idea of setting and presentation. You have to really think about how to present yourself and where to do the interviews in such a way that it'll make your interviewee most comfortable and most likely to give you honest answers. So just to give you an example with children. If you sit kids down in a room, that's great, and has great chairs, and looks like it's meant for adults. And if you wear lab coat while asking them questions, you're going to get a lot of demand characteristics. So that means, the kid is really going to be thinking about, wow, this person has a lot of power over me, where they want to hear from me. Let me make sure that I tell them what they want to hear from me, and that's not the kinds of answers that you want. So for example, when I interview kids, I try to find a kid friendly place. So something that has a lot of color, that kid size chairs for example, and I try to dress very casual so I might wear my University of Minnesota t-shirt for example. Now, on the other hand, if I was interviewing high powered CEO's about how to use technology to manage their day, I may want to wear something different. I may want to wear a fancy pants suit or at least my awesome blazer while I meet them. So it's something to think about ahead of time to think about how is the participant going to perceive you. Are they going to be comfortable giving you honest answers, are you going to be taken seriously, and all that kind of stuff. So the third step is actually conducting the interviews. So I usually prefer to audio record the interview, which frees me up to be thinking about follow-up questions. Be really listening to what the participant's saying, instead of madly writing. Of course, this is only with participant permission, and most people, I find are okay with you audio recording, if you say that this is just going to be used to generate transcriptions, it's not going to end up on NPR or YouTube later. I do still take some loose notes but I really focus mostly on listening and thinking about what I'm going to ask as a follow-up when the participant is talking. One thing that you kind of learn as you do a few of these, is strictly follow up on hunches or unusual responses. So sometimes the participant will answer a question but you can't sarcasm on what they said. So follow up on that so you know like you, said you really enjoy the tools that you have currently available for communicating with your child, but I also sense some sarcasm in that, can you tell me more? So that can be a really good way of actually kind of getting to some underlying issues. Sometimes laughter is an unusual response. So you'll ask a question and the participant will laugh first and then answer it. And that could be something to follow on as well. So notice you laugh here tell me a little bit more about that. Why did you do that? The main lesson to learn in during interviews. And this is something that I'm myself struggle with. And definitely when I'm listen to myself after the interviews still find moments around, my God, I just interrupted the participant when they say something interesting. But the main lesson is actually being comfortable with silence. So even three seconds of silence in an interview can feel like an interminable length of time. So for example, here's three seconds of silence. That was really uncomfortable. I was ready to jump in with talking again. Now, in an interview, when a participant has finished answering a question or when you think they have finished answering a question or when you stated the question and they haven't started talking, it maybe really tempting to jump in with a clarification of the question or with the next question or some other follow up, but you really need to give that silence time because the participant is actually thinking, and to them that silence may not be as interminable. And maybe they're thinking, there's one thing I could say but I'm not going to say because it doesn't seem that interesting. That silence may actually kind of get them to say whatever's on their mind. So I would say count to ten. It's very uncomfortable, but it does actually lead to better interviews. And overall, this is kind of a more a learned skill or a craft or an art than it is a science. Learning how to interview well is really about practicing doing it. So one of the things that I like to do, and this is again also very uncomfortable, is to take that audio recording afterwards. And either as I'm transcribing it or if somebody else is transcribing, just listening into it myself on the train, I'm just listening through the whole interview recording and critiquing myself and my skills as an interviewer. Did I jump in somewhere and interrupt the participant? That's something I frequently do. Did I give enough time? Did I seize an opportunity for a follow up based on something the participant said that might not have been obvious? I take those notes and I hope to learn from them and improve as I do the next interview studies. So, analysis is the last step and it's complicated. We'll actually cover it in it's own lecture. We'll have an entire lecture on qualitative analysis and so for now, I'll just let you know that after you have done the interviews, you need to analyze them. So the other thing I wanted to cover a little bit was this idea of digital tools for interviews. So I really do believe that interviews are best done in person, so if you're sitting across the table from somebody you can read their body responses and you can see where they're looking, you can see if they're distracted or with you, you can see if there's anything else going on in their context that might be changing how they're responding. But it's not always possible. Sometimes the participants you want are scattered throughout the country or the world. And those interviews need to be done some other way. So I have in the past done interviews by phone or video chat and audio recorded them and kind of followed the same process as I generally would. I do prefer video chat to the telephone because it does give me at least a little bit of a window into the context of the participant and at least a little bit of insight into up their expressions and kind of where their attention is. But I find that the richest interviews are really get in person, even though it does have that inconvenience of having to travel to the participant or having them travel to the lab. So the one thing I would say is that don't interview over email or chat or some other text based medium. Or if you do maybe don't call it an interview, call it a questionnaire or survey. Basically, anytime the person has to type, you go and get severely abbreviated responses and you're going to get responses that are kind of edited by the participant because When we speak, we speak freely. When we write, we imagine that being seen by others outside of the intended audience. So, in general, I would definitely avoid that and I think for me if somebody says they interviewed over email, that definitely raises a red flag in terms of how that might have biased their data and biased the participant responses. So, I promised that we'd also talk about group interviews which are also known as focus groups. So let's cover that a little bit as well. So sometimes there may be an advantage to interviewing not just one person at a time but a whole group of people. So typically it would be something like three to ten people depending on the context. And you kind of pose a question to the whole group and have them answer it at the same time or having them discuss it. Typically, these groups are very carefully selected, based on the role and demographic of the participants depending on what your research questions are. And the one thing I'd like you to take away about these kind of group selection is, thinking about whether for your questions, for your context, it is more appropriate to have a heterogeneous group or a homogenous group. And what I mean by that is, do you want to have a group where everybody has the same role? So for example, let's say that I, if I was doing my study with kids and parents as a group interview, I didn't do that, but if I did I may think about inviting a group of five dads to talk, and maybe a group of five moms to talk, and maybe a group of five kids to talk. And so those are groups that are homogeneous. Everybody there has the same role. Now on the other hand, I may take a different approach. I may have a heterogeneous group where I have the same family where everybody has multiple roles all be in the same group interview. So a single family with let's say mom, dad, and the kids being in the same room as I'm interviewing them and kind of providing me feedback of what's going on. So there are advantages of each. I think in general it's just important to think kind of what questions are you asking. If you're asking a lot of questions that are really directed at the dads but you have a heterogeneous group, then half of that group or more than half of that group is not going to be able to contribute to that question or to that answer, and so that may not be what you're going for. S,o how do you decide whether to interview people as individuals or to interview them in a group? So I've kind of made a little bit of a comparison between the two. So definitely it's more time and effort that's spent with each participant when you're interviewing individuals. And that's both a pro and a con. So if you don't have a lot of time, you are spending that hour, that hour and a half, or maybe even two hours if you can, kind of getting there and getting back with each participant. But that also means that you're getting richer responses, because you do have that time to have that silence after responses or to really give the participant time to think before they jump in with their answer. With individuals, it is easier to coordinate the logistics because typically it's just fine to have two people meet up, the interviewer and the interviewee. And it's finding a place that works for both of you, finding a time that works for both of you. It's not that hard. And I think it's also a lot more privacy for the participants. The responses they're giving are not being heard by everybody else in the room, for example. So they might be more honest. On the other hand, in groups, it is frequently easier to solicit more feedback because, if you ask a question and maybe in the group of five, one participant doesn't actually have an answer to the question, the other four will jump in and start talking. And perhaps through that discussion, that fifth participant will figure out something interesting to say about this point as well. I really find that these discussions between the participants that occur when they're answering questions can really highlight things like their assumptions or what's really going on or kind of underlying issues. And these group discussions can really be fascinating sources of data. Now, one negative however is that you may need to deal with confounding personalities. So, sometimes you may have somebody who's really quiet in the group or somebody who's incredibly outgoing has a lot of opinions. And that might be a problem because the quiet person might not feel like they can offer any of their opinions, at worst your whole interview is kind of dominated by some one dominant personality. And lastly, there's this issue of what we call presentation of self. So, for example if you're interviewing parents about parenting, parents frequently feel judged for how they parent. So they may want to present themselves in the best light to the other parents in the group interview. So they may not give you entirely honest answers or may really try to kind of gloss over their actual parenting practices and present it in a particular light. And typically, when you're interviewing, that's not what you're going for. You really want honest responses from participants. So, we're wrapping up on this video. I know this is a long one. So bear with me. Well, we just finished talking about the strengths and weaknesses of interview methods. The strength of it is that it can work with no knowledge or hypothesis before starting. This is not a hypothesis driven method, this is not a method where you kind of know what's going to happen before you conduct the interview. But rather one where you're really trying to gather more information from the participants. It does provide you with that rich and nuanced data about participants' lives. You're really getting them to tell their stories for you. And that can be a really powerful experience, and because you're really getting them to articulate their experience, it reveals all those internal invisible aspects of their experience. So whether it's motivation, why did they do something? Affect, how did that really feel? Identity, is this who you are, how does this match with who you are? Their priorities, their preferences, their mental models. And it also allows them to reflect on rare past experiences rather than something that happens to them every day that you can just observe. Now of course there are also weaknesses to this method. One big weakness is that recall may be biased. So, you're asking participants to think about something that's happened in the past, or to reflect on something that's happened in the past. They may not be remembering it accurately or correctly. Their reflection in reporting may also be biased. So, whether that's about presenting themselves in the best light to you or really kind of considering how, if let's say their boss saw the answers they give to the interviewer how that would reflect on them. And it's not really what they actually do, it's kind of how they reflect on that. So, one example of this is if you ask me, what does a professor do during the day? What do you during your day? In an interview I might give a set of answers and I'm trying to be honest but I'll say I spend 20% of my time doing teaching. I spend this amount of time doing research, I spent some time on service. Whereas in reality if you actually followed me throughout my day, you would see that the one pervasive activity that I spend the most time on is actually email. Which is not one of the things that I listed because maybe I didn't think about it at all. So, sometimes when you're asking people to reflect, or report, they will give you a different answer from what actually happened in real life. And lastly, interviews are a qualitative method. In order to get that nuanced data, that rich data, you really do have to spend a lot of time and effort with each user. So you can usually only get to only a few small numbers of users. You're really not looking for a representative sample here. You're really not trying to say something that's quantitatively significant about the entire population. You're really just trying to get the stories that your participants can provide you about their daily every day experience. So, if you'd like more information, this is my favorite book about being an interviewer, Interviewing as Qualitative Research, I think it's really good and it's really simply written so if you end up using this method, I definitely recommend you take a look at this book. It's quite good. Well, thank you for bearing with me for this long video. And I'll see you next time.
