[MUSIC] Welcome back, so now we're going to basically see a use case in the realm of the domain of people analytics. Where in some sense, a machine is trained to provide certain answers. We saw in the earlier half of today's session direct problem solving from real world questions to real world answers through experimentation. Now, we're going to take the detour. We're going to take the longer route. So we will in some sense, create a model, analyze the model using analytics, interpret the model results. And thereby, come up with the real world answers to those real world questions. Survey research has been around for a long time in marketing research. It basically has two core components, questionnaire design and sampling. Many questionnaires use what is called a Likert scale to try to measure and integrate various dimensions of relatively complex constructs. Some may find survey methods, questionnaires, and so on. And the Likert scale quite low tech, and frankly, quite off-putting, right? I mean, come on, this is rooted in the last century. Well, let's see one real-life instance where humble, simple Likerts led the way for some relatively fancy analysis. Predictive analytics actually, where machines could be trained. And that's where we are headed now. Think of a call center. Yeah, that's what a call center might look like. A bunch of people, and so on. Yeah, it could be inbound, could be outbound. You are the hiring manager, let's say for a call center. How would you go about making your staffing decisions? One of the things you might want to do perhaps, is call for resumes of prospective candidates. And once the resume arrives, you might look at things like experience, education, references, and so on, right? Demographic things perhaps, okay. But that may no be enough. So you in some sense, also want to hold a personal interview to get a better sense of what the person is like. All right, so while a resume might tell you demographic things about a candidate, right? Things like education, gender, age, and so on. What about the psychographic aspects? What about things like values, attitudes, interests, lifestyle, and so on. Which might or might not but typically might, at least in a call center kind of a job, have a bearing on job performance perhaps. After all, psychographics are quite important. And that, in some senses, where the Likert is coming, because Likert have been used to measure conflicts, constructs things like attitude, things like values. What you see here is a set of Likert psychographic questions, right? On a 1 to 5 scale, people just answer them based on whether they sound true or not for their particular context, for their self perception. Now, when looking for workers to staff its call centers, Xerox Corporation basically, used traditional hiring methods. It's calling for resumes, doing interviews, and so on. It valued past experience on the job. And then a computer program from a startup called Evolv, incubated by Stanford, came along and said what you're doing isn't rigorous, we can do better. One of the controversial things they said, was that experience on the job actually doesn't matter. Now, Evolv designed this algorithm that crunches, and basically, matches psychographic profiles with work outcomes. So your profiles, your outcomes, and we're going to find that part of your profile that basically predicts a work outcome. The claim is that conventional hiring methods are short of rigor. A statistical approach to hiring should actually improve results by, in some sense, eliminating managers' hiring biases. So let us see what happen, okay? What did Xerox do? Xerox basically said, you know what you're saying is interesting, here's what we'll do. We'll give you three months and we'll basically, give you 20% quota in some sense, right? So you fill up, fill it up. And then we will compare how you did or how the hires that you made did vis-á-vis, the hires that we made through our traditional procedures. And then we will see what happens. Evlov's method basically requires job applicants to take a 30-minute test that screens them for personality traits. And these are all questions like, I ask more than. The more questions that people most of the people do. People tend to trust what I say. One to seven, yes no, and so on. Now, the firm has a database of current and past worker psychographics and of job performance. What will it do? It will statistically match the responses given to the psychographic questions, look for patterns, and basically match them with work outcomes. It assigns a score, green means great hire. Red means no. Yellow means hold to each applicant so the scoring mechanism that's coming about in the end. So this is what is going on. Yeah, so in some sense, it has data on inputs psychographics and other things demographics. It has data on outputs, outcomes, and it's going to statistically match them. Okay, what happened? Did it work? Turns out it did. Attrition dropped 20%, and that is a significant number for something like a call center. I mean, the sector sees a lot of churn, a lot of attrition. On-job performance improved by a similar amount, by 20%. Not just that attrition goes down, performance goes up by 20%. The algorithm uncovers correlations the naked eye cannot see. Here are some fun facts, right? Who would you rather hire A or B? Candidate who used Internet Explorer or Safari to answer your questionnaire. Or basically a candidate who used Firefox or Chrome? Who would you rather hire? Turns out, well, the algorithm says that it is better to hire candidate B. Now, people have post [INAUDIBLE] come out with explanations for why they're maybe none, there is no theory involved. It's basically just using data matching. One of the interesting explanations people came out with, is because perhaps using Firefox or Chrome requires you to go out of your way showing a sheet of a download, and then install another browser. Maybe that has an impact on job performance, I don't know. Nobody does, all right, nobody knows. Here is another question, who would you rather hire, A or B? A is joined 1-2 social networks, B is on 3-4? It turns out A is a better hire in this case, nobody knows why, it's basically data pattern speaking, right? The algorithm doesn't know why. Questions based on Evolv-Xerox, how did a humble set of Likert basically inputs to this fancy analysis? What in the data enables such analysis? This has implications for the value of data and data collection. And about machine learning, how does it work? What is it anyway? All right, broadly speaking, what do we mean when we say, train the machine? So let's, in some sense, get here. Here's a question for you. This is based on the continuing Evolv's Xerox example, right? So consider the data Evolv had. It had data of two types. It had data in the individual side, demographics, psychographics. It had data on the firm side, training costs, productivity, turnover rates, and so on. Given this data, Evolv is somehow able to connect job outcomes to psychographic profiles, right? So your outcomes and your inputs, psychographics, demographics, personality profiles. And thereby, make a prediction. I want you to speculate on how this might work. What is the principle that drives Evol's approach, right? So freeform, what is the first thing that comes to your mind? How might this work? Feel free to write a few lines or bullet points, or build a framework or a flowchart. Use symbols if you want to, I'm perfectly fine, right? We'll take a shot at this, take a small break, think about it, write it down. Okay, welcome back again. Recall where we were. Real world question, real world answer. And we are taking the longer road, the detour. All right, we've actually done the abstraction part. So we have qualified personality in cycle graphics into a set of metric Likerts. Now, we're getting to the analytics part, the mathematical system is going to yield mathematical conclusions. So let's see how that works. Call it a virtuous cycle, all right? We start with better measurement. The fact that you were measuring psychographics and personality et al. It starts there. What happens next? Because of better measurement, you end up with better database and this naturally follows. You are measuring both inputs and outcomes, important. Better database records basically mean better analysis. Higher quality X and Y basically means a higher quality function. Better analysis in turn, right? Gives you better predictabilities, right? Your first card prediction will be way off, right? It takes the gap as an input into its next cut prediction. And the next time around, it's going to predict a smaller error. And the third time around, an even smaller error. So basically, better data, better databases, better analysis, better measurements mean better predictions. And each time that happens, your better predictions would imply better outcomes now, right. And each time that happens, you again, feed back into better database records, right? Because they all become measurements again for the next cycle. Stop there and see what happens, right? What just happened? The moment that cycle completed, something very interesting happened. In some sense, it is referencing itself or a course of cycle, right? So once the cycle is complete, right? Once we basically have this whole thing come together, the machine evolves, it learns. It is being trimmed based on the set of outcomes, right? And it is coming up with the sort of function weight, that we as individuals would not be able to figure out, we couldn't program it. The machine learns on its own. It kind of tries to figure out what is that set of functions? What is that set of weights within functions that will yield the best prediction? Think of what the core of the predictive algorithm is, right? That's a brute force computation approach. And what you're doing is crunching millions of possible combinations. If I have 50 questions in my psychographic profile, imagine the number of possible combinations. Questions 1, 17, 21, and 23 lead to, or these and not that lead to. The number of patterns that is possible is enormous. And it is basically, crunching through a lot of thing, right? The predictive models superior to intuition or experience-based mental models, by which basically comes out with the whole question on what do we mean by experience anyway? It means that overtime, we get a sense of what works, what inputs and what back on lead to what outcomes? What the machine is doing rather than relying intuition or experience. It is using brute force to crunch through, and emerge with that function directly. The core point is that the algorithm discovers connections that only years of experience or maybe not even that call of giveness. The implication are potentially profound. And we will see this when we get to the learning curve immediately following. [MUSIC]
