So in this module, we want to talk about performance evaluation.

And as most of you know from having worked into organizations,

there are couple of purposes.

One is rewards and punishment.

Mostly, firms tie rewards whether it's a raise or bonus or sometimes a promotion

to some kind of performance evaluation, but also firms do this for feedback.

They want employees to get better at their jobs.

This is a means of giving that kind of feedback.

The kind of feedback necessary to get better at your job, but

this is performance evaluation and we're going to focus on performance

evaluation as opposed to talent management.

It's really tough to compare employees if they're not in identical situations.

So, we'll talk elsewhere in the people analytics course about talent management

and comparing employees.

Right now, we really want to focus on evaluating performance management of

a given employee.

For this starting place, for this lecture and a little bit for life,

you might beginning by assuming all employees are equal ability.

That's a rough decent starting place for what we want to do here.

Begin by assuming all employees are equal ability and now we're trying

to figure out how hard are they working, and how can we improve them at their job.

So, the fundamental challenge in performance evaluation

is that performance measures are noisy.

Whatever means we want to use for evaluating how are employee is performing

is imperfectly related to what they're actually doing with how hard they're

working or how smart they're working and that slippage is essentially noise.

This is how a statistician will think about it.

This is how an economist will think about it.

There are noise in the performance measures and

this is going to be a problem for us, if we're trying to infer

from these performance measures something about the employee.

You can think about it as a normal distribution.

So for any given level of effort,

you might see the performance measure line up perfectly with that level of effort or

you might see it come in a little high or a little low.

So, a range of outcomes can occur due to factors outside of the employee's control.

So no matter how hard they work or how smart they work, they have competitors,

they have team members,

they have bosses, they have an economy that they're interacting with.

All of these things are outside the employees control.

And so there's noise, there's slippage between what the employee tries to do.

How hard they work, how smart they work and

the performance measure that results as a consequence of that.

So this is our challenge, separating skill from luck.

And for a person like me, this is one of the great challenges in life.

You're forever trying to separate from what we see, how much is luck and

how much is skill.

There's an economist at the University of Chicago,

John Huizinga who said recently in an interview.

This distinction between ascertaining skill and luck shows up all the time.

Who do you give your money to for investing?

How much do you pay for a certain employee?

It's everywhere.

It isn't just about sports, it's about life.

So, that's really what we're trying to do here in this performance

evaluation module.

We're trying to separate skill from luck.

You might think of it as signal versus noise.

We want to understand the true signal even though we only get noisy performance

measures.

So in some environments, this isn't so hard.

For example, in this picture, I've shown two normal distributions and

you can think of this as what might result if an employee gives high effort or

if the employee gets low effort.

And we're going to use, for our simple model here, we're going to use effort, but

that's just a simplification.

You can think of this as working smart.

You can think of this, you can even though we want to avoid for

the moment thinking of it as being a high talent versus low talent.

So for the moment, let's just think of it as high effort.

Are they working hard or are they not working hard?

In the world I've described in this picture,

the outcome measures that result from high effort are low effort are quite different.

So you see that the high effort distribution is quite a bit higher,

quite a bit closer to the good outcomes than is the low effort distribution.

So that when you get a performance measure,

that's pretty high on the good outcomes scale.

You can be pretty sure that came from high effort, so

you can infer this employee did something well, this employee did something right.

You can give them positive feedback.

Conversely, if you see something in this world that is very,

very low on the outcome scale close to the bad outcomes end,

then you can be quite confident that,

that came from low effort just because these signals are quite separate.

So in this environment,

it's not very hard to tell whether your employee is working hard or not.

You get pretty clean signals.

Now, it's not perfect.

If you get an average performance measure, it can be ambiguous and

that's going to be the case often.

But for broad range here, a broad range of outcomes,

you can be pretty confident about what kind of effort the employee gave.

But another environment says, brutal to determine whether the employee gave high

effort or low effort, because of how overlapping the distributions are.

So for example, in this situation,

the good outcome measure more likely came from high effort, but

there's still a very high reason of probability that it came from low effort.

That maybe someone didn't put in much work, but

the economy broke just right for them or the bus covered them or

the team they worked with happened to be really on that matter.

All of those things are outside the employees control that can lead to a low

effort performance resulting in positive consequence and the vice versa.

You may get a bad outcome and think on average that came from low effort, but

there still a very reasonable chance, because of the noise in the measure.

That is came from a high effort.

Again, it could be that they put in 80 hours that week, but

their team mates let them down or their boss made a bad decision or

their competitor made an unexpected move.

So in these environments, it's very difficult to ascertain the level of effort

from a noisy performance measure.

This, in essence is the challenge with performance evaluation.

It's not always quite this difficult.

There's not always this much overlap, but there's always some level of noise and

it makes it difficult to give the right kind of feedback.

We want to give you some help sorting this environment, because all of you are been

judged in this way to some extent and all of you're judging employees in this way.

So this is going to motivate everything that come's next, essentially.

We're going to give you some examples, some techniques.

And importantly,

some biases that people have whenever they try to make these kinds of inferences.

So the module plan, first is to give you an extended example and

then drop into four specific issues and unpack those, because there are biases

associated with each of those issues and that will lead to some prescriptions,

which we provide as we go along and then we summarize in the last stage.

Performance Evaluation: the Challenge of Noisy Data (People Analytics)

商业分析毕业项目

与讲师见面