The Wisdom of Crowds: Signal Independence (People Analytics)

Loading...

来自 University of Pennsylvania 的课程

商业分析毕业项目

300 个评分

University of Pennsylvania

商业分析毕业项目

300 个评分

课程 5（共 5 门，Specialization 商业分析）

The Business Analytics Capstone Project gives you the opportunity to apply what you've learned about how to make data-driven decisions to a real business challenge faced by global technology companies like Yahoo, Google, and Facebook. At the end of this Capstone, you'll be able to ask the right questions of the data, and know how to use data effectively to address business challenges of your own. You’ll understand how cutting-edge businesses use data to optimize marketing, maximize revenue, make operations efficient, and make hiring and management decisions so that you can apply these strategies to your own company or business. Designed with Yahoo to give you invaluable experience in evaluating and creating data-driven decisions, the Business Analytics Capstone Project provides the chance for you to devise a plan of action for optimizing data itself to provide key insights and analysis, and to describe the interaction between key financial and non-financial indicators. Once you complete your analysis, you'll be better prepared to make better data-driven business decisions of your own.

从本节课中

Module 3: Your Strategy

In Module 3, you will focus on creating your recommended strategy for GYF to address adblockers. Your strategy does not have to be lengthy, but it must be clear, and it must address the problem. (Hint: if you have a clearly defined problem, your strategy is much more likely to be clearly defined as well). You'll be submitting your strategy for peer review, and then also reviewing the work of at least 3 of your peers. It's OK if reviewing the strategies of other learners in this course gives you further ideas for revising your own strategy. One of the primary benefits of peer review is to expand the range of feedback you can get, and we designed this Module around peer review so that you can get as much feedback as possible before moving on to the next phase of the project. You may find the resources and lectures below helpful in formulating your strategy and considering how data can be leveraged and appropriately understood. You are strongly encouraged to complete and include your response to Application Exercise 2 as part of your Strategy.

Performance Evaluation: the Challenge of Noisy Data (People Analytics)6:46

Finding Persistence: Regression to the Mean (People Analytics)11:56

Extrapolating from Small Samples (People Analytics)5:53

The Wisdom of Crowds: Signal Independence (People Analytics)5:56

Process vs. Outcome (People Analytics)7:47

Hiring 1 (People Analytics)8:50

Hiring 2 (People Analytics)9:07

与讲师见面

Wharton Teaching Staff
Educators
The Wharton School

A third issue that we want to emphasize in performance evaluation is signal

independence.

And to do this I want to begin with this notion of the wisdom of crowds

which has become very popular in recent years.

The name comes from a book by James Surowiecki.

He draws on research that's been going on for probably 100 years.

But the fact that Surowiecki wrote this book led many academics to do

more research in this area.

And it has become a very hot area of research.

The basic observation is that the average of a large number of forecasts

reliably outperforms the average individual forecast.

So the motivating example are these historical county fairs.

Where you might have a large cow, for example, in a pen.

And everyone who comes to the fair gets to guess the weight of the cow.

And the interesting bit, the fascinating bit really,

is that even though most people were quite wrong in their guesses.

The average of their guesses is remarkably accurate.

And this has been shown in county fairs, but also in many other domains, that

the result of all these bets is that the idiosyncratic errors offset each other.

I might be a little bit high.

You might guess a little bit low.

If there's enough of you and

me making these guesses then we tend to get very close to the truth.

So this has been studied in many domains now, and shown in many places, and

it provides a way to get closer to the truth by getting more signals.

Getting more people involved.

Getting more judgements from more people.

This is something that some firms do in performance evaluation, and

more firms should do more of.

But there's one very important caveat, and that is that the value of the crowd

critically depends on the independence of their opinions.

So if, for example in the county fair if, everyone who walked up to the booth

talked with each other before they made their guess,

the value to the crowd would greatly diminish.

Those idiosyncratic errors would be a little bit less idiosyncratic.

Now they'd be related to each other.

If one person was loudly arguing, that he knew because of the breed of cow this was

and the last cow he saw, that this was a particular weight.

He would influence everybody's opinion, their opinions wouldn't be independent,

and the value of those independent signals would be washed away.

So it's not merely a crowd that you need, you need a crowd that is,

to the extent possible, independent of one another.

So independence here means uncorrelated.

If the opinions are actually correlated,

then the value of each additional opinion quickly diminishes.

So if you think you have the opinion of 100 people, if they're actually highly

correlated, then you might only have the opinion of 5 or 10.

It's striking how quickly you rob

a crowd of its value when those opinions are correlated.

Here's a chart of that, this comes from Bob Clemen and Bob Winkler, down at Duke.

1985 study where they just kind of worked out mathematically,

what's the equivalent number of experts, for a given number of experts, and

the degree to which those experts are correlated.

So, here's what they found.

If the correlation is 0, in other words if the experts are perfectly independent,

then every expert you add, creates that much new value.

So the equivalent number of independent experts is the same.

But when those expert opinions become correlated even at 0.2 which is a pretty

low correlation, even at 0.2 you quickly lose the value of adding experts.

So, for example, in this chart it shows that as you go from 0 experts to 9,

if you have a correlation of 0.2, you never quite get above 4.

You asymptote there where you have 9 judges, but because there's a little

correlation between them, the effective number of judges is only 3, 3 and a half.

If the correlation is 0.4 it plateaus much more around 2.

If the correlation is 0.8,

you're not getting much more than 1 opinion even though you've got 9 experts.

This should be very sobering to us.

So we want to push you towards crowds.

We want to push you towards more opinions,

more assessments because that's going to help.

But you've got to simultaneously try to maintain the independence.

Of those opinions.

In a recent study by Florian Zimmerman and colleagues from Zurich,

they found that even when you tell people the correlation.

Even when you tell them the exact structure of where the correlation is,

how strong the correlation is, people don't properly adjust.

It is sobering to think about how people deal with

correlated opinions where it's actually not laid out for them explicitly.

This is not something people deal with very naturally.

Where does the correlation come from?

How can we think of signal independence in terms that

we are familiar with in organizations?

Well, the most obvious is that it happens when people have discussed

the issue together.

When they've had conversations, when they've bounced their opinions off of each

other, as soon as that's happened, they no longer have independent opinions.

It can happen if they simply talk to the same people.

Maybe they don't talk to each other, but they both talk to the same third party.

Or they share the same friends.

Or they went to dinner with the same group on different nights before.

All of these things break down independence,

all of these things increase correlation.

It even happens when people have the same background.

If they're from the same place, if they've trained the same way,

if they have the same advisers, if they have the same historical experiences.

All of these things tend to increase the correlation in their opinions.

Therefore, all of these things break down the independence and

decrease the value of adding additional voices to whatever process you're running.

So, we push towards a broader sample.

We push towards more sources, more assessments.

But with the very important caveat, you need to build those as independently

as possible and then design processes to keep them as independent as possible.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2018 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取