Bayesian multiple regression

Loading...

来自 Duke University 的课程

Bayesian Statistics

545 个评分

Duke University

Bayesian Statistics

545 个评分

课程 4（共 5 门，Specialization Statistics with R）

This course describes Bayesian statistics, in which one's inferences about parameters or hypotheses are updated as evidence accumulates. You will learn to use Bayes’ rule to transform prior probabilities into posterior probabilities, and be introduced to the underlying theory and perspective of the Bayesian paradigm. The course will apply Bayesian methods to several practical problems, to show end-to-end Bayesian analyses that move from framing the question to building models to eliciting prior probabilities to implementing in R (free statistical software) the final posterior distribution. Additionally, the course will introduce credible regions, Bayesian comparisons of means and proportions, Bayesian regression and inference using multiple models, and discussion of Bayesian prediction. We assume learners in this course have background knowledge equivalent to what is covered in the earlier three courses in this specialization: "Introduction to Probability and Data," "Inferential Statistics," and "Linear Regression and Modeling."

从本节课中

Bayesian Regression

This week, we will look at Bayesian linear regressions and model averaging, which allows you to make inferences and predictions using several models. By the end of this week, you will be able to implement Bayesian model averaging, interpret Bayesian multiple linear regression and understand its relationship to the frequentist linear regression approach.

Bayesian simple linear regression8:11

Checking for outliers4:04

Bayesian multiple regression4:47

与讲师见面

Mine Çetinkaya-Rundel
Associate Professor of the Practice
Department of Statistical Science
David Banks
Professor of the Practice
Statistical Science
Colin Rundel
Assistant Professor of the Practice
Statistical Science
Merlise A Clyde
Professor
Department of Statistical Science

Next we discuss Bayesian inference and multiple regression.

We will use a reference prior that provides a default or baseline analysis

that provides a correspondence between Bayesian and frequentist analysis.

We will illustrate this using a data set from earlier on kids cognitive scores,

where we predicted the value of the kids cognitive score from

the mom's high school status mom's IQ score, whether or

not the mom worked during the first three years of the kid's life, and mom's age.

Where the betas are the unknown regression coefficients, and

epsilon is the error term.

For Bayesian inference, we need to specify a prior distribution for epsilon.

Since the cognitive score is continuous,

we'll use a normal distribution with mean 0 and

assume that the variance of the errors is constant across all of the individuals.

We will also need to specify a prior distribution for

all of the unknown regression coefficients and the unknown variant sigma squared.

The conjugate family for this model is a multivariate normal distribution.

Given sigma squared for all the regression coefficients and

a gamma distribution for the inverse of the variants.

For an informative prior distribution, we would need to specify the prior mean,

variance and covariances of all other regression coefficients.

We also need to specify the prior parameters for

the gamma distribution, such as the prior guess of sigma squared and

how many degrees of freedom the distribution should be based on.

This elicitation can be quite involved

because I do not have prior information about the prior variances and

co-variances or other prior parameters in this distribution.

I'm going to adapt a reference prior that is a limiting

case of the conjugate family.

The reference prior that we will use has a uniform, or

flat distribution for all of the coefficients in the regression function,

similar to what we used in the simple linear regression example.

We also use the reference prior on sigma squared that is proportional

to 1 over sigma squared, or 1 over the variance.

The reference posterior distribution in the multiple regression case parallels

simple linear regression, where the marginal posterior distribution for

each of the coefficients is a student t-distribution.

The degrees of freedom for the student t-distribution are n- p- 1.

From the residual degree of freedom where we lose p

plus 1 degrees of freedom for estimating the intercept and

the p coefficients associated with the p predictors.

The posterior mean is equal to the ordinarily least squares estimate.

And the scale in the t-distribution is given by the standard error of beta

from the OLS regression.

These can be obtained from r using the lm function.

Let's look at the posterior distribution for the four predictors in the cognitive

scores example, based on the marginal t-distributions.

These are all centered at their respective OLS estimates of beta,

with the spread of the distribution related to the standard error.

Now it may not always be convenient to show the posterior distributions, and

instead posterior summaries may be more useful for reporting.

Here we have the table of the point estimates given by the posterior means,

the posterior standard deviations, and

95% credible intervals for each of the predictors.

As in the simple linear aggression, the posterior estimates from the reference

prior, that are in the table, are equivalent to the numbers reported from

the LM function in R, or using the confident function.

And these are given by the posterior mean plus or minus the appropriate t quantile

with n-p-1 degrees of freedom times the posterior standard deviation.

For example, given this data we believe there's a 95% Chance that the kids' score

increases by 0.44 to 0.68 with an increase of the mom's IQ score.

The mom's high school status has a larger effect where we believe that there's

a 95% chance that scores of 0.55 up to 9.64 points higher for

moms that have three or more years of high school.

In summary, we've provided a Bayesian analysis for

multiple regression using a default reference prior.

The quantities of the posterior distributions are easy to obtain from

any program that provides OLS estimates.

And while the values are the same for the reference Bayesian analysis in OLS,

credible intervals and confidence intervals have different interpretations.

The credible intervals for the coefficients for work and

age both included 0, suggesting that one or

both variables could perhaps be dropped from the model.

In the next video,

we will explore model selection using the Bayes information criterion.

探索我们的目录

免费加入并获得个性化推荐、更新和优惠。

Coursera 致力于普及全世界最好的教育，它与全球一流大学和机构合作提供在线课程。

© 2019 Coursera Inc. 保留所有权利。

通过 App Store 下载

通过 Google Play 获取