Skip to content

MANE 3332.03

Lecture 26, April 28

  • Topics:

    • Linear Regression (Chapters 11 and 12)
    • Assignments:
    • New: Chapter 9, Case 3 Upper Quiz (assigned 4/29/2025, due 5/1/2025)
    • New: Chapter 9, Case 3 Lower Quiz (assigned 4/29/2025, due 5/1/2025)
    • New: Chapter 9, Case 3 2-sided Quiz (assigned 4/29/2025, due 5/1/2025)
    • Technical Report Two (assigned 4/17/2025, due 5/15/2025)
    • Attendance
    • Questions?

Handouts

Class Schedule

Tuesday Lecture Thursday Lecture
4/29: Chapter 11 5/1: Chapter 11
5/6: Review 5/8: Dead Day (no class)

3 Sessions plus final exam

Final Exam: Tuesday May 13, 2025 10:15 am - 12:00 pm

Technical Report Two: Thursday May 15, 2025 11:59 pm

Simple Linear Regression

  • Regression analysis is a statistical technique for modeling and investigating the relationship between two or more variables.

  • Simple linear regression considers the relationship between a single independent variable and a dependent variable

  • A good tool to examine the relationship is a scatter diagram

Empirical Models

  • An empirical model is a model that captures the relationship between regressor inputs and a response variable that is not based upon theoretical knowledge

  • There are many types of empirical models

  • Discuss wind-powered generator

Simple Linear Regression Model

  • A simple linear regression model is shown below
\[ Y=\beta_0+\beta_1x+\varepsilon \]

where \(Y\) is the dependent (or response) variable, \(x\) is the independent (or regressor) variable and \(\varepsilon\) is the random error term

  • We can use this model to predict \(Y\) for a given value of \(x\)
\[ E(Y|x)=\mu_{Y|x}=\beta_0+\beta_1x \]
  • Assuming \(\varepsilon\) has zero mean and variance \(\sigma^2\)
\[ \begin{aligned} E(Y|x)&=&E(\beta_0+\beta_1x+\varepsilon)=\beta_0+\beta_1x+E(\varepsilon)\\ &=&\beta_0+\beta_1x\\ V(Y|x)&=&V(\beta_0+\beta_1x+\varepsilon)=V(\beta_0+\beta_1x)+V(\varepsilon)\\ &=&0+\sigma^2 \end{aligned} \]
  • Examine the graphic shown below

Figure 11-2

Figure 11-2 on page 283

Figure 11-2

Method of Least Squares

  • The method use to estimate values for \(\beta_0\) and \(\beta_1\) is called least squares and was developed by Gauss

  • Examine figure shown below

  • Minimize

\[ L=\sum_{i=1}^n\varepsilon_i^2=\sum_{i=1}^n\left(y_i-\beta_0-\beta_1x_i\right)^2 \]

  • The solution to this problem is called the least squares normal equations

  • Examine the graphics shown below

Figure 11-3, page 285

Figure 11-3

Equations 11-7 and 11-8 on page 285

Equations

R

  • In this course, we will use R to estimate the parameters and calculate sums of squares quantities

  • Example Problem

Example Problem

Creating Regression Data in R

Hypothesis Test

  • It is possible to perform a hypothesis involving the slope parameter, \(\beta_1\)
\[ \begin{aligned} H_0:\beta_1&=&\beta_{1,0}\\ H_1:\beta_1&\neq&\beta_{1,0}\\\end{aligned} \]

where \(\beta_{1,0}\) is a constant (often 0).

  • Requires the assumption that \(\varepsilon\sim\mbox{NID}(0,\sigma^2)\)

  • The test statistic is a \(t\)-random variable

\[ t_0=\frac{\hat{\beta}_1-\beta_{1,0}}{se(\hat{\beta}_1)} \]
  • A similar test can be formed for \(\beta_0\)

Examining Model Adequacy

  • Two major concerns

    • Does the model provide an adequate explanation of the data?

    • Are the model assumptions satisfied?

Residual Analysis

  • The residuals are defined to be
\[ e_i=y_i-\hat{y}_i=y_i-\hat{\beta}_0-\hat{\beta}_1x \]
  • Examine normality assumption by generating a normal probability plot of residuals

Residual Analysis - Constant Variance

  • Examine the assumption of constant variance by plotting residuals versus fitted values and residuals vs \(x\)

  • Examine if additional terms are required (such as quadratic) by examining residuals vs \(x\)

  • Residuals are often standardized

Lack of Fit Test

  • If there are repeated observations (identical values of \(x\)) a lack of fit test can be performed
\[ \begin{aligned} H_0&:&\mbox{The model is correct}\\ H_1&:&\mbox{The model is NOT correct} \end{aligned} \]
  • The repeated observations allows the \(SS_E\) error term to be partitioned
\[ SS_E=SS_{PE}+SS_{LOF} \]
  • The test statistic is
\[ F_0=\frac{MS_{LOF}}{MS_{PE}} \]