Skip to content

MANE 3332.03

Lecture 18, April 1

Agenda

  • Midterm exam not graded
  • Chapter 5 Material
  • Chapter 6, time permitting
  • Linear Combinations Practice Problems - due April 3, 2025
  • Attendance
  • Questions?

Handouts

Class Schedule

Tuesday Lecture Thursday Lecture
4/1: Chapter 5 4/3: Chapters 7 & 8
4/8: Chapter 8, Case 1 4/10: Chapter 8, Case 2
4/15: Chapter 8, Case 3 4/17: Chapter 9, Case 1
4/22: Chapter 9, Case 2 4/24: Chapter 9, Case 3
4/29: Chapter 11 5/1: Chapter 11
5/6: Review 5/8: Dead Day (no class)

11 Sessions plus final exam

Final Exam: Tuesday May 13, 2025 10:15 am - 12:00 pm

Chapter Five

  • Joint Probability Distributions
  • Contains eight sections
  • We will only examine 5.4 (Covariance and Correlation) and 5.6 (linear functions of random variables)

Covariance and Correlation

Covariance

  • When two or more variables are defined on a probability space, it is useful to describe how they vary together

  • A common measure of the relationship between two random variables is the covariance

\[ \begin{aligned} \sigma_{XY}&=&E(XY)-\mu_X\mu_Y \end{aligned} \]

Covariance, continued

  • Theoretically for two continuous random variables with joint probability distribution function \(f_{XY}(x,y)\), the covariance is found by
\[ \begin{aligned} \sigma_{XY}&=&\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}xyf_{XY}(x,y)dxdy-\mu_X\mu_Y \end{aligned} \]

Covariance and Independence

  • If \(X\) and \(Y\) are independent random variables,
\[ \sigma_{XY}=0 \]

  • However, \(\sigma_{XY}=0\) does not imply that \(X\) and \(Y\) are independent. Textbook mentions Figure 5-13(d)

  • SPECIAL CASE. IF \(X\) and \(Y\) are normal random variables and have \(\sigma_{XY}=0\), then \(X\) and \(Y\) are independent

Sample Covariance

  • To calculate the sample covariance use
\[ s_{XY}=\frac{1}{n}\sum_{i=1}^n\left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right) \]
  • Easily done in software

Correlation

  • The correlation between two random variables \(X\) and \(Y\) is
\[ \rho_{XY}=\frac{\mbox{cov}(X,Y)}{\sqrt{V(X)V(Y)}}=\frac{\sigma_{XY}}{\sigma_X\sigma_Y} \]
  • For any two random variables \(X\) and \(Y\)
\[ -1\leq\rho_{XY}\leq 1 \]
  • If \(X\) and \(Y\) are independent \(\rho_{XY}=0\). The converse is not true.

Sample Correlation Coefficient

  • To calculate the sample correlation coefficient,
\[ r_{XY}=\frac{s_{XY}}{\sqrt{S^2_XS^2_Y}} \]

Summary

  • Correlation is a linear measure and will not work for non-linear relationships

  • Correlation is a measure of association; it does not prove cause and effect relationships

    -Examine examples at Spurious Correlations website

Linear Functions of Random Variables

Functions of Random Variables

  • Additive System. Let \(X\) be a random variable with mean \(\mu\) and variance \(\sigma^2\). Define a new random variable \(Y\)
\[ Y=X+c \]

It follows that

\[ \begin{aligned} E(Y)&=&E(X)+c=\mu+c\\ V(Y)&=&V(X)+0=\sigma^2 \end{aligned} \]

Linear Functions of Random Variables

Functions of Random Variables

  • Multiplicative System. Consider the new random variable \(Y\)
\[ Y=cX \]

It follows that

\[ \begin{aligned} E(Y)&=&E(cX)=cE(X)=c\mu\\ V(Y)&=&V(cX)=c^2V(x)=c^2\sigma^2 \end{aligned} \]

Linear Combination

  • A linear combination of the random variables \(X_1,X_2,\ldots,X_n\) is
\[ Y=c_1X_1+c_2X_2+\cdots+c_nX_n \]
  • The mean of a linear combination of random variables is
\[ E(Y)=c_1\mu_1+c_2\mu_2+\cdots+c_n\mu_n \]
  • The variance of a linear combination of random variables is
\[ V(Y)=c_1^2\sigma_1^2+c_s^2\sigma_s^2+\cdots+c_n^2\sigma_n^2 \]

Linear Combination of Non-independent R.V.

Let \(X_1,X_2,\ldots,X_n\) be random variables with means \(E(X_i)=\mu_i\), variances \(V(X_i)=\sigma^2_i\) and covariances \(\mbox{Cov}(X_i,X_j)\) for \(i,j=1,2,\ldots,n\) with \(i<j\)

  • The linear combination is defined to be
\[ Y=c_1X_1+c_2X_2+\cdots+c_nX_n \]
  • The mean of \(Y\) is
\[ E(Y)=c_1\mu_1+c_2\mu_2+\cdots+c_n\mu_n \]

Linear Combination of Non-independent R.V.

  • and the variance is
\[ \begin{aligned} V(Y)&=&c^2_1\sigma^2+c^2_2\sigma^2+\cdots+c^2_n\sigma^2_n\\ &&+2{\sum\sum}_{i<j}c_ic_j\mbox{Cov}(X_i,X_j) \end{aligned} \]

Linear Combination Problem

linear combination problem

Linear Combination Practice Problems

Central Limit Theorem

If \(X_1,X_2,\ldots,X_n\) is a random sample of size \(n\) taken from a population with mean \(\mu\) and variance \(\sigma^2\), and if \(\overline{X}\) is the sample mean, the limiting form of the distribution of

\[ Z=\frac{\overline{X}-\mu}{\sigma/\sqrt{n}} \]

as \(n\rightarrow\infty\), is the standard normal distribution

  • Incredibly useful theorem

  • See example below

  • \(n\) often does not have to be very large

    • If the population is continuous, unimodal and symmetric, often \(n\) can be as small as 4 or 5

    • Larger samples will be required in other situations

    • If \(n\geq 30\) the normal approximation will work satisfactorily regardless of the shape of the population

CLT Illustration

central limit theorem illustration