Início » Análise de Regressão » 1. Regressão Linear Simples

1.3 Propriedades dos Estimadores

Os estimadores de mínimos quadrados $ \widehat\beta_{0} $ e $ \widehat\beta_{1} $ possuem importantes propriedades: são não viciados e têm variância mínima entre todos os estimadores não viciados que são combinações lineares dos Yi (Teorema de Gauss-Markov). Desta forma, os estimadores de mínimos quadrados são frequentemente ditos "melhores estimadores lineares não viciados".


1. Valor esperado (média) de $ \widehat{\beta}_{1} $:

Definindo

$$C_i=\dfrac{x_i-\bar{x}}{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2}\ \ i=1,\dots,n,$$

segue que, 

$$\widehat{\beta}_1=\dfrac{\displaystyle\sum\limits_{i=1}^n(x_i - \bar{x})Y_i}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=\sum\limits_{i=1}^n C_i Y_i.$$

Desta forma,

$$E(\widehat{\beta}_1)=E(\sum\limits_{i=1}^n C_i Y_i)=\sum\limits_{i=1}^n C_i E(Y_i)=\sum\limits_{i=1}^n C_i (\beta_0 + \beta_1 x_i)=\beta_0 \sum\limits_{i=1}^n C_i + \beta_1 \sum\limits_{i=1}^n C_i x_i.$$

Como

$$\sum\limits_{i=1}^n C_i = \dfrac{\displaystyle\sum\limits_{i=1}^n(x_i -\bar{x})}{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2}= \dfrac{\displaystyle\sum\limits_{i=1}^n x_i -n\bar{x}}{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2}= \dfrac{\displaystyle\sum\limits_{i=1}^n x_i -\sum\limits_{i=1}^n x_i}{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2}=0$$

$$\text{e\ } \sum\limits_{i=1}^n C_i x_i= \dfrac{\displaystyle\sum\limits_{i=1}^n(x_i -\bar{x})x_i}{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2}= \dfrac{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2}{\displaystyle\sum\limits_{i=1}^n (x_i -\bar{x})^2} = 1,$$

concluímos que $ E(\widehat{\beta}_1)=\beta_{1} $ (estimador não viciado).


2. Variância de $ \widehat{\beta}_{1} $:

De (1) temos que

$$Var(\widehat{\beta}_1)=Var \left(\sum\limits_{i=1}^n C_i Y_i \right).$$

Como Yi, i=1,...,n são variáveis independentes, segue que

$$Var(\widehat{\beta}_1)=\sum\limits_{i=1}^n C_i^2 Var(Y_i) = \displaystyle\sum\limits_{i=1}^n \left(\dfrac{(x_i - \bar{x})}{ \displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}\right)^2\sigma^2=\dfrac{\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}.$$

Considerando n pares de valores observados (x1,y1),...,(xn,yn), podemos escrever

$$Var(\widehat{\beta}_1)=\dfrac{\sigma^2}{S_{xx}}.$$


3. Valor esperado (média) de $ \widehat{\beta}_{0} $:

$$E(\widehat{\beta}_0)=E\left[\bar{Y}-\widehat{\beta}_1\bar{x}\right]=E(\bar{Y})-\bar{x}E(\widehat{\beta}_1)=E\left(\sum\limits_{i=1}^n\dfrac{Y_i}{n}\right)-\bar{x}\beta_1= \sum\limits_{i=1}^n\dfrac{E(Y_i)}{n}-\bar{x}\beta_1.$$

Como $ E(Y_i)=(\beta_0+\beta_1x_i) $, segue que

$$E(\widehat{\beta}_0)= \dfrac{1}{n}\sum\limits_{i=1}^n(\beta_0+\beta_1x_i)-\bar{x}\beta_1= \beta_0+\beta_1 \dfrac{\displaystyle\sum\limits_{i=1}^n x_i}{n}-\bar{x}\beta_1= \beta_0.$$

(estimador não viciado). 


4. Variância de $ \widehat{\beta}_{0} $:

$$Var(\widehat{\beta}_0)=Var \left(\bar{Y}-\widehat{\beta}_1 \bar{x}\right) = Var(\bar{Y}) + Var(\widehat{\beta}_1 \bar{x})-2 Cov(\bar{Y},\widehat{\beta}_1\bar{x}).$$

Notemos que

$$Cov(\bar{Y},\widehat{\beta}_1\bar{x})=E(\bar{Y}\widehat{\beta}_1\bar{x})-E(\bar{Y})E(\widehat{\beta}_1\bar{x})=E(\bar{x}\bar{Y}\widehat{\beta}_1)-E\left( \sum_{i=1}^n\dfrac{Y_i}{n}\right)\bar{x}\beta_1$$

$$=E\left(\bar{x}\dfrac{\displaystyle\sum\limits_{i=1}^n(\beta_0+\beta_1x_i+\varepsilon_i)}{n}\widehat{\beta}_1\right)-\dfrac{\bar{x}\beta_1}{n}\sum\limits_{i=1}^n(\beta_0+\beta_1x_i)=\dfrac{\bar{x}}{n}\sum\limits_{i=1}^n \left[ \beta_0\beta_1 + x_i\beta_1^2+E(\varepsilon_i \widehat{\beta}_1)\right]$$

$$-\dfrac{\bar{x}~\beta_1}{n}\sum_{i=1}^n(\beta_0+\beta_1x_i)=\dfrac{\bar{x}~\beta_1}{n}\sum_{i=1}^n(\beta_0+\beta_1x_i)+\dfrac{\bar{x}}{n}\sum_{i=1}^nE(\varepsilon_i\widehat{\beta}_1)-\dfrac{\bar{x}~\beta_1}{n}\sum_{i=1}^n(\beta_0+\beta_1x_i)=\dfrac{\bar{x}}{n}\sum_{i=1}^nE(\varepsilon_i\widehat{\beta}_1).$$

Como

$$E(\varepsilon_i\widehat{\beta}_1)=E\left[\varepsilon_i\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})Y_j}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}\right]=\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})E\left[\varepsilon_iY_j\right]}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})E\left[\varepsilon_i(\beta_0+\beta_1x_j+\varepsilon_j)\right]}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}$$

 

$$=\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})[\beta_0 E(\varepsilon_i)+\beta_1x_jE(\varepsilon_i)+E(\varepsilon_j\varepsilon_i)]}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})E(\varepsilon_j\varepsilon_i)}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=0,$$

já que para $ i\neq j $, 

$$E(\varepsilon_j\varepsilon_i)=0\Rightarrow E(\varepsilon_i\widehat{\beta}_1)=0.$$

e para $ i=j $,

$$E(\varepsilon_j\varepsilon_i)=\sigma^2\Rightarrow E(\varepsilon_i\widehat{\beta}_1)=\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=\sigma^2\dfrac{\displaystyle\sum\limits_{j=1}^k(x_j-\bar{x})}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=0;$$

segue que 

$$Cov(\bar{Y},\widehat{\beta}_1\bar{x})=0.$$

Desta forma, 

$$Var(\widehat{\beta}_0)=Var(\bar{Y})+ Var(\widehat{\beta}_1 \bar{x})=Var\left(\sum_{i=1}^n \frac{Y_i}{n}\right)+ \bar{x}^2 Var(\widehat{\beta}_1).$$

Como Yi, i=1,...,n são independentes, segue que

$$Var(\widehat{\beta}_0)=\dfrac{1}{n^2}\sum\limits_{i=1}^n Var(Y_i)+ \bar{x}^2 \dfrac{\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=\dfrac{n \sigma^2}{n^2}+ \bar{x}^2 \dfrac{\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}=\sigma^2 \left( \dfrac{1}{n} + \dfrac{\bar{x}^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}\right).$$

Novamente, dados n pares de valores (x1,y1),...,(xn,yn) escrevemos

$$Var(\widehat{\beta}_0)=\sigma^2\left(\dfrac{1}{n}+\dfrac{\bar{x}^2}{S_{xx}}\right).$$


5. Covariância entre $ \widehat{\beta}_{0} $ e $ \widehat{\beta}_{1} $: 

$$Cov(\widehat{\beta}_0, \widehat{\beta}_1)=E(\widehat{\beta}_0\widehat{\beta}_1)-E(\widehat{\beta}_0)E(\widehat{\beta}_1)=E[(\bar{Y}-\widehat{\beta}_1\bar{x})\widehat{\beta}_1]-\beta_0\beta_1=E[\bar{Y}\widehat{\beta}_1-\bar{x}\widehat{\beta}_1^2]-\beta_0\beta_1= E(\bar{Y}\widehat{\beta}_1)$$

$$-\bar{x}E(\widehat{\beta}_1^2)-\beta_0\beta_1= E\left[\dfrac{1}{n}\sum\limits_{i=1}^n Y_i\widehat{\beta}_1\right]-\bar{x}[Var(\widehat{\beta}_1)+ (E(\widehat{\beta}_1))^2]-\beta_0\beta_1=\dfrac{1}{n}\sum\limits_{i=1}^n E[(\beta_0+\beta_1x_i+\varepsilon_i)\widehat{\beta}_1]$$

$$-\bar{x}\left[ \dfrac{\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}+\beta_1^2\right] -\beta_0\beta_1=\dfrac{1}{n}\sum\limits_{i=1}^n E[\beta_0\widehat{\beta}_1 + \beta_1\widehat{\beta}_1x_i+\varepsilon_i\widehat{\beta}_1]-\dfrac{\bar{x} \, \sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2} - \bar{x} \beta_1^2 - \beta_0 \beta_1$$

$$= \dfrac{1}{n} \sum\limits_{i=1}^n \left[\beta_0 \beta_1 + \beta_1^2 x_i + E(\varepsilon_i \widehat{\beta}_1) \right] - \dfrac{\bar{x} \, \sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2} - \bar{x} \beta_1^2 - \beta_0 \beta_1$$

$$= \beta_0 \beta_1 + \beta_1^2 \bar{x} + \dfrac{1}{n} \sum\limits_{i=1}^n E(\varepsilon_i \widehat{\beta}_1)-\dfrac{\bar{x} \, \sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2} - \bar{x} \beta_1^2 - \beta_0 \beta_1=\dfrac{1}{n} \sum\limits_{i=1}^n E(\varepsilon_i \widehat{\beta}_1)-\dfrac{\bar{x} \, \sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}.$$

De (4) temos que

$$E(\varepsilon_i \widehat{\beta}_1)=0$$

 

e portanto, 

$$Cov(\widehat{\beta}_0,\widehat{\beta}_1)=-\dfrac{\bar{x}\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}.$$


6. Distribuição amostral para $ \widehat{\beta}_{1} $:

Em (1), definimos 

$$\widehat{\beta}_{1}=\sum\limits_{i=1}^n C_i Y_i.$$

Como $ \widehat{\beta}_{1} $ é combinação linear de normais independentes (combinação linear dos Yi), segue que $ \widehat{\beta}_{1} $ também tem distribuição normal com média e variância dadas respectivamente em (1) e (2) e portanto, 

$$\widehat{\beta}_1 \sim N \left(\beta_1,\frac{\sigma^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}\right).$$

 

 

7. Distribuição amostral para $ \widehat{\beta}_{0} $:

Como em (6), $ \widehat{\beta}_{0} $ também é combinação linear de normais independentes Yi e portanto, também tem distribuição normal. A média e a variância de $ \widehat{\beta}_{0} $ são apresentadas em (3) e (4), respectivamente. Desta forma, 

$$\widehat{\beta}_{0} \sim N \left[\beta_0,\sigma^2\left(\frac{1}{n}+\frac{\bar{x}^2}{\displaystyle\sum\limits_{i=1}^n (x_i - \bar{x})^2}\right)\right].$$


Em relação ao estimador da variância $ \sigma^2 $,

8. Valor esperado (média) de QME:

$$QME=\dfrac{SQE}{n-2}$$

Assim,

$$E(QME)=E\left(\dfrac{SQE}{n-2}\right)=\dfrac{E(SQE)}{n-2}.$$

 

Sabemos que

$$SQE = \sum e_i^{2}=\sum_{i=1}^n (Y_i-\widehat{Y}_i)^{2}=\sum_{i=1}^n(Y_i-(\widehat{\beta}_0+\widehat{\beta}_1x_i))^{2}=\sum_{i=1}^n[Y_i^{2}-2 Y_i(\widehat{\beta}_0 +\widehat{\beta}_1 x_i) + (\widehat{\beta}_0 +\widehat{\beta}_1 x_i)^{2}]$$

$$=\sum_{i=1}^n[Y_i^{2}-2 Y_i(\bar{Y} - \widehat{\beta}_1 \bar{x} + \widehat{\beta}_1 x_i) + (\bar{Y} - \widehat{\beta}_1 \bar{x} +\widehat{\beta}_1 x_i)^{2}]=\sum_{i=1}^n[Y_i^{2}-2 Y_i\bar{Y} +2Y_i\bar{x}\widehat{\beta}_1 - 2Y_ix_i \widehat{\beta}_1+ [\bar{Y} - \widehat{\beta}_1 (\bar{x} - x_i)^{2}]$$

$$=\sum_{i=1}^n[Y_i^{2}-2 Y_i\bar{Y} +2 Y_i\bar{x}\widehat{\beta}_1- 2Y_ix_i \widehat{\beta}_1+ \bar{Y}^2 - 2 \bar{Y} \widehat{\beta}_1 (\bar{x} - x_i) + \widehat{\beta}_1^2 (\bar{x} - x_i)^2$$

$$=\sum_{i=1}^n[(Y_i -\bar{Y})^2 - 2 \widehat{\beta}_1 (x_iY_i -\bar{x} Y_i + \bar{Y}\bar{x} - \bar{Y}x_i) + \widehat{\beta}_1^2 (\bar{x} - x_i)^2]$$

$$= \sum_{i=1}^n(Y_{i}-\bar{Y})^2- 2\widehat{\beta}_1 \sum_{i=1}^n[(\bar{x}-x_i)(\bar{Y}-Y_i)]+\widehat{\beta}_1^{2} \sum_{i=1}^n(x_i-\bar{x})^2$$

$$=\sum_{i=1}^n(Y_i-\bar{Y})^{2}-2\widehat{\beta}_1\sum_{i=1}^n(x_i-\bar{x})Y_{i}+\widehat{\beta}_1^{2}(\sum_{i=1}^n x_i^{2}-n\bar{x}^{2})= \sum_{i=1}^n(Y_i -\bar{Y})^2-2\widehat{\beta}_1\sum_{i=1}^n(x_i-\bar{x})Y_i+\widehat{\beta}_1^{2} \sum_{i=1}^n(x_i-\bar{x})^2$$

$$=\sum_{i=1}^n(Y_i-\bar{Y})^2 - 2\widehat{\beta}_1\sum_{i=1}^n(x_i-\bar{x})Y_i+\widehat{\beta}_1^{2} \dfrac{\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})Y_i}{\widehat{\beta}_1}=\sum_{i=1}^n(Y_i-\bar{Y})^2 -\widehat{\beta}_1\sum_{i=1}^{n}(x_i-\bar{x})Y_i.$$


Desta forma,  

 

$$E(SQE)=E\left(\sum_{i=1}^{n}(Y_{i}-\bar{Y})^2 - \widehat{\beta}_{1}\sum_{i=1}^{n}(x_{i}-\bar{x})Y_{i}\right)$$

$$=E\left(\sum\limits_{i=1}^{n}(Y_{i}-\bar{Y})^2\right)-E\left(\dfrac{\displaystyle\sum\limits_{i=1}^{n}(x_{i}-\bar{x})Y_{i}}{\displaystyle\sum_{i=1}^n(x_{i}-\bar{x})^2}\sum_{i=1}^n(x_i-\bar{x})Y_{i}\right)$$

$$=\sum_{i=1}^nE(Y_{i}^2)-nE(\bar{Y}^2)-\dfrac{1}{\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})^2}E\left[\left(\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})Y_i\right)^2\right]$$

$$=\sum_{i=1}^n[Var(Y_i)+E^2(Y_i)]-n[Var(\bar{Y})+E^2(\bar{Y})]$$

$$-\dfrac{1}{\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})^2}\left[Var\left(\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})Y_i\right)+E^2\left(\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})Y_i\right)\right].$$

Como

$$E\left(\sum\limits_{i=1}^n(x_i-\bar{x})Y_i\right)=\sum_{i=1}^n(x_i-\bar{x})E(Y_i)=\sum_{i=1}^n(x_i-\bar{x})(\beta_0+\beta_1x_i)$$

e

$$Var\left(\sum_{i=1}^n(x_i-\bar{x})Y_i\right)=\sum_{i=1}^n(x_i-\bar{x})^2Var(Y_i)=\sum_{i=1}^n(x_i-\bar{x})^2\sigma^2.$$

Segue que

$$E(SQE)=\left(\sum_{i=1}^n\sigma^2+(\beta_0+\beta_1x_i)^2\right)-n\left(\dfrac{\sigma^2}{n}+(\beta_0+\beta_1\bar{x})^2\right)$$

$$-\dfrac{1}{\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})^2}\sum_{i=1}^n(x_i-\bar{x})^2\sigma^2-\dfrac{1}{\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})^2}\left[\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})(\beta_0+\beta_1x_i)\right]^2$$

$$=n\sigma^2+\sum_{i=1}^n(\beta_0+\beta_1x_i)^2-\sigma^2-n(\beta_0+\beta_1\bar{x})^2-\sigma^2-\dfrac{1}{\displaystyle\sum\limits_{i=1}^n(x_i-\bar{x})^2}\left[\beta_0\sum_{i=1}^n(x_i-\bar{x})+\beta_1\sum_{i=1}^n(x_i-\bar{x})x_i\right]^2$$

$$=\sigma^2(n-2)+\sum_{i=1}^n(\beta_0+\beta_1x_i)^2-n(\beta_0+\beta_1\bar{x})^2-\beta_1^2\sum_{i=1}^n(x_i-\bar{x})^2=\sigma^2(n-2).$$

Portanto,

$$E(QME)=\dfrac{\sigma^2(n-2)}{n-2}=\sigma^2.$$

(estimador não viciado).


Exemplo 1.3.1

Para os dados do exemplo da "Motivação 1", obter estimativas para a variância dos estimadores $ \widehat{\beta}_0 $ e $ \widehat{\beta}_1 $. O valor de QME foi calculado no "Exemplo 1.2.2". Já os valores de $ \bar{x}^2 $ e $ S_{xx} $ foram calculados no "Exemplo 1.2.1". 

clique aqui para efetuar o download dos dados utilizados nesse exemplo

Solução:

$$\widehat{Var}(\widehat{\beta}_0)=\widehat{\sigma}^2\left[\dfrac{1}{n}+\dfrac{\bar{x}^2}{S_{xx}}\right]=QME \left[\dfrac{1}{n} + \dfrac{\bar{x}^2}{S_{xx}} \right] ~=~2,2866 \left[\frac{1}{20} + \frac{51756,25}{625} \right] ~=~189,4732.$$

$$\widehat{Var}(\widehat{\beta}_1)=\dfrac{\widehat{\sigma}^2}{S_{xx}}=\dfrac{QME}{S_{xx}}~=~\frac{2,2866}{625} ~=~0,003658.$$

Utilizando o Software Action obtemos os seguintes resultados:

 Para entender como executar essa função do Software Action, você pode consultar o manual do usuário.