Most equity risk models applied in practice assume stable return correlations over time. However, there is considerable evidence suggesting that correlations among stock returns and hence, variance-covariance matrices (VCMs) become unstable over time. In this paper, we account for correlation instabilities in US stock returns and derive VCMs from time-varying factor model estimates. To do so, we use three different estimation approaches: (1) moving window least squares, (2) flexible least squares and (3) the random walk model. Our empirical results suggest that a time-varying estimation of return correlations fits the data considerably better than time-invariant estimation and thus, increases the efficiency of risk estimation and portfolio selection.