6.2.1. statsmodels.sandbox.datarich

6.2.1.1. Econometrics for a Datarich Environment

6.2.1.1.1. Introduction

In many cases we are performing statistical analysis when many observed variables are available, when we are in a data rich environment. Machine learning has a wide variety of tools for dimension reduction and penalization when there are many varibles compared to the number of observation. Chemometrics has a long tradition of using Partial Least Squares, NIPALS and similar in these cases. In econometrics the same problem shows up when there are either many possible regressors, many (weak) instruments or when there are a large number of moment conditions in GMM.

This section is intended to collect some models and tools in this area that are relevant for the statical analysis and econometrics.

6.2.1.2. Covariance Matrices

Several methods are available to reduce the small sample noise in estimated covariance matrices with many variable. Some applications: weighting matrix with many moments, covariance matrix for portfolio choice

6.2.1.3. Dimension Reduction

Principal Component and Partial Least Squares try to extract the important low dimensional factors from the data with many variables.

6.2.1.4. Regression with many regressors

Factor models, selection of regressors and shrinkage and penalization are used to improve the statistical properties, when the presence of too many regressors leads to over-fitting and too noisy small sample estimators and statistics.

6.2.1.5. Regression with many moments or many instruments

The same tools apply and can be used in these two cases. e.g. Tychonov regularization of weighting matrix in GMM, similar to Ridge regression, the weighting matrix can be shrunk towards the identity matrix. Simplest case will be part of GMM. I don’t know how much will be standalone functions.

6.2.1.6. Intended Content

6.2.1.6.1. PLS

what should be available in class?

6.2.1.6.2. Factormodel and supporting helper functions

6.2.1.6.2.1. PCA based

First version based PCA on Stock/Watson and Bai/Ng, and recent papers on the selection of the number of factors. Not sure about Forni et al. in approach. Basic support of this needs additional results for PCA, error covariance matrix of data on reduced factors, required for criteria in Bai/Ng. Selection criteria based on eigenvalue cutoffs.

Paper on PCA and structural breaks. Could add additional results during find_nfact to test for parameter stability. I haven’t read the paper yet.

Idea: for forecasting, use up to h-step ahead endogenous variables to directly get the forecasts.

Asymptotic results and distribution: not too much idea yet. Standard OLS results are conditional on factors, paper by Haerdle (abstract seems to suggest that this is ok, Park 2009).

Simulation: add function to simulate DGP of Bai/Ng and recent extension. Sensitivity of selection criteria to heteroscedasticity and autocorrelation.

Bai, J. & Ng, S., 2002. Determining the Number of Factors in
Approximate Factor Models. Econometrica, 70(1), pp.191-221.
Kapetanios, G., 2010. A Testing Procedure for Determining the Number
of Factors in Approximate Factor Models With Large Datasets. Journal of Business and Economic Statistics, 28(3), pp.397-409.
Onatski, A., 2010. Determining the Number of Factors from Empirical
Distribution of Eigenvalues. Review of Economics and Statistics, 92(4), pp.1004-1016.
Alessi, L., Barigozzi, M. & Capasso, M., 2010. Improved penalization
for determining the number of factors in approximate factor models. Statistics & Probability Letters, 80(23-24), pp.1806-1813.
Breitung, J. & Eickmeier, S., Testing for structural breaks in dynamic
factor models. Journal of Econometrics, In Press, Accepted Manuscript. Available at: http://www.sciencedirect.com/science/article/B6VC0-51G3W92-1/2/f45ce2332443374fd770e42e5a68ddb4 [Accessed November 15, 2010].
Croux, C., Renault, E. & Werker, B., 2004. Dynamic factor models.
Journal of Econometrics, 119(2), pp.223-230.
Forni, M. et al., 2009. Opening the Black Box: Structural Factor
Models with Large Cross Sections. Econometric Theory, 25(05), pp.1319-1347.
Forni, M. et al., 2000. The Generalized Dynamic-Factor Model:
Identification and Estimation. Review of Economics and Statistics, 82(4), pp.540-554.
Forni, M. & Lippi, M., The general dynamic factor model: One-sided
representation results. Journal of Econometrics, In Press, Accepted Manuscript. Available at: http://www.sciencedirect.com/science/article/B6VC0-51FNPJN-1/2/4fcdd0cfb66e3050ff5d19bf2752ed19 [Accessed November 15, 2010].
Kapetanios, G., 2010. A Testing Procedure for Determining the Number
of Factors in Approximate Factor Models With Large Datasets. Journal of Business and Economic Statistics, 28(3), pp.397-409.
Onatski, A., 2010. Determining the Number of Factors from Empirical
Distribution of Eigenvalues. Review of Economics and Statistics, 92(4), pp.1004-1016.
Park, B.U. et al., 2009. Time Series Modelling With Semiparametric
Factor Dynamics. Journal of the American Statistical Association, 104(485), pp.284-298.

6.2.1.6.2.2. other factor algorithm

PLS should fit in reasonably well.

Bai/Ng have a recent paper, where they compare LASSO, PCA, and similar, individual and in combination. Check how much we can use scikits.learn for this.

6.2.1.6.2.3. miscellaneous

Time series modeling of factors for prediction, ARMA, VARMA. SUR and correlation structure What about sandwich estimation, robust covariance matrices? Similarity to Factor-Garch and Go-Garch Updating: incremental PCA, ...?

6.2.1.7. TODO next

MVOLS
: OLS with multivariate endogenous and identical exogenous variables.
rewrite and expand current varma_process.VAR
PCA
: write a class after all, and/or adjust the current donated class
and keep adding required statistics, e.g. residual variance, projection of X on k-factors, ... updating ?
FactorModelUnivariate
: started, does basic principal component regression,
based on standard information criteria, not Bai/Ng adjusted
FactorModelMultivariate
: follow pattern for univariate version and use
MVOLS