Inference for Regression with Variables Generated from Unstructured Data

arXiv:2402.15585v1 Announce Type: cross
Abstract: The leading strategy for analyzing unstructured data uses two steps. First, latent variables of economic interest are estimated with an upstream information retrieval model. Second, the estimates are treated as "data" in a downstream econometric model. We establish theoretical arguments for why this two-step strategy leads to biased inference in empirically plausible settings. More constructively, we propose a one-step strategy for valid inference that uses the upstream and downstream models jointly. The one-step strategy (i) …

abstract arxiv data econ.em economic generated inference information leads regression retrieval stat.ml strategy type unstructured unstructured data variables