Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inference. (arXiv:2009.09961v3 [cs.CL] UPDATED)

Leveraging text, such as social media posts, for causal inferences requires
the use of NLP models to 'learn' and adjust for confounders, which could
otherwise impart bias. However, evaluating such models is challenging, as
ground truth is almost never available. We demonstrate the need for empirical
evaluation frameworks for causal inference in natural language by showing that
existing, commonly used models regularly disagree with one another on real
world tasks. We contribute the first such framework, generalizing several
challenges across …

adjusting arxiv causal inference challenges evaluation framework inference text