ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback. (arXiv:2210.12329v1 [cs.CL])

Recently, dataset-generation-based zero-shot learning has shown promising
results by training a task-specific model with a dataset synthesized from large
pre-trained language models (PLMs). The final task-specific model often
achieves compatible or even better performance than PLMs under the zero-shot
setting, with orders of magnitude fewer parameters. However, synthetic datasets
have their drawbacks. They have long been suffering from low-quality issues
(e.g., low informativeness and redundancy). This explains why the massive
synthetic data does not lead to better performance -- a …

arxiv context dataset dataset generation feedback