Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation

arXiv:2404.04232v1 Announce Type: new
Abstract: Compositional generalization, representing the model's ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect controllable text generation (MCTG) methods. Nonetheless, a comprehensive compositional generalization evaluation benchmark of MCTG is still lacking. We propose CompMCTG, a benchmark encompassing diverse multi-aspect labeled datasets and a crafted three-dimensional evaluation protocol, to holistically evaluate the compositional generalization of MCTG approaches. We observe that existing MCTG …

abstract arxiv benchmark benchmarking cs.cl data evaluation generate improving property text text generation training training data type