Semantic Text Transmission via Prediction with Small Language Models: Cost-Similarity Trade-off

arXiv:2403.00290v1 Announce Type: cross
Abstract: We consider the communication of natural language text from a source to a destination over noiseless and character-erasure channels. We exploit language's inherent correlations and predictability to constrain transmission costs by allowing the destination to predict or complete words with potential dissimilarity with the source text. Concretely, our objective is to obtain achievable $(\bar{c}, \bar{s})$ pairs, where $\bar{c}$ is the average transmission cost at the source and $\bar{s}$ is the average semantic similarity measured via …

abstract arxiv channels communication correlations cost costs cs.ai cs.it cs.lg exploit language language models math.it natural natural language prediction semantic small small language models text trade trade-off type via words