Towards Image Semantics and Syntax Sequence Learning

Convolutional neural networks and vision transformers have achieved outstanding performance in machine perception, particularly for image classification. Although these image classifiers excel at predicting image-level class labels, they may not discriminate missing or shifted parts within an object. As a result, they may fail to detect corrupted images that involve missing or disarrayed semantic information in the object composition. On the contrary, human perception easily distinguishes such corruptions. To mitigate this gap, we introduce the concept of "image grammar", consisting …

class classification classifiers convolutional neural networks cs.cv excel image images labels machine machine perception networks neural networks perception performance semantics syntax transformers vision vision transformers