StreaMulT: Streaming Multimodal Transformer for Heterogeneous and Arbitrary Long Sequential Data

arXiv:2110.08021v2 Announce Type: replace
Abstract: The increasing complexity of Industry 4.0 systems brings new challenges regarding predictive maintenance tasks such as fault detection and diagnosis. A corresponding and realistic setting includes multi-source data streams from different modalities, such as sensors measurements time series, machine images, textual maintenance reports, etc. These heterogeneous multimodal streams also differ in their acquisition frequency, may embed temporally unaligned information and can be arbitrarily long, depending on the considered system and task. Whereas multimodal fusion has …

abstract arxiv challenges complexity cs.cl cs.lg cs.mm data data streams detection diagnosis etc images industry industry 4.0 machine maintenance multimodal predictive predictive maintenance reports sensors series source data streaming systems tasks textual time series transformer type