[D] Is "feature dilution" a recognised phenomenon in deep neural networks and how to combat it

I've been grappling with a challenge related to data integration and multimodal neural networks, and I'd love your insights. Here's the scenario: I have a feature matrix with multiple types of features, including 5 continuous variables within the range of 0 to 1. Additionally, I've concatenated an embedding vector with 1024 dimensions into the same feature matrix, where the embedding values are also continuous.

My concern is whether the presence of the high-dimensional embedding features dilutes the effect or importance …

challenge continuous data data integration feature features insights integration love machinelearning matrix multimodal multiple networks neural networks types variables