Sensitivity Analysis On Loss Landscape

arXiv:2403.01128v1 Announce Type: new
Abstract: Gradients can be employed for sensitivity analysis. Here, we leverage the advantages of the Loss Landscape to comprehend which independent variables impact the dependent variable. We seek to grasp the loss landscape by utilizing first, second, and third derivatives through automatic differentiation. we know that Spearman's rank correlation coefficient can detect the monotonic relationship between two variables. However, I have found that second-order gradients, with certain configurations and parameters, provide information that can be visualized …

abstract advantages analysis arxiv correlation cs.lg derivatives differentiation impact independent landscape loss sensitivity through type variables