Learning under Singularity: An Information Criterion improving WBIC and sBIC

arXiv:2402.12762v1 Announce Type: cross
Abstract: We introduce a novel Information Criterion (IC), termed Learning under Singularity (LS), designed to enhance the functionality of the Widely Applicable Bayes Information Criterion (WBIC) and the Singular Bayesian Information Criterion (sBIC). LS is effective without regularity constraints and demonstrates stability. Watanabe defined a statistical model or a learning machine as regular if the mapping from a parameter to a probability distribution is one-to-one and its Fisher information matrix is positive definite. In contrast, models …

abstract arxiv bayes bayesian constraints criterion cs.lg information novel singular singularity stability statistical stat.ml type