[R] Announcing CROWDLAB: open-source tools for data labeled by multiple annotators

Hi Redditors! Many of us use multiple annotations to get higher quality labels for our data — yet AFAIK there is no open-source python package for **data labeled by multiple annotators** — so we [built one](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html), [benchmarked it](https://cleanlab.ai/blog/multiannotator/), and released [the CROWDLAB paper](https://cleanlab.github.io/multiannotator-benchmarks/paper.pdf).

[CROWDLAB produces a consensus label, confidence, and annotator score for data labeled by multiple annotators.](https://preview.redd.it/04xhqh1kj0s91.png?width=1152&format=png&auto=webp&s=3325dd1b12fed0c4deb740f1de0657b52f3686d0)

After many long nights, I'm psyched to share the new easy-to-use and effective CROWDLAB algorithm that can use **any classifier** to estimate: …

data machinelearning tools