Announcing CROWDLAB: open-source tools for data labeled by multiple annotators

Hi Redditors! Many of us in computer vision use multiple annotations to get higher quality labels for our images — yet AFAIK there is no open-source python package for **data labeled by multiple annotators** — so we [built one](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html), [benchmarked it](https://cleanlab.ai/blog/multiannotator/), and released [the CROWDLAB paper](https://cleanlab.github.io/multiannotator-benchmarks/paper.pdf).

[CROWDLAB produces a consensus label, confidence, and annotator score for data labeled by multiple annotators.](https://preview.redd.it/im7kymc2m0s91.png?width=1152&format=png&auto=webp&s=d4c08adc8073333358dcab3f5f674a68e88a5563)

After many long nights, I'm psyched to share the new easy-to-use and effective CROWDLAB algorithm that can use **any …

computervision data tools