[P] Automatic Translation of Comics ( Bande Dessinée, Manga, Webtoons, etc) with Speech Bubble Detection, Text Segmentation, OCR and Inpainting

I'd like to share what i've been working on for a while. A python desktop app for automatically translating comics in a variety of formats (Image, Pdf, Epub and comic book archives) and in multiple languages. It uses 2 yolov8 models i trained for detection and segmentation, a suite of models for OCR depending on the language and a finetuned lama checkpoint for Inpainting.

repo - [https://github.com/ogkalu2/comic-translate](https://github.com/ogkalu2/comic-translate)

GUI

https://preview.redd.it/1gq7j7r8smec1.png?width=576&format=png&auto=webp&s=29790a1c2768ee274ade20945ba0ee9edfe0ba5a

​

​

app archives book bubble comics desktop detection etc image inpainting languages machinelearning manga multiple ocr pdf python segmentation speech text translation yolov8