SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation

arXiv:2312.16272v2 Announce Type: replace
Abstract: Recent advancements in subject-driven image generation have led to zero-shot generation, yet precise selection and focus on crucial subject representations remain challenging. Addressing this, we introduce the SSR-Encoder, a novel architecture designed for selectively capturing any subject from single or multiple reference images. It responds to various query modalities including text and masks, without necessitating test-time fine-tuning. The SSR-Encoder combines a Token-to-Patch Aligner that aligns query inputs with image patches and a Detail-Preserving Subject Encoder …

arxiv cs.cv encoder encoding representation ssr type