From Unimodals to Multimodality: DIY Techniques for Building Foundational Models

A comprehensive tutorial: Using advanced techniques like prompt adaptation and adapters to transform open-source unimodal models into multimodal ones, including all variants of LLaMA-Adapters, LLaVa, MiniGPT-4, and more.

Authors: Elahe Aghapour, Salar Rahili

INTRODUCTION

With recent advancements in large language models (LLMs), AI has become the spotlight of technology. We’re now more eager than ever to reach AGI-level intelligence. Yet, achieving a human-like understanding of our surroundings involves much more than just mastering language and text comprehension. Humans use …

advanced authors become building diy foundational foundational models language language models large language large language models llama llava llms minigpt minigpt-4 multimodal multimodality ones open source prompt spotlight technology tutorial variants