Multi-Lingual Malaysian Embedding: Leveraging Large Language Models for Semantic Representations

In this work, we present a comprehensive exploration of finetuning Malaysian language models, specifically Llama2 and Mistral, on embedding tasks involving negative and positive pairs. We release two distinct models tailored for Semantic Similarity and Retrieval-Augmented Generation (RAG).
For Semantic Similarity, our 600 million parameter Llama2 model outperforms OpenAI text-embedding-ada-002 across all recall@k metrics for b.cari.com.my, c.cari.com.my, Malay news, and Malaysian Twitter test sets.
In the realm of RAG models, our approach proves competitive with OpenAI text-embedding-ada-002 in the Malaysian …

ada cs.cl cs.lg embedding exploration finetuning language language models large language large language models llama2 mistral negative openai positive rag release retrieval retrieval-augmented semantic tasks text work