Retrieval-Augmented Generation (RAG) and LLM

Cevher Dogan

·

Follow

Published in

Self Study Notes

·

4 min read

·

Dec 10, 2024

59

Listen

Share

Following up my recent post in Participation in GitHub Projects

Below is a structured guide to deep dive into Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs). The focus is on foundational understanding, implementation practices, and hands-on experimentation.

1. Understanding RAG (Retrieval-Augmented Generation)

Key Concept:

RAG combines:

• Retrievers (to fetch relevant data from external sources like knowledge bases or documents).
• Generators (to create coherent and context-aware outputs using retrieved information).

This bridges the gap between retrieval-based and generative systems, making RAG suitable for knowledge-intensive tasks.

A. Key Papers to Read

1. Lewis et al. (2020): Retrieval-augmented generation for knowledge-intensive NLP tasks.

• Introduces RAG. Uses Dense Passage Retrieval (DPR) with BART for generating evidence-based responses. Paper

1. Karpukhin et al. (2020): Dense Passage Retrieval for Open-Domain Question Answering.

• Focuses on the retriever component, optimizing retrieval through dense embeddings…