Retrievals Documentation

GitHub

Retrievals is an easy-to-use python framework supporting state-of-the-art embeddings, especially for information retrieval and rerank in NLP/LLM, based on PyTorch and Transformers.

• Contrastive learning enhanced embeddings

• LLM embeddings

Installation

Install the Prerequisites

• transformers

• peft

• faiss / faiss-gpu

Now you are ready, proceed with

$ pip install open-retrievals

Examples

Contents:

• Quick start
  • Use the pretrained weight as embedding
  • Fine tune the transformer pretrained weight by contrastive learning
  • Query search by faiss
  • Rerank to enhance the performance
  • Langchain example for RAG
• Finetune
  • Pointwise
• Rerank
• Information Retrieval
• RAG
  • Build an RAG Application
  • Enhance RAG Performance