Retrieval
============================

.. _retrieval:

1. Pipeline
----------------------------

generate data -> train -> eval

pretrained encoding -> build hard negative -> train -> eval -> indexing -> retrieval

pretrain -> fine tuning -> distill


2. Offline indexing
----------------------------


3. Retrieval
----------------------------


Faiss retrieval
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


BM25 retrieval
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Elastic search retrieval
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


Ensemble retrieval
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

we can use `RRF_fusion` to ensemble multiple retrievals to improve the retrieval performance.