FlashRank

FlashRank is a Python library that runs reranking models locally on CPU, without Torch or Transformers. It's for developers building search systems or retrieval-augmented generation (RAG) apps who want to reorder retrieved passages by relevance before passing them to an LLM.

It works with existing full-text, vector and hybrid search pipelines. The retrieval system finds candidate passages; FlashRank evaluates them against the query and returns a revised order. Its small model footprint makes it a candidate for applications where an extra ranking stage needs to keep memory use and response time low, including serverless deployments such as AWS Lambda.

Model choice covers compact cross-encoders and larger LLM-based rerankers. The default is ms-marco-TinyBERT-L-2-v2, with ms-marco-MiniLM-L-12-v2 and rank-T5-flan available for other ranking needs. For listwise ranking, which evaluates passages as a group, it supports rank_zephyr_7b_v1_full in a 4-bit GGUF format.

For multilingual search, ms-marco-MultiBERT-L-12 provides language coverage beyond English, while miniReranker_arabic_v1 targets Arabic. The ce-esci-MiniLM-L12-v2 model is fine-tuned on Amazon's ESCI dataset for product search. Ranking speed depends on passage length and model depth, so the model choice affects the overhead added to retrieval.

FlashRank is open source under Apache 2.0. Its pairwise ranking output retains passage IDs and optional metadata alongside relevance scores. Its current rank_zephyr integration accepts at most 20 passages per pass; sliding-window support is not yet implemented.