# Cohere Rerank 3.5

Cohere Rerank 3.5 is a reranking model released December 2, 2024 that scores candidate documents against a query and returns them ordered by relevance. It targets English-language search over long-form text, tables, and JSON, with a per-document context of 4.1K tokens.

- **Model ID:** `cohere/rerank-v3.5`
- **Type:** reranking
- **Providers:** bedrock
- **Context window:** 4,096
- **Maximum output tokens:** 4,096
- **Pricing:** See the provider details below
- **Canonical page:** https://vercel.com/ai-gateway/models/rerank-v3.5

## Supported parameters

Detailed capability metadata has not been reported for this model.

## Example

```ts
import { rerank } from 'ai';

const result = await rerank({
  model: 'cohere/rerank-v3.5',
  query: 'What is the capital of France?',
  documents: [
    'Paris is the capital of France.',
    'Berlin is the capital of Germany.',
    'Madrid is the capital of Spain.',
  ],
})
```

## About

Cohere Rerank 3.5 is the December 2024 update to the Cohere Rerank family. Cohere positions it as a reasoning-focused reranker for complex enterprise search where the difference between the right document and a near-miss matters.

Reranking is a cross-encoder step that runs after first-pass retrieval. A vector search, BM25 index, or hybrid retriever returns a candidate set, then Cohere Rerank 3.5 scores each candidate against the query and returns a relevance-ordered list. Cross-attention between the query and the full document text picks up signal that bi-encoder embedding similarity misses on under-specified or multi-part queries.

Cohere Rerank 3.5 handles English long-form text and semi-structured data including JSON, tables, and email-style records. The per-document context window is 4.1K tokens, shared between query and document. Documents that exceed the limit are chunked automatically and the highest-scoring chunk drives the document's final rank.

In a RAG pipeline, the common pattern is to retrieve 50 to 200 candidates with an embedding model and then rerank them down to the top 5 to 20 documents passed to the generative model. That reduces the prompt token count sent to the LLM while improving the quality of the context, which often offsets the reranking call's cost.

See https://aws.amazon.com/blogs/machine-learning/cohere-rerank-3-5-is-now-available-in-amazon-bedrock-through-rerank-api/ for the API contract, including how to format queries, documents, and the optional `return_documents` and `top_n` parameters.

## What to consider

Rerankers sit after an initial retrieval step, not in place of it. Pair Cohere Rerank 3.5 with an embedding model (or any first-pass search) so it has a candidate pool to score. The per-document context of 4.1K tokens covers both the query and document tokens, so very long documents need chunking before they reach the model.

## When to use

### Best For

- English-language RAG pipelines that already retrieve a candidate set and need higher-precision top-k selection
- Search over semi-structured data like JSON records, support tickets, and tabular documents where field-level relevance matters
- Long-document corpora where chunked passages need final ordering before being passed to a generative model
- Enterprise knowledge bases that rely on complex multi-part queries the embedding model alone struggles to disambiguate
- Hybrid retrieval stacks combining BM25 and vector search where a single relevance score is needed across the merged candidate list

### Consider Alternatives When

- Your corpus is primarily non-English, in which case `rerank-v4-fast` or `rerank-v4-pro` cover multilingual content explicitly
- Latency budgets are tight at high query volume and a lighter reranker would meet the SLA
- You only need the embedding model's first-pass results and the marginal accuracy gain doesn't justify a second model call

## Best for

- **English RAG top-k refinement:** Reorder a candidate pool from vector or keyword search into a sharper top-k for the generative model
- **JSON and tabular search:** Score semi-structured records where field content drives the match
- **Long-document corpora:** Chunked passages need final ordering before they hit the LLM context window
- **Complex enterprise queries:** Multi-part questions benefit from cross-attention over the full document
- **Hybrid retrieval merging:** Unify BM25 and vector candidates under a single relevance score

## Consider alternatives

- **Multilingual workloads:** `rerank-v4-fast` and `rerank-v4-pro` are tuned for English and non-English documents
- **Latency-critical pipelines:** A lighter reranker may fit a tighter response budget at high query volume
- **Strict cost ceilings:** Skip reranking when first-pass retrieval already meets your accuracy bar
- **Image or multimodal retrieval:** Use a multimodal embedding or retrieval model instead

## Frequently asked questions

### What does Cohere Rerank 3.5 actually do?

It takes a query and a list of candidate documents, scores each document for semantic relevance to the query, and returns the list reordered. The model itself doesn't retrieve; it refines results from a first-pass retriever like a vector index or BM25 search.

### How does Cohere Rerank 3.5 differ from an embedding model?

Embedding models produce a vector per document and a vector per query, then similarity is computed offline. Cohere Rerank 3.5 reads the query and a candidate document together through cross-attention, which catches relevance signal that bi-encoder similarity misses on complex queries. The tradeoff is that reranking runs per-candidate at query time, so it's used on a shortlist rather than the full corpus.

### What types of documents does Cohere Rerank 3.5 support?

English-language text including long-form documents, semi-structured JSON, tables, and email-style records. The per-document context window is 4.1K tokens, shared between the query and the document.

### How many documents should I send per rerank call?

Typical pipelines retrieve 50 to 200 candidates from a first-pass index and rerank them down to a top-k of 5 to 20 for the generative model. The exact numbers depend on your latency budget and how noisy the first-pass retriever is.

### Can I use Cohere Rerank 3.5 without a separate embedding model?

You can pair it with any retriever, including BM25 or hybrid search. An embedding model is the most common first stage, but Cohere Rerank 3.5 only needs a candidate set, not a specific retrieval method.

### How much does Cohere Rerank 3.5 cost on AI Gateway?

Reranking is billed per search query. See the pricing section on this page for the current per-query rate on AI Gateway.

### Does Cohere Rerank 3.5 support Zero Data Retention?

Yes, Zero Data Retention is available for this model. Zero Data Retention is offered on a per-provider basis. See https://vercel.com/docs/ai-gateway/capabilities/zdr for details.

## Links

- [Model page](https://vercel.com/ai-gateway/models/rerank-v3.5)
- [AI Gateway documentation](https://vercel.com/docs/ai-gateway)
- [Provider model documentation](https://aws.amazon.com/blogs/machine-learning/cohere-rerank-3-5-is-now-available-in-amazon-bedrock-through-rerank-api/)
- [Provider pricing](https://aws.amazon.com/bedrock/pricing/)


If you encounter incorrect information, can’t find what you need, need a capability Vercel doesn’t currently offer, or have any other feedback to report, please do it by following the instructions at https://vercel.com/feedback.md.