Rerank
Rerank request bodies on a batch line. There is no live POST /v1/rerank.
Not a live HTTP endpoint
Do not POST to /v1/rerank. Put that path on each JSONL line as url, then submit the file with
POST /v1/batches and "endpoint": "/v1/rerank".
A Lararouter extension rather than part of the OpenAI surface, so the official SDKs have no method for
it. Like chat and embeddings, it runs as the body of a batch line.
Reranking is the second half of retrieval. Vector search is cheap and approximate, so you fetch fifty candidates, then have a cross-encoder read the query and each document together and reorder them. The top five after reranking are usually noticeably better than the top five from the vector index alone.
Requires the batches:write scope on the key that creates the batch.
{"custom_id":"question_118","method":"POST","url":"/v1/rerank","body":{"model":"command-r-35b","query":"How long do refunds take?","documents":["Shipping is free on orders over $50.","Refunds are issued within 5 business days of approval.","Our warehouse is open Monday to Friday."],"top_n":2}}Submit the file:
POST /v1/batches HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY
Content-Type: application/json
{
"input_file_id": "file_9Hs4TnQ2",
"endpoint": "/v1/rerank",
"completion_window": "24h"
}Parameters
Prop
Type
Response
{
"custom_id": "question_118",
"response": {
"status_code": 200,
"request_id": "req_01JD8XK4M2",
"body": {
"object": "rerank",
"model": "command-r-35b",
"results": [
{ "index": 1, "relevance_score": 0.9814 },
{ "index": 0, "relevance_score": 0.0231 }
],
"usage": { "prompt_tokens": 96, "total_tokens": 96, "cost_usd": 0.0000384 }
}
},
"error": null
}index refers to the position in the documents array you sent. Results come back sorted by
relevance_score descending, so the order of results is not the order of documents. That's the
whole point, and it's the one thing to be careful about when wiring the response back into your data.
relevance_score runs 0 to 1 but isn't calibrated across models. A 0.6 from one model doesn't mean
what a 0.6 from another does, so tune any cutoff per model rather than sharing a constant.
Retrieval pipeline
Embed, search, rerank, answer. That is the standard shape, except each model step is its own batch and the search happens in your own database in between.
This is a precompute pattern
Chaining three batches means three completion windows, so this builds an answer cache overnight rather than serving a live query. Precomputing answers to a known question set, or backfilling summaries over a corpus, is where it pays off.
namespace App\Actions\Search;
use App\Models\Document;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Http;
use Throwable;
class RankContextForQuestions
{
/** Stage two: turn embedded questions into a rerank batch. */
public function handle(Collection $questions): string
{
try {
$lines = $questions->map(function (Question $question) {
// The vector came back from the embeddings batch that ran before this one.
$candidates = Document::nearestTo($question->embedding)->limit(50)->get();
$question->update(['candidate_ids' => $candidates->pluck('id')->all()]);
return [
'custom_id' => "question_{$question->id}",
'method' => 'POST',
'url' => '/v1/rerank',
'body' => [
'model' => 'command-r-35b',
'query' => $question->body,
'documents' => $candidates->pluck('body')->all(),
'top_n' => 5,
],
'metadata' => ['entity' => "user:{$question->user_id}"],
];
});
return $this->submit($lines, '/v1/rerank');
} catch (Throwable $e) {
report($e);
throw $e;
}
}
}Storing candidate_ids alongside the question is what makes the next stage work: when the rerank
results land, index points into that saved array, and the reranked context becomes the system prompt
for a chat completions batch.
Every stage carries the same entity, so all three land under user:{$question->user_id} in
usage reporting and the cost of answering one question is a number you can actually
see.