Batches

Submit a job, poll it, and collect the results. Every invocation on Lararouter runs this way.

A batch is the unit of work on Lararouter. You upload a JSONL file, create a batch against it, and Lararouter returns every result inside the completion window: 24 hours on Pay-as-you-go, 4 hours on Enterprise.

There is no synchronous inference endpoint. Running everything as a scheduled job is what keeps per-token pricing below OpenRouter batch inference.

POST /v1/files HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY
Content-Type: multipart/form-data; boundary=----LararouterBatch

POST /v1/batches HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY
Content-Type: application/json

GET /v1/batches/batch_5Rj8Lm2Q HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY

GET /v1/files/file_2Wp7Kd91/content HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY

Create

POST /v1/batches HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY
Content-Type: application/json

{
  "input_file_id": "file_9Hs4TnQ2",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h",
  "metadata": {
    "entity": "team:acme",
    "tags": "job=nightly-summaries"
  }
}

Requires the batches:write scope.

curl https://REGION.api.lararouter.com/v1/batches \
  -H "Authorization: Bearer $LARAROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file_9Hs4TnQ2",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h",
    "metadata": {
      "entity": "team:acme",
      "tags": "job=nightly-summaries"
    }
  }'

Prop

Type

The response is the batch object, immediately, with a status of validating and no results yet.

{
  "id": "batch_5Rj8Lm2Q",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "status": "validating",
  "input_file_id": "file_9Hs4TnQ2",
  "output_file_id": null,
  "error_file_id": null,
  "completion_window": "24h",
  "request_counts": { "total": 4200, "completed": 0, "failed": 0 },
  "cost_usd": 0,
  "region": "REGION",
  "created": 1754150400,
  "expires_at": 1754236800,
  "metadata": { "entity": "team:acme", "tags": "job=nightly-summaries" }
}

Use an idempotency key

A retried POST /v1/batches without an Idempotency-Key creates a second batch and bills the whole file twice. Derive the key from the job (the date, the queue job ID) and a retry replays the original batch object instead.

Status

GET /v1/batches/batch_5Rj8Lm2Q HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY

Requires the batches:read scope.

curl https://REGION.api.lararouter.com/v1/batches/batch_5Rj8Lm2Q \
  -H "Authorization: Bearer $LARAROUTER_API_KEY"

{
  "id": "batch_5Rj8Lm2Q",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "status": "completed",
  "input_file_id": "file_9Hs4TnQ2",
  "output_file_id": "file_2Wp7Kd91",
  "error_file_id": "file_6Yt3Bn05",
  "completion_window": "24h",
  "request_counts": { "total": 4200, "completed": 4187, "failed": 13 },
  "cost_usd": 3.8412,
  "region": "REGION",
  "created": 1754150400,
  "in_progress_at": 1754150520,
  "completed_at": 1754169300,
  "expires_at": 1754236800,
  "metadata": { "entity": "team:acme", "tags": "job=nightly-summaries" }
}

Status	Meaning
`validating`	Reading the input file. Nothing has been billed.
`failed`	The input file didn't validate. `error_file_id` says which lines. Nothing billed.
`queued`	Validated and waiting to run.
`in_progress`	Running. `request_counts` moves as lines finish.
`completed`	Result files are ready. Some lines may still have failed. Check `request_counts.failed`.
`expired`	The window closed with work outstanding. Finished lines are in `output_file_id` and billed; the rest are in `error_file_id` and are not.
`cancelling`	A cancel was accepted. In-flight lines are allowed to finish.
`cancelled`	Stopped. Whatever completed before the cancel is in `output_file_id` and billed.

cost_usd is a Lararouter addition and is the authoritative number for the job. It only ever counts lines that produced a result, so an expired or cancelled batch bills for the part that ran.

Polling

Poll on a schedule, not in a loop. A minute between checks is plenty for a job measured in hours, and GET /v1/batches/{id} counts against your rate limit like any other call.

curl https://REGION.api.lararouter.com/v1/batches/batch_5Rj8Lm2Q \
  -H "Authorization: Bearer $LARAROUTER_API_KEY"

Poll until the status is terminal. A minute between checks is enough; tighter loops only spend your rate limit.

Reading results

Download output_file_id with GET /v1/files/{id}/content. Each line pairs your custom_id with the response that request would have returned.

{"custom_id":"ticket_4821","response":{"status_code":200,"request_id":"req_01JD8XK4M2","body":{"id":"chatcmpl_01JD8XK4M2","object":"chat.completion","model":"llama-3.3-70b","choices":[{"index":0,"message":{"role":"assistant","content":"The customer is locked out following a password reset…"},"finish_reason":"stop"}],"usage":{"prompt_tokens":284,"completion_tokens":96,"total_tokens":380,"cost_usd":0.00042}}},"error":null}

error_file_id uses the same envelope with response null and error populated:

{"custom_id":"ticket_4913","response":null,"error":{"type":"invalid_request_error","code":"context_length_exceeded","message":"Input plus max_tokens exceeds the model window.","param":"messages"}}

Lines fail individually. One malformed prompt in 4,200 doesn't stop the other 4,199, which is why completed and a non-zero request_counts.failed go together routinely. Always read both files.

Every line also lands in the audit log with its own request_id, so a result you lost is refetchable long after the file is gone.

Cancel

POST /v1/batches/batch_5Rj8Lm2Q/cancel HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY

Stops scheduling new lines. Anything already in flight runs to completion, so the batch sits in cancelling briefly before reaching cancelled.

Requires the batches:write scope.

curl -X POST https://REGION.api.lararouter.com/v1/batches/batch_5Rj8Lm2Q/cancel \
  -H "Authorization: Bearer $LARAROUTER_API_KEY"

Cancelling is not a refund. Lines that already produced a result are billed and appear in output_file_id.

List

GET /v1/batches?status=in_progress&limit=20 HTTP/1.1
Host: REGION.api.lararouter.com
Authorization: Bearer $LARAROUTER_API_KEY

Cursor paginated, newest first, filterable by status.

Requires the batches:read scope.

curl -G https://REGION.api.lararouter.com/v1/batches \
  -H "Authorization: Bearer $LARAROUTER_API_KEY" \
  -d "status=in_progress" \
  -d "limit=20"

{
  "object": "list",
  "data": [{ "id": "batch_5Rj8Lm2Q", "object": "batch", "status": "in_progress" }],
  "has_more": false,
  "next_cursor": null
}

The completion window

expires_at is created plus the window, and it is a ceiling rather than an estimate. Most batches finish well inside it; nothing is allowed to run past it.

Plan for expiry, not against it

An expired batch is a partial success, not an error. Import what's in output_file_id, read the batch_expired lines out of error_file_id, and resubmit those in the next job. Splitting a very large file into several smaller batches makes this cheaper to recover from.

Budgets are checked at submit as well as during the run: a batch whose projected cost would breach a limit is rejected with 429 usage_limit_exceeded rather than starting and stopping halfway.