For the complete documentation index, see [llms.txt](https://docs.parasail.io/parasail-docs/llms.txt). This page is also available as [Markdown](https://docs.parasail.io/parasail-docs/use-cases/batch-processing.md).

Process thousands or millions of inferences offline at 50% off serverless pricing. Batch is ideal for evaluations, data processing, content generation, and any workload that doesn't need real-time responses.

## Why batch?

- **50% off** serverless token pricing
- **Additional 50% off** for cached tokens
- **Up to 50,000 requests** and **1000 MB** per input file
- **OpenAI-compatible**—use the same batch file format and API
- **Any HuggingFace model**—no dedicated deployment needed
- **Resumable**—submit in one process, monitor in another

## Quickstart

Copy

```
import random
from openai_batch import Batch

with Batch() as batch:
    objects = ["cat", "robot", "coffee mug", "spaceship", "banana"]
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Tell me a joke about a {random.choice(objects)}"}]
        )
    result, output_path, error_path = batch.submit_wait_download()
    print(f"Batch completed with status {result.status}")
```

See the [batch quickstart](https://docs.parasail.io/parasail-docs/quickstart/batch) for the full guide.

## What you can batch

| Task               | Example models                  | Guide                                                          |
|--------------------|----------------------------------|----------------------------------------------------------------|
| Chat completions    | Any HuggingFace transformer      | [Batch full guide](https://docs.parasail.io/parasail-docs/products/quickstart) |
| Embeddings          | GritLM, GTE-Qwen2               | [Batch embedding models](https://docs.parasail.io/parasail-docs/products/quickstart#batch-embedding-models) |
| Image understanding  | Qwen3-VL-8B-Instruct            | [Batch image understanding](https://docs.parasail.io/parasail-docs/products/quickstart#batch-image-understanding) |
| Private models      | Any private HuggingFace repo     | [Batch private models](https://docs.parasail.io/parasail-docs/products/quickstart#private-models) |

## Pricing

Batch is billed at **50% of serverless pricing**, based on parameter count. FP8 models incur no additional cost; FP16 models are 30% more. See [full pricing details](https://docs.parasail.io/parasail-docs/billing/pricing#batch-pricing).

## Relevant pages

- [Batch quickstart](https://docs.parasail.io/parasail-docs/products/quickstart)—comprehensive guide
- [Batch helper GitHub repo](https://github.com/parasail-ai/openai-batch)—library source and README
- [File format](https://docs.parasail.io/parasail-docs/products/quickstart/file-format)—JSONL input format
- [API reference](https://docs.parasail.io/parasail-docs/api-reference/batch-api)—OpenAI-compatible batch API
- [Priority](https://docs.parasail.io/parasail-docs/api-reference/batch-api#batch-priority)—batch priority levels
- [Troubleshooting](https://docs.parasail.io/parasail-docs/products/quickstart/troubleshooting)