For the complete documentation index, see [llms.txt](https://docs.parasail.io/parasail-docs/llms.txt). This page is also available as [Markdown](https://docs.parasail.io/parasail-docs/quickstart/batch.md).

Process thousands or millions of LLM inferences at 50% off serverless pricing. Batch is ideal for evaluations, data processing, and any workload that doesn't require real-time responses.

## 1. Get an API key

1. Go to [https://www.saas.parasail.io/keys](https://www.saas.parasail.io/keys).
2. Click **Create API Key**.
3. Copy the key—it's shown only once.

Store it in your environment:

```
export PARASAIL_API_KEY="your-api-key-here"
```

## 2. Install the batch helper library

```
pip install openai-batch
```

The library is 100% compatible with both Parasail and OpenAI. Source code: [github.com/parasail-ai/openai-batch](https://github.com/parasail-ai/openai-batch).

## 3. Submit your first batch

```
import random
from openai_batch import Batch

with Batch() as batch:
    objects = ["cat", "robot", "coffee mug", "spaceship", "banana"]
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Tell me a joke about a {random.choice(objects)}"}]
        )
    result, output_path, error_path = batch.submit_wait_download()
    print(f"Batch completed with status {result.status} and stored in {output_path}")
```

Run it:

```
PARASAIL_API_KEY=<your-api-key> python3 test_batch.py
```

## 4. Monitor via the Batch UI

The Parasail Batch UI lets you upload batches, track status, and download results. Open the **Batch** tab in the dashboard.

## Key features

- **50% off serverless pricing**—cached tokens get an additional 50% off.
- **OpenAI-compatible**—works with the OpenAI batch file format and API.
- **Any HuggingFace model**—no need for a dedicated deployment; just specify the HuggingFace ID.
- **Embeddings**—supports embedding models, not just chat.
- **Resumable**—submit a batch, then monitor and download in a separate process using the batch ID.
- **Up to 50,000 requests** and **1000 MB** per input file.

## Next steps

- [Batch full guide](https://docs.parasail.io/parasail-docs/products/quickstart)—comprehensive batch processing guide
- [Batch helper GitHub repo](https://github.com/parasail-ai/openai-batch)—library source and README
- [Batch file format](https://docs.parasail.io/parasail-docs/products/quickstart/file-format)—JSONL input format (OpenAI-compatible)
- [Batch API reference](https://docs.parasail.io/parasail-docs/api-reference/batch-api)—OpenAI-compatible batch API spec
- [Batch with private models](https://docs.parasail.io/parasail-docs/products/quickstart#private-models)—batch processing with private HuggingFace repos