For the complete documentation index, see llms.txt. Markdown versions of documentation pages are available by appending .md to page URLs; this page is available as Markdown.

Batch API

Parasail's batch API is a direct drop-in replacement for OpenAI's batch API. The same documentation, guides, and client libraries apply.

Base URL

https://api.parasail.io/v1

Client setup

from openai import OpenAI

client = OpenAI(
    base_url="https://api.parasail.io/v1",
    api_key="<PARASAIL_API_KEY>",
)

Key differences from OpenAI

• Use the Parasail URL instead of OpenAI's.
• Use your Parasail API key (create one here).
• Use an open source model from Hugging Face instead of ChatGPT.
• Up to 50,000 requests and 1000 MB per input file (vs OpenAI's 250 MB).

OpenAI API reference

• OpenAI Batch API reference
• OpenAI Files API reference

Batch helper library

Parasail provides a Python batch helper library that simplifies batch submission and monitoring:

pip install openai-batch

See the openai-batch GitHub repository and batch full guide for details.

Batch file format

The batch input file is a JSONL file where each line is a request. The format is 100% compatible with OpenAI. See Batch file format.

Batch priority

Batches have a default priority of 0. Set a higher priority by passing the PARASAIL_PRIORITY metadata key when creating the batch.

from openai_batch import Batch

with Batch() as batch:
    for i in range(100):
        batch.add_to_batch(
            model="NousResearch/DeepHermes-3-Mistral-24B-Preview",
            messages=[{"role": "user", "content": f"Prompt #{i}"}],
        )

batch.submit(metadata={"PARASAIL_PRIORITY": "1"})

Use priority sparingly for urgent jobs. Standard jobs should omit this metadata and use the default priority.