troubleshooting.md

Troubleshooting

This page covers the most common problems you'll hit when submitting and processing batch jobs, and how to resolve them. Parasail's batch service is 100% compatible with the OpenAI batch API, so the file formats, status values, and error codes described here match the OpenAI batch model.

JSONL validation failures

A batch input file is a .jsonl file where each line is a single, complete JSON object representing one request. A batch will fail validation if any line is malformed or missing required fields.

Each request line must include:

Common causes of validation failures:

To debug, validate the file locally before submitting:

import json

with open("example_batch_input.jsonl") as f:
    for i, line in enumerate(f, start=1):
        try:
            req = json.loads(line)
        except json.JSONDecodeError as e:
            print(f"Line {i} is not valid JSON: {e}")
            continue
        for key in ("custom_id", "method", "url", "body"):
            if key not in req:
                print(f"Line {i} is missing required key: {key}")

See Batch file format for the full request and response schema.

File-size and request-count limits exceeded

Parasail limits the size of each batch input file:

Workloads that exceed either limit must be split into multiple batches. If you use the Parasail Batch Helper Library, add_to_batch raises a ValueError when the file-size or request-count limit for the provider is exceeded. Catch it and roll over into a new batch:

from openai_batch import Batch

def submit_in_chunks(requests):
    batch_ids = []
    batch = Batch()
    for req in requests:
        try:
            batch.add_to_batch(**req)
        except ValueError:
            # Current batch is full; submit it and start a new one.
            batch_ids.append(batch.submit())
            batch = Batch()
            batch.add_to_batch(**req)
    batch_ids.append(batch.submit())
    return batch_ids

If you submit through the API or Batch UI directly, split the .jsonl file yourself so that each file stays under both limits.

Jobs stuck in validating or in_progress

A batch moves through validating, then in_progress, then a terminal state (completed, failed, expired, or cancelled). Jobs run on Parasail's servers, so a long-running batch is processed to completion even if your local script crashes, resets, or exits.

You do not need to keep a process running to wait for results. Record the batch ID at submission time and resume monitoring later from a separate process:

from openai_batch import Batch
import time

with Batch(batch_id="batch-tclfzwczcd",
           output_file="mybatch.jsonl") as batch:
    while True:
        status = batch.status()
        print(f"Batch status: {status.status}")
        if status.status in ["completed", "failed", "expired", "cancelled"]:
            break
        time.sleep(60)  # Check every minute

output_path, error_path = batch.download()
    print(f"Output saved to: {output_path}")

You can also track progress, view metadata, or cancel a queued or running batch from the Batch UI.

Partial failures

A batch can complete with some requests succeeding and others failing. Failures are reported per request rather than failing the whole batch, so always inspect each output line.

Two things to keep in mind when reading output:

Each output line contains a response object with a status_code. A 200 means the request succeeded; any other code is the same HTTP error you would have received for that request interactively.

import json

results = {}
with open("mybatch.jsonl") as f:
    for line in f:
        record = json.loads(line)
        custom_id = record["custom_id"]
        response = record.get("response") or {}
        status = response.get("status_code")
        if status == 200:
            results[custom_id] = response["body"]
        else:
            print(f"{custom_id} failed with status {status}: {record.get('error')}")

Failed requests can be collected and resubmitted in a new batch.

Quota errors

If a batch can't be scheduled because your organization is out of GPU capacity, you'll see an "Insufficient quota" message. By default, each organization has a quota of 4 GPUs, shared across both Batch and Dedicated services regardless of GPU type. A single batch that requires more GPUs than your remaining quota will be rejected.

To resolve this:

Authentication errors (401)

The batch tooling reads your key from the PARASAIL_API_KEY environment variable. A 401 response, or an error about a missing API key, usually means the key isn't set or isn't valid.

  PARASAIL_API_KEY=<your-key> python3 test_batch.py

  import os
  from openai import OpenAI

client = OpenAI(
      base_url="https://api.parasail.io/v1",
      api_key=os.environ["PARASAIL_API_KEY"],
  )

Output parsing

Results are returned as an output .jsonl file, with one response per line. Parse it line by line as JSON and pull response.body out of each record (see the partial-failures snippet above for matching on custom_id).

For embeddings, Parasail strongly recommends submitting requests with "encoding_format": "base64", which returns each embedding as a Base64-encoded binary array instead of a plain-text float array. This typically reduces output file size by nearly 4x with no loss of precision, but it means you must decode the value before use:

import base64
import json
import numpy as np

with open("embeddings-output.jsonl") as f:
    for line in f:
        record = json.loads(line)
        for item in record["response"]["body"]["data"]:
            vector = np.frombuffer(base64.b64decode(item["embedding"]), dtype=np.float32)
            # use vector ...

Next steps