model recommendations.md

Model Selection

Model availability and capabilities change frequently. Instead of relying on a static recommendation table, use this guide to choose a short list of candidates, check availability with the live models endpoint, and evaluate the models against your own prompts before committing to production.

To list currently available serverless models, use the /v1/models endpoint.

Start with the workload

Workload	What to optimize for	Start here
Chat and text generation	Instruction following, latency, context length, and cost	Chat and text generation
RAG and embeddings	Embedding quality, retrieval latency, output size, and batch throughput	RAG and embeddings
Batch evaluation or offline inference	Throughput, price, retry handling, and output reconciliation	Batch processing
Agents and tool calling	Tool-call reliability, structured outputs, multi-turn behavior, and latency	Agents and tool calling
Vision and multimodal understanding	Image input quality, input size, and task success	Multi-Modal
Image generation and editing	Image quality, edit fidelity, batch throughput, and prompt sensitivity	Image Generation

Choose the right product tier

Tier	Best fit	Notes
Serverless	Fast experiments and production traffic on models already hosted by Parasail	Use the models endpoint to check availability before wiring a model into an application.
Dedicated Instances	Private models, custom Hugging Face models, predictable capacity, or model settings that need full endpoint control	Start in the UI when possible so the compatibility checker can validate model and hardware choices.
Batch	Large offline jobs, evaluation sets, embeddings, and workloads that do not need immediate responses	Use `custom_id` values so outputs can be matched back to inputs and failed requests can be retried selectively.

Check live availability

Use the models endpoint before committing to a model name:

curl https://api.parasail.io/v1/models \
  -H "Authorization: Bearer $PARASAIL_API_KEY"

The response lists the serverless models available to your account. For Dedicated Instances, use the deployment flow or the Dedicated compatibility checks to validate public or private Hugging Face model IDs.

Evaluate candidates

Use the same representative prompts, documents, tool schemas, or images for every candidate. Track these dimensions:

Quality: task success rate, factuality, formatting reliability, and human review scores.
Latency: time to first token for streaming and total response time for non-streaming calls.
Cost: token usage, output length, deployment size, and whether the workload can run through Batch.
Context fit: maximum input size, retrieval chunk size, and expected output length.
Feature support: structured output, tool calling, multimodal input, streaming, and Responses API support where required.
Operational fit: rate limits, quota requirements, retry behavior, and monitoring needs.

Run a repeatable evaluation

Use one evaluation set for every candidate model.

Collect representative prompts, documents, images, tool schemas, or expected JSON schemas from your product.
Run the same inputs through each candidate model.
Save the raw request, raw response, latency, token usage, and pass/fail result for each input.
Review failures by category: wrong answer, bad format, missing tool call, hallucination, timeout, or cost outlier.
Promote only models that meet your quality, latency, cost, and feature-support thresholds.

For small evaluations, call Serverless or Dedicated endpoints directly. For large offline evaluations, use Batch so you can process many requests at lower cost and retry failed rows by custom_id.

For private or fine-tuned models, start with Dedicated Instances and validate the model ID in the deployment flow before running a full evaluation.

Validate feature-specific requirements

Some workloads need more than general text quality:

For JSON or schema-constrained output, test the model with Structured Output.
For function calling and agent loops, test the model with Tool/Function Calling and the Responses API.
For multimodal prompts, test with the exact image or document types described in Multi-Modal.
For embeddings, measure retrieval quality on your own corpus before scaling through Batch.

Next steps

Models endpoint—list available serverless models
Serverless overview—make interactive OpenAI-compatible calls
Dedicated Instances overview—deploy public or private models
Batch Quickstart—run large offline jobs