dedicated.md
Dedicated Instances
Deploy a private GPU endpoint with full control over the model, hardware, and scaling. Dedicated instances are ideal for production workloads, custom models, and models not available on serverless.
1. Get an API key
- Go to https://www.saas.parasail.io/keys.
- Click Create API Key.
- Copy the key—it's shown only once.
2. Navigate to the Dedicated tab
In the Parasail dashboard, click the Dedicated tab to open the deployment configuration page.
3. Configure your deployment
| Field | Description |
|---|---|
| Deployment Name | A name for your endpoint. |
| HuggingFace ID / URL | The model to load. For example, neuralmagic/Qwen2-72B-Instruct-FP8 or the full HuggingFace URL. |
| HuggingFace Token | Required only for private HuggingFace repos. See Private HuggingFace Models. |
| Hardware Service Tier | Choose from low-cost, low-latency, or ultra-low-latency GPU configurations. |
If the model is supported, you'll see a green confirmation message.
4. (Optional) Set advanced options
- Speculative Decoding Draft Model—speed up output with a draft model. See Speculative Decoding.
- Replicas—spin up multiple load-balanced replicas for high throughput.
- Scale Down Policy—automatically shut down after idle time to avoid unexpected costs.
- Max Concurrent Connections—set a per-replica concurrency limit to maintain QoS.
5. Deploy
Click Deploy. If the model weights have been cached previously, deployment takes 3–5 minutes. If weights need to be downloaded from HuggingFace, it can take 10 minutes to several hours depending on model size.
6. Call your endpoint
Once the endpoint is live, call it using the OpenAI SDK:
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.parasail.io/v1",
api_key=os.environ["PARASAIL_API_KEY"],
)
response = client.chat.completions.create(
model="your-deployment-name",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
Next steps
- Dedicated instances overview—full guide to dedicated endpoints
- Auto-scaling—scale replicas with traffic
- FP8 quantization—reduce cost with quantization
- Management API—manage endpoints programmatically