---
name: Novitaai
description: Use when building AI applications with LLM APIs, deploying GPU instances, running isolated code with Agent Sandbox, or generating images, videos, and audio. Reach for this skill when agents need to call language models, manage inference infrastructure, execute untrusted code safely, or integrate with AI model APIs.
metadata:
    mintlify-proj: novitaai
    version: "1.0"
---

# Novita AI Skill

## Product Summary

Novita AI is an AI-native cloud platform for running models, scaling GPUs, and building AI agents. It provides OpenAI-compatible LLM APIs (10,000+ models), GPU instance management, serverless GPU endpoints, and Agent Sandbox for isolated code execution. Access via REST APIs with Bearer token authentication (`Authorization: Bearer sk_*`). Base URL: `https://api.novita.ai` (OpenAI-compatible: `https://api.novita.ai/openai`). Install SDKs: `pip install novita-sandbox` (Python) or `npm i novita-sandbox` (JS/TS). CLI: `npm i -g novita-sandbox-cli`. Primary docs: https://novita.ai/docs

## When to Use

Use Novita AI when:
- **Building LLM applications**: Chat completions, embeddings, function calling, structured outputs, vision/multimodal tasks
- **Running inference at scale**: Need cost-effective model serving with 10,000+ open-source models
- **Deploying GPU workloads**: Training, fine-tuning, or custom model inference on dedicated or serverless GPUs
- **Executing untrusted code safely**: Agent Sandbox provides isolated environments with full state preservation
- **Batch processing**: Large-scale async LLM jobs with 48-hour completion windows
- **Integrating with agent frameworks**: OpenAI SDK compatibility, LangChain, Claude Code, Codex, and 25+ integrations

## Quick Reference

### Authentication & Setup
| Task | Command/Code |
|------|--------------|
| Create API key | Console → Settings → Key Management → Create API Key (copy immediately, shown once only) |
| Set environment variable | `export NOVITA_API_KEY="sk_..."` (Linux/macOS) or `setx NOVITA_API_KEY "sk_..."` (Windows) |
| Bearer header format | `Authorization: Bearer sk_<your-key>` |
| Check balance | `curl https://api.novita.ai/v1/user/balance -H "Authorization: Bearer $NOVITA_API_KEY"` |
| Key expiration options | Permanent / 90 days / 30 days / 24 hours (set at creation, cannot change) |

### LLM API Endpoints
| Endpoint | Purpose | OpenAI Compatible |
|----------|---------|-------------------|
| `/openai/v1/chat/completions` | Chat-based LLM inference | Yes |
| `/openai/v1/completions` | Text completion | Yes |
| `/openai/v1/embeddings` | Generate embeddings | Yes |
| `/v1/models` | List available models | No |
| `/v1/batches` | Async batch processing | No |

### Common Model Patterns
| Use Case | Model Examples | Key Parameter |
|----------|---|---|
| Fast reasoning | `deepseek/deepseek-r1`, `minimax/minimax-m2` | `stream: true/false` |
| Vision/multimodal | `qwen/qwen-vl-max`, `gpt-4-vision` | `messages[].content[].type: "image_url"` |
| Function calling | Any model | `tools: [{type: "function", ...}]` |
| Structured output | Any model | `response_format: {type: "json_schema", ...}` |

### Sandbox Lifecycle
| State | Behavior | Billing |
|-------|----------|---------|
| Running | Active, executes commands, serves connections | Per-second CPU/RAM |
| Paused | Suspended, state preserved, can resume | Storage only |
| Killed | Terminated, resources released, cannot resume | None |

### GPU Instance Types
| Type | Use Case | Scaling |
|------|----------|---------|
| GPU Instance | Persistent VMs, direct runtime control, training/fine-tuning | Manual |
| Serverless GPU | Stateless inference endpoints, auto-scaling | Automatic |

## Decision Guidance

### When to Use X vs Y

| Decision | Use This | When | Use That | When |
|----------|----------|------|----------|------|
| **LLM API vs GPU Instance** | LLM API | Need inference on pre-deployed models, cost-sensitive, quick integration | GPU Instance | Running custom models, need persistent state, training workloads |
| **Streaming vs Non-Streaming** | Streaming (`stream: true`) | Long outputs, real-time UX, prevent timeout | Non-streaming (`stream: false`) | Short responses, need full result at once, simpler parsing |
| **Chat vs Completion** | Chat Completion | Multi-turn dialogue, system prompts, structured messages | Completion | Simple text generation, prompt-only input |
| **Batch vs Real-Time** | Batch API | 1000+ requests, cost optimization, 48-hour window acceptable | Real-Time API | <100ms latency required, interactive workflows |
| **Sandbox vs GPU Instance** | Sandbox | Run untrusted code, agents, file/browser workflows, state preservation | GPU Instance | Long-running services, custom model serving, training |
| **Pause vs Kill Sandbox** | Pause | Resume later, preserve state, reduce cost temporarily | Kill | Permanent cleanup, free resources immediately |

## Workflow

### 1. Set Up Authentication
1. Log into https://novita.ai/user/login
2. Go to Settings → Key Management
3. Click "Create API Key", name it (e.g., `production`), copy immediately
4. Store in environment variable: `export NOVITA_API_KEY="sk_..."`
5. Verify: `curl https://api.novita.ai/v1/user/balance -H "Authorization: Bearer $NOVITA_API_KEY"`

### 2. Make Your First LLM API Call
1. Choose a model from https://novita.ai/models (e.g., `deepseek/deepseek-r1`)
2. Set base URL to `https://api.novita.ai/openai` (OpenAI-compatible)
3. Initialize client with API key:
   ```python
   from openai import OpenAI
   client = OpenAI(base_url="https://api.novita.ai/openai", api_key=os.environ["NOVITA_API_KEY"])
   ```
4. Call chat completion:
   ```python
   response = client.chat.completions.create(
       model="deepseek/deepseek-r1",
       messages=[{"role": "user", "content": "Hello"}],
       max_tokens=512
   )
   ```
5. Parse response: `response.choices[0].message.content`

### 3. Deploy a Sandbox for Code Execution
1. Install SDK: `pip install novita-sandbox`
2. Create sandbox from template:
   ```python
   from novita_sandbox import Sandbox
   sandbox = Sandbox.create(template="python-3.11")
   ```
3. Execute command: `result = sandbox.run_command("python script.py")`
4. Access filesystem: `sandbox.filesystem.write("/tmp/file.txt", "content")`
5. Pause when done: `sandbox.pause()` (resume later with `sandbox.resume()`)
6. Kill when finished: `sandbox.kill()`

### 4. Deploy a GPU Instance
1. Go to Console → GPU Instances
2. Click "Create Instance", select GPU type (e.g., RTX 4090), region, image
3. Wait for instance to start (status: Running)
4. Connect via SSH or web terminal
5. Install dependencies, run workloads
6. Stop when idle: `instance.stop()` (billing pauses)
7. Delete when done: `instance.delete()`

### 5. Submit a Batch Job
1. Prepare JSONL file with requests (one per line):
   ```json
   {"custom_id": "1", "params": {"model": "deepseek/deepseek-r1", "messages": [{"role": "user", "content": "Q1"}]}}
   ```
2. Upload file: `file_id = client.files.create(file=open("batch.jsonl", "rb")).id`
3. Create batch: `batch = client.batches.create(input_file_id=file_id)`
4. Poll status: `client.batches.retrieve(batch.id).status` (completes within 48 hours)
5. Retrieve results: `client.files.content(batch.output_file_id).text`

## Common Gotchas

- **API key shown only once**: Copy immediately after creation. If lost, create a new key; old one cannot be recovered.
- **Expiration cannot be changed**: Set expiration at creation time. Expired keys are rejected; create a new key to restore access.
- **Environment variable not found**: Ensure you ran `export` in the same terminal session, or set permanently in shell profile (`.bashrc`, `.zshrc`) and restart terminal. Services (Docker, systemd) need their own env config.
- **429 Rate Limit errors**: Implement exponential backoff on retries. Check if limit is TPM (tokens/minute) or RPM (requests/minute). Contact support to raise limits.
- **Streaming timeout**: Long outputs may timeout without streaming. Use `stream: true` for responses >5000 tokens.
- **Sandbox auto-kill on idle**: Idle timeout fires when no client connected for configured duration. Set `idle_timeout` to pause instead of kill to preserve state.
- **Batch 48-hour window**: Batch jobs must complete within 48 hours. Plan accordingly for large datasets.
- **Async image/video tasks**: Image and video generation return `task_id` immediately. Poll `/v1/tasks/{task_id}` to check status and retrieve results.
- **Model access policies**: If key has model access restrictions, requests to unlisted models return 403. Check key policy: `GET /v1/keys/{key_id}/model-access`
- **Network access policies**: If key has IP restrictions, requests from unlisted IPs return 403. Whitelist your IP in key settings.

## Verification Checklist

Before submitting work with Novita AI:

- [ ] API key is set in environment variable and not hardcoded
- [ ] Bearer token format is correct: `Authorization: Bearer sk_...`
- [ ] Base URL matches use case: `https://api.novita.ai/openai` (LLM) or `https://api.novita.ai` (other APIs)
- [ ] Model name is valid (check https://novita.ai/models or `GET /v1/models`)
- [ ] Account has sufficient balance (check `GET /v1/user/balance`)
- [ ] For streaming: `stream: true` is set and response is parsed as chunks
- [ ] For async tasks (image/video): polling loop checks `status == "completed"` before reading results
- [ ] For batch jobs: input file is valid JSONL with required fields
- [ ] For Sandbox: state is preserved (paused) or cleaned up (killed) after use
- [ ] For GPU instances: instance is stopped when not in use to reduce billing
- [ ] Error responses are checked: 401 (auth), 403 (access), 429 (rate limit), 400 (validation)

## Resources

**Comprehensive navigation**: https://novita.ai/docs/llms.txt

**Critical pages**:
1. [LLM API Guide](https://novita.ai/docs/guides/llm-api) — OpenAI-compatible chat/completion endpoints, streaming, parameters
2. [API Reference Overview](https://novita.ai/docs/api-reference/api-reference-overview) — all endpoints, authentication, error codes
3. [Sandbox Overview](https://novita.ai/docs/guides/sandbox-overview) — isolated execution, lifecycle, state management

---

> For additional documentation and navigation, see: https://novita.ai/docs/llms.txt