ExplainerUpdated November 18, 2026

Private LLM API, three ways to get one

You can get a private LLM API by running an open-weight model yourself, getting zero retention from a lab, or using an API that keeps no prompts by default.

Running your own keeps every prompt in your hands but costs hardware and upkeep.

The three routes

RouteWho holds your promptsWhat it takesModels
Run it yourselfYouGPUs, an inference server, upkeepOpen weights small enough to run locally
Zero retention from a labThe lab, for the length of the requestApproval, a sales request or a contractThat lab's models
An API that keeps nothing by defaultThe API, for the length of the requestAn accountThe open-weight models it serves

1. Run an open-weight model yourself

llama.cpp and vLLM both serve an OpenAI-compatible endpoint, so your code doesn't change. Sizes are for 4-bit builds, from each model's Hugging Face card.

ModelParametersSize at 4-bitRuns onLicense
Qwen 3.8 27B Abliterated27.8Babout 17 GBone 24 GB GPU or a 32 GB MacApache 2.0
Qwen 3.6 35B-A3B Abliterated36B total, 3B activeabout 22 GBone 24 GB GPUApache 2.0
Gemma 4 31B Heretic31.3Babout 19 GBone 24 GB GPUApache 2.0
Llama 3.1 8B Abliterated8Babout 5 GBan 8 GB GPU or a laptopLlama 3.1 Community License

The largest open-weight models, such as Kimi K3.1 Pro at 2,780B parameters, need a rack of GPUs.

2. Get zero retention from a lab

Until a lab grants zero retention, its default applies.

LabKeeping nothing takesDefault until then
OpenAI APIPrior approvalPlus additional requirementsUp to 30 days
Claude APIA sales requestCovered Models keep 30 daysWithin 30 days
Mistral APIA written requestAt Mistral's discretionNo period published
xAI APIA console settingTeam-wide, not yet on for every team30 days
Gemini APISettings, or Vertex AIVertex AI for a guarantee55 days

See how to get zero data retention for each lab's steps.

3. Use an API that keeps nothing by default

This gets you large open-weight models without hardware or an approval step.

Silk ComputeZero data retention
"Silk Compute does not store your prompts or model outputs."

It serves six open-weight models, Kimi K3.1 Pro among them, with a 1,048,576-token context.

silkcompute.com

abliteration.ai and Featherless also say they keep none. The ranking compares all three, and the switching guide shows the code.