How to use Darkbloom with LiteLLM

At a glance

How do I use LiteLLM with Darkbloom in Python?

Call litellm.completion() with model="openai/<model-id>", api_base="https://api.darkbloom.dev/v1" and your Darkbloom key as api_key. It takes about five minutes.

1. Get a Darkbloom API key. Open the Darkbloom console and sign in, add credits (the minimum is $0.50, and Darkbloom says credits are non-refundable and never expire), then choose API console in the console’s left menu and Create API key (Darkbloom’s docs call this page Settings → API Keys). The console shows the key only once, so copy it right away. Every key spends from your prepaid balance, so add only what you are willing to spend while you test, and watch the first runs, because a loop or a batch job can send far more requests than you expect.
2. Install LiteLLM. It needs Python 3.10 or newer. On an older Python, pip can quietly install an old LiteLLM release without an error (LiteLLM docs), so check python --version first.

   Copy

   pip install litellm

3. Put your key in an environment variable, so it stays out of your code. Use the terminal where you will run Python:

   macOS and LinuxCopy

   export DARKBLOOM_API_KEY="YOUR_DARKBLOOM_API_KEY"

   Windows (then open a new terminal)Copy

   setx DARKBLOOM_API_KEY "YOUR_DARKBLOOM_API_KEY"

4. See which models you can use. Darkbloom’s model list gives each ID with its context window and prices:

   Copy

   curl -s https://api.darkbloom.dev/v1/models \
     -H "Authorization: Bearer $DARKBLOOM_API_KEY"

5. Make a request. Save this as app.py and run python app.py, with an ID from that list and the openai/ prefix:

   app.pyCopy

   import os
   import litellm
   
   response = litellm.completion(
       model="openai/gemma-4-26b",
       api_base="https://api.darkbloom.dev/v1",
       api_key=os.environ["DARKBLOOM_API_KEY"],
       messages=[{"role": "user", "content": "Say hello in five words."}],
   )
   print(response.choices[0].message.content)

Streaming and tool calls work the same way. Add stream=True to get chunks as they arrive, or pass a tools list. In our test gemma-4-26b answered a request with a tool by returning a tool call (finish_reason “tool_calls”). The Features column below shows which models Darkbloom lists with tool support.

To avoid passing api_base and api_key in every call, you can set the environment variables OPENAI_API_BASE and OPENAI_API_KEY instead, and LiteLLM uses them for every openai/ model in that process (checked). The catch is that you then cannot call OpenAI itself from the same process with the same variables.

How do I use Darkbloom through the LiteLLM proxy?

The LiteLLM proxy is a local server that speaks the OpenAI API and forwards requests to the models in its config.yaml. Use it when several apps or tools should share one Darkbloom key and one URL. Add Darkbloom to the config, start the proxy, and point any OpenAI-compatible client at it. You need a Darkbloom key from step 1 above.

1. Install LiteLLM with the proxy extras.

   Copy

   pip install "litellm[proxy]"

2. Create a config file named config.yaml, with one entry per model. model_name is the name your apps will ask for, and you can choose it freely. os.environ/DARKBLOOM_API_KEY makes LiteLLM read the key from that environment variable, so the key is not written in the file.

   config.yamlCopy

   model_list:
     - model_name: darkbloom-gemma
       litellm_params:
         model: openai/gemma-4-26b
         api_base: https://api.darkbloom.dev/v1
         api_key: os.environ/DARKBLOOM_API_KEY

3. Set two environment variables in the terminal that will run the proxy: your Darkbloom key, and a master key that your apps will use to call the proxy. Make up a long random string that starts with sk-, and keep a copy, because your apps need it.

   macOS and LinuxCopy

   export DARKBLOOM_API_KEY="YOUR_DARKBLOOM_API_KEY"
   export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 16)"

   Windows (then open a new terminal)Copy

   setx DARKBLOOM_API_KEY "YOUR_DARKBLOOM_API_KEY"
   setx LITELLM_MASTER_KEY "sk-YOUR_LONG_RANDOM_STRING"

4. Start the proxy. It listens on port 4000. --host 127.0.0.1 makes it reachable from your own computer only.

   Copy

   litellm --config config.yaml --host 127.0.0.1

5. Call it as you would call OpenAI, with the master key as the API key and your model_name as the model:

   curlCopy

   curl http://127.0.0.1:4000/v1/chat/completions \
     -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "darkbloom-gemma",
       "messages": [{"role": "user", "content": "Say hello in five words."}]
     }'

   OpenAI Python libraryCopy

   import os
   from openai import OpenAI
   
   client = OpenAI(base_url="http://127.0.0.1:4000/v1", api_key=os.environ["LITELLM_MASTER_KEY"])
   reply = client.chat.completions.create(
       model="darkbloom-gemma",
       messages=[{"role": "user", "content": "Say hello in five words."}],
   )
   print(reply.choices[0].message.content)

Any other tool that takes a base URL and a key can use the proxy as well: give it http://127.0.0.1:4000/v1 and the master key, and ask for your model_name.

Which Darkbloom model should I use with LiteLLM?

Use any ID that GET /v1/models returns. On October 23, 2026 it listed 12 models, shown here from the cheapest output price up, in US dollars per million tokens. Check the live prices before you rely on a number; that public list also names models that are not online. A key can be limited to some models, and Darkbloom refuses any other with a 403 (see the errors table).

In the proxy, apps ask for the model_name from your config, such as darkbloom-gemma, and not for Darkbloom’s ID. To offer another model, add another entry with its own model_name and model: openai/<model-id>.

Can one proxy entry cover every Darkbloom model?

Yes, with a wildcard entry (LiteLLM docs). This one sends any model name that starts with darkbloom/ to Darkbloom and uses the rest of the name as the model ID, so darkbloom/gemma-4-26b reaches gemma-4-26b:

model_list:
  - model_name: "darkbloom/*"
    litellm_params:
      model: "openai/*"
      api_base: https://api.darkbloom.dev/v1
      api_key: os.environ/DARKBLOOM_API_KEY

This works for any ID Darkbloom serves, including IDs that contain a slash, such as darkbloom/EigenLabs/Qwen3.8-27B-4bit-mtp. A key limited to some models still gets the 403 for the others. One thing does not work well: the proxy’s own /v1/models does not list Darkbloom’s models. With this entry it listed 211 names that start with darkbloom/ and are OpenAI model names, and none of Darkbloom’s. An app that fills its model menu from the proxy would show those, so type the model name yourself, or use one named entry per model as in the config above.

How much does LiteLLM cost with Darkbloom, and what will it report?

LiteLLM adds no charge. Darkbloom bills the tokens you use from your prepaid balance: (input tokens × input price + output tokens × output price) ÷ 1,000,000, with a minimum of $0.0001 per request. At the October 23, 2026 prices, a request with 1,000 tokens in and 200 out is worth $0.000086 on gemma-4-26b, so Darkbloom bills the $0.0001 minimum, while 20,000 tokens in and 2,000 out costs about $0.00128. Because of the minimum, 10,000 small requests cost at least $1.00 on any model. Each of our small test requests was billed exactly $0.0001 in Darkbloom’s usage log.

To give the proxy the prices, add model_info to the entry (these are the October 23, 2026 list prices for gemma-4-26b, in dollars per token). The proxy then reports the estimate for each response in the x-litellm-response-cost header. The entry below also sets max_tokens, which the next paragraph explains:

model_list:
  - model_name: darkbloom-gemma
    litellm_params:
      model: openai/gemma-4-26b
      api_base: https://api.darkbloom.dev/v1
      api_key: os.environ/DARKBLOOM_API_KEY
      max_tokens: 500
    model_info:
      input_cost_per_token: 0.000000042
      output_cost_per_token: 0.00000022

In Python, pass the same numbers with a call and read the estimate from the response:

response = litellm.completion(
    model="openai/gemma-4-26b",
    api_base="https://api.darkbloom.dev/v1",
    api_key=os.environ["DARKBLOOM_API_KEY"],
    messages=[{"role": "user", "content": "Say hello in five words."}],
    max_tokens=500,
    input_cost_per_token=0.000000042,
    output_cost_per_token=0.00000022,
)
print(response._hidden_params["response_cost"])

LiteLLM sends no max_tokens unless you set one (in a call, or in litellm_params for every request through the proxy, as above). Without it Darkbloom adds 8192 (Darkbloom docs) and reserves balance for the whole amount while the request runs, so a limit that fits your use keeps that reservation small.

How do I keep my API keys out of my code?

Export DARKBLOOM_API_KEY in the terminal that runs Python or the proxy, as in the steps above. Read it with os.environ in code and os.environ/DARKBLOOM_API_KEY in config.yaml. LiteLLM also loads a .env file, but only if it finds one, and where it looks depends on where LiteLLM is installed. In our test it found .env in the project folder when the virtual environment was inside that folder (python -m venv .venv), and it did not find it when LiteLLM was installed somewhere else. Exporting the variables works in both cases.

If you do use a .env file, add it to .gitignore so the keys are never committed.

echo ".env" >> .gitignore

What do the common errors mean?

These are the messages you are most likely to see, with what causes each and the fix. All but the empty-balance row were reproduced with LiteLLM 1.106.2. Through the proxy, an error from Darkbloom arrives as an HTTP error with LiteLLM’s text inside.

Is Darkbloom private, and is it ready for production?

Not yet for sensitive work. Darkbloom’s quickstart calls the service “an experimental research prototype” and says not to use it in production applications, and its console labels it Public alpha. Darkbloom says prompts are encrypted to a verified provider Mac. Its security overview says the coordinator re-encrypts each request for the chosen provider and runs in a confidential VM. An independent review by Wavect (September 2026) reads Darkbloom’s technical documentation as saying that both the coordinator and the provider handle your prompt as plaintext inside protected boundaries.

LiteLLM sends the messages you give it to Darkbloom. If you turn on LiteLLM’s logging or callbacks, those can store your prompts too. Do not send secrets, customer data or code you are not allowed to share.

How this guide was checked

• A paid request from LiteLLM to api.darkbloom.dev worked on October 22, 2026. LiteLLM 1.106.2 on Linux, in a fresh Python 3.13 environment, sent requests to api.darkbloom.dev with openai/gemma-4-26b from Python (litellm.completion, also with streaming and with a tool call) and through the LiteLLM proxy (curl, the OpenAI Python library and a wildcard entry). Each one got an answer, and Darkbloom’s usage log shows each small request billed at the $0.0001 minimum.
• The setup steps were compared with LiteLLM’s own docs (linked under Sources) on October 22, 2026.
• Every macOS and Linux command, Python snippet and config on this page was run from a fresh Python 3.13 virtual environment, installed with pip install "litellm[proxy]", which gave LiteLLM 1.106.2, the newest release on PyPI that day. That covers the Python call, streaming, a tool call, the proxy with each config shown, requests to the proxy with curl and with the OpenAI Python library, the price settings and the wildcard entry.
• On the wire, against a local test server that behaves like an OpenAI-compatible API, LiteLLM sent POST /v1/chat/completions with a Bearer header, removed the openai/ prefix and sent no max_tokens unless one was set. It read OPENAI_API_BASE and OPENAI_API_KEY when api_base and api_key were not given.
• The “not permitted to use model” row, the 404 row and the model-prefix row were reproduced against Darkbloom itself. The “invalid API key” row used a test server that returns Darkbloom’s real reply for a wrong key. Darkbloom’s own text for an empty balance was not captured, because that needs an empty balance.
• The model list, context windows, features and prices come from GET /v1/models on October 22, 2026 (it needs a key). The key steps in the console come from the console’s own page text. We tested with a key that already existed, so we did not click through key creation ourselves.
• Not run: the Windows commands, Docker, uv tool install, LiteLLM virtual keys and its spend database.
• Our Darkbloom link was opened in a clean browser profile on October 22, 2026. It saved the referral code and the page said the code “is saved and will be applied after you sign in”. We did not create an account to test the sign-in step itself.

About this guide

BloomGauge is a free Mac app for people who run Darkbloom providers, so we benefit when Darkbloom has more paying demand. We are independent of Darkbloom and not endorsed by it. Our Darkbloom sign-in links carry our referral code, BLOOMGAUGE. If you sign in to Darkbloom after opening one, Darkbloom attaches the code to your account and pays us a share of your eligible token spend. Darkbloom funds that reward, so it does not change your price.

Sources

• LiteLLM docs: OpenAI-compatible endpoints
• LiteLLM docs: proxy CLI quick start
• LiteLLM docs: wildcard routing
• LiteLLM docs: custom LLM pricing
• LiteLLM on PyPI
• Darkbloom docs: quickstart
• Darkbloom docs: chat completions
• Darkbloom docs: authentication and API keys
• Darkbloom docs: add credits
• Darkbloom docs: usage history
• Darkbloom live prices (GET /v1/pricing)

Found something out of date? Tell us in #bloomgauge on the Darkbloom Slack or at Support.