Skip to main content

​
LiteLLM SDK

You can repoint LiteLLM through Butter using its completion function, which accepts the necessary base URL and headers. See it in the example below.
Beware of passing a single message (not OpenAI Chat Completions API compatible) rather than messages in the completion.
import os
from litellm import completion

response = completion(
    base_url = "https://proxy.butter.dev/v1", 
    extra_headers={"Butter-Auth": f"Bearer {os.getenv('BUTTER_API_KEY')}"},
    model="openai/gpt-4",
    messages = [{'role': 'user', 'content': "What is the English word for mantequilla?"}]
)

print(response)

​
LiteLLM Proxy

Butter also works when stacked with other Chat Completions endpoints like LiteLLM Proxy. LiteLLM Proxy supports a custom pass_through_endpoints configuration to route requests to Butter. See it in the example below.
litellm_config.yaml
model_list: []

general_settings:
  master_key: sk-1234

  pass_through_endpoints:
    - path: "/butter/v1/chat/completions"
      target: "https://proxy.butter.dev/v1/chat/completions"
      forward_headers: false
      forward_response: true 
      headers:
        Butter-Auth: "Bearer os.environ/BUTTER_API_KEY"
        Authorization: "Bearer os.environ/OPENAI_API_KEY"

import openai

client = openai.OpenAI(
    api_key="sk-1234",
    base_url="http://0.0.0.0:4000/butter/v1"
)

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "What is the English word for mantequilla?"}]
)

print(response.choices[0].message.content)