OpenAI compatible API endpoints
Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions
) and text embedding models (/v1/embeddings
). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.
Usage
Workers AI
Normally, Workers AI requires you to specify the model name in the cURL endpoint or within the env.AI.run
function.
With OpenAI compatible endpoints,you can leverage the openai-node sdk to make calls to Workers AI. This allows you to use Workers AI by simply changing the base URL and the model name.
OpenAI SDK Exampleimport OpenAI from "openai";
const openai = new OpenAI({ apiKey: env.CLOUDFLARE_API_KEY, baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1` });
const chatCompletion = await openai.chat.completions.create({ messages: [{ role: "user", content: "Make some robot noises" }], model: "@cf/meta/llama-3-8b-instruct", });
const embeddings = await openai.embeddings.create({ model: "@cf/baai/bge-large-en-v1.5", input: "I love matcha" });
cURL examplecurl --request POST \ --url https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions \ --header "Authorization: Bearer {api_token}" \ --header "Content-Type: application/json" \ --data ' { "model": "@cf/meta/llama-3-8b-instruct", "messages": [ { "role": "user", "content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible" } ] }'
AI Gateway
These endpoints are also compatible with AI Gateway.