LlamaIndex

LlamaIndex reaches OpenAI-compatible services through OpenAILike from the llama-index-llms-openai-like package. It can't look up a model's details from an ID it doesn't know, so you set two of them yourself: that the model is a chat model, and its context window.

Set it up

The current LlamaIndex packages need Python 3.10 or later.

pip install llama-index-llms-openai-like

Look up the context length on the model's page, for example deepseek-v4-1-flash, and put it in CONTEXT_WINDOW:

import os
from llama_index.core.llms import ChatMessage
from llama_index.llms.openai_like import OpenAILike

CONTEXT_WINDOW = ...  # the context length from the model's page, in tokens

llm = OpenAILike(
    model="deepseek-v4-1-flash",
    api_base="https://api.cheapaimodel.ai/v1",
    api_key=os.environ["CHEAPAIMODEL_API_KEY"],
    is_chat_model=True,
    context_window=CONTEXT_WINDOW,
)

response = llm.complete("Explain recursion in one sentence.")
print(response.text)

# Streaming
messages = [ChatMessage(role="user", content="Write a haiku about the sea.")]
for chunk in llm.stream_chat(messages):
    print(chunk.delta, end="", flush=True)

With is_chat_model=True, every call, complete included, goes to /v1/chat/completions.

Use it across LlamaIndex

Set it on Settings to make it the default for indexes, query engines and chat engines:

from llama_index.core import Settings

Settings.llm = llm

Vector indexes also need an embedding model, and Settings.embed_model defaults to OpenAI's. cheapaimodel has no embeddings models yet, so set Settings.embed_model to another provider's embedding model or a local one. Don't point it at our base URL.

Common mistakes

• Requests go to /v1/completions. is_chat_model defaults to False, which sends complete and even chat to the legacy completions endpoint as a single prompt. Our models list only /v1/chat/completions and /v1/responses. Set is_chat_model=True.
• ValueError: Unknown model 'deepseek-v4-1-flash'. Please provide a valid OpenAI model name. The LLM is OpenAI from llama_index.llms.openai, which accepts only OpenAI's own model names. Use OpenAILike.
• Could not load OpenAI embedding model when building an index. Settings.embed_model is still the OpenAI default. Set it to another provider's or a local embedding model.
• 503 with model_not_found on /v1/embeddings. Same cause, with OPENAI_API_KEY and OPENAI_API_BASE set to your cheapaimodel key and our base URL: the default embedding model reads them and asks us for an embeddings model. Pass api_key and api_base to OpenAILike instead, and set Settings.embed_model.
• 400 from kimi-k3. OpenAILike always sends temperature, 0.1 by default, and kimi-k3 accepts only 1 or 0.6. Pass temperature=1.0. See Sampling limits.
• One query makes several requests. context_window defaults to 3900 tokens, and LlamaIndex splits retrieved text to fit, so a query engine sends more requests than it needs to. Set it to the model's context length.

For base URL mistakes such as a missing /v1, see Base URL.