Chat Completions - Cerebras Inference (https://inference-docs.cerebras.ai/api-reference/chat-completions) citeturn153view1 [wordlim: 200] Crawled: today; Content type: text/html; Source: open({"ref_id":"https://inference-docs.cerebras.ai/api-reference/chat-completions","lineno":null}); Total lines: 988 L0: > ## Documentation Index L1: > L2: > Fetch the complete documentation index at: 【0†/llms.txt】 L3: > L4: > Use this file to discover all available pages before exploring further. L5: L6: 【1†iframe†www.googletagmanager.com】【2†Skip to main content】 L7: L8: 【3†Cerebras Inference home page】 L9: L10: Search... L11: L12: Navigation L13: L14: Chat L15: L16: Chat Completions L17: L18: * 【4†Python SDK†github.com】 L19: * 【5†Node.js SDK†github.com】 L20: L21: ### Introduction L22: L23: * 【6†Authentication】 L24: * 【7†Versions】 L25: L26: ### Chat L27: L28: * 【8†POST Chat Completions】 L29: L30: ### Completions L31: L32: * 【9†POST Completions】 L33: ### Models L34: L35: * 【10†GET List models】 L36: * 【11†GET Retrieve model】 L37: * 【12†GET Public models】 L38: L39: ### Batch L40: L41: Preview L42: L43: * 【13†POST Create batch】 L44: * 【14†GET Retrieve batch】 L45: * 【15†GET List batch】 L46: * 【16†DEL Cancel batch】 L47: L48: ### Files L49: L50: Preview L51: L52: * 【17†POST Upload file】 L53: * 【18†GET List files】 L54: * 【19†GET Retrieve file】 L55: * 【20†GET Retrieve file content】 L56: * 【21†DEL Delete file】 L57: L58: ### Metrics L59: L60: * 【22†GET Retrieve metrics】 L61: ### Management L62: L63: Preview L64: L65: * 【23†GET List model architectures】 L66: * 【24†GET List model versions】 L67: * 【25†POST Upload model version】 L68: * 【26†GET Retrieve model version status】 L69: * 【27†DEL Delete model version】 L70: * 【28†PATCH Update model version aliases】 L71: * 【29†GET List endpoints】 L72: * 【30†POST Deploy model to endpoint】 L73: * 【31†GET Retrieve endpoint status】 L74: # Chat Completions L75: L76: Generate conversational responses using a structured message format with roles (system, user, assistant, developer, tool). Best for chatbots, assistants, and multi-turn conversations. L77: L78: POST L79: L80: / L81: L82: v1 L83: L84: / L85: L86: chat L87: L88: / L89: L90: completions L91: L92: Python L93: Default L94: L95: `from cerebras.cloud.sdk import Cerebras L96: import os L97: L98: client = Cerebras(api_key=os.environ.get("CEREBRAS_API_KEY")) L99: L100: chat_completion = client.chat.completions.create( L101: model="qwen-3.8-27b", L102: reasoning_effort="none", L103: messages=[ L104: {"role": "user", "content": "Hello!"} L105: ], L106: ) L107: print(chat_completion)` L108: L109: 200 L110: Default L111: L112: `{ L113: "id": "chatcmpl-b8d624a5-43d4-477a-8b94-61be750e2872", L114: "choices": [ L115: { L116: "finish_reason": "stop", L117: "index": 0, L118: "message": { L119: "role": "assistant", L120: "content": "Hello! How can I assist you today?", L121: "tool_calls": null L122: }, L123: "logprobs": null L124: } L125: ], L126: "created": 1775679664, L127: "model": "qwen-3.8-27b", L128: "object": "chat.completion", L129: "system_fingerprint": "fp_4c26d27ac5dbffe28c72", L130: "usage": { L131: "prompt_tokens": 69, L132: "completion_tokens": 45, L133: "total_tokens": 114 L134: }, L135: "time_info": { L136: "queue_time": 0.000630072, L137: "prompt_time": 0.006339716, L138: "completion_time": 0.01440533, L139: "total_time": 0.022999626968, L140: "created": 1775679664.285658 L141: } L142: }` L143: Supported reasoning levels, defaults, and response formats vary by model. See 【32†Reasoning】 for model-specific tables, request examples, and conversation-history guidance. For image formats, limits, and model-specific support for images in user and tool messages, see 【33†Image Inputs】. L144: #### Authorizations L145: L146: Authorization L147: L148: string L149: L150: header L151: L152: required L153: L154: API key for authentication. Obtain your key from the Cerebras Cloud console and pass it as `Authorization: Bearer YOUR_API_KEY`. L155: #### Headers L156: L157: Content-Type L158: L159: enum L160: L161: default:application/json L162: L163: The media type of the request body. L164: L165: Available options: L166: L167: `application/json`, L168: L169: `application/vnd.msgpack` L170: L171: Content-Encoding L172: L173: enum L174: L175: The compression encoding applied to the request body. When set, the request body must be gzip-compressed. Can be combined with any supported `Content-Type`. L176: L177: Available options: L178: L179: `gzip` L180: L181: queue_threshold L182: L183: string L184: Controls the queue time threshold for requests using the `flex` or `auto` service tiers. Requests are preemptively rejected if the rolling average queue time exceeds this threshold. Valid range: `50`–`20000` (milliseconds). Private Preview. L185: #### Body L186: L187: application/json L188: L189: messages L190: L191: Chat message · object[] L192: L193: required L194: L195: A list of messages comprising the conversation so far. Depending on the model you use, different message types (modalities) are supported, like text and images. L196: L197: Hide child attributes L198: L199: messages.role L200: L201: enum L202: L203: required L204: L205: The role of the message author. L206: L207: Available options: L208: L209: `system`, L210: L211: `user`, L212: L213: `assistant`, L214: L215: `tool`, L216: L217: `developer` L218: L219: messages.content L220: L221: [Select] L222: The message content. This is required for system, user, developer, and tool messages. Assistant messages may omit content when they include tool calls. Images are supported in user messages and, on enabled `gemma-4-31b`, `kimi-k2.7-code`, `kimi-k3`, and `kimi-k3.1-pro` endpoints, tool messages. Keep the complete assistant response immediately before its tool results, with one result for each returned tool-call ID. L223: L224: messages.name L225: L226: string L227: L228: An optional name for the message author. L229: L230: messages.reasoning L231: string | null L232: L233: Reasoning from an assistant response supplied back in conversation history. Supported behavior varies by model. L234: L235: messages.tool_calls L236: L237: object[] L238: L239: Tool calls generated in an assistant message. L240: L241: Hide child attributes L242: L243: messages.tool_calls.id L244: L245: string L246: L247: required L248: L249: messages.tool_calls.type L250: L251: enum L252: L253: required L254: L255: Available options: L256: L257: `function` L258: L259: messages.tool_calls.function L260: L261: object L262: L263: required L264: L265: Hide child attributes L266: L267: messages.tool_calls.function.name L268: L269: string L270: L271: required L272: messages.tool_calls.function.arguments L273: L274: string L275: L276: required L277: L278: JSON-encoded arguments for the function call. L279: L280: messages.tool_call_id L281: L282: string L283: L284: The tool call this message responds to. Required when `role` is `tool`. L285: L286: model L287: L288: string L289: L290: required L291: L292: The model or endpoint ID to use for generating a response. For Shared Inference, use a model ID from the 【34†model catalog】. For 【35†Dedicated Inference】, use the endpoint ID assigned to your organization. L293: L294: frequency_penalty L295: L296: number | null L297: L298: default:0 L299: A number between -2.0 and 2.0. Positive values reduce the likelihood of the model repeating tokens by applying a penalty proportional to how frequently each token has already appeared in the generated output. L300: L301: Required range: `-2 <= x <= 2` L302: L303: logit_bias L304: L305: object | null L306: Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token. L307: Hide child attributes L308: L309: logit_bias.{key} L310: L311: number L312: L313: logprobs L314: L315: boolean L316: L317: default:false L318: L319: Whether to return log probabilities of the output tokens. L320: L321: max_completion_tokens L322: L323: integer | null L324: L325: The maximum number of tokens that can be generated in the completion, including reasoning tokens. The total length of input tokens and generated tokens is limited by the model's context length. L326: L327: max_tokens L328: L329: integer | null L330: L331: An alias for `max_completion_tokens`. Do not send both parameters in the same request. L332: min_completion_tokens L333: L334: integer | null L335: L336: The minimum number of tokens to generate. If omitted or set to 0, the model decides when to stop. Setting this value to -1 requests generation up to the maximum sequence length. L337: L338: min_tokens L339: L340: integer | null L341: L342: An alias for `min_completion_tokens`. Do not send both parameters in the same request. L343: L344: n L345: L346: enum | null L347: L348: default:1 L349: L350: The number of completion choices to generate. Cerebras currently supports only `1`. L351: L352: Available options: L353: L354: `1` L355: L356: parallel_tool_calls L357: boolean | null L358: L359: default:true L360: L361: Whether to enable parallel function calling during tool use. When enabled (default), the model can request multiple tool calls simultaneously in a single response. When disabled, only one tool call is made at a time. L362: L363: prediction L364: L365: object | null L366: Configuration for a 【36†Predicted Output】, which can greatly speed up response times when large parts of the model response are known in advance. Most common when regenerating a file with mostly minor changes. Available by request on dedicated endpoints. Predicted Outputs are not available for `qwen-3.8-27b` with Shared Inference or for `kimi-k2.7-code`. L367: L368: Hide child attributes L369: L370: prediction.type L371: L372: enum L373: L374: required L375: L376: The type of predicted content. Always `content`. L377: L378: Available options: L379: L380: `content` L381: prediction.content L382: L383: [Select] L384: L385: required L386: L387: The content that should be matched when generating a model response. L388: L389: presence_penalty L390: L391: number | null L392: L393: default:0 L394: L395: A number between -2.0 and 2.0. Positive values reduce the likelihood of the model repeating tokens that have already appeared in the output, encouraging the model to introduce new topics. L396: L397: Required range: `-2 <= x <= 2` L398: L399: prompt_cache_key L400: L401: string | null L402: An opaque identifier that groups related requests so they reuse the same 【37†prompt cache】. Requests sharing the same `prompt_cache_key` are routed together, which increases cache hits and reduces time to first token. L403: L404: Set it to a stable identifier such as a conversation, session, or workflow ID. Prompt caching works automatically when this field is omitted. This field is an optional routing hint and is available for all models. L405: L406: Maximum string length: `1024` L407: L408: reasoning_effort L409: L410: enum | null L411: Selects a model-specific reasoning level. Effort levels don't reserve or guarantee an exact reasoning-token budget. L412: L413: Supported values and defaults vary by model. L414: Model | Supported values | Default L415: --- | --- | --- L416: `qwen-3.8-27b` | `none`, `low`, `medium`, `high` | `high` L417: `kimi-k2.7-code` | Accepted for compatibility; ignored | Not applicable L418: `kimi-k3` | `low`, `high`, `max` | `max` L419: `kimi-k3.1-pro` | `low`, `medium`, `high`, `max` | `max` L420: `zai-glm-5.3-flash` | `low`, `high`, `max` | `max` L421: `gpt-oss-120b` | `low`, `medium`, `high` | `medium` L422: `gemma-4-31b` | `none`, `low`, `medium`, `high` | `none` L423: For `qwen-3.8-27b`, `high` selects the model's native `xhigh` mode. For `gemma-4-31b`, `low`, `medium`, and `high` enable reasoning equivalently. L424: L425: The value `none` disables reasoning for `qwen-3.8-27b` and `gemma-4-31b`. For `kimi-k2.7-code`, all accepted values, including `none`, are ignored and reasoning stays enabled. The other listed models don't support `none`. L426: L427: Reasoning tokens count toward `max_completion_tokens` and `usage.completion_tokens`. See 【32†Reasoning】 for model-specific examples. L428: Available options: L429: L430: `low`, L431: L432: `medium`, L433: L434: `high`, L435: L436: `max`, L437: L438: `none` L439: L440: reasoning_format L441: L442: enum | null L443: L444: Selects how the API returns reasoning. Support varies by model. L445: L446: Value | Behavior L447: --- | --- L448: `parsed` | Returns reasoning separately in `message.reasoning` or streaming `delta.reasoning` L449: `raw` | Includes reasoning in the final-answer content when supported L450: `hidden` | Omits reasoning text while still generating and counting reasoning tokens L451: `none` | Uses the model's default response format L452: Setting this field to `none` doesn't disable reasoning. Use `reasoning_effort` to disable reasoning on supported models. L453: * `qwen-3.8-27b` returns reasoning separately, including when `raw` is selected. `hidden` isn't supported. L454: * `kimi-k2.7-code` defaults to `parsed` and supports `raw`. `hidden` isn't supported. Do not combine `raw` with `json_object` or `json_schema` response formats. L455: * `kimi-k3` and `kimi-k3.1-pro` support `parsed` only. Do not send `raw` or `hidden`. L456: * `zai-glm-5.3-flash` and `gpt-oss-120b` default to `parsed` and support `raw` and `hidden`. L457: * `gemma-4-31b` doesn't support `raw` or `hidden`. L458: See 【38†Reasoning formats】 for the complete model comparison. L459: L460: Available options: L461: L462: `parsed`, L463: L464: `raw`, L465: L466: `hidden`, L467: L468: `none` L469: L470: clear_thinking L471: L472: boolean | null L473: L474: default:false L475: L476: Controls whether historical assistant reasoning is removed from the conversation before prompting the model. Assistant final-answer content is retained. L477: Model | `false` or omitted (default) | `true` L478: --- | --- | --- L479: `qwen-3.8-27b` | Preserves historical assistant reasoning | Removes historical assistant reasoning L480: `zai-glm-5.3-flash` | Preserves historical assistant reasoning | Removes assistant reasoning before the latest user message L481: L482: Not supported for `gemma-4-31b`, `kimi-k2.7-code`, `kimi-k3`, or `kimi-k3.1-pro`. Do not send this field for those models. L483: L484: See 【39†Multi-turn reasoning】 for an example that resends assistant reasoning. L485: response_format L486: L487: Text · object L488: L489: An object that controls the format of the model response. L490: L491: Setting `{ "type": "json_schema", ... }` enables Structured Outputs, which enforces schema compliance. See 【40†Structured Outputs】 for details. L492: Setting `{ "type": "json_object" }` enables legacy JSON mode, which ensures the model returns valid JSON but does not enforce a specific schema. To use `json_object`, include a system or user message specifying the desired format. `json_object` is not compatible with streaming, so `stream` must be set to `false`. L493: L494: * Text L495: L496: * JSON Schema L497: L498: * JSON Object L499: L500: Hide child attributes L501: L502: response_format.type L503: L504: enum L505: L506: required L507: L508: The type of response format being defined. Always `text`. L509: L510: Available options: L511: `text` L512: L513: seed L514: L515: integer | null L516: L517: If specified, the system will make a best effort to sample deterministically so repeated requests with the same `seed` and parameters return the same result. Determinism is not guaranteed. L518: L519: service_tier L520: L521: enum | null L522: L523: default:default L524: Controls request prioritization. `default` uses the request's standard priority and quota. `priority` attempts the configured higher priority and can retry at the standard priority. `auto` starts with standard priority and quota, then can fall back to `flex`. `flex` uses the lowest configured priority and separate flex rate limits. L525: L526: Service tiers are not available with Shared Inference on the Developer Tier. L527: Note: This feature is in Private Preview. For access or more information, 【41†contact us†www.cerebras.ai】 or reach out to your account representative. L528: L529: Available options: L530: L531: * `priority` - Highest priority processing. Requires an endpoint with service-tier support enabled. L532: * `default` - Standard priority processing L533: * `auto` - Automatically uses the highest available service tier L534: * `flex` - Lowest priority processing L535: L536: See 【42†Service Tiers】 for more information. L537: L538: Available options: L539: L540: `priority`, L541: `default`, L542: L543: `auto`, L544: L545: `flex` L546: L547: stop L548: L549: [Select] L550: L551: Up to four sequences that cause the API to stop generating further tokens. The returned text does not contain the stop sequence. L552: L553: stream L554: L555: boolean | null L556: L557: Streams partial message deltas when set to `true`. L558: L559: temperature L560: L561: number | null L562: L563: Sampling temperature between 0 and 2. Higher values, such as 0.8, make output more random; lower values, such as 0.2, make it more focused and deterministic. We recommend changing either this value or `top_p`, but not both. L564: Required range: `0 <= x <= 2` L565: L566: tool_choice L567: L568: [Select] L569: L570: Controls which (if any) tool is called by the model. L571: L572: * `none`: No tool is called. L573: * `auto`: The model chooses whether to call a tool. This is the default when tools are present. L574: * `required`: Forces a tool call. L575: L576: A specific tool can be forced by passing an object specifying the function name. L577: L578: Available options: L579: L580: `none`, L581: L582: `auto`, L583: L584: `required` L585: L586: tools L587: L588: object[] | null L589: A list of tools the model may call. Use this to provide a list of functions the model may generate JSON inputs for. L590: L591: Currently, only functions are supported. Specifying tools consumes prompt tokens; too many may degrade performance or hit context length limits. L592: L593: Hide child attributes L594: L595: tools.type L596: L597: enum L598: L599: required L600: L601: The type of the tool. Currently only `function` is supported. L602: L603: Available options: L604: L605: `function` L606: L607: tools.function L608: L609: object L610: L611: required L612: L613: Hide child attributes L614: L615: tools.function.name L616: L617: string L618: required L619: L620: The name of the function to call. Supported characters: `a-z`, `A-Z`, `0-9`, `_`, `-`. Maximum length: 64. L621: L622: tools.function.description L623: L624: string L625: L626: A description of what the function does, used by the model to choose when and how to call the function. L627: L628: tools.function.parameters L629: L630: object L631: L632: The parameters the function accepts, described as a JSON Schema object. Omitting parameters defines a function with an empty parameter list. L633: L634: tools.function.strict L635: L636: boolean L637: L638: default:false L639: When true, constrained decoding guarantees that tool arguments conform to the supported subset of the JSON Schema in `parameters`. See 【43†Tool Calling】 and 【40†Structured Outputs】 for requirements and model-specific limitations. L640: L641: top_logprobs L642: L643: integer | null L644: L645: An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. If using this parameter, `logprobs` must also be set to `true`. L646: L647: Required range: `0 <= x <= 20` L648: L649: top_p L650: number | null L651: L652: Nucleus sampling parameter between 0 and 1. The model considers only tokens comprising the top `top_p` probability mass. For example, 0.1 means only tokens comprising the top 10% probability mass are considered. We recommend changing either this value or `temperature`, but not both. L653: L654: Required range: `0 <= x <= 1` L655: L656: user L657: L658: string | null L659: L660: A unique identifier representing your end-user, which can help monitor and detect abuse. L661: #### Response L662: L663: 200 - application/json L664: L665: Successful response L666: L667: id L668: L669: string L670: L671: A unique identifier for the chat completion. L672: L673: choices L674: L675: object[] L676: L677: A list containing the chat completion choice. Cerebras currently supports `n: 1` only. L678: L679: Hide child attributes L680: L681: choices.finish_reason L682: L683: enum L684: L685: The reason the model stopped generating tokens. L686: L687: Available options: L688: L689: `stop`, L690: L691: `length`, L692: L693: `content_filter`, L694: L695: `tool_calls` L696: L697: choices.index L698: L699: integer L700: L701: The index of the choice in the list of choices. L702: L703: choices.logprobs L704: object | null L705: L706: Log probability information for the output tokens. L707: L708: choices.reasoning_logprobs L709: L710: object | null L711: