prefix completion - HorayAI

1. Use Cases

In prefix-based completion, users provide desired prefix information, enabling the model to complete the remaining content based on the provided prefix. With this capability, the model demonstrates improved instruction-following ability, meeting user requirements for specific formats in certain scenarios.

2. How to Use

How to Use

extra_body={"prefix": "prefix content"}

3. Supported Models

Currently, large language models support the above parameter. Note: The list of supported models may change. Please refer to this documentation for the latest supported models.

4. Example Usage

Below is an example of using prefix-based completion with the OpenAI library:

client = OpenAI(
    api_key="Your API KEY", # get from https://dash.horay.ai/account/ak
    base_url="https://api.horay.ai/v1"
)
 
messages = [
    {"role": "user", "content": "Please write quick sort code"},
]

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V2.5",
    messages=messages,
    extra_body={"prefix":"```python\n"}
)

print(response.choices[0].message.content)