​
1. Use Cases

FIM (Fill In the Middle) completion allows users to provide both the preceding and following content, enabling the model to fill in the middle content. This is typically used in scenarios such as code completion and filling in missing text within a document.

​
2. How to Use

​
2.1 Using in the chat/completions API

{ 
    "model": "model info",
    "messages": "prompt message",
    "params": "params",
    "extra_body": {"prefix":"prefix content", "suffix":"suffix content"}
}

​
2.2 Using in the completions API

{
    "model": "model info",
    "prompt": "prefix content",
    "suffix": "suffix content"
}

​
3. Supported Models

  • Deepseek Series:

    • deepseek-ai/DeepSeek-V2.5

  • Qwen Series:

    • Qwen/Qwen2.5-Coder-7B-Instruct
    • Qwen/Qwen2.5-Coder-32B-Instruct
Note: The list of supported models may change. Please refer to this documentation for the latest supported models.

​
4. Example Usage

​
4.1 Using FIM completion with the OpenAI chat.completions API:

client = OpenAI(
    api_key="Your API KEY", # get from https://dash.horay.ai/account/ak
    base_url="https://api.horay.ai/v1"
)
 
messages = [
    {"role": "user", "content": "Please write quick sort code"},
]

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V2.5",
    messages=messages,
    extra_body={
            "prefix": f"""
def quick_sort(arr):
    # If the array length is less than or equal to 1, return the array.
    if len(arr) <= 1:
        return arr
    else:
""",
            "suffix": f"""
# test quick_sort function
arr = [3, 6, 8, 10, 1, 2, 1]
sorted_arr = quick_sort(arr)
print("Sorted array:", sorted_arr)
"""
    },
    stream=True,
    max_tokens=4096
)

for chunk in response:
    print(chunk.choices[0].delta.content, end='')

​
4.2 Using FIM completion with the OpenAI completions API:

client = OpenAI(
    api_key="Your API KEY", # get from https://dash.horay.ai/account/ak
    base_url="https://api.horay.ai/v1"
)

response = client.completions.create(
    model="deepseek-ai/DeepSeek-V2.5",
    prompt=f"""
def quick_sort(arr):
    # If the array length is less than or equal to 1, return the array.
    if len(arr) <= 1:
        return arr
    else:
""",
    suffix=f"""
# 测试 quick_sort 函数
arr = [3, 6, 8, 10, 1, 2, 1]
sorted_arr = quick_sort(arr)
print("Sorted array:", sorted_arr)
""",
    stream=True,
    max_tokens=4096
)

for chunk in response:
    print(chunk.choices[0].text, end='')