Muhammad
05/17/2024, 10:14 AMAI
05/17/2024, 7:51 AMagihealth
05/17/2024, 5:08 AMConnor Davis | Outbox Solutions
05/17/2024, 4:28 AMJohn_Rodriguez_0906
05/17/2024, 4:10 AMA.Salem
05/17/2024, 1:09 AMMaster of None
05/17/2024, 12:51 AMSangy
05/17/2024, 12:00 AMEthan Tan
05/16/2024, 10:21 PMTerry Ruch
05/16/2024, 10:09 PMga media
05/16/2024, 8:19 PMjosh
05/16/2024, 7:03 PMjosh
05/16/2024, 6:44 PMjosh
05/16/2024, 6:33 PMA.Salem
05/16/2024, 5:55 PMTinjira
05/16/2024, 2:44 PMCelemiba
05/16/2024, 2:42 PMWDGrimm
05/16/2024, 2:20 PMSemantic
05/16/2024, 1:27 PMJM
05/16/2024, 10:59 AMJuzTim
05/16/2024, 10:21 AMJotham
05/16/2024, 9:30 AMStakh
05/16/2024, 9:25 AMimport time
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route('/chat/completions', methods=['POST'])
def basic_custom_llm_route():
request_data = request.get_json()
response = {
"id": "chatcmpl-8mcLf78g0quztp4BMtwd3hEj58Uof",
"object": "chat.completion",
"created": int(time.time()),
"model": "gpt-3.5-turbo-0613",
"system_fingerprint": None,
"choices": [
{
"index": 0,
"delta": {"content": "This is some test content"},
"logprobs": None,
"finish_reason": "stop"
}
]
}
return jsonify(response), 201
if __name__ == "__main__":
app.run(debug=True, port=5000)
Accessible with ngrok (https://cead-37-168-11-222.ngrok-free.app)
Testing the endpoint with with Postman works (see image attached), I'm getting the expected response:
{
"choices": [
{
"delta": {
"content": "This is some test content"
},
"finish_reason": "stop",
"index": 0,
"logprobs": null
}
],
"created": 1715851284,
"id": "chatcmpl-8mcLf78g0quztp4BMtwd3hEj58Uof",
"model": "gpt-3.5-turbo-0613",
"object": "chat.completion",
"system_fingerprint": null
}
But when experimenting inside Vapi, I get no answer from the agent.
Am I doing something wrong?
Also why is the message's content stored in completion.choices[0].delta.content
in your examples, contrary to OpenAI's API endpoint which stores it in completion.choices[0].message.content
.
Thank you for your answer.
https://cdn.discordapp.com/attachments/1240596039006687232/1240596039396888586/image.png?ex=66472298&is=6645d118&hm=e30750e603436253a1f622212a7d67bb1b41d12255dbd759cbe64f0b0620153c&Caith
05/16/2024, 9:06 AMsumitguha
05/16/2024, 7:48 AMBAARUD
05/16/2024, 7:09 AMNushaine
05/16/2024, 6:55 AMLucas
05/16/2024, 6:02 AMMartinL
05/16/2024, 2:36 AMLouis
05/16/2024, 2:13 AM