Utilizing the OpenAI Formatted Endpoint
Introduction
In version 1.13 of Layar, OpenAI formatted endpoint was introduced to allow developers the ability to use generic OpenAI endpoints and payload formatting. This allows you to easily reuse code that works with OAI endpoints, inside of your Certara instance.
OAI Endpoint
The main endpoint is /layar/oai/v1/. Certara utilizes the V1 endpoints in OAI. Ensure you are utilizing V1 endpoint formats when utilizing this endpoint: V1 Documentation
There are some endpoints you will not be able to utilize. Specifically the Audio and Image endpoints. The primary endpoints you will want to use are as follows.
LLM Compatability
If you are using an older version of a model, IE Mistral or Mixtral. The OAI endpoint will not work with these models because they do not conform OAI completion formats. Please reach out to AI support at [email protected] if you have questions about upgrading your model.
Utilizing Tools
The OpenAI endpoints allow you to utilize binding tools, which the model can use when generating responses. This allows you to make an agentic chat bot that can utilize tools to do a wider range of operations. In order to utilize tools, you must be utilizing a Llama 3.3 model or higher. In addition, these model arguments must be added to the layar.config file.
--enable-auto-tool-choice --tool-call-parser llama3_json
LangChain/LangGraph
The OAI endpoint allows developers to utilize open source libraries like Langchain and Langgraph to interact with the served models. Refer to this recipe for steps on how to use these tools: OAI Endpoint In LangChain
Updated 8 days ago