Utilizing External Provider Models

Introduction

In Layar 1.17 the ability to utilize external providers like Azure, OpenAi, and Anthropic was added. This guide will go over to how to configure these if you do not have the required hardware to serve the model yourself.

ALL_EXTERNAL_MODEL_INFO

This is a layar.config parameter that must be provided if you want to use an external provider. The paramter is a JSON dictionary that looks as follows.

{
  model_name: the API name of the model in the external provider e.g. "certaraai-dev-oai-deployment-gpt4o",
  model_provider: the type of external provider ("azure","openai", or "anthropic"),
  model_base: the common name for the model e.g. "gpt-4o",
  total_max_len: total context length for the model,
  max_generation_tokens: maximum length of generated tokens for the model,
  disallowed_generation_args: usually this is just [] i.e. nothing but some models require specific generation arguments to be filtered out from the general model call proces
}

OpenAI

To use OpenAI you must also provide you OpenAI API key in the OPENAI_KEY parameter in the layar.config file.

OPENAI_KEY: YOUR_API_KEY

ALL_EXTERNAL_MODEL_INFO: {
  "model_name":"gpt-4o",
  "model_provider":"openai",
  "model_base":"gpt-4o"
}

The possible model_name can be as follows.

{"model_name":"gpt-4.1",
 "model_provider":"openai",
 "model_base":"gpt-4.1",
 "total_max_len":1047576,
 "max_generation_tokens":32768,
 "disallowed_generation_args":[]},
{"model_name":"gpt-4o",
 "model_provider":"openai",
 "model_base":"gpt-4o",
 "total_max_len":128000,
 "max_generation_tokens":16384,
 "disallowed_generation_args":[]},
{"model_name":"gpt-5",
 "model_provider":"openai",
 "model_base":"gpt-5",
 "total_max_len":400000,
 "max_generation_tokens":128000,
 disallowed_generation_args: ["logprobs","temperature","top_p","frequency_penalty"]},
{"model_name":"gpt-5-nano",
 "model_provider":"openai",
 "model_base":"gpt-5",
 "total_max_len":400000,
 "max_generation_tokens":128000,
 disallowed_generation_args: ["logprobs","temperature","top_p","frequency_penalty"]},
{"model_name":"gpt-5-mini",
 "model_provider":"openai",
 "model_base":"gpt-5",
 "total_max_len":400000,
 "max_generation_tokens":128000,
 disallowed_generation_args: ["logprobs","temperature","top_p","frequency_penalty"]},
{"model_name":"o3",
 "model_provider":"openai",
 "model_base":"o3",
 "total_max_len":200000,
 "max_generation_tokens":100000,
 disallowed_generation_args: ["logprobs","temperature","top_p","frequency_penalty"]},
{"model_name":"o4-mini",
 "model_provider":"openai",
 "model_base":"o4-mini",
 "total_max_len":200000,
 "max_generation_tokens":100000,
 disallowed_generation_args: ["logprobs","temperature","top_p","frequency_penalty"]}

Azure

The Azure setup is very similar; it requires an API key to be given in the AZURE_KEYparameter. You must also provide the AZURE_ENDPOINTparameter, which will be the URL to the Azure environment. You must also provide ALL_EXTERNAL_MODEL_INFO.

AZURE_ENDPOINT: the endpoint for the clients azure e.g. https://certaraai-dev-oai.openai.azure.com/
AZURE_KEY: YOUR_API_KEY

ALL_EXTERNAL_MODEL_INFO: {
  "model_name":"certaraai-dev-oai-deployment-gpt4o",
  "model_provider":"azure",
  "model_base":"gpt-4o"
}

model_name and model_base will be different in the Azure environment. The possible model_basevalues are the same as the OpenAI values.

Anthropic

Anthropic requires an ANTHROPIC_KEYand ALL_EXTERNAL_MODEL_INFO.

ANTHROPIC_KEY: YOUR_API_KEY

ALL_EXTERNAL_MODEL_INFO: {
  "model_name":"claude-3-haiku-20240307",
  "model_provider":"anthropic",
  "model_base":"claude"
}

The following models can be used.

{ "model_name": "claude-3-7-sonnet-20250219",
  "model_provider": "anthropic",
  "model_base": "claude",
  "total_max_len": 200000,
  "max_generation_tokens": 64000,
  "disallowed_generation_args": [ ] },
{ "model_name": "claude-sonnet-4-20250514",
  "model_provider": "anthropic",
  "model_base": "claude",
  "total_max_len": 200000,
  "max_generation_tokens": 64000,
  "disallowed_generation_args": [ ] },
{ "model_name": "claude-3-5-haiku-20241022",
  "model_provider": "anthropic",
  "model_base": "claude",
  "total_max_len": 200000,
  "max_generation_tokens": 8192,
  "disallowed_generation_args": [ ] },
{ "model_name": "claude-opus-4-20250514",
  "model_provider": "anthropic",
  "model_base": "claude",
  "total_max_len": 200000,
  "max_generation_tokens": 32000,
  "disallowed_generation_args": [ ] },
{ "model_name": "claude-opus-4-1-20250805",
  "model_provider": "anthropic",
  "model_base": "claude",
  "total_max_len": 200000,
  "max_generation_tokens": 32000,
  "disallowed_generation_args": [ "temperature" ] }

vLLM OAI Endpoint

If you are on version 3.2 or higher, you can utilize a vLLM OAI endpoint as a model provider.

🛑
vLLM Compatability
On version 3.2 of Layar, vLLM 0.20 was used to test this external functionality. If the external vLLM is running on a version higher then this, there may be issues communicating between Layar and the external vLLM provider.

The ALL_EXTERNAL_MODEL_INFO would look like this, if you are utilizing a vLLM OAI Endpoint.

{
"model_name":"gpt-oss-120b",
"model_provider":"vllm",
"model_alias":"gpt-oss-external",
"total_max_len":"128000",
"model_server_url": "https://LLM_OAI_ENDPOINT/oai",
"model_server_auth":"regular api bearer token",
"supports_reasoning": true
}

model_name

The full name of the model. Unlike internal configuration, you do not need to provide the producer of the model, IE OpenAI / Meta / Nvidia

model_provider

A string that will always be vllm .

model_alias

A string denoting the name of the model users will see.

total_max_len

A string containing the maximum number of tokens that can be submitted to the models.

model_server_url

A string containing the URL to the OAI endpoint that serves the model.

model_server_auth

A string containing the auth token needed to access the endpoint.

supports_reasoning

A boolean value denoting if the model supports reasoning or not. This can be found on the models huggingface page.