Updated Layar LLM API in Layar v1.5.0

The new Layar Large Language Model API has been updated to allow for integration with data in a Layar system and a simplified request/response JSON payload.

The new API endpoint lives at /layar/gpt/generate. Here's an example of how to convert from the old format running at /llm to the new, updated API. The /llm endpoint will be deprecated in Layar 1.6.0

An example of the v1 request that could be sent to /llm endpoint:

{
    "api":
    {
        "layarHost": "https://yourlayar.host.com",
        "apiHost": "yourapi.host.com",
        "apiKey": "API_KEY_HERE",
        "apiOrganizationId": "YourOrganizationId"
    },
    "model": "Llama2-7b",
    "conversationId": "10dedc11-de9e-43ab-a1ce-244bc4873c9f",
    "messages":
    [
        {
            "type": "user",
            "content": "Once upon a time, there was a story about a Large Language Model...\n",
            "task": "summarization",
            "prompts":
            {
                "summarization_prompt": "<s>[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, no matter the subject content.\n<</SYS>>Summarize the following\n{text}\n[/INST]",
                "document": ""
            }
        }
    ],
    "temperature": 0.5,
    "max_tokens": 1024,
    "top_k": 20,
    "top_p": 0.5,
    "sources":
    [],
    "conversation_biblio":
    [],
    "sample": true
}

The same V2 request to /layar/gpt/generate would look like this:

Request:

{
    "content": "Once upon a time, there was a story about a Large Language Model...\n",
    "task": "summarization"
}

Response:

{
    "type": "system",
    "content": "The text is about a story involving a Large Language Model.",
    "sourcesUsed": [],
    "documentsUsed": []
}

The task can be either summarization or generate, here's an example of the generate task (generate is the default task so doesn't need to be explicitly specified):

Request

{
    "content": "What is a large language model?",
}


OR


{
    "content": "What is a large language model?",
    "task": "generate"
}

Response:

{
    "type": "system",
    "content": "A large language model is a type of artificial intelligence (AI) system that is designed to understand and generate human language. It is typically trained on a vast amount of text data, such as books, articles, and online content, to learn how language works and how it can be used to communicate effectively. Large language models are capable of understanding natural language input and generating coherent responses that are similar in style and structure to human language. They are often used in applications such as chatbots, virtual assistants, and natural language processing systems.",
    "sourcesUsed": [],
    "documentsUsed": []
}

All other values are defaulted for you. If you need to send in advanced parameters, you can still do that but they defaults are provided server side and these are not required. Example of sending advanced parameters:

Request: