Prompting a Model
Introduction
The layar/gpt/generate/
endpoint is used to prompt the model. This guide will go over the basics to prompting via the API.
Pre-Reqs
Before a document search can be done the API requests must be authenticated. Make sure you have already followed the instructions for importing dependencies and authentication from the Getting Started Guide.
Check Your Imported Modules
Make sure you have imported the
requests
andjson
module before proceeding with this guide.
The following header can be used in your request.
header = {'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': f"Bearer {token}",
'X-Vyasa-Client': 'layar',
'X-Vyasa-Data-Providers' : 'sandbox.certara.ai',
'X-Vyasa-Data-Fabric' : 'YOUR_FABRIC_ID'
}
Request Body
The body of the request needs a minimal amount of parameters to get a valid response.
body = {'content' : 'What disease is JAK2 protein associated with?',
'task' : 'generate',
'sources'[{'documentID' : '1234567ABCDEF'
}]
}
Generate Endpoint Parameters
There are a large variety of parameters that can help control the quality of the prompt as well as the throughput. Please review Generate Parameters for more info.
Utilizing a Specific Model
If you are on Layar 1.10, the body of the request can utilize the model
parameter. This allows you to dictate what model to use when generating a response.
Pre-Reqs
Before you can utilize this parameter please review the guide Assigning Models to GPUs.
Request Body
The body of the request utilizing a specific model would look as follows.
body = {'content' : 'What disease is JAK2 protein associated with?',
'task' : 'generate',
'model' :'mixtral-instruct-awq',
'sources' :[{'documentID' : '1234567ABCDEF'}]
}
Correct Model Name
In order to use the
model
parameter correctly, you need to ensure you are using the correct model name. For example, if the model name being used in the layar.config file iscasperhansen/mixtral-instruct-awq
the name you would use in the request ismixtral-instruct-awq
.
POST Request
Now that the body is constructed a POST can be made to the endpoint.
generateURL = f'{envurl}/layar/gpt/generate'
respose = requests.post(generateURL,
headers = header,
json = body)
#Optional
print(json.dumps(response, indent=4)
Generate Endpoint Response
For more information on the response, please review Generate Response for more info
Updated about 1 month ago