HomeGuidesRecipesAPI EndpointsRelease NotesCommunity
Log In

Submitting a bulk question job

🚧

Have Your Dataset Prepared

Before asking questions to a bulk collection of text documents, make sure you have your set of documents prepared for analysis. See creating datasets and importing data for more details.

Part I. Set Up

Import Dependencies

To start, let's upload our dependencies.

# add dependencies
import layar_api
from layar_api.rest import ApiException
import requests
from pprint import pprint

Configure Authentication

Next, we'll want to configure our session with our authentication keys. Copy the following commands and only swap out the strings for base_host, client_id, and client_secret. The base_host is the Layar instance you're working within (e.g. 'demo.vyasa.com'), and the client ID and secret are your provided authentication keys.

To learn how to get your authentication keys, please reference this document.

# set up your authentication credentials
base_host = 'BASE_URL' # your Layar instance (e.g. 'demo.vyasa.com')
client_id = 'AbcDEfghI3' # example developer API key
client_secret = '1ab23c4De6fGh7Ijkl8mNoPq9' #example developer API secret

# configure oauth access token for authorization
configuration = layar_api.Configuration()
configuration.host = f"https://{base_host}"
configuration.access_token = configuration.fetch_access_token(
    client_id, client_secret)

# Make your life easier for the next task: instantiating APIs!
client = layar_api.ApiClient(configuration)

Instantiate Your APIs

Finally, we'll want to instantiate the APIs we are going to call in future commands. We'll be using the SourceDocumentApi and SavedListApi.

# Instantiate APIs
sourceDocApi = layar_api.SourceDocumentApi(client)
questionApi = layar_api.QuestionApi(client)
setsApi = layar_api.SavedListApi(client)

Part 2. Create the Bulk Job Request

Identify Your Document Batch

Similar to how you would ask a new question in Curate, you're going to want to identify what documents you'll be asking this question in. You can find the documents based on a number of parameters in the SourceDocumentSearchCommand. Here, we've used the saved_list_ids parameter (which is the same thing as the Layar Set ID if you've created a Layar Set with all of your documents).

## Search Documents & Create a Document Batch for Querying
setId = ['AYBrf_ZWKBnV9heqHzEu'] # Layar ID for the Set of Documents to Be Queried
documentBatch = layar_api.SourceDocumentSearchCommand(saved_list_ids=setId)

📘

Pro Tip

If you're interested in section aware question answering, such as asking a question of a specific set of PDF sections like methods or inclusion criteria, you can use the sourceDocument parameter section_searches to specify which sections you're interested in asking QA of! For more details, check out this tutorial here.

Input "Ask a Question" Parameters

Just like you would see in the "Ask a Question" modal in Curate:

  1. Provide a question key (questionKey)
  2. Add a natural language question string (and any variations of that string) as the questionStrings parameter.
  3. Finally, if you anticipate the answers will likely fall into a concept type, provide those as a list in the conceptTypes parameter.
# Provide the Question Parameters
questionKey = ['Study Drug'] #The question key you would like to name this job as
questionStrings = ['What is the drug or therapeutic being evaluated?'] # If there are multiple variations of the question you wish to try under the same bulk job
conceptTypes = ["CHEMICAL", "CHEBI"] # Add any concept types by their ID if you expect an answer to fall into a certain concept type

Build the Request Body

The body needed when calling the start_batch endpoint is the BulkQuestionCommand, which includes three optional parameters: the question you're asking (bulk_questions), the documents you're asking the question of (source_document_search_command), and the batch key to place the question key under (question_grouping_key).

The bulk_questions parameter is actually a BulkQuestion object input, whose parameters look very similar to the Curate "Ask a Question" modal. Those question parameters you defined above are placed into a single list (q).

# Build the BulkQuestion Body
q = layar_api.BulkQuestion(question_key = questionKey, question_string_variations = questionStrings, concept_types = conceptTypes)
job = layar_api.BulkQuestionCommand(bulk_questions = [q], source_document_search_command = documentBatch, question_grouping_key='API Demo Batch') # BulkQuestionCommand |  (optional)

Submit Your BulkQuestion Job

Now that you have your request body prepped (job), it's time to submit your API request! Make a call to the questionApi class using the start_batch endpoint (see SDK documentation).

# Submit BulkQuestion Job
try:
    # submit a request to ask a batch of questions
    api_response = questionApi.start_batch(body=job)
    pprint(api_response)
    print("Success!") 
except ApiException as e:
    print("Exception when calling QuestionApi->start_batch: %s\n" % e)

Console Response

You should see the following as a console response. The job_id is a submitted question batch job, whose status can be viewed by using the QuestionBatchSearch command and using the job_id as a search parameter.

{'job_id': '410aa856-c819-4061-bf59-6dd68a8b6cfe'}
Success!

You can also see this new job directly within Curate. Head over to your instance's Curate URL (here it's demo.vyasa.com/curate), click into the batch (question_grouping_key), and you'll see your new question (questionKey)!

3104

Before submitting the job, within the batch "Demo Batch".

3104

Before submitting the job, within the batch "Demo Batch", you can now see the question key for "Study Drug".

Part 3. Check Question Batch Job Status

If you're interested in checking in on your QA job while it's running, you can search for the bulk question and get its job status details. Using the BulkQuestionSearchCommand, you can get details on:

  • A specific batch (batch_grouping_key).
  • Any specific question keys within that batch (question_keys).
  • Specific Layar sets within a batch (saved_list_ids). This is helpful if you've submitted a QA job that queries two or more Layar Sets and you want to compare results between those two or more sets.
  • A specific QA run (based on the job ID).
# create an instance of the api class
body = layar_api.BulkQuestionSearchCommand(
    batch_grouping_key = 'Clinical Trial API Demo' # Optional. The batch name (only one batch at a time)
    question_keys = ['Disease', 'Disease by Section'] # Optional. A list of specific question keys within the batch (e.g. 'Disease).
    saved_list_ids = ['AYBrf_ZWKBnV9heqHzEu'] # Optional. Any specific Layar Sets used in the bulk QA job. Helpful if you had multiple saved_lists in your QA job. 
    job_id = 'd6f1c63b-d190-46c1-86e2-dc6533130bd2', # Optional. If you're interested in a specific QA run, you can pull up the details for that Job ID
    )

try:
    # Search for question batches
    api_response = questionApi.search_question_batch(body=body)
    pprint(api_response)
except ApiException as e:
    print("Exception when calling QuestionApi->search_question_batch: %s\n" % e)

You can expect a request to give you a list of batches that meet your search criteria for the search_question_batch method.

Below is an example batch question detail:

{'batch_grouping_key': 'Clinical Trial API Demo',
 'created_by_user': 25000,
 'date_indexed': datetime.datetime(2022, 6, 29, 19, 30, 45, 402000, tzinfo=tzutc()),
 'date_published': datetime.datetime(2022, 6, 29, 19, 30, 45, 402000, tzinfo=tzutc()),
 'date_updated': datetime.datetime(2022, 6, 29, 19, 34, 45, 314000, tzinfo=tzutc()),
 'id': None,
 'job_id': 'd6f1c63b-d190-46c1-86e2-dc6533130bd2',
 'name': None,
 'question_count': None,
 'question_keys': 'Disease',
 'questions_answered': None,
 'questions_completed': None,
 'questions_failed': None,
 'questions_queued': None,
 'questions_skipped': None,
 'saved_list_ids': 'AYBrf_ZWKBnV9heqHzEu'},

Up Next

Now that you've submitted a bulk question job, learn how you can retrieve the answers (model and human curated) for each document within a given batch!