HomeGuidesRecipesAPI EndpointsRelease NotesCommunity
Log In

Curation Reporting

πŸ“˜

Curations Required

When you run the following tutorial, it will only return a response if there are actually curations within your set. Go into the test set you're working with and make a few curations.

PS: Add variety! Try selecting some model predictions, select from the text, add some "additional answers" (aka custom curations), and a few "No Answer Applicable". You can also assign yourself as a Moderator and/or Curator to try out those inputs in the response as well.

High Level Curation Overview

Get a high level overview of the curation being done on a set, you can run /layar/question/search. Using this call, you will learn the following:

  • Who has been assigned to that document/question (and what their role is)
  • Whether or not there is 'discord' between the curators
  • What percent the curation has been completed
  • Whether a document/question has been moderated
  • And much more!
response = requests.post(f"{envUrl}/layar/question/search", # note the set ID
    data = json.dumps({
    "batchGroupingKeys": ["AYYJ-UFin02FJXK6Tk3W"],
    "hasUserCurations": ["TRUE"], # Set to TRUE to include both custom answers (Select from Text, Additional Answers, No Answer Applicable) *as well as* when curators have accepted a model prediction
    "rows": 100, # Number of results to return (max 5K before requiring pagination with 'start' param)
    "start": 0}), 
    headers = {
        'accept':'application/json',
        'content-Type':'application/json',
        'authorization':f"Bearer {token}",
        'X-Vyasa-Client': 'curate'
    }
)

  
temp = response.json()
df = pd.DataFrame.from_dict(temp)
df = df.drop(['queryString', 'minimumRequestedAnswers','pageIterationCount', 'createdByUser', 'searchCommandJsonString', 'truncateTextToLength'], axis=1)
df
// this is a single example pulled from the response
[{'queryString': 'Where is the osteonecrosis located on the body?',
  'jobId': 'b1a6e243-d633-4504-b258-b5f790f244b9',
  'batchGroupingKey': 'AYYJ-UFin02FJXK6Tk3W',
  'questionKey': 'Where is the osteonecrosis located on the body?',
  'singleDocQuestionDocumentId': 'pubmed_36464628',
  'typeOfSearch': 'DOCUMENT_AS_PARAGRAPHS',
  'dataProviders': 'master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com',
  'truncateTextToLength': 20000,
  'searchCommandJsonString': '{"documentIds":["pubmed_36464628"],"highlight":false,"highlightPreTag":"<em>","highlightPostTag":"</em>","start":0,"sort":"dateIndexed","sortOrder":"desc"}',
  'minimumRequestedAnswers': 5,
  'pageIterationCount': 0,
  'advancedParams': {'chunk_size': 5, 'null_threshold': 0},
  'complete': True,
  'assignedUserIds': [],
  'moderatorUserIds': [],
  'assignedUserCount': 0,
  'curationDiscord': False,
  'moderated': False,
  'userCurations': [{'createdByUser': 25036,
    'dateIndexed': '2023-01-31T22:54:18.161+0000',
    'curations': [{'curationType': 'CUSTOM',
      'customString': 'test custom curation'}]}],
  'curatedAnswerUserCount': 1,
  'curatedAnswerTotalCount': 1,
  'curatedAnswerUniqueCount': 1,
  'curationPercentComplete': 0.0,
  'deepLearningModelId': 'AX9184SyOBLJbFhFZLqp',
  'bestAnswers': ['test custom curation'],
  'hasAnswers': True,
  'id': 'AYYJ_Fxqn02FJXK6Tk6K',
  'dateIndexed': '2023-01-31T22:39:39.113+0000',
  'datePublished': '2023-01-31T22:39:39.113+0000',
  'dateUpdated': '2023-01-31T22:54:18.075+0000',
  'createdByUser': 25036}]

Example dataframe:

|    | jobId                                | batchGroupingKey     | questionKey                                      | singleDocQuestionDocumentId   | typeOfSearch           | dataProviders                                                                   | sectionKeywords   | complete   | assignedUserIds   | moderatorUserIds   |   assignedUserCount | curationDiscord   | moderated   | userCurations                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |   curatedAnswerUserCount |   curatedAnswerTotalCount |   curatedAnswerUniqueCount |   curationPercentComplete | bestAnswers                                                                                    | hasAnswers   | id                   | dateIndexed                  | datePublished                | dateUpdated                  | advancedParams                         | deepLearningModelId   |
|---:|:-------------------------------------|:---------------------|:-------------------------------------------------|:------------------------------|:-----------------------|:--------------------------------------------------------------------------------|:------------------|:-----------|:------------------|:-------------------|--------------------:|:------------------|:------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------:|--------------------------:|---------------------------:|--------------------------:|:-----------------------------------------------------------------------------------------------|:-------------|:---------------------|:-----------------------------|:-----------------------------|:-----------------------------|:---------------------------------------|:----------------------|
|  0 | 3ce7e538-69e2-4eba-a88a-d7d3aa5fb09d | AYYJ-UFin02FJXK6Tk3W | Heme - Binary Text Classification                | pubmed_36259635               | PARAGRAPH              | master-pubmed.vyasa.com                                                         | []                | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:55.627+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_Neyn02FJXK6Tk-S_heme'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |                        1 |                         1 |                          1 |                         0 | ['heme']                                                                                       | True         | AYYJ_Neyn02FJXK6Tk-S | 2023-01-31T22:40:10.657+0000 | 2023-01-31T22:40:10.657+0000 | 2023-01-31T22:54:55.539+0000 | nan                                    | nan                   |
|  1 | 3ce7e538-69e2-4eba-a88a-d7d3aa5fb09d | AYYJ-UFin02FJXK6Tk3W | Heme - Binary Text Classification                | pubmed_36455995               | PARAGRAPH              | master-pubmed.vyasa.com                                                         | []                | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:49.697+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_Neyn02FJXK6Tk9t_heme'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |                        1 |                         1 |                          1 |                         0 | ['heme']                                                                                       | True         | AYYJ_Neyn02FJXK6Tk9t | 2023-01-31T22:40:10.655+0000 | 2023-01-31T22:40:10.655+0000 | 2023-01-31T22:54:49.614+0000 | nan                                    | nan                   |
|  2 | 3ce7e538-69e2-4eba-a88a-d7d3aa5fb09d | AYYJ-UFin02FJXK6Tk3W | Heme - Binary Text Classification                | pubmed_36464628               | PARAGRAPH              | master-pubmed.vyasa.com                                                         | []                | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:52.162+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_Neyn02FJXK6Tk9p_heme'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |                        1 |                         1 |                          1 |                         0 | ['heme']                                                                                       | True         | AYYJ_Neyn02FJXK6Tk9p | 2023-01-31T22:40:10.654+0000 | 2023-01-31T22:40:10.654+0000 | 2023-01-31T22:54:52.081+0000 | nan                                    | nan                   |
|  3 | 453f834e-7d59-4140-9728-0d13e8ea7db5 | AYYJ-UFin02FJXK6Tk3W | Cancer - Binary Text Classification              | pubmed_36455995               | PARAGRAPH              | master-pubmed.vyasa.com                                                         | []                | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:40.484+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_NGkn02FJXK6Tk78_not_cancer'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                        1 |                         1 |                          1 |                         0 | ['not_cancer']                                                                                 | True         | AYYJ_NGkn02FJXK6Tk78 | 2023-01-31T22:40:09.107+0000 | 2023-01-31T22:40:09.107+0000 | 2023-01-31T22:54:40.391+0000 | nan                                    | nan                   |
|  4 | 453f834e-7d59-4140-9728-0d13e8ea7db5 | AYYJ-UFin02FJXK6Tk3W | Cancer - Binary Text Classification              | pubmed_36464628               | PARAGRAPH              | master-pubmed.vyasa.com                                                         | []                | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:43.031+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_NGkn02FJXK6Tk74_not_cancer'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                        1 |                         1 |                          1 |                         0 | ['not_cancer']                                                                                 | True         | AYYJ_NGkn02FJXK6Tk74 | 2023-01-31T22:40:09.107+0000 | 2023-01-31T22:40:09.107+0000 | 2023-01-31T22:54:42.948+0000 | nan                                    | nan                   |
|  5 | 453f834e-7d59-4140-9728-0d13e8ea7db5 | AYYJ-UFin02FJXK6Tk3W | Cancer - Binary Text Classification              | pubmed_36518180               | PARAGRAPH              | master-pubmed.vyasa.com                                                         | []                | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:45.494+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_NGkn02FJXK6Tk7p_cancer'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |                        1 |                         1 |                          1 |                         0 | ['cancer']                                                                                     | True         | AYYJ_NGkn02FJXK6Tk7p | 2023-01-31T22:40:09.106+0000 | 2023-01-31T22:40:09.106+0000 | 2023-01-31T22:54:45.408+0000 | nan                                    | nan                   |
|  6 | b1a6e243-d633-4504-b258-b5f790f244b9 | AYYJ-UFin02FJXK6Tk3W | Where is the osteonecrosis located on the body?  | pubmed_36259635               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:28.728+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_H2zn02FJXK6Tk68_femoral_head'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |                        1 |                         1 |                          1 |                         0 | ['femoral head']                                                                               | True         | AYYJ_H2zn02FJXK6Tk68 | 2023-01-31T22:39:47.599+0000 | 2023-01-31T22:39:47.599+0000 | 2023-01-31T22:54:28.634+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
|  7 | b1a6e243-d633-4504-b258-b5f790f244b9 | AYYJ-UFin02FJXK6Tk3W | Where is the osteonecrosis located on the body?  | pubmed_36455995               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:07.152+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_GBOn02FJXK6Tk6O_jaw'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                        1 |                         1 |                          1 |                         0 | ['jaw']                                                                                        | True         | AYYJ_GBOn02FJXK6Tk6O | 2023-01-31T22:39:40.109+0000 | 2023-01-31T22:39:40.109+0000 | 2023-01-31T22:54:07.056+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
|  8 | b1a6e243-d633-4504-b258-b5f790f244b9 | AYYJ-UFin02FJXK6Tk3W | Where is the osteonecrosis located on the body?  | pubmed_36464628               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:18.161+0000', 'curations': [{'curationType': 'CUSTOM', 'customString': 'test custom curation'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                        1 |                         1 |                          1 |                         0 | ['test custom curation']                                                                       | True         | AYYJ_Fxqn02FJXK6Tk6K | 2023-01-31T22:39:39.113+0000 | 2023-01-31T22:39:39.113+0000 | 2023-01-31T22:54:18.075+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
|  9 | b1a6e243-d633-4504-b258-b5f790f244b9 | AYYJ-UFin02FJXK6Tk3W | Where is the osteonecrosis located on the body?  | pubmed_36518180               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:22.535+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_FJXn02FJXK6Tk57_both_legs'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                        1 |                         1 |                          1 |                         0 | ['both legs']                                                                                  | True         | AYYJ_FJXn02FJXK6Tk57 | 2023-01-31T22:39:36.524+0000 | 2023-01-31T22:39:36.524+0000 | 2023-01-31T22:54:22.446+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
| 10 | b1a6e243-d633-4504-b258-b5f790f244b9 | AYYJ-UFin02FJXK6Tk3W | Where is the osteonecrosis located on the body?  | pubmed_36518932               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:26.649+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_E9Sn02FJXK6Tk54_femoral_head'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |                        1 |                         1 |                          1 |                         0 | ['femoral head']                                                                               | True         | AYYJ_E9Sn02FJXK6Tk54 | 2023-01-31T22:39:35.761+0000 | 2023-01-31T22:39:35.761+0000 | 2023-01-31T22:54:26.472+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
| 11 | b1a6e243-d633-4504-b258-b5f790f244b9 | AYYJ-UFin02FJXK6Tk3W | Where is the osteonecrosis located on the body?  | pmc_9675136                   | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:54:31.237+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_EI9n02FJXK6Tk5t_femoral_head'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |                        1 |                         1 |                          1 |                         0 | ['femoral head']                                                                               | True         | AYYJ_EI9n02FJXK6Tk5t | 2023-01-31T22:39:32.411+0000 | 2023-01-31T22:39:32.411+0000 | 2023-01-31T22:54:31.156+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
| 12 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | AYYJ-UFin02FJXK6Tk3W | What is the disease or indication being studied? | pubmed_36259635               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:53:17.070+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_Cmpn02FJXK6Tk5K_delayed_acetabular_fracture'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                        1 |                         1 |                          1 |                         0 | ['delayed acetabular fracture']                                                                | True         | AYYJ_Cmpn02FJXK6Tk5K | 2023-01-31T22:39:26.116+0000 | 2023-01-31T22:39:26.116+0000 | 2023-01-31T22:53:16.847+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
| 13 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | AYYJ-UFin02FJXK6Tk3W | What is the disease or indication being studied? | pubmed_36455995               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:53:08.649+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ_AI8n02FJXK6Tk4e_autoimmune_and_inflammatory_disorders'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                        1 |                         1 |                          1 |                         0 | ['autoimmune and inflammatory disorders']                                                      | True         | AYYJ_AI8n02FJXK6Tk4e | 2023-01-31T22:39:16.026+0000 | 2023-01-31T22:39:16.026+0000 | 2023-01-31T22:53:08.399+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
| 14 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | AYYJ-UFin02FJXK6Tk3W | What is the disease or indication being studied? | pubmed_36518932               | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:53:12.289+0000', 'curations': [{'curationType': 'ACCEPTED_EXISTING', 'answerId': 'AYYJ--U7n02FJXK6Tk3__ONFH'}]}]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |                        1 |                         1 |                          1 |                         0 | ['ONFH']                                                                                       | True         | AYYJ--U7n02FJXK6Tk3_ | 2023-01-31T22:39:08.600+0000 | 2023-01-31T22:39:08.600+0000 | 2023-01-31T22:53:12.183+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |
| 15 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | AYYJ-UFin02FJXK6Tk3W | What is the disease or indication being studied? | pmc_9675136                   | DOCUMENT_AS_PARAGRAPHS | master-pubmed.vyasa.com,master-pmc-oa.vyasa.com,master-clinicaltrials.vyasa.com | nan               | True       | []                | []                 |                   0 | False             | False       | [{'createdByUser': 25036, 'dateIndexed': '2023-01-31T22:53:51.327+0000', 'curations': [{'curationType': 'SELECT_FROM_TEXT', 'customString': 'end-stage hip diseases', 'customStringContext': {'documentId': 'pmc_9675136', 'provider': 'master-pmc-oa.vyasa.com', 'startOffset': 500, 'endOffset': 522}}, {'curationType': 'SELECT_FROM_TEXT', 'customString': 'femoral head necrosis', 'customStringContext': {'documentId': 'pmc_9675136', 'provider': 'master-pmc-oa.vyasa.com', 'startOffset': 500, 'endOffset': 521}}, {'curationType': 'SELECT_FROM_TEXT', 'customString': 'osteoarthritis', 'customStringContext': {'documentId': 'pmc_9675136', 'provider': 'master-pmc-oa.vyasa.com', 'startOffset': 500, 'endOffset': 514}}, {'curationType': 'SELECT_FROM_TEXT', 'customString': 'femoral neck fracture', 'customStringContext': {'documentId': 'pmc_9675136', 'provider': 'master-pmc-oa.vyasa.com', 'startOffset': 500, 'endOffset': 521}}]}] |                        1 |                         4 |                          4 |                         0 | ['end-stage hip diseases', 'femoral head necrosis', 'osteoarthritis', 'femoral neck fracture'] | True         | AYYJ-9Rxn02FJXK6Tk3z | 2023-01-31T22:39:04.304+0000 | 2023-01-31T22:39:04.304+0000 | 2023-01-31T22:53:51.219+0000 | {'chunk_size': 5, 'null_threshold': 0} | AX9184SyOBLJbFhFZLqp  |

Retrieve Curator's Answers

You can also zoom into the 'userCurations' portion to look at the specific curations completed for each document:question. You can learn what type of curation was done (accepted model answer, select from text, etc.), where there answer was located in the text, what the probability of an answer chosen by the curator was, and more.

response = requests.post(f"{envUrl}/layar/question/search", # note the set ID
    data = json.dumps({
    "batchGroupingKeys": ["AYYJ-UFin02FJXK6Tk3W"],
    "hasUserCurations": ["TRUE"],
    "rows" : 100,
    "start" : 0}), # Set to TRUE to include both custom answers (Select from Text, Additional Answers, No Answer Applicable) *as well as* when curators have accepted a model prediction
    headers = {
        'accept':'application/json',
        'content-Type':'application/json',
        'authorization':f"Bearer {token}",
        'X-Vyasa-Client': 'curate'
    }
)

  
data = response.json() 

# Flatten data into readable dataframe
df0 = pd.json_normalize(data) # Entire Question objects for all questions that meet the QuestionSearchCommand criteria above
df = pd.json_normalize(data, record_path =['userCurations']) # userCurations dict, specifically for those questions

df = pd.json_normalize(df['curations']) # Curations to be cleaned up into dataframe
df = df[0].apply(pd.Series) # Create a new, Curation-specific dataframe
df = df.rename(columns={
    "curationType": "Curation Type", 
    "customString": "Curator's Answer", 
    "customStringContext.documentId": "Custom Answer Document ID", # Only relevant for Select from Text answers, can be removed
    "customStringContext.provider": "Custom Answer Data Source", # Only relevant for Select from Text answers, can be removed
    "customStringContext.startOffset": "Select from Text Start Offset", # Only relevant for Select from Text answers, can be removed
    "customStringContext.endOffset" : "Select from Text End Offset", # Only relevant for Select from Text answers, can be removed
    "answerId" : "Model Prediction ID"}) # Accepted Model Prediction ID, will be needed later
df.insert(0, 'Document ID', df0['singleDocQuestionDocumentId'])
df.insert(1, 'Job ID', df0['jobId'])
curations = df

# Add a little sort based on document ID and reset the index
curations = curations.sort_values(by=['Document ID']).reset_index().drop(['index'], axis=1)

Example dataframe output:

|    | Document ID     | Job ID                               | Curation Type     | Model Prediction ID                                        | Curator's Answer       | Custom Answer Document ID   | Custom Answer Data Source   |   Select from Text Start Offset |   Select from Text End Offset |
|---:|:----------------|:-------------------------------------|:------------------|:-----------------------------------------------------------|:-----------------------|:----------------------------|:----------------------------|--------------------------------:|------------------------------:|
|  0 | pmc_9675136     | b1a6e243-d633-4504-b258-b5f790f244b9 | ACCEPTED_EXISTING | AYYJ_EI9n02FJXK6Tk5t_femoral_head                          | nan                    | nan                         | nan                         |                             nan |                           nan |
|  1 | pmc_9675136     | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | SELECT_FROM_TEXT  | nan                                                        | end-stage hip diseases | pmc_9675136                 | master-pmc-oa.vyasa.com     |                             500 |                           522 |
|  2 | pubmed_36259635 | 3ce7e538-69e2-4eba-a88a-d7d3aa5fb09d | ACCEPTED_EXISTING | AYYJ_Neyn02FJXK6Tk-S_heme                                  | nan                    | nan                         | nan                         |                             nan |                           nan |
|  3 | pubmed_36259635 | b1a6e243-d633-4504-b258-b5f790f244b9 | ACCEPTED_EXISTING | AYYJ_H2zn02FJXK6Tk68_femoral_head                          | nan                    | nan                         | nan                         |                             nan |                           nan |
|  4 | pubmed_36259635 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | ACCEPTED_EXISTING | AYYJ_Cmpn02FJXK6Tk5K_delayed_acetabular_fracture           | nan                    | nan                         | nan                         |                             nan |                           nan |
|  5 | pubmed_36455995 | 3ce7e538-69e2-4eba-a88a-d7d3aa5fb09d | ACCEPTED_EXISTING | AYYJ_Neyn02FJXK6Tk9t_heme                                  | nan                    | nan                         | nan                         |                             nan |                           nan |
|  6 | pubmed_36455995 | 453f834e-7d59-4140-9728-0d13e8ea7db5 | ACCEPTED_EXISTING | AYYJ_NGkn02FJXK6Tk78_not_cancer                            | nan                    | nan                         | nan                         |                             nan |                           nan |
|  7 | pubmed_36455995 | b1a6e243-d633-4504-b258-b5f790f244b9 | ACCEPTED_EXISTING | AYYJ_GBOn02FJXK6Tk6O_jaw                                   | nan                    | nan                         | nan                         |                             nan |                           nan |
|  8 | pubmed_36455995 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | ACCEPTED_EXISTING | AYYJ_AI8n02FJXK6Tk4e_autoimmune_and_inflammatory_disorders | nan                    | nan                         | nan                         |                             nan |                           nan |
|  9 | pubmed_36464628 | 3ce7e538-69e2-4eba-a88a-d7d3aa5fb09d | ACCEPTED_EXISTING | AYYJ_Neyn02FJXK6Tk9p_heme                                  | nan                    | nan                         | nan                         |                             nan |                           nan |
| 10 | pubmed_36464628 | 453f834e-7d59-4140-9728-0d13e8ea7db5 | ACCEPTED_EXISTING | AYYJ_NGkn02FJXK6Tk74_not_cancer                            | nan                    | nan                         | nan                         |                             nan |                           nan |
| 11 | pubmed_36464628 | b1a6e243-d633-4504-b258-b5f790f244b9 | CUSTOM            | nan                                                        | test custom curation   | nan                         | nan                         |                             nan |                           nan |
| 12 | pubmed_36518180 | 453f834e-7d59-4140-9728-0d13e8ea7db5 | ACCEPTED_EXISTING | AYYJ_NGkn02FJXK6Tk7p_cancer                                | nan                    | nan                         | nan                         |                             nan |                           nan |
| 13 | pubmed_36518180 | b1a6e243-d633-4504-b258-b5f790f244b9 | ACCEPTED_EXISTING | AYYJ_FJXn02FJXK6Tk57_both_legs                             | nan                    | nan                         | nan                         |                             nan |                           nan |
| 14 | pubmed_36518932 | b1a6e243-d633-4504-b258-b5f790f244b9 | ACCEPTED_EXISTING | AYYJ_E9Sn02FJXK6Tk54_femoral_head                          | nan                    | nan                         | nan                         |                             nan |                           nan |
| 15 | pubmed_36518932 | 05db6aaf-d97e-4d49-97f9-20ef2e166ff6 | ACCEPTED_EXISTING | AYYJ--U7n02FJXK6Tk3__ONFH                                  | nan                    | nan                         | nan                         |                             nan |                           nan |

Up Next

Learn how to apply a series of additional analytics to your results from Curate! Group answers to identify common results in your set, and map your results to uploaded Layar ontologies.