cr-connect-bpmn

Occasionally there is a need to access data saved in one workflow in another workflow. Since task data is only accessible within the workflow, the ability to store data and retrieve it is available, aka the datastore. The datastore is a key value pair with three available types; Study, User and File. A datastore is scoped to the process level, i.e., will not be the same in a sub-process or Call Activity.

Terms

type - the type of datastore to set; study, user, or file
key - variable name used to identify the value being stored
value - the data value to be associated with the key
file_id - the id of the associated file (only used when type is set to file)
default - value to return if key value pair has not been stored yet

A Datastore List is available to document information about each key value pair used in CR Connect 2.

Datastore Set

To set a datastore, use the data_store_set script. It takes 3 required keyword arguments; type, key, and value. If the type is file, it requires a fourth keyword argument of file_id.

Examples:

File

data_store_set(type='file', file_id=123, key='my_key', value='my_value')

Study

data_store_set(type='study', key='my_key', value='my_value')

User

data_store_set(type='user', key='my_key', value='my_value')

Datastore Get

To retrieve a datastore value, use the data_store_get script. It requires 2 keyword arguments; type and key. If the type is file, it requires a third keyword argument of file_id. You can set an optional argument of default that is returned if no record is found.

Examples:

File

data_store_get(type='file', file_id=123, key='my_key')

data_store_get(type='file', file_id=123, key='my_key', default='my_default_value')

Study

data_store_get(type='study', key='my_key')

data_store_get(type='study', key='my_key', default='my_default_value')

User

data_store_get(type='user', key='my_key')

data_store_get(type='user', key='my_key', default='my_default_value')

Study Datastore

The study datastore allows a user to save data in one workflow within a study and retrieve it in another workflow in the same study. The format for script use is:

Store: study_data_set(key,value)
Retrieve: study_data_get(key,default)

User Datastore

The user datastore allows a user to save data in one workflow within a study and retrieve it in another workflow in any study. The format for script use is:

Store: user_data_set(key,value)
Retrieve: user_data_get(key,default)

File Datastore

file_id - the unique id of a file associated with a workflow
key - variable name used to identify the value being stored.
default - value to return if key value pair has not been stored yet. (Optional)

The file datastore allows a user to save metadata in one workflow within a study and retrieve it in another workflow in that study.

Store: file_data_set(file_id=x,key='somekey',value='somevalue')
Retrieve: file_data_get(file_id=x,key='somekey', default='xyz')

Delete Variables

Using delete_variables() instead of del() does not throw an error if the variable listed is not found.

Documents

Checking if File/Document is Uploaded

It is possible to detect if a file has been uploaded for the current study, using the code in the Documents reference file. For example:

is_file_uploaded('Consent_AdultParentalPermission') would return true if the Adult Parental Permission Consent form has been uploaded during any workflow related to the current study.

Deleting a File/Document

You can delete a file using the value found in the code column in the Documents reference file. For example: delete_file('Consent_AdultParentalPermission') A list of documents can also be passed in. If you like named arguments delete_file(code='Consent....') is also acceptable.

Note: This deletes all files with this code for the entire study.

If you only want to delete files for the current workflow, you can add a study_wide parameter. For example:

delete_file('Consent_AdultParentalPermission', study_wide=False)

Generate Document Template

To dynamically generate a Word document from a template, add a Script Task with an Inline Script expression:

complete_template('template_file_name.docx', 'irb_document_code')

Replace template_file_name.docx with your template file name.

We use the template file name to name the generated document. If you want to use a different name, you can pass an optional third parameter.

complete_template('template_file_name.docx', 'irb_document_code', 'some_other_file_name.docx')

You may add new templates to a workflow specification using the "Upload Template File" button in the Configurator.

Task Data

Use Jinja template syntax to display data gathered in a particular workflow task.

Images

If a workflow specification contains a form field that allows users to upload image files, you may display those files in a Word template using the following syntax:

complete_template('template_file_name.docx', irb_document_code [form_field_id_1,form_field_id_2]

Replace form_field_id_1,form_field_id_2 with a comma-delimited list (with no spaces) of the fields you'd like to display in the Word document. The images will be available within the document template in a dictionary called images. The image file IDs will be available in a list if you used a files (multiple file) uploader, and as a single ID for a file (single file) uploader.

For multiple-file uploader field:

{% for image_id in form_field_id_1 %}
{{ images[image_id].image }}
{% endfor %}

For a single-file uploader field:

{{ images[form_field_id_2].image }}

Email Messages

Creates an email, using the provided subject and recipients arguments, which are required. The Element Documentation field in the script task must contain markdown/jinja that becomes the body of the email message.

You can also provide cc, bcc, reply_to and attachments arguments. The cc, bcc, reply_to, and attachments arguments are not required.

The recipients, cc, and bcc arguments can contain an email address or list of email addresses. In place of an email address, we accept the string 'associated', in which case we look up the users associated with the study who have send_email set to True.

The reply_to argument can contain an email address. Each project will be setup to have have a default reply_to email address. For CRConnect it is: uvacrconnect@virginia.edu.

Attachments

The attachments are based on doc_codes. The attachments argument can include a doc_code or list of doc_codes.

attachments='Study_App_Doc'

attachments=['Study_App_Doc', 'Study_Protocol_Document']

Filtering attachments

Normally, we include all files for each doc_code.

To filter the files for a particular doc code, you can send a doc_code tuple instead of a doc_code. A doc_code tuple contains a doc_code and list (of file_names).

attachments=('Study_App_Doc', []) -- This would send all files for Study_App_Doc

attachments=('Study_App_Doc', ['some_file']) -- This would only send the file named 'some_file'

attachments=('Study_App_Doc', ['some_file', 'some_other_file']) -- This would send both 'some_file' and 'some_other_file'

You can also pass a list of these doc_code tuples

attachments=[('Study_App_Doc', []),('Study_Protocol_Document', [])]

You can mix the doc_code strings and doc_code tuples any way you want.

attachments=['Study_App_Doc', ('Study_Protocol_Document', ['some_file'])]

Examples:

email_model = email(subject="My Subject", recipients=["dhf8r@virginia.edu", pi.email, 'associated'])

email_model = email(subject="My Subject", recipients="user@example.com", cc='associated', bcc='test_user@example.com)

email_model = email(subject="My Subject", recipients="user@example.com", reply_to="reply_to@example.com")

email_model = email(subject="My Subject", recipients="user@example.com", attachments='Study_App_Doc')

email_model = email(subject="My Subject", recipients="user@example.com", attachments=['Study_App_Doc', Study_Protocol_Document])

email_model = email(subject="My Subject", recipients="user@example.com", attachments=('Study_App_Doc', []))

email_model = email(subject="My Subject", recipients="user@example.com", attachments=('Study_App_Doc', ['some_file']))

email_model = email(subject="My Subject", recipients="user@example.com", attachments=[('Study_App_Doc', []), ('Study_Protocol_Document',['some_file'])])

email_model = email(subject="My Subject", recipients="user@example.com", attachments=['Study_App_Doc', ('Study_Protocol_Document', ['some_file'])])

Returns an email_model. You can pass email_model.id to the get_email_data script.

In non-production environments, the email will be delivered to a shared mailtrap account that authorized users can check to see if the generated email looks correct. Access to mailtrap can be requested from the current development team. https://mailtrap.io/

Data about Email Messages

Retrieves data about one or more email messages.

If you know the email_id, you can get data about a specific email.

email_data = get_email_data(email_id=email_id)

This returns a list with 1 email model.

You can also get data about all emails associated with a workflow spec.

email_data = get_email_data(workflow_spec_id=workflow_spec_id)

This returns a list of 1 or more email models.

Personnel Study Access / Emails

It is possible to allow more than the owner of a workflow to log in and complete workflow tasks. To do add a new study assosciate, you can call:

update_study_associate(uid='sbp3ey',role='Unicorn Herder',send_email=False, access=True) The role is for information purposes only, and can be anything. setting send_email to True will mean that all correspondence directed at the study owner will also be sent to this associate.

Be sure to set the access to true for this to have any effect on their ability to log in and see a study. You can set this to false to remove a users ability to edit workflows.

Updating / Removing associates

You can modify the list of all associates in one call by sending in an array to this command: update_study_associates([{'uid':'sbp3ey','role':'Unicorn Herder', 'send_email': False, 'access':True}]) Calling this with an empty array will clear out all users associated with a study.

Retrieving a list of Associates

Returns all people associated with the study - Will always return the study owner as associated example : get_study_associates() => [{'uid':'sbp3ey','role':'Unicorn Herder', 'send_email': False, 'access':True}]

Check for a specific associate

Returns person assocated with study or an error if one is not associated. example : get_study_associate('sbp3ey') => {'uid':'sbp3ey','role':'Unicorn Herder', 'send_email': False, 'access':True}

LDAP

Attempts to create a dictionary with person details, using the provided argument (a UID) and look it up through LDAP. If no UID is provided, then returns information about the current user. Example: supervisor_info = ldap(supervisor_uid) sets the supervisor information to ldap details for the given uid."

#IRB APIs

Study Info

investigators = study_info('investigators') pi = study_info('investigators','PI') scs = study_info('investigators','SC')

IRB Info

It is possible to access details from IRB Online about studies as well.

irb_info = get_irb_info()

Which will return a dictionary of details we can get from IRB Oneline, similar to the following:

{
    "AGENDA_DATE": "2021-04-15T00:00:00+00:00",
    "DATE_MODIFIED": "2021-04-15T00:00:00+00:00",
    "IRBEVENT": "IRB Event 1",
    "IRB_ADMINISTRATIVE_REVIEWER": "IRB Admin Reviewer 1",
    "IRB_OF_RECORD": "IRB of Record 1",
    "IRB_REVIEW_TYPE": "IRB Review Type 1",
    "IRB_STATUS": "IRB Status 1",
    "STUDYIRBREVIEWERADMIN": "Study IRB Review Admin 1",
    "UVA_IRB_HSR_IS_IRB_OF_RECORD_FOR_ALL_SITES": 1,
    "UVA_STUDY_TRACKING": "UVA Study Tracking 1"
  },

Main Dashboard URL

get_dashboard_url()

Get Localtime

Converts a UTC time to a different timezone. It defaults to US/Eastern

new_time = get_localtime(timestamp=utc_timestamp_string)

new_time = get_localtime(timestamp=utc_timestamp_string, timezone=timezone_string)

Enumeration Label

When you have a dropdown list of options, or an autocomplete field, you may want to find the full label the user selected, rather than just having the raw value that was selected. This allows you to look that back up. For instance, you might have a user task called "PetForm" with a dropdown field called 'pet', and the options are: dog => Lovely friendly Dogs cat => Feisty mean old kitties swallow => Lovely beautiful birds

When the user selects "Lovely friendly dogs" the value 'dog' will be stored. If you want to retrieve the original label displayed you can do so at anytime, if you know the task, field and value.

(note that after the form is completed, the variable Pet will be set to 'dog')

pet_label = enum_label(task="PetForm",field="Pet",value=Pet);

or the less verbose, and less clear:

pet_label = enum_label("PetForm", "Pet", Pet);

Logging

Adding Log Entries

You can add log entries from a script task. The log script takes 3 arguments; level, code, and message.

Level must be metrics, debug, info, warning, error or critical.

Code is a short string meant for searching the logs. By convention, it is lower case with underscores.

Message is a more descriptive string, including any info you want to log.

Examples:

log_result = log(level='info', code='missing_info', message='You must include the correct info!')

log_result = log(level='info', code='missing_info', message=f'You must include the correct info! We received {some_var}')

log_result = log(level='metrics', code='start_workflow', message='The workflow started!')

Note: Returning 'log_result' is optional.

Retrieving Log Entries

You can retrieve log entries for the current workflow or study. Please note that all log entries for a study are available on the Study Dashboard under the logs tab for everyone to see.

Current Workflow

The get_logs_for_workflow script returns logs for the current workflow. It accepts 3 optional arguments; level, code, and size, in that order. You can also pass them with keywords. The level and code arguments filter the records. Size limits the number of records returned.

Examples:

log_results = get_logs_for_workflow()

log_results = get_logs_for_workflow('metrics')

log_results = get_logs_for_workflow(level='info', code='missing_info', size=10)

Current Study

The get_logs_for_study script returns logs for the current study. It accepts 3 optional arguments; level, code, and size, in that order. You can also pass them with keywords. The level and code arguments filter the records. Size limits the number of records returned.

Examples:

logs = get_logs_for_study()

logs = get_logs_for_study('metrics', 'missing_info', 10)

logs = get_logs_for_study(level='metrics', code='missing_info', size=10)

Modify Spreadsheet

Add data to an existing (IRB Document) spreadsheet

Examples:

modify_spreadsheet('Finance_BCA', 'C4', 'This is my inserted text')

modify_spreadsheet(irb_doc_code='Finance_BCA', cell='C4', text='This is my inserted text')

Study Status

Get Study Status

Get the status for the current study.

Study status will be one of:

Study Status	UI Text
`in_progress`	In Progress
`open_for_enrollment`	IRB Open for Enrollment
`cr_connect_complete`	CRConnect Complete
`hold`	Hold
`abandoned`	Abandoned

Example:

study_status = get_study_status()

Set Study Status

Set the study status. Requires a study status.

Study status must be 1 of 'in_progress', 'hold', 'open_for_enrollment', 'abandoned', or 'cr_connect_complete'

Example:

set_study_status('cr_connect_complete')

Study Progress Status

Get and set progress status for the current study.

Progress status is only used when the study status is in_progress.

Progress status can be one of

Progress Status	UI Text
`in_progress`	In Progress
`ready_for_pre_review`	Ready for Pre-Review
`submitted_for_pre_review`	Submitted for Pre-Review
`in_pre_review`.	In pre-Review
`returned_from_pre_review`	Returned from Pre-Review
`resubmitted_for_pre_review`	Resubmitted for Pre-Review
`pre_review_complete`	Pre-Review Complete
`agenda_date_set`	Agenda Date Set
`approved`	Approved
`approved_with_conditions`	Approved With Conditions
`deferred`	Deferred
`disapproved`	Disapproved

Get Study Progress Status

Get progress status for the current study.

Example:

progress_status = get_study_progress_status()

Set Study Progress Status

Set progress status for the current study.

Example:

progress_status = set_study_progress_status(new_status='approved')

Workflow

Workflow Status

Get the current status of a workflow for the current study.

You must pass in a workflow_spec_id.

Examples:

workflow_status = get_workflow_status('my_workflow_spec_id')

workflow_status = get_workflow_status(workflow_spec_id='my_workflow_spec_id')

Status List

not_started = "not_started"
user_input_required = "user_input_required"
waiting = "waiting"
complete = "complete"
erroring = "erroring"

Get Workflow Spec Information

Get workflow spec information from a workflow spec id

Example:

spec_info = get_spec_from_id(spec_id)

Workflow - Reset

Will reset any workflow to an unstarted/untouched state. It will optionally clear any previously uploaded files or data.

Examples:

reset_workflow('workflow_spec_id')

reset_workflow(workflow_spec_id='workflow_spec_id')

reset_workflow('workflow_spec_id', clear_data=True)

Workflow - Start

Start a workflow programmatically. Requires a workflow_spec_id.

Examples:

start_workflow('a_workflow_spec_id')

start_workflow(workflow_spec_id='a_workflow_spec_id')

Note: Previously stated workflows will not be restarted. They must be in the "not_started" state

Zipped Files

Creates a zip file from a list of file_ids and a doc_code. Returns a file model.

This is meant to use as an attachment to an email message.

file_ids = [1, 2, 3]

zip_file_model = get_zipped_files(file_ids=file_ids, doc_code='CRC2_IRBSubmission_ZipFile')

Takes an optional keyword argument of filename that is used to name the generated file. Otherwise, attachments.zip is used.

zip_file_model = get_zipped_files(file_ids=file_ids, doc_code='CRC2_IRBSubmission_ZipFile', filename='my_file_name.zip')

Get Instance

Returns the value of the INSTANCE_NAME environment variable.

Currently, this is staging for the staging server and testing for the testing server.

Example:

instance = get_instance()

Workflow State

Returns a dictionary containing state and message for a workflow.

{'state': 'required', 'message': 'This workflow is required.'}

{'state': None, 'message': None}

Requires a workflow_spec_id.

Examples:

workflow_state = get_workflow_state('simple_workflow')

workflow_state = get_workflow_state(workflow_spec_id='simple_workflow')

Current User

Returns information about the current user, including id, uid, is_admin, and ldap_info.

If the current user is actually an admin impersonating another user, then we also return information about the impersonator.

Example:

current_user = get_current_user()

Pre Review

Retrieve information about submissions returned to the PI during Pre Review.

Example:

pre_reviews = get_pre_reviews()

Get User Studies

Returns the list of studies for a user in Protocol Builder

Example:

user_studies = get_user_studies(user_id)

The script returns a data object like this:

[
  {
    "DATECREATED": "2002-02-19T14:26:49.127756",
    "DATELASTMODIFIED": "2012-02-19T14:26:49.127756",
    "STUDYID": 11111,
    "TITLE": "An old study from the good old days that should not show up.",
    "REVIEW_TYPE": 2
  },
  {
    "DATECREATED": "2020-02-19T14:26:49.127756",
    "DATELASTMODIFIED": "2020-02-19T14:26:49.127756",
    "STUDYID": 54321,
    "TITLE": "Another study about the effect of a naked mannequin on software productivity",
    "REVIEW_TYPE": 2
  }
]

Get the Study IDs

You can easily pull all the Study IDs--or any other attribute, with a list comprehension

Example:

user_studies = get_user_studies(user_id)

user_study_ids = [x.STUDYID for x in user_studies]