tests
This commit is contained in:
parent
d2f9013491
commit
8bb8d3dd34
|
@ -30,3 +30,4 @@ and use it to learn about the different parts of the CR Connect Workflow code ba
|
||||||
02_services
|
02_services
|
||||||
03_api
|
03_api
|
||||||
04_errors/00_index.rst
|
04_errors/00_index.rst
|
||||||
|
05_tests/00_index.rst
|
||||||
|
|
|
@ -0,0 +1,17 @@
|
||||||
|
.. _index-tests:
|
||||||
|
|
||||||
|
=============
|
||||||
|
Writing Tests
|
||||||
|
=============
|
||||||
|
|
||||||
|
CR Connect Workflow uses **pytest** for testing. It also has a **BaseTest** class.
|
||||||
|
|
||||||
|
`BaseTest` gives you access to a test database, and has fixtures for loading sample data, as well as working with users, studies, and workflows.
|
||||||
|
|
||||||
|
Our general approach is to have full coverage at the unit-test level.
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 2
|
||||||
|
|
||||||
|
01_layout
|
||||||
|
02_examples/00_index.rst
|
|
@ -0,0 +1,33 @@
|
||||||
|
------
|
||||||
|
Layout
|
||||||
|
------
|
||||||
|
|
||||||
|
Tests are in the **/tests** directory.
|
||||||
|
|
||||||
|
The test directory has 6 subdirectories; **data**, **emails**, **files**, **ldap**, **study**, and **workflow**.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
/tests
|
||||||
|
|____data
|
||||||
|
|
|
||||||
|
|----emails
|
||||||
|
|
|
||||||
|
|----files
|
||||||
|
|
|
||||||
|
|----ldap
|
||||||
|
|
|
||||||
|
|----study
|
||||||
|
|
|
||||||
|
|----workflow
|
||||||
|
|
|
||||||
|
|----base_test.py
|
||||||
|
...
|
||||||
|
|
||||||
|
The `data` directory holds workflows (BPMN files) for tests.
|
||||||
|
|
||||||
|
The other directories organize tests by topic. Some tests are in the tests directory itself.
|
||||||
|
|
||||||
|
This is not a perfect system.
|
||||||
|
|
||||||
|
Our long-term goal for tests is to mimic the layout of the crc directory.
|
|
@ -0,0 +1,13 @@
|
||||||
|
.. _index-test-examples:
|
||||||
|
|
||||||
|
--------
|
||||||
|
Examples
|
||||||
|
--------
|
||||||
|
|
||||||
|
One of the best ways to learn about writing tests is to look at existing tests.
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 2
|
||||||
|
|
||||||
|
01_hello_world
|
||||||
|
02_email_script
|
|
@ -0,0 +1,138 @@
|
||||||
|
Hello World
|
||||||
|
===========
|
||||||
|
|
||||||
|
One pattern we use to test new features is to create a workflow that uses the new feature.
|
||||||
|
|
||||||
|
Then, we can test the new feature by running the workflow and making assertions on the results.
|
||||||
|
|
||||||
|
Here is a simple test of a workflow.
|
||||||
|
|
||||||
|
Code
|
||||||
|
----
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
from tests.base_test import BaseTest
|
||||||
|
|
||||||
|
|
||||||
|
class TestHelloWorld(BaseTest):
|
||||||
|
|
||||||
|
def test_hello_world(self):
|
||||||
|
workflow = self.create_workflow('hello_world')
|
||||||
|
|
||||||
|
workflow_api = self.get_workflow_api(workflow)
|
||||||
|
first_task = workflow_api.next_task
|
||||||
|
|
||||||
|
self.complete_form(workflow, first_task, {'name': 'asdf'})
|
||||||
|
|
||||||
|
workflow_api = self.get_workflow_api(workflow)
|
||||||
|
second_task = workflow_api.next_task
|
||||||
|
|
||||||
|
self.assertEqual('Hello asdf', second_task.documentation)
|
||||||
|
|
||||||
|
Setup
|
||||||
|
-----
|
||||||
|
|
||||||
|
The code starts with three things we do in any test file.
|
||||||
|
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
from tests.base_test import BaseTest
|
||||||
|
|
||||||
|
class TestHelloWorld(BaseTest):
|
||||||
|
|
||||||
|
def test_hello_world(self):
|
||||||
|
|
||||||
|
- We import BaseTest.
|
||||||
|
- We create a class that extends BaseTest. The class name must begin with **Test**.
|
||||||
|
- We define a method within our class with a name that begins with **test_**.
|
||||||
|
|
||||||
|
|
||||||
|
Our test class is named **TestHelloWorld** and our test method is named **test_hello_world**.
|
||||||
|
|
||||||
|
You can have more than one test method in a test class.
|
||||||
|
|
||||||
|
Each test method must begin with **test_**.
|
||||||
|
|
||||||
|
|
||||||
|
Detail
|
||||||
|
------
|
||||||
|
|
||||||
|
Our test is relatively simple, but the method and approach is used in many of our tests.
|
||||||
|
|
||||||
|
- We create a workflow from the create_workflow method. This will be of type WorkflowModel.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
workflow = self.create_workflow('hello_world')
|
||||||
|
|
||||||
|
.. Note::
|
||||||
|
|
||||||
|
This means that we have a workflow named **hello_world.bpmn**,
|
||||||
|
and it is inside a directory name **hello_world**,
|
||||||
|
inside the /test/data directory.
|
||||||
|
|
||||||
|
So, /tests/data/hello_world/hello_world.bpmn
|
||||||
|
|
||||||
|
- We create a workflow_api object from the workflow.
|
||||||
|
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
workflow_api = self.get_workflow_api(workflow)
|
||||||
|
|
||||||
|
A workflow_api object is similar to a workflow, but it has additional attributes for the frontend such as **status**, **next_task**, and **navigation**.
|
||||||
|
|
||||||
|
- We grab the next task.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
first_task = workflow_api.next_task
|
||||||
|
|
||||||
|
This is the first task that needs a human interaction, usually a UserTask or ManualTask.
|
||||||
|
|
||||||
|
In this case, it is a UserTask with a form asking for a name.
|
||||||
|
|
||||||
|
- We complete a form by calling **complete_form**.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
self.complete_form(workflow, first_task, {'name': 'asdf'})
|
||||||
|
|
||||||
|
We pass 3 items to complete_form; the workflow and task objects, and a dictionary.
|
||||||
|
The dictionary is how we pass data to the form.
|
||||||
|
|
||||||
|
The complete_form method is defined in BaseTest. Among other things, complete_form calls the
|
||||||
|
/workflow/{workflow_id}/task/{task_id}/data API endpoint.
|
||||||
|
|
||||||
|
- We again call get_workflow_api and get the next task.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
workflow_api = self.get_workflow_api(workflow)
|
||||||
|
second_task = workflow_api.next_task
|
||||||
|
|
||||||
|
This time next_task gives us the second task that requires human interaction.
|
||||||
|
|
||||||
|
It is a ManualTask that has some information in the Element Documentation.
|
||||||
|
|
||||||
|
- We check the value using an assert statement.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
self.assertEqual('Hello asdf', second_task.documentation)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
----------------
|
||||||
|
Example Workflow
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Here is a workflow you can use in this test.
|
||||||
|
|
||||||
|
.. image:: /_static/05/02/process_hello_world.png
|
||||||
|
|
||||||
|
.. image:: /_static/05/02/task_get_name.png
|
||||||
|
|
||||||
|
.. image:: /_static/05/02/task_say_hello.png
|
|
@ -0,0 +1,99 @@
|
||||||
|
Email Script
|
||||||
|
============
|
||||||
|
|
||||||
|
The Hello World example gives us a great introduction to the basics of writing tests in CR Connect Workflow,
|
||||||
|
and the scaffolding available in BaseTest.
|
||||||
|
|
||||||
|
These next examples test the email script, and they each show us something more than the basics.
|
||||||
|
|
||||||
|
Validation
|
||||||
|
----------
|
||||||
|
|
||||||
|
In this test, we run the validation script. This is the same as clicking on the shield in the Configurator.
|
||||||
|
|
||||||
|
It shows us how to call an API endpoint.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
class TestEmailScript(BaseTest):
|
||||||
|
|
||||||
|
def test_email_script_validation(self):
|
||||||
|
# This validates scripts.email.do_task_validate_only
|
||||||
|
# It also tests that we don't overwrite the default email_address with random text during validation
|
||||||
|
# Otherwise json would have an error about parsing the email address
|
||||||
|
self.load_example_data()
|
||||||
|
spec_model = self.load_test_spec('email_script')
|
||||||
|
rv = self.app.get('/v1.0/workflow-specification/%s/validate' % spec_model.id, headers=self.logged_in_headers())
|
||||||
|
self.assertEqual([], rv.json)
|
||||||
|
|
||||||
|
with ... outbox
|
||||||
|
---------------
|
||||||
|
|
||||||
|
When testing email, we don't want to actually send emails.
|
||||||
|
|
||||||
|
Flask mail has a way to intercept the emails and show you the results.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
def test_email_script(self):
|
||||||
|
with mail.record_messages() as outbox:
|
||||||
|
|
||||||
|
self.assertEqual(0, len(outbox))
|
||||||
|
|
||||||
|
workflow = self.create_workflow('email_script')
|
||||||
|
|
||||||
|
workflow_api = self.get_workflow_api(workflow)
|
||||||
|
first_task = workflow_api.next_task
|
||||||
|
|
||||||
|
self.complete_form(workflow, first_task, {'subject': 'My Email Subject', 'recipients': 'test@example.com'})
|
||||||
|
|
||||||
|
self.assertEqual(1, len(outbox))
|
||||||
|
self.assertEqual('My Email Subject', outbox[0].subject)
|
||||||
|
self.assertEqual(['test@example.com'], outbox[0].recipients)
|
||||||
|
self.assertIn('Thank you for using this email example', outbox[0].body)
|
||||||
|
|
||||||
|
Exception
|
||||||
|
---------
|
||||||
|
|
||||||
|
We can test for error conditions.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
def test_bad_email_address_1(self):
|
||||||
|
workflow = self.create_workflow('email_script')
|
||||||
|
first_task = self.get_workflow_api(workflow).next_task
|
||||||
|
|
||||||
|
with self.assertRaises(AssertionError):
|
||||||
|
self.complete_form(workflow, first_task, {'recipients': 'test@example'})
|
||||||
|
|
||||||
|
|
||||||
|
Add data
|
||||||
|
--------
|
||||||
|
|
||||||
|
We can add extra data for our test.
|
||||||
|
|
||||||
|
Here, we need a second user to add as an associate.
|
||||||
|
|
||||||
|
.. code-block::
|
||||||
|
|
||||||
|
|
||||||
|
def test_email_script_associated(self):
|
||||||
|
workflow = self.create_workflow('email_script')
|
||||||
|
workflow_api = self.get_workflow_api(workflow)
|
||||||
|
|
||||||
|
# Only dhf8r is in testing DB.
|
||||||
|
# We want to test multiple associates, and lb3dp is already in testing LDAP
|
||||||
|
self.create_user(uid='lb3dp', email='lb3dp@virginia.edu', display_name='Laura Barnes')
|
||||||
|
StudyService.update_study_associates(workflow.study_id,
|
||||||
|
[{'uid': 'dhf8r', 'role': 'Chief Bee Keeper', 'send_email': True, 'access': True},
|
||||||
|
{'uid': 'lb3dp', 'role': 'Chief Cat Herder', 'send_email': True, 'access': True}])
|
||||||
|
|
||||||
|
first_task = workflow_api.next_task
|
||||||
|
|
||||||
|
with mail.record_messages() as outbox:
|
||||||
|
self.complete_form(workflow, first_task, {'subject': 'My Test Subject', 'recipients': ['user@example.com', 'associated']})
|
||||||
|
|
||||||
|
self.assertEqual(1, len(outbox))
|
||||||
|
self.assertIn(outbox[0].recipients[0], ['user@example.com', 'dhf8r@virginia.edu', 'lb3dp@virginia.edu'])
|
||||||
|
self.assertIn(outbox[0].recipients[1], ['user@example.com', 'dhf8r@virginia.edu', 'lb3dp@virginia.edu'])
|
||||||
|
self.assertIn(outbox[0].recipients[2], ['user@example.com', 'dhf8r@virginia.edu', 'lb3dp@virginia.edu'])
|
Binary file not shown.
After Width: | Height: | Size: 871 KiB |
Binary file not shown.
After Width: | Height: | Size: 852 KiB |
Binary file not shown.
After Width: | Height: | Size: 849 KiB |
Loading…
Reference in New Issue