tests

03_cr_connect_workflow/00_index.rst

@@ -30,3 +30,4 @@ and use it to learn about the different parts of the CR Connect Workflow code ba
    02_services
    03_api
    04_errors/00_index.rst
+   05_tests/00_index.rst

03_cr_connect_workflow/05_tests/00_index.rst

@@ -0,0 +1,17 @@
+.. _index-tests:
+
+=============
+Writing Tests
+=============
+
+CR Connect Workflow uses **pytest** for testing. It also has a **BaseTest** class.
+
+`BaseTest` gives you access to a test database, and has fixtures for loading sample data, as well as working with users, studies, and workflows.
+
+Our general approach is to have full coverage at the unit-test level.
+
+.. toctree::
+   :maxdepth: 2
+
+   01_layout
+   02_examples/00_index.rst

03_cr_connect_workflow/05_tests/01_layout.rst

@@ -0,0 +1,33 @@
+------
+Layout
+------
+
+Tests are in the **/tests** directory.
+
+The test directory has 6 subdirectories; **data**, **emails**, **files**, **ldap**, **study**, and **workflow**.
+
+.. code-block::
+
+  /tests
+   |____data
+   |
+   |----emails
+   |
+   |----files
+   |
+   |----ldap
+   |
+   |----study
+   |
+   |----workflow
+   |
+   |----base_test.py
+   ...
+
+The `data` directory holds workflows (BPMN files) for tests.
+
+The other directories organize tests by topic. Some tests are in the tests directory itself.
+
+This is not a perfect system.
+
+Our long-term goal for tests is to mimic the layout of the crc directory.

03_cr_connect_workflow/05_tests/02_examples/00_index.rst

@@ -0,0 +1,13 @@
+.. _index-test-examples:
+
+--------
+Examples
+--------
+
+One of the best ways to learn about writing tests is to look at existing tests.
+
+.. toctree::
+   :maxdepth: 2
+
+   01_hello_world
+   02_email_script

03_cr_connect_workflow/05_tests/02_examples/01_hello_world.rst

@@ -0,0 +1,138 @@
+Hello World
+===========
+
+One pattern we use to test new features is to create a workflow that uses the new feature.
+
+Then, we can test the new feature by running the workflow and making assertions on the results.
+
+Here is a simple test of a workflow.
+
+Code
+----
+
+.. code-block::
+
+    from tests.base_test import BaseTest
+
+
+    class TestHelloWorld(BaseTest):
+
+        def test_hello_world(self):
+            workflow = self.create_workflow('hello_world')
+
+            workflow_api = self.get_workflow_api(workflow)
+            first_task = workflow_api.next_task
+
+            self.complete_form(workflow, first_task, {'name': 'asdf'})
+
+            workflow_api = self.get_workflow_api(workflow)
+            second_task = workflow_api.next_task
+
+            self.assertEqual('Hello asdf', second_task.documentation)
+
+Setup
+-----
+
+The code starts with three things we do in any test file.
+
+
+.. code-block::
+
+    from tests.base_test import BaseTest
+
+    class TestHelloWorld(BaseTest):
+
+        def test_hello_world(self):
+
+- We import BaseTest.
+- We create a class that extends BaseTest. The class name must begin with **Test**.
+- We define a method within our class with a name that begins with **test_**.
+
+
+Our test class is named **TestHelloWorld** and our test method  is named **test_hello_world**.
+
+You can have more than one test method in a test class.
+
+Each test method must begin with **test_**.
+
+
+Detail
+------
+
+Our test is relatively simple, but the method and approach is used in many of our tests.
+
+- We create a workflow from the create_workflow method. This will be of type WorkflowModel.
+
+.. code-block::
+
+    workflow = self.create_workflow('hello_world')
+
+.. Note::
+
+    This means that we have a workflow named **hello_world.bpmn**,
+    and it is inside a directory name **hello_world**,
+    inside the /test/data directory.
+
+    So, /tests/data/hello_world/hello_world.bpmn
+
+- We create a workflow_api object from the workflow.
+
+
+.. code-block::
+
+    workflow_api = self.get_workflow_api(workflow)
+
+A workflow_api object is similar to a workflow, but it has additional attributes for the frontend such as **status**, **next_task**, and **navigation**.
+
+- We grab the next task.
+
+.. code-block::
+
+    first_task = workflow_api.next_task
+
+This is the first task that needs a human interaction, usually a UserTask or ManualTask.
+
+In this case, it is a UserTask with a form asking for a name.
+
+- We complete a form by calling **complete_form**.
+
+.. code-block::
+
+    self.complete_form(workflow, first_task, {'name': 'asdf'})
+
+We pass 3 items to complete_form; the workflow and task objects, and a dictionary.
+The dictionary is how we pass data to the form.
+
+The complete_form method is defined in BaseTest. Among other things, complete_form calls the
+/workflow/{workflow_id}/task/{task_id}/data API endpoint.
+
+- We again call get_workflow_api and get the next task.
+
+.. code-block::
+
+    workflow_api = self.get_workflow_api(workflow)
+    second_task = workflow_api.next_task
+
+This time next_task gives us the second task that requires human interaction.
+
+It is a ManualTask that has some information in the Element Documentation.
+
+- We check the value using an assert statement.
+
+.. code-block::
+
+    self.assertEqual('Hello asdf', second_task.documentation)
+
+
+
+----------------
+Example Workflow
+----------------
+
+Here is a workflow you can use in this test.
+
+.. image:: /_static/05/02/process_hello_world.png
+
+.. image:: /_static/05/02/task_get_name.png
+
+.. image:: /_static/05/02/task_say_hello.png

03_cr_connect_workflow/05_tests/02_examples/02_email_script.rst

@@ -0,0 +1,99 @@
+Email Script
+============
+
+The Hello World example gives us a great introduction to the basics of writing tests in CR Connect Workflow,
+and the scaffolding available in BaseTest.
+
+These next examples test the email script, and they each show us something more than the basics.
+
+Validation
+----------
+
+In this test, we run the validation script. This is the same as clicking on the shield in the Configurator.
+
+It shows us how to call an API endpoint.
+
+.. code-block::
+
+    class TestEmailScript(BaseTest):
+
+        def test_email_script_validation(self):
+            # This validates scripts.email.do_task_validate_only
+            # It also tests that we don't overwrite the default email_address with random text during validation
+            # Otherwise json would have an error about parsing the email address
+            self.load_example_data()
+            spec_model = self.load_test_spec('email_script')
+            rv = self.app.get('/v1.0/workflow-specification/%s/validate' % spec_model.id, headers=self.logged_in_headers())
+            self.assertEqual([], rv.json)
+
+with ... outbox
+---------------
+
+When testing email, we don't want to actually send emails.
+
+Flask mail has a way to intercept the emails and show you the results.
+
+.. code-block::
+
+    def test_email_script(self):
+        with mail.record_messages() as outbox:
+
+            self.assertEqual(0, len(outbox))
+
+            workflow = self.create_workflow('email_script')
+
+            workflow_api = self.get_workflow_api(workflow)
+            first_task = workflow_api.next_task
+
+            self.complete_form(workflow, first_task, {'subject': 'My Email Subject', 'recipients': 'test@example.com'})
+
+            self.assertEqual(1, len(outbox))
+            self.assertEqual('My Email Subject', outbox[0].subject)
+            self.assertEqual(['test@example.com'], outbox[0].recipients)
+            self.assertIn('Thank you for using this email example', outbox[0].body)
+
+Exception
+---------
+
+We can test for error conditions.
+
+.. code-block::
+
+    def test_bad_email_address_1(self):
+        workflow = self.create_workflow('email_script')
+        first_task = self.get_workflow_api(workflow).next_task
+
+        with self.assertRaises(AssertionError):
+            self.complete_form(workflow, first_task, {'recipients': 'test@example'})
+
+
+Add data
+--------
+
+We can add extra data for our test.
+
+Here, we need a second user to add as an associate.
+
+.. code-block::
+
+
+    def test_email_script_associated(self):
+        workflow = self.create_workflow('email_script')
+        workflow_api = self.get_workflow_api(workflow)
+
+        # Only dhf8r is in testing DB.
+        # We want to test multiple associates, and lb3dp is already in testing LDAP
+        self.create_user(uid='lb3dp', email='lb3dp@virginia.edu', display_name='Laura Barnes')
+        StudyService.update_study_associates(workflow.study_id,
+                                             [{'uid': 'dhf8r', 'role': 'Chief Bee Keeper', 'send_email': True, 'access': True},
+                                              {'uid': 'lb3dp', 'role': 'Chief Cat Herder', 'send_email': True, 'access': True}])
+
+        first_task = workflow_api.next_task
+
+        with mail.record_messages() as outbox:
+            self.complete_form(workflow, first_task, {'subject': 'My Test Subject', 'recipients': ['user@example.com', 'associated']})
+
+            self.assertEqual(1, len(outbox))
+            self.assertIn(outbox[0].recipients[0], ['user@example.com', 'dhf8r@virginia.edu', 'lb3dp@virginia.edu'])
+            self.assertIn(outbox[0].recipients[1], ['user@example.com', 'dhf8r@virginia.edu', 'lb3dp@virginia.edu'])
+            self.assertIn(outbox[0].recipients[2], ['user@example.com', 'dhf8r@virginia.edu', 'lb3dp@virginia.edu'])