sartography
/
spiff-arena
mirror of https://github.com/sartography/spiff-arena.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,828 Commits 45 Branches 61 Tags 129 MiB
Commit Graph

1004 Commits

Author SHA1 Message Date
Dan 7c12dffe41 Merge remote-tracking branch 'origin/main' into feature/message_fixes 2023-02-23 14:26:04 -05:00
Dan 0030a46938 run_pyl 2023-02-23 14:17:22 -05:00
Dan 0f3ef00d72 BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names. 
SpiffWorkflow -
    - start_messages function should return message names, not ids.
    - don't catch external thrown messages within the same workflow process
    - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)
    - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.
    - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events
 Backend
    - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.
    - Slight improvement to API Errors
    - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.
    - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.
    - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.
    - A process instance is not connected to message correlations.
    - Message Instances are not always tied to a process instance (for example, a Send Message from an API)
    - API calls to create a message use the same logic as all other message catching code.
    - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than
    churning through all of the tasks)
    - One giant mother of a migration.
 2023-02-23 13:53:03 -05:00
jasquat 216bd799bf do not call serialize if we can use the cached bpmn_json instead w/ burnettk 2023-02-23 13:48:27 -05:00
jasquat 7dd72c72e1 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-23 11:19:26 -05:00
jasquat 2bf31fc135 turn on sentry detailed tracing for task-data w/ burnettk 2023-02-23 11:19:22 -05:00
burnettk 55f3b765ee update spiff 2023-02-23 11:02:40 -05:00
Kevin Burnett c65aeafbe3 Merge pull request #145 from sartography/data_size_script 
Data size script
 2023-02-22 17:36:31 -08:00
jasquat 44f31ffc36 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-22 15:35:02 -05:00
jasquat a3126da7ef set git user configs explicitly using the env vars w/ burnettk 2023-02-22 15:34:47 -05:00
burnettk 1a29e123c1 so we do not lose our minds when debugging on a server 2023-02-22 15:16:13 -05:00
jasquat 9b14510cd1 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-22 14:42:46 -05:00
jasquat c6b45dbdc3 show help text for textareas as well w/ burnettk 2023-02-22 14:42:43 -05:00
burnettk 9fb490c7f9 why not dict, too 2023-02-22 14:36:26 -05:00
Jon Herron 7fe8955adf Cleanup 2023-02-22 13:42:45 -05:00
Jon Herron 7f2d69163e Getting ./bin/pyl to pass 2023-02-22 13:34:26 -05:00
Jon Herron 488abc03ea Provide info about keys in task data and python env 2023-02-22 13:13:28 -05:00
Jon Herron 2afc19175e Unfactor to fix size calculation/type hint issues 2023-02-22 11:17:23 -05:00
Jon Herron a4c0dd52ae Getting ./bin/pyl to pass 2023-02-22 11:04:07 -05:00
Jon Herron d5fa8eae21 Adding python env size 2023-02-22 10:55:09 -05:00
Jon Herron 2fa4623876 Add cumulative task data size to script 2023-02-22 10:39:35 -05:00
burnettk bb6dd35bbd remove start and end events from simple logs view 2023-02-21 22:43:23 -05:00
burnettk 54248ca568 copy env so we are doing additive stuff rather than completely rewriting it 2023-02-21 22:00:02 -05:00
jasquat 012f5c20aa fix git bash script unbound error w/ burnettk 2023-02-21 17:08:09 -05:00
jasquat 90de749ab9 more git config updates w/ burnettk 2023-02-21 16:43:52 -05:00
jasquat c5d611f03b disable strict host checking for git commands w/ burnettk 2023-02-21 16:42:11 -05:00
jasquat e5b83d0919 fixed tests w/ burnettk 2023-02-21 15:28:54 -05:00
jasquat b0d132fc1e give access to download process data files if a user has access to start a process model w/ burnettk 2023-02-21 15:08:19 -05:00
Jakub Sokołowski 819415d487 backend: create SSH key file when contents provided 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-02-21 20:02:16 +01:00
Jakub Sokołowski 56a913e33c backend: use sensible lock filename for git 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-02-21 19:58:39 +01:00
Jakub Sokołowski 1bbf3a264c backend: specify --rebase when using git pull 
Otherwise it fails with:
```
Pulling without specifying how to reconcile divergent branches is discouraged.
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-02-21 19:58:38 +01:00
Jakub Sokołowski 1e8e6a7968 backend: fix use of SSH private key for git ops 
Primarily this is supposed to fix the `git pull` aciton triggered by
webhooks from GitHub. But in addition to that the point is to simplify
that committing wrapper which has far too much in it.

Instead of passing everything as CLI arguments one should make use of
already supported environment variables and the `env` argument to
`subprocess` functions like `run()`. Writing extra logic in the wrapper
only makes it unnecessarily complicated.

By passing both user, email, and the SSH options in `run_shell_command`
we avoid the need to repeat the same boilerplate to provide Git config
and SSH credentials.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-02-21 19:58:36 +01:00
jbirddog 01a95011ad Look in task data/script engine environment for data, not just workflow data. (#143) 2023-02-20 18:12:50 -05:00
Dan 790483a421 run_pyl 2023-02-20 12:34:42 -05:00
Dan 5f6a61c93f * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys. But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code. So Messages now contain a dictionary of correlation properties only. 
* SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.

* When comparing Correlation Property values - we are storing these values as strings in the database and can't convert them back to integers later, so I'm changing everying everywhere to compare after conversion to a string.  Don't feel great about this one.
* By using an SQL Alchemy join table, there is a lot of db queries we don't need to write.
* A few handy fucntions on db models to make it easier to work with correlations.
* Updated tests because I changed some of the BPMN models we were testing against.
* Database migration to use the new constraint names with the alternate form of the join table between correlation mesages to instance messages.
 2023-02-20 11:50:35 -05:00
burnettk a600736e67 delint 2023-02-20 10:58:04 -05:00
Jakub Sokołowski fb024a49f7 backend/git_service: accept webhook test requests 
https://docs.github.com/en/webhooks-and-events/webhooks/testing-webhooks

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-02-20 13:11:21 +01:00
Jakub Sokołowski e47c0752db backend/git_service: check repo URLs from webhook 
Since we are cloning a private repo we are using `ssh_url` in our case.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-02-20 13:02:54 +01:00
Dan d46502cd2b Allow people to run commands like "flask db upgrade" without setting specific environment variables like FLASK_SESSION_SECRET_KEY everytime - they just need to add in their own /instance/config.py with their local configuration. 2023-02-19 10:37:00 -05:00
Dan 2a800e844f * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me. 
* Clear out complex get_message_instance_receive how that many-to-many works.
* Create decent error messages when correlations fail
* Move correlation checks into the MessageInstance class
* The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.
 2023-02-18 13:09:58 -05:00
Dan f2ca47e798 bump spiffworkflow hash 2023-02-17 13:08:45 -05:00
Dan ecbd30af00 failing test. 2023-02-17 12:10:32 -05:00
Dan ef4297331d Merge branch 'main' into feature/message_fixes 2023-02-17 12:01:38 -05:00
Dan 8e257600af work in progress - 
* Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy
* Use the correlation keys, not the process id when accepting api messages.
 2023-02-17 10:45:01 -05:00
jasquat b22c33e1d6 put the env vars in the env section of the github action configs w/ burnettk 2023-02-16 15:19:01 -05:00
jasquat 0839214fa5 added test for quickstart guide w/ burnettk 2023-02-16 15:05:55 -05:00
jasquat 435dc147df pyl 2023-02-16 14:38:19 -05:00
jasquat f5a5059ce0 Merge remote-tracking branch 'origin/main' into feature/support_local_openid_in_cypress_tests 2023-02-16 14:25:51 -05:00
burnettk 9f3bb283b3 use _GIT_SOURCE_BRANCH instead of _GIT_BRANCH 2023-02-16 14:10:31 -05:00
burnettk 888547642e fix unprefixed config 2023-02-16 13:56:48 -05:00
burnettk 5918501083 update configs for publish feature 2023-02-16 13:47:26 -05:00
jasquat a98892be32 added some support for using the backend openid server for cypress tests w/ burnettk 2023-02-16 12:09:44 -05:00
burnettk 76a701e9dd add new uses and delete ppg.ba.sme1 2023-02-16 11:53:08 -05:00
jasquat 39763c8c79 the equal sign is not apart of the env var w/ burnettk 2023-02-16 11:18:50 -05:00
jasquat cc17f31958 updated terraform deployed env config name w/ burnettk 2023-02-16 10:51:05 -05:00
jasquat 641d237e83 more config updates w/ burnettk 2023-02-16 10:21:02 -05:00
jasquat 97f23e6d4f cleaned up the default config file a bit 2023-02-16 09:05:40 -05:00
jasquat 6a6c17d71e pyl 2023-02-16 08:33:39 -05:00
jasquat c9c1ae4c16 fixed some env vars for ci 2023-02-16 07:59:51 -05:00
jasquat 5707e2c4e5 pyl 2023-02-16 07:39:40 -05:00
jasquat 7689380411 Merge remote-tracking branch 'origin/main' into feature/update_configs_with_prefix 2023-02-16 07:17:39 -05:00
burnettk 1650486a32 remove unused import 2023-02-16 06:34:03 -05:00
burnettk 01668368e7 all tests pass w/ spiff lib upgrade, thanks for the help elizabeth 2023-02-15 22:56:12 -05:00
burnettk 3cfd81efa6 format 2023-02-15 22:35:55 -05:00
jasquat 26875f13bf updated remaining configs manually w/ burnettk 2023-02-15 17:21:37 -05:00
jasquat 9b93504945 updated secret config names as well w/ burnettk 2023-02-15 17:17:47 -05:00
jasquat 3e4fc1259f updated configs to use the prefix using the script w/ burnettk 2023-02-15 17:07:12 -05:00
jasquat 909060e985 updated default user group var name for better prefix w/ burnettk 2023-02-15 16:43:40 -05:00
jasquat 09f3921236 save task data to spiff step details when copmleting a user task as well w/ burnettk jbirddog 2023-02-15 15:34:51 -05:00
jasquat 5a2e810fd6 append OPEN_ID to TENANT_SPECIFIC_FIELDS for clarification on what it is from w/ burnettk 2023-02-15 11:12:30 -05:00
jasquat 7b96335221 do not remove the default user group when refreshing permissions w/ burnettk 2023-02-15 11:09:04 -05:00
Dan 759b2bffa5 Merge branch 'main' of github.com:sartography/spiff-arena into main 2023-02-14 19:39:46 -05:00
Dan c8a83f80b6 The aud in a json web token can be a string or an array -- so also deal with the case that it is an array 2023-02-14 19:39:35 -05:00
jasquat 84f27082d4 updates to user import scripts w/ burnettk 2023-02-14 17:39:42 -05:00
jasquat 4dd8896de7 commit user if it has changed any attributes w/ burnettk 2023-02-14 15:08:25 -05:00
jasquat 4d81907966 fix for mypy on python 3.9 w/ burnettk 2023-02-14 11:34:45 -05:00
jasquat 91bd2d0710 give some leeway to iat when validating auth w/ burnettk 2023-02-14 10:51:11 -05:00
jasquat 35f8e6bfde add in debug logging when failing to login to help debug auth failures 2023-02-14 09:59:29 -05:00
jbirddog 646d803be0 Don't try to complete a ready task to move the process instance diagram to the next yellow. (#136) 2023-02-13 18:52:06 -05:00
Dan Funk 6621a7979b Merge pull request #137 from sartography/failing_test 
Check the End Event to fix the failing test
 2023-02-13 17:04:32 -05:00
jasquat 8b7a408f21 allow overriding keycloak url in add user script w/ burnettk 2023-02-13 16:16:43 -05:00
jasquat c9cfe8cd0b updated add test users script to use realm admin w/ burnettk 2023-02-13 16:04:57 -05:00
Jon Herron 491d30b9b3 The End Event has a spiff step 2023-02-13 14:13:34 -05:00
jasquat 683c1ee2c1 added script to import users w/ burnettk 2023-02-13 12:15:00 -05:00
jasquat 098aa69216 added api endpoint to allow logging in for the first time with an openid access_token w/ burnettk 2023-02-13 11:57:31 -05:00
Dan Funk 4caa880603 Merge pull request #135 from sartography/bug/len 
This should fix the len issue - though there may be better ways to fix
 2023-02-13 10:38:32 -05:00
Dan 6373761a4c run_pyl 2023-02-13 10:38:21 -05:00
burnettk bee3872842 delete 15 users that are no longer needed 2023-02-13 10:36:03 -05:00
Dan af8caeb985 Merge remote-tracking branch 'origin/main' into bug/len 2023-02-13 10:35:20 -05:00
burnettk 0e612e53f9 commit before removing users 2023-02-13 10:30:07 -05:00
Dan aaa402579f This should fix the len issue - though there may be better ways to fix 
this.
This reverts commit 52b496d105.
 2023-02-13 10:13:51 -05:00
Dan 52b496d105 Revert "more finally for better len" 
This reverts commit feb25c286d.
 2023-02-13 10:12:31 -05:00
Dan feb25c286d more finally for better len 
added a fixme for raising errors
fixed up an error message
 2023-02-13 10:10:23 -05:00
burnettk 45ea3b76d9 fix employee id 2023-02-12 23:43:56 -05:00
Dan 76bd40cb09 Update Spiffworkflow so we have the better error messages on service tasks. 2023-02-12 16:19:40 -05:00
Dan 014d24fbda don't default to a broken environment. 2023-02-12 15:00:17 -05:00
burnettk e18a0702d4 add five test users, rename peopleops users, delete two 2023-02-12 13:35:09 -05:00
Dan 0d0e26281d Remove version from docker-compose, as this is causing issues for people still using docker-compose v2. 2023-02-12 12:24:25 -05:00
burnettk 47ee1f73cc turn off profiling for now to see if that is what is causing segfaults 2023-02-10 09:52:23 -05:00
jbirddog ed52f07251 Save logs in finally (#134) 2023-02-10 09:18:04 -05:00
jasquat 1748a0ae7f link to the spiff step from a task on the frontend and use the correct db in ci 2023-02-09 17:07:36 -05:00
jasquat cd41be4455 pyl w/ burnettk 2023-02-09 16:03:26 -05:00
jasquat 0c30751946 call activities are also working w/ burnettk 2023-02-09 15:52:00 -05:00
jasquat 012d2bd367 tasks and subprocesses can are setting the task states properly now when getting task data w/ burnettk jbirddog 2023-02-09 15:29:45 -05:00
jasquat 7347c73d6a pyl w/ burnettk 2023-02-09 10:28:36 -05:00
Jon Herron 70f5836a2d Merge branch 'save_step_data' of github.com:sartography/spiff-arena into save_step_data 2023-02-09 09:45:50 -05:00
Jon Herron a5dc669509 Fix bug where deletes from the environment were not removed from the task data as well 2023-02-09 09:45:31 -05:00
jasquat f0de4959ae Merge remote-tracking branch 'origin/main' into save_step_data 2023-02-09 09:37:03 -05:00
Dan Funk 3ba9827670 Merge pull request #132 from sartography/feature/no_more_current_user 
Feature/no more current user
 2023-02-08 18:23:25 -05:00
jasquat df5451685f updated controller to use spiff step details to find correct task data to show w/ burnettk 2023-02-08 17:31:20 -05:00
Dan 18c481e3d4 run_pyl 2023-02-08 17:10:57 -05:00
Dan eb39b61488 remove the "current_user" from being added to the task_data. 2023-02-08 15:53:14 -05:00
jasquat 56249ac68a Merge remote-tracking branch 'origin/main' into save_step_data 2023-02-08 14:08:27 -05:00
jasquat c7b132ffc5 Merge remote-tracking branch 'origin/main' into save_step_data 2023-02-08 14:02:24 -05:00
burnettk d5a8475be6 pyl 2023-02-08 14:02:17 -05:00
Dan Funk a6cd94d011 Merge pull request #131 from sartography/feature/improved_service_task_errors 
Feature/improved service task errors
 2023-02-08 13:38:51 -05:00
Dan cbbaab482a If you have an instance/config.py to override local development it would also override testing. 2023-02-08 13:37:56 -05:00
Dan b5cd11ef37 Use a mock when making external calls in tests. 2023-02-08 13:14:42 -05:00
Dan b1d4eee781 Merge remote-tracking branch 'origin/main' into feature/improved_service_task_errors 2023-02-08 13:11:53 -05:00
burnettk 1921d0d174 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-08 12:51:52 -05:00
burnettk f9de69e511 catch up with environment change 2023-02-08 12:51:48 -05:00
jasquat 1d4a7a9b94 added additional columns to spiff_step_details w/ burnettk jbirddog 2023-02-08 12:43:30 -05:00
Dan f46909ecfb fixing some missing types 2023-02-08 12:27:55 -05:00
Dan c270ababd9 run_pyl 2023-02-08 12:00:27 -05:00
Dan 443a855782 Assure that when something goes wrong calling a service task that we get as much good information about the problem as possible. 2023-02-08 11:53:20 -05:00
jasquat 2dc1e00d69 Merge remote-tracking branch 'origin/main' into save_step_data 2023-02-08 11:47:11 -05:00
jasquat 9b3a2e2fb2 enable faulthandler to hopefully see seg faults in the logs w/ burnettk jbirddog 2023-02-08 11:42:11 -05:00
Jon Herron a0e36854bf Merge branch 'main' into save_step_data 2023-02-08 10:14:07 -05:00
burnettk 368330f561 remove dup 2023-02-08 09:52:40 -05:00
burnettk 81a379a2e9 add users 2023-02-08 08:04:30 -05:00
jasquat 4bf191af31 Merge pull request #114 from sartography/frontend/use-api-subpath 
frontend: use /api subpath instead of subdomain
 2023-02-07 17:20:45 -05:00
jasquat ec283a2558 allow setting configs for the frontend through env vars w/ burnettk 2023-02-07 17:17:45 -05:00
jasquat 0162821b4c allow configuring gunicorn configs in boot_server_in_docker w/ burnettk 2023-02-07 15:38:29 -05:00
burnettk a644a5b447 sentences need spaces between them 2023-02-07 15:09:49 -05:00
jasquat 559d9a78d8 renamed development env to local_development and testing to unit_testing w/ burnettk 2023-02-07 15:02:47 -05:00
jasquat 80555872ce set up qa2 to only do path based routing w/ burnettk 2023-02-07 14:30:13 -05:00
jasquat c659cc86fc added backend url to qa2 configs 2023-02-07 13:46:54 -05:00
jasquat 884f8bb872 added qa2 configs to use keycloak on qa1 to test different domains w/ burnettk 2023-02-07 11:30:18 -05:00
Dan Funk 462008da5e Merge pull request #128 from sartography/feature/bug_fixes 
Feature/bug fixes
 2023-02-07 08:20:16 -05:00
Kevin Burnett 721e86f351 Update tasks_controller.py 2023-02-06 16:21:55 -08:00
burnettk c74719d13d update get_token 2023-02-06 17:55:07 -05:00
jasquat 9d8dcc20f1 pass tenant attributes when creating a user w/ burnettk 2023-02-06 17:34:55 -05:00
jasquat 0fa91dc761 pyl w/ burnettk 2023-02-06 16:26:42 -05:00
jasquat d1bf3b69b9 Merge remote-tracking branch 'origin/main' into feature/tenant-specific-fields-from-openid 2023-02-06 16:24:59 -05:00
jasquat ad503ae2cd added bambooid to status users w/ burnettk 2023-02-06 16:24:48 -05:00
Jon Herron 7070503239 POC for saving some data about each step 2023-02-06 15:59:26 -05:00
jasquat b42fa26f14 allow added custom attributes to users when adding to keycloak w/ burnettk 2023-02-06 15:34:55 -05:00
jbirddog f728fc64d5 Backend do_engine_steps performance improvements (#129) 
Co-authored-by: Dan <daniel.h.funk@gmail.com>
 2023-02-06 15:25:49 -05:00
jasquat 5452b48d08 use our json encoder to dump the user for get_current_user script w/ burnettk 2023-02-06 14:11:15 -05:00
Dan a2f91999e2 Use the same markdown library for displaying as for editing - could enable a security run_pyl 2023-02-06 12:20:29 -05:00