1108 Commits

Author SHA1 Message Date
burnettk
ab631aab99 lint 2023-02-23 15:01:45 -05:00
burnettk
973be69df5 add test users 2023-02-23 14:58:14 -05:00
Dan
8466d823c2 Adding Migration. 2023-02-23 14:42:00 -05:00
jasquat
49fb41eff9 remove task size check since it can take a long time to run and we do not do anything with it w/ burnettk jbirddog 2023-02-23 14:41:45 -05:00
Dan
7c12dffe41 Merge remote-tracking branch 'origin/main' into feature/message_fixes 2023-02-23 14:26:04 -05:00
Dan
0030a46938 run_pyl 2023-02-23 14:17:22 -05:00
Dan
0f3ef00d72 BPMN.io -- Just show the message names not the ids - to assure we are only exposing the names.
SpiffWorkflow -
    - start_messages function should return message names, not ids.
    - don't catch external thrown messages within the same workflow process
    - add an expected value to the Correlation Property Model so we can use this well defined class as an external communication tool (rather than building an arbitrary dictionary)
    - Added a "get_awaiting_correlations" to an event, so we can get a list of the correlation properties related to the workflows currently defined correlation values.
    - workflows.waiting_events() function now returns the above awaiting correlations as the value on returned message events
 Backend
    - Dropping MessageModel and MessageCorrelationProperties - at least for now.  We don't need them to send / receive messages though we may eventually want to track the messages and correlations defined across the system - these things (which are ever changing) should not be directly connected to the Messages which may be in flux - and the cross relationships between the tables could cause unexpected and unceissary errors.  Commented out the caching logic so we can turn this back on later.
    - Slight improvement to API Errors
    - MessageInstances are no longer in a many-to-many relationship with Correlations - Each message instance has a unique set of message correlations specific to the instance.
    - Message Instances have users, and can be linked through a "counterpart_id" so you can see what send is connected to what recieve.
    - Message Correlations are connected to  recieving message instances.  It is not to a process instance, and not to a message model.  They now include the expected value and retrieval expression required to validate an incoming message.
    - A process instance is not connected to message correlations.
    - Message Instances are not always tied to a process instance (for example, a Send Message from an API)
    - API calls to create a message use the same logic as all other message catching code.
    - Make use of the new waiting_events() method to check for any new recieve messages in the workflow (much easier than
    churning through all of the tasks)
    - One giant mother of a migration.
2023-02-23 13:53:03 -05:00
jasquat
216bd799bf do not call serialize if we can use the cached bpmn_json instead w/ burnettk 2023-02-23 13:48:27 -05:00
jasquat
7dd72c72e1 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-23 11:19:26 -05:00
jasquat
2bf31fc135 turn on sentry detailed tracing for task-data w/ burnettk 2023-02-23 11:19:22 -05:00
burnettk
55f3b765ee update spiff 2023-02-23 11:02:40 -05:00
Kevin Burnett
c65aeafbe3 Merge pull request #145 from sartography/data_size_script
Data size script
2023-02-22 17:36:31 -08:00
jasquat
44f31ffc36 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-22 15:35:02 -05:00
jasquat
a3126da7ef set git user configs explicitly using the env vars w/ burnettk 2023-02-22 15:34:47 -05:00
burnettk
1a29e123c1 so we do not lose our minds when debugging on a server 2023-02-22 15:16:13 -05:00
jasquat
9b14510cd1 Merge branch 'main' of github.com:sartography/spiff-arena 2023-02-22 14:42:46 -05:00
jasquat
c6b45dbdc3 show help text for textareas as well w/ burnettk 2023-02-22 14:42:43 -05:00
burnettk
9fb490c7f9 why not dict, too 2023-02-22 14:36:26 -05:00
Jon Herron
7fe8955adf Cleanup 2023-02-22 13:42:45 -05:00
Jon Herron
7f2d69163e Getting ./bin/pyl to pass 2023-02-22 13:34:26 -05:00
Jon Herron
488abc03ea Provide info about keys in task data and python env 2023-02-22 13:13:28 -05:00
Jon Herron
2afc19175e Unfactor to fix size calculation/type hint issues 2023-02-22 11:17:23 -05:00
Jon Herron
a4c0dd52ae Getting ./bin/pyl to pass 2023-02-22 11:04:07 -05:00
Jon Herron
d5fa8eae21 Adding python env size 2023-02-22 10:55:09 -05:00
Jon Herron
2fa4623876 Add cumulative task data size to script 2023-02-22 10:39:35 -05:00
burnettk
bb6dd35bbd remove start and end events from simple logs view 2023-02-21 22:43:23 -05:00
burnettk
54248ca568 copy env so we are doing additive stuff rather than completely rewriting it 2023-02-21 22:00:02 -05:00
jasquat
012f5c20aa fix git bash script unbound error w/ burnettk 2023-02-21 17:08:09 -05:00
jasquat
90de749ab9 more git config updates w/ burnettk 2023-02-21 16:43:52 -05:00
jasquat
c5d611f03b disable strict host checking for git commands w/ burnettk 2023-02-21 16:42:11 -05:00
jasquat
e5b83d0919 fixed tests w/ burnettk 2023-02-21 15:28:54 -05:00
jasquat
b0d132fc1e give access to download process data files if a user has access to start a process model w/ burnettk 2023-02-21 15:08:19 -05:00
819415d487 backend: create SSH key file when contents provided
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 20:02:16 +01:00
56a913e33c backend: use sensible lock filename for git
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 19:58:39 +01:00
1bbf3a264c backend: specify --rebase when using git pull
Otherwise it fails with:
```
Pulling without specifying how to reconcile divergent branches is discouraged.
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 19:58:38 +01:00
1e8e6a7968 backend: fix use of SSH private key for git ops
Primarily this is supposed to fix the `git pull` aciton triggered by
webhooks from GitHub. But in addition to that the point is to simplify
that committing wrapper which has far too much in it.

Instead of passing everything as CLI arguments one should make use of
already supported environment variables and the `env` argument to
`subprocess` functions like `run()`. Writing extra logic in the wrapper
only makes it unnecessarily complicated.

By passing both user, email, and the SSH options in `run_shell_command`
we avoid the need to repeat the same boilerplate to provide Git config
and SSH credentials.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-21 19:58:36 +01:00
jbirddog
01a95011ad Look in task data/script engine environment for data, not just workflow data. (#143) 2023-02-20 18:12:50 -05:00
Dan
790483a421 run_pyl 2023-02-20 12:34:42 -05:00
Dan
5f6a61c93f * SpiffWorkflow event_definitions wanted to return a message event's correlation properties mested within correlation keys. But messages are directly related to properties, not to keys - and it forced a number of conversions that made for tricky code. So Messages now contain a dictionary of correlation properties only.
* SpiffWorkflow did not serialize correlations - so they were lost between save and retrieve.

* When comparing Correlation Property values - we are storing these values as strings in the database and can't convert them back to integers later, so I'm changing everying everywhere to compare after conversion to a string.  Don't feel great about this one.
* By using an SQL Alchemy join table, there is a lot of db queries we don't need to write.
* A few handy fucntions on db models to make it easier to work with correlations.
* Updated tests because I changed some of the BPMN models we were testing against.
* Database migration to use the new constraint names with the alternate form of the join table between correlation mesages to instance messages.
2023-02-20 11:50:35 -05:00
burnettk
a600736e67 delint 2023-02-20 10:58:04 -05:00
fb024a49f7 backend/git_service: accept webhook test requests
https://docs.github.com/en/webhooks-and-events/webhooks/testing-webhooks

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-20 13:11:21 +01:00
e47c0752db backend/git_service: check repo URLs from webhook
Since we are cloning a private repo we are using `ssh_url` in our case.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-20 13:02:54 +01:00
Dan
d46502cd2b Allow people to run commands like "flask db upgrade" without setting specific environment variables like FLASK_SESSION_SECRET_KEY everytime - they just need to add in their own /instance/config.py with their local configuration. 2023-02-19 10:37:00 -05:00
Dan
2a800e844f * Re-work message tests so I could wrap my simple head around what was happening - just needed an example that made sense to me.
* Clear out complex get_message_instance_receive how that many-to-many works.
* Create decent error messages when correlations fail
* Move correlation checks into the MessageInstance class
* The APIError could bomb out ugly if it hit a workflow exception with not Task Spec.
2023-02-18 13:09:58 -05:00
Dan
f2ca47e798 bump spiffworkflow hash 2023-02-17 13:08:45 -05:00
Dan
ecbd30af00 failing test. 2023-02-17 12:10:32 -05:00
Dan
ef4297331d Merge branch 'main' into feature/message_fixes 2023-02-17 12:01:38 -05:00
Dan
8e257600af work in progress -
* Link between message instance and correlations is now a link table and many-to-many relationships as recommended by SQLAlchemy
* Use the correlation keys, not the process id when accepting api messages.
2023-02-17 10:45:01 -05:00
jasquat
b22c33e1d6 put the env vars in the env section of the github action configs w/ burnettk 2023-02-16 15:19:01 -05:00
jasquat
0839214fa5 added test for quickstart guide w/ burnettk 2023-02-16 15:05:55 -05:00