338 Commits

Author SHA1 Message Date
jasquat
449fe87d43 Feature/deny macro on permission (#672)
* look for deny in the permission action rather than in the target uri and updated the documentation w/ burnettk

* added depecation warning if allowed_permissions is being used intead of actions w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-15 11:46:56 -05:00
jasquat
1d37001727 Feature/no data for finished spiff tasks (#661)
* do not add data to spiff tasks if that task is finished w/ burnettk

* build docker image for this branch w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-14 12:47:20 -08:00
jbirddog
c122926e01 Support creating unique top level keys for KKV data store (#613) 2023-11-13 11:37:24 -05:00
jasquat
39edd60fd6 Feature/hide private data objects (#581)
* added test and some additional support for deny permissions w/ burnettk

* added support for deny through permissions-check api w/ burnettk

* support DENY at the beginning of a permission target marcro

* do not look up permissions using grant type, only use the uniqueness key

* added support in frontend to display a nice error if user does not have access to a data object value w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-25 16:31:06 -04:00
jasquat
2f9aa12906 Feature/error boundary (#552)
* WIP for error boundary support w/ burnettk

* updates to catch error boundary events and send them back out w/ burnettk

* fixed broken test w/ burnettk

* use connector-http and spiffworkflow-proxy from main w/ burnettk

* updated smtp and slack connectors in connector-proxy-demo w/ burnettk

* added more tests for the service task service call connector to test errors w/ burnettk

* added spiffworkflow-connector-command and some code clean up

* updated the connectors in the proxy demo

* use SpiffWorkflow from git instead of locally

* some cleanup while code reviewing

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-18 14:00:12 -04:00
jbirddog
a80abc4c47 Keyed key/value data store implementation (#548) 2023-10-17 10:59:01 -04:00
jasquat
c2dc4a738e Feature/data migrator tests (#546)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* attempting to add tests for data migrator and fix 1.3 for postgres

* run version_1_3 migration differently from postgres versus mysql and sqlite

* look up the task model again to make sure it is fresh w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:29:51 -04:00
jasquat
4d842e8dbf Improvement/better serialization (#540)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* switch SpiffWorkflow back to main

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:14:02 -04:00
jasquat
8bf92f7a39 Feature/regex support in permissions (#530)
* support wildcards when adding users to groups from waiting table

* moved the user route to authentication_controller to avoid having so many user routes and this controller was all about login

* added test to ensure regexes work for permissions - still need to remove old ones on refresh

* moved token related code out of authorization service and into authentication service w/ burnettk

* remove old user group assignment waiting entries when refreshing permissions w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-06 13:47:40 -04:00
jasquat
cba4a19d3b Feature/remove group service (#529)
* removed group service in favor of user service and messing around with ruff and pre-commeit w/ burnettk

* pyl can succeed now w/ burnettk

* removed debug code w/ burnettk

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-05 13:27:38 -04:00
jbirddog
34a0323c4d Upsearch for data stores (#520) 2023-10-04 09:42:25 -04:00
Kevin Burnett
fa5109ff57 update permission check to hopefully improve performance (#523)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-10-03 13:00:13 -07:00
jasquat
5d4713fc0e feature/use-context-with-git-commands (#524)
* use the c option with git commands instead of using cd from python w/ burnettk

* removed the cd method since we should not be using it since it is not threadsafe

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-03 16:00:03 -04:00
jasquat
8bf38aaa1c Feature/new reference cache (#511)
* added basic model for new reference cache w/ burnettk

* switched out SpecReferenceCache for ReferenceCacheModel w/ burnettk jbirddog

* pyl w/ burnettk jbirddog

* save items to the db using the new cache with generation table w/ burnettk

* bulk save for performance

* tests are passing

* actually use the new generation table - we still need a test to ensure we are using it

* added test to ensure using new cache generation

* corrected reference interface on frontend w/ burnettk

* do not perform git pull in webhook if the revision is the same as the current w/ burnettk jbirddog

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-22 11:28:59 -04:00
jasquat
1826cc4b6c Improvement/flexible task iteration (#507)
* updated to use new spiff branch and fixed broken tests w/ burnettk essweine

* updated spiffworkflow with new main build w/ burnettk essweine

* initial updates to new spiff branch w/ burnettk essweine

* more updates for new spiff w/ burnettk essweine

* fixed some linting issues w/ burnettk essweine

* fixed some failing tests w/ burnettk

* updated spiffworkflow for cancel fix w/ burnettk

* Improvement/flexible task iteration 2 (#504)

* wip

* consistent failure, mostly

* removing test code and tests

* removing unused test bpmn files

* removing unused test bpmn files

* minor cleanup of commented code

* spaces and unused imports

* go back to spiff on main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>

* lint

* updated test to reflect storing predicted tasks w/ burnettk

* add some orders so postgres does not do whatever it wants, and clear log

* fix lint

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: danfunk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-21 14:47:18 -07:00
jasquat
f6d3bc8e73 Feature/api keys (#489)
* some initial work to support user api keys w/ burnettk

* some updates to store and use service accounts - migrations do not work in sqlite atm

* pyl

* minor tweak to the migration

* refactored user route

* this is working if returning user that created the service account

* put back migrations from main w/ burnettk

* tests pass with new migration w/ burnettk

* do not remove service account permissions on refresh_permissions w/ burnettk

* added new component to make some api calls to populate child components and routes w/ burnettk

* allow displaying extensions in configuration tab w/ burnettk

* removed service accounts controller in favor of extension and encrypt the api keys

* add fuzz to username to make deleting and recreating service accounts easier

* allow specifying the process id to use when running an extension w/ burnettk

* allow extensions to navigate to each other on form submit w/ burnettk

* removed commented out debug code

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-15 10:10:57 -04:00
jasquat
a72ed7ffb5 disable element units by default and added test that was failing due to it w/ burnettk (#480)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-08 13:47:31 -07:00
jasquat
912886f3e6 Improvement/task and spec relations (#477)
* updated to use new spiff branch and fixed broken tests w/ burnettk essweine

* updated spiffworkflow with new main build w/ burnettk essweine

* work around dictionary changed size during iteration error in parallel tests

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-08 12:54:43 -04:00
jasquat
9ea90a94bf updated the support user permissions to disallow authentications as well and updated webui to nav to auth page if auth is available but secrets are not w/ burnettk (#454)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-07 10:33:56 -04:00
Dan Funk
80ad92a0c3 Threads (#439)
* WIP - threading with dan

* WIP - cleanup with dan

* Fixing a minor bug and adding a comment and taking the win, even though it was all actually done by @jbirddog with help from @essweine

* adding a couple of simple tests

* * Threaded execution of multiple ready engine tasks is now the default behavior for all execution strategies by default (the skip-one strategry does not do it)
* Assure that tasks in the db match those in the spiffworkflow bpmn on save (remove all predicted and pruned tasks)

with @jbirddog, @jlantz, @burnettk

* run_pyl

---------

Co-authored-by: Jon Herron <jon.herron@yahoo.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-07 10:12:56 -04:00
jasquat
9bb9ce47f8 Feature/business end states (#333)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

* translated mysql last milestone query to sqlalchemy w/ burnettk

* fixed last milestone query so instances still return if no milestone found and moved some code from the main report method to own methods

* added some comments

* added last milestone column to process instances table

* display last milestone in instance list table w/ burnettk

* remove 3 characters when truncating last milestone for ellipsis

* make sure we have a current processor so we don't return null

* remove sleep

* The background processor now only picks up processes that were last updated more than a minute ago to avoid conflicting with the interstitial page.  With the understanding that we can rmeove this limitation when we can refactor to allow the backend processes to provide updates on what they are doing.

* pyl w/ burnettk

* cache last milestone on instances

* pyl

* added test for last milestone and added it to the proces instance show page w/ burnettk

* fixed broken test w/ burnettk

* fixed last milestone header

* removed duplicated column

* fixed broken test

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: Kevin Burnett <18027+burnettk@users.noreply.github.com>
Co-authored-by: danfunk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-07 10:10:44 -04:00
jasquat
d58391e21d added permissions for authentication endpoints 2023-08-24 08:37:01 -04:00
jasquat
29620cb17f Feature/add support permissions (#445)
* added support perm macro which removes secrets perms w/ burnettk

* support perm macro inherits from basic now and updated docs on permissions to be more accurate w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-22 08:53:26 +05:30
jasquat
17309fb7fd Feature/allow markdown in extension results (#435)
* allow markdown in extensions results w/ burnettk

* fixed tests

* moved our rjsf form to component so extensions can also use it w/ burnettk

* added ability to create extensions that can download files w/ burnettk

* added test for extensions-get-data endpoint w/ burnettk

* make user optional when getting process instance reports

* added extensions-get-data to elevated perm macro and raise an error if user is not specified when needed when running a report

* fixed typeguard test

* push extensions branch

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-08-21 21:59:25 +05:30
Dan Funk
a0202c6a5b Feature/onboarding ephemeral (#442)
* The onboarding controller should not save the process model to the database.  It creates a pile of pointless noise.  So just cleaning that up.

* run_pyl

* assure we can handle user tasks if they happen during on-boarding, while keeping the list of processes clean.
Lots of weird stuff getting run_pyl going.

* pyl

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-08-16 12:11:38 -04:00
Kevin Burnett
9313a9f73a Feature/event payloads part 2 (#401)
* Revert "Revert "Feature/event payloads (#393)""

This reverts commit 95fafb7af118cbe81ca20600bbb83e54e0936a5a.

* Revert "Revert "poet not available in container""

This reverts commit 140220498c284163dc02f8075fac949dff4de9e5.

* Revert "Revert "Run event payloads data migration from background processor (#399)""

This reverts commit 2afced3a51cda18491bc23b344bf2bada41393d5.

* Revert "Revert "using new spiff api to get info about events. w/ elizabeth""

This reverts commit af857fee229fc89824e45a5d36ab0178e284ed44.

* Revert "Revert "fix tests for waiting_event_can_be_skipped""

This reverts commit 886e6bd42a94390bf4d863ec79bff0a3831f6fcf.

* push image for preview env

* default scripts to localhost w/ burnettk

* use the bugfix/update-split-task-inputs spiffworkflow branch w/ burnettk

* removed debug json files

* use main for spiffworkflow

* do not attempt to highlight non-diagram boundary items w/ burnettk

* updated SpiffWorkflow to fix multiple signal event issue w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-10 18:24:49 +05:30
jasquat
1c84e5bb47 Feature/run error handler from dequeued (#424)
* run the error handler from the dequeued method instead of marking all failed process instances as error w/ burnettk

* do not commit some_cache_dir w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-02 14:05:11 +05:30
jasquat
796dc9dbec Feature/extensions api (#423)
* added api and test to run a process model and return task data synchronously w/ burnettk

* added test to make sure we can access db models from extensions api w/ burnettk

* added extensions api to elevated permissions

* fixed permission tests

* do not add extensions permission to all permission for a pg or pm w/ burnettk

* added configs for extensions api w/ burnettk

* added the basis for an extensions list api w/ burnettk

* added tests for extenstions api and do not use serialized as a property

* allow giving a body to an extension when running and some support in frontend to use extensions

* added ability to display markdown and rjsf on extensions page

* added ability to submit the extension form and display the resulting task data

* made frontend extension urls have 2 pieces of information so we can specify multiple routes for the same process-model w/ burnettk

* do not save process instances when running extensions w/ burnettk

* add extension input to a task not the process w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-08-02 14:02:53 +05:30
jbirddog
43beb916a3 Basic scaffolding for onboarding (#400)
* Wedge between InProgressInstances for default view customization

* Rename to OnboardingView, stubbed out api in the backend

* Flip between InProgressInstances and MyTasks via the backend

* WIP

* WIP

* Basic human task handling

* FE lint

* Getting ./bin/pyl to pass

* Suppress any exceptions during onboarding request

* Script to skip onboarding if already started

* Getting ./bin/pyl to pass

* Better default location

* PR feedback

* PR feedback

* PR feedback - add new endpoint to basic permissions

* Fix basic permissions test

* Add integration tests

* Getting bin_pyl to pass
2023-07-27 11:54:30 +05:30
burnettk
886e6bd42a Revert "fix tests for waiting_event_can_be_skipped"
This reverts commit 2900a9a5f82bb7a0a2b58b59b12c7b4e2e2f5452.
2023-07-19 11:43:17 -04:00
burnettk
2900a9a5f8 fix tests for waiting_event_can_be_skipped 2023-07-18 17:07:14 -04:00
Dan Funk
73aed82f24 Feature/view data stores (#388)
* * Added /data-stores/ endpoint accessible by privileged users that will return a list of all data stores known to the system.
* Added a /data-stores/[type]/[name] endpoint that will return a list of all data stored within a data-store.
* Granted users with "elevated permissions" the right to access the data store.
* Added a "Data Store" link to the navigation bar beside messages.
* And a few useful tests.
* Still a little front end work to do to get it all looking pretty.

* Added a Data Store List component that, well, displays, you guessed it! A list of data stores and their contents.

Also, Carbon's paginator doesn't care how many pages you have, it's going to build the mother of all drop down lists - you got 60,000,000 records? Showing 5 at a time? It's going to be a dropdown list that contains all numbers between 1 and 12,000,000, because that makes sense! So, yea, not doing that, cutting it off at 1000 pages - you got more pages than that, the paginator can't take you there.  As you can show 100 items per page, that means you can access 100,000 items instantly.

* renaming data_store_items_list => data_store_item_list
2023-07-13 20:54:10 +05:30
jasquat
a7d0fbb38c Feature/sanitize jinja template (#389)
* added a helpers class that can be used in jinja templates

* added jinja helpers to script task imports

* added some comments for jinja helper usage

* added tests for jinja helpers

* pyl

* pass jinja helper mappings in so helpers can be used consistently

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-13 20:35:49 +05:30
jasquat
285294967d Feature/error to suspended state (#384)
* fixed suspended message w/ burnettk

* allow manually executing errored tasks when suspended w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-12 19:50:24 +05:30
jasquat
841f3ccc8c Feature/task assignment (#352)
* added an api to assign a list of users to a task w/ burnettk

* use the modal submit and close buttons when saving task data on the instance show page w/ burnettk

* switch save and cancel buttons on secrets new page w/ burnettk

* add some icons, tho still missing event stuff

* finished adding imporoved icons and fixing up task modal w/ burnettk

* added some user search options to assig tasks to w/ burnettk

* cleaned up task details modal and added call to backend to add potential users w/ burnettk

* fixed broken tests w/ burnettk

* removed some merge comments w/ burnettk

* process instance id is an int not a str w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-12 19:44:01 +05:30
jasquat
25eb8b30f1 give user access to pi for-me with all perm to process group w/ burnettk (#371)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-05 14:47:15 -04:00
jasquat
d810d9b01f fixed up the unit test permission yml to better match the way we do permissions (#363)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-29 14:21:15 -04:00
jasquat
f80c52ba38 update basic permission set for callers to allow for processes to be passed in (#362)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-29 19:05:18 +05:30
jasquat
3a8cfd2642 disable form submit buttons when appropriate, lock process instance when sending events, and ensure return events match ones associated with desired guids w/ burnettk (#359)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-29 09:36:47 +05:30
jasquat
51cfcfa2a5 Feature/autosave and null form data (#346)
* autosave form data on change w/ burnettk

* tests are now passing and pyl

* do not do anything on autosave success w/ burnettk

* merged autosave and null form data branches together w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-26 18:26:56 +05:30
jbirddog
2c99e17381 Show callers for non primary files (#334)
* WIP - working with multiple process ids, need to return a file's process ids

* Send bpmn process ids back with bpmn files

* Getting ./bin/pyl to pass

* Fix up unit test

* Wire up the front end

* Pre pr cleanup
2023-06-23 11:45:23 +05:30
Kevin Burnett
7de0d97303 Feature/better subworkflow management part 2 (#341)
* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf00241201c515b54444fe9fb373c7f4.

* updated SpiffWorkflow to fix infinite loop in task trace w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-20 10:37:49 -04:00
burnettk
48dcde8faf Revert "Feature/better subworkflow management (#331)"
This reverts commit 23af5cbbe4015f8bea28fb2b26bbd0c04bfa04e1.
2023-06-15 15:10:48 -04:00
jasquat
23af5cbbe4 Feature/better subworkflow management (#331)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-15 10:31:42 -04:00
burnettk
0848c64299 if lane owner name is an actual group, assume that we want tasks assigned to group 2023-06-13 21:59:17 -04:00
jasquat
a14072c0b6 added remaining path segment all perms to elevated options (#326)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-12 08:05:33 -07:00
jasquat
121b3c7cc9 some fixes and updates to help with running an acceptance test model (#323)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-09 11:19:02 -07:00
jasquat
4b2970170c updated permission macros to give admins access to task-data and proc… (#314)
* updated permission macros to give admins access to task-data and process-data w/ burnettk

* do not check for write to process-models if diagram is readonly anyway w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-08 11:39:14 -04:00
jbirddog
27447e533c Fix issue when timer start event is greater than a day (#303)
* Fix issue when start time is greater than a day

* Add some cycle timer tests

* Add some more tests

* Getting ./bin/pyl to pass
2023-06-06 18:25:26 -07:00
jasquat
2f4e824863 synced unit tests with sample and added support for subprocesses for bpmn unit tests w/ burnettk 2023-06-02 11:20:41 -04:00