Commit Graph

362 Commits

Author SHA1 Message Date
jasquat 4ee2a289c3
delete message triggerable processes if they no longer exist within the corresponding file w/ burnettk (#1070)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-20 07:44:02 -08:00
jasquat 27ea21b81a
create process groups json file at root if it does not exist when creating a data store w/ burnettk (#1056)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-16 15:20:16 -05:00
jasquat cc3d10f340
Feature/generic webhook (#1020)
* added api endpoint for a generic webhook w/ burnettk

* added the start of test for testing webhooks w/ burnettk

* the initial test for webhooks is now working w/ burnettk

* added test to prove we can run a message send from a non-persistent process instance w/ burnettk

* pyl w/ burnettk

* updated connector-http for patch command w/ burnettk

* make the webhook persistent so the message instance can be created w/ burnettk

* make sure we commit the message instance to the db in the webhook code w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-12 13:44:52 -05:00
jasquat 4b2fbd8938
when starting a process model from a message start event cancel all start events that do not match w/ burnettk (#1005)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-08 14:55:35 -05:00
jbirddog 353b5a84a6
Support top level data stores, not exposed via the UI yet (#986) 2024-02-07 08:37:38 -08:00
jbirddog 305a6fb67a
KKV Revamp (#968) 2024-02-07 09:01:52 -05:00
jasquat 5ae59367cb
Feature/reset pi go button (#978)
* delete human task for task that is being reset to in a process instance w/ burnettk

* added script to remove duplicate human tasks from the database w/ burnettk

* test that human tasks are not duplicated during pi reset w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-06 15:50:00 -05:00
Kevin Burnett a61f2d6927
future tasks should not cause anything to happen if the instance is suspended (#952)
* Reapply "future tasks should not cause anything to happen if the instance is suspended"

This reverts commit 05b50df2b3.

* lint

* add required approval and merge instead of mark as auto merge

* lint

* added server default to new future_task column so default is actually added to the table schema w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-06 11:43:17 -08:00
burnettk 05b50df2b3
Revert "future tasks should not cause anything to happen if the instance is suspended"
This reverts commit b627567add.
2024-02-02 17:59:37 -05:00
burnettk b627567add
future tasks should not cause anything to happen if the instance is suspended 2024-02-02 17:58:53 -05:00
Kevin Burnett 020519a724
support bpmn executable property (#934)
* support executable property

* process model with no primary file is not executable

* filter by executable and primary file and avoid snagging g.user from service

* kill Start buttons on process group list as well

* use more similar code to check is executable and write integration test

* add a test and improve a variable name

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2024-02-01 10:57:12 -08:00
Kevin Burnett 6a3f8a212f
update spiff and remove dot notation (#933)
* update spiff and remove dot notation

* update script_engine dependency based on spiff lib update

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2024-02-01 06:51:19 -08:00
burnettk 40b9570e22
process instances for-me is a basic permissions thing needed by everyone 2024-01-30 12:16:46 -05:00
jbirddog a8a32b60fa
Add data store at the process group level (#859) 2024-01-10 09:48:31 -05:00
jbirddog ad3e63367f
Make file upload detection more flexible, add integration test (#854) 2024-01-04 11:32:13 -05:00
jasquat a7a48ee9fc
feature/get-pg-of-readable-pm (#832)
* get parent process groups of process models that the user has access to w/ burnettk

* use the process group list to get the info we need for the group show page for permissions w/ burnettk

* clear the browser cache when updating a process group w/ burnettk

* fixed broken test w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-12-22 13:17:40 -05:00
jasquat d2b39b10c6
feature/handle-null-bpmn-process-on-pi (#827)
* do not error out and allow process instances to recover if the bpmn_process is null but the definition is set w/ burnettk

* fixed another flakey test w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-12-20 16:16:08 -05:00
jasquat 06bf1f24df
fixed flakey task model test 2023-12-20 13:54:20 -05:00
burnettk d688e2414d
make it so we can delete TaskInstructionsForEndUserModel on instance delete 2023-12-19 15:28:02 -05:00
jasquat 3fce735d4f
Feature/task table drop (#823)
* removed id from task and still working on getting the migration working w/ burnettk

* fixed migration to work on postgres and sqlite as well w/ burnettk

* fixed tests w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-12-19 08:59:29 -08:00
jbirddog d4e0fc69fd Test params (#789) 2023-12-07 08:09:44 -05:00
jasquat 18600189c8 Feature/background proc with celery (#788)
* WIP: some initial test code to test out celery w/ burnettk

* some cleanup for celery and added base model to put tasks waiting on timers

* removed dup bpmn file

* some more cleanup and added strategy to queue instructions

* some minor code changes w/ burnettk

* remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine

* added migration for future tasks and added test to make sure we are inserting into it w/ burnettk essweine

* ensure future task run at time can be updated w/ burnettk

* added table to queue instructions for end user in w/ burnettk

* added test to ensure we are storing instructions for end users w/ burnettk

* added progress page to display new instructions to user

* ignore dup instructions on db insert w/ burnettk

* some more updates for celery w/ burnettk

* some pyl and test fixes w/ burnettk

* fixed tests w/ burnettk

* WIP: added in page to show instructions on pi show page w/ burnettk

* pi show page is fully using not interstitial now w/ burnettk

* fixed broken test w/ burnettk

* moved background processing items to own module w/ burnettk

* fixed apscheduler start script

* updated celery task queue to handle future tasks and upgraded black and set its line-length to match ruff w/ burnettk

* added support to run future tasks using countdown w/ burnettk

* build image for celery branch w/ burnettk

* poet does not exist in the image w/ burnettk

* start blocking scheduler should always start the scheduler w/ burnettk

* add init and stuff for this branch

* make this work not just on my mac

* send other args to only

* added running status for process instance and use that on fe to go to show page and added additional identifier to locking system to isolate celery workers better w/ burnettk

* fixed typing error that typeguard found, not sure why mypy did not w/ burnettk

* do not check for no instructions on interstitial page for cypress tests on frontend w/ burnettk

* do not queue process instances twice w/ burnettk

* removed bad file w/ burnettk

* queue tasks using strings to avoid circular imports when attmepting to queue w/ burnettk

* only queue imminent new timer events and mock celery

* some keyboard shortcut support on frontend and added ability to force run a process instance over the api w/ burnettk

* some styles added for the shortcut menu w/ burnettk

* pyl w/ burnettk

* fixed test w/ burnettk

* removed temporary celery script and added support for celery worker in run server locally w/ burnettk

* cleaned up migrations w/ burnettk

* created new migration to clean up old migrations

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-05 11:41:59 -05:00
jbirddog 8d72ef5cfd Feature flag support (#787) 2023-12-05 10:29:54 -05:00
jasquat 91ee554543 ensure we do not delete user group assignments from regexes w/ burnettk (#690)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-20 08:27:28 -08:00
jasquat 449fe87d43 Feature/deny macro on permission (#672)
* look for deny in the permission action rather than in the target uri and updated the documentation w/ burnettk

* added depecation warning if allowed_permissions is being used intead of actions w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-15 11:46:56 -05:00
jasquat 1d37001727 Feature/no data for finished spiff tasks (#661)
* do not add data to spiff tasks if that task is finished w/ burnettk

* build docker image for this branch w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-14 12:47:20 -08:00
jbirddog c122926e01 Support creating unique top level keys for KKV data store (#613) 2023-11-13 11:37:24 -05:00
jasquat 39edd60fd6 Feature/hide private data objects (#581)
* added test and some additional support for deny permissions w/ burnettk

* added support for deny through permissions-check api w/ burnettk

* support DENY at the beginning of a permission target marcro

* do not look up permissions using grant type, only use the uniqueness key

* added support in frontend to display a nice error if user does not have access to a data object value w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-25 16:31:06 -04:00
jasquat 2f9aa12906 Feature/error boundary (#552)
* WIP for error boundary support w/ burnettk

* updates to catch error boundary events and send them back out w/ burnettk

* fixed broken test w/ burnettk

* use connector-http and spiffworkflow-proxy from main w/ burnettk

* updated smtp and slack connectors in connector-proxy-demo w/ burnettk

* added more tests for the service task service call connector to test errors w/ burnettk

* added spiffworkflow-connector-command and some code clean up

* updated the connectors in the proxy demo

* use SpiffWorkflow from git instead of locally

* some cleanup while code reviewing

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-18 14:00:12 -04:00
jbirddog a80abc4c47 Keyed key/value data store implementation (#548) 2023-10-17 10:59:01 -04:00
jasquat c2dc4a738e Feature/data migrator tests (#546)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* attempting to add tests for data migrator and fix 1.3 for postgres

* run version_1_3 migration differently from postgres versus mysql and sqlite

* look up the task model again to make sure it is fresh w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:29:51 -04:00
jasquat 4d842e8dbf Improvement/better serialization (#540)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* switch SpiffWorkflow back to main

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:14:02 -04:00
jasquat 8bf92f7a39 Feature/regex support in permissions (#530)
* support wildcards when adding users to groups from waiting table

* moved the user route to authentication_controller to avoid having so many user routes and this controller was all about login

* added test to ensure regexes work for permissions - still need to remove old ones on refresh

* moved token related code out of authorization service and into authentication service w/ burnettk

* remove old user group assignment waiting entries when refreshing permissions w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-06 13:47:40 -04:00
jasquat cba4a19d3b Feature/remove group service (#529)
* removed group service in favor of user service and messing around with ruff and pre-commeit w/ burnettk

* pyl can succeed now w/ burnettk

* removed debug code w/ burnettk

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-05 13:27:38 -04:00
jbirddog 34a0323c4d Upsearch for data stores (#520) 2023-10-04 09:42:25 -04:00
Kevin Burnett fa5109ff57 update permission check to hopefully improve performance (#523)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-10-03 13:00:13 -07:00
jasquat 5d4713fc0e feature/use-context-with-git-commands (#524)
* use the c option with git commands instead of using cd from python w/ burnettk

* removed the cd method since we should not be using it since it is not threadsafe

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-03 16:00:03 -04:00
jasquat 8bf38aaa1c Feature/new reference cache (#511)
* added basic model for new reference cache w/ burnettk

* switched out SpecReferenceCache for ReferenceCacheModel w/ burnettk jbirddog

* pyl w/ burnettk jbirddog

* save items to the db using the new cache with generation table w/ burnettk

* bulk save for performance

* tests are passing

* actually use the new generation table - we still need a test to ensure we are using it

* added test to ensure using new cache generation

* corrected reference interface on frontend w/ burnettk

* do not perform git pull in webhook if the revision is the same as the current w/ burnettk jbirddog

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-22 11:28:59 -04:00
jasquat 1826cc4b6c Improvement/flexible task iteration (#507)
* updated to use new spiff branch and fixed broken tests w/ burnettk essweine

* updated spiffworkflow with new main build w/ burnettk essweine

* initial updates to new spiff branch w/ burnettk essweine

* more updates for new spiff w/ burnettk essweine

* fixed some linting issues w/ burnettk essweine

* fixed some failing tests w/ burnettk

* updated spiffworkflow for cancel fix w/ burnettk

* Improvement/flexible task iteration 2 (#504)

* wip

* consistent failure, mostly

* removing test code and tests

* removing unused test bpmn files

* removing unused test bpmn files

* minor cleanup of commented code

* spaces and unused imports

* go back to spiff on main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>

* lint

* updated test to reflect storing predicted tasks w/ burnettk

* add some orders so postgres does not do whatever it wants, and clear log

* fix lint

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: danfunk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-21 14:47:18 -07:00
jasquat f6d3bc8e73 Feature/api keys (#489)
* some initial work to support user api keys w/ burnettk

* some updates to store and use service accounts - migrations do not work in sqlite atm

* pyl

* minor tweak to the migration

* refactored user route

* this is working if returning user that created the service account

* put back migrations from main w/ burnettk

* tests pass with new migration w/ burnettk

* do not remove service account permissions on refresh_permissions w/ burnettk

* added new component to make some api calls to populate child components and routes w/ burnettk

* allow displaying extensions in configuration tab w/ burnettk

* removed service accounts controller in favor of extension and encrypt the api keys

* add fuzz to username to make deleting and recreating service accounts easier

* allow specifying the process id to use when running an extension w/ burnettk

* allow extensions to navigate to each other on form submit w/ burnettk

* removed commented out debug code

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-15 10:10:57 -04:00
jasquat a72ed7ffb5 disable element units by default and added test that was failing due to it w/ burnettk (#480)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-08 13:47:31 -07:00
jasquat 912886f3e6 Improvement/task and spec relations (#477)
* updated to use new spiff branch and fixed broken tests w/ burnettk essweine

* updated spiffworkflow with new main build w/ burnettk essweine

* work around dictionary changed size during iteration error in parallel tests

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-08 12:54:43 -04:00
jasquat 9ea90a94bf updated the support user permissions to disallow authentications as well and updated webui to nav to auth page if auth is available but secrets are not w/ burnettk (#454)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-07 10:33:56 -04:00
Dan Funk 80ad92a0c3 Threads (#439)
* WIP - threading with dan

* WIP - cleanup with dan

* Fixing a minor bug and adding a comment and taking the win, even though it was all actually done by @jbirddog with help from @essweine

* adding a couple of simple tests

* * Threaded execution of multiple ready engine tasks is now the default behavior for all execution strategies by default (the skip-one strategry does not do it)
* Assure that tasks in the db match those in the spiffworkflow bpmn on save (remove all predicted and pruned tasks)

with @jbirddog, @jlantz, @burnettk

* run_pyl

---------

Co-authored-by: Jon Herron <jon.herron@yahoo.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-07 10:12:56 -04:00
jasquat 9bb9ce47f8 Feature/business end states (#333)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

* translated mysql last milestone query to sqlalchemy w/ burnettk

* fixed last milestone query so instances still return if no milestone found and moved some code from the main report method to own methods

* added some comments

* added last milestone column to process instances table

* display last milestone in instance list table w/ burnettk

* remove 3 characters when truncating last milestone for ellipsis

* make sure we have a current processor so we don't return null

* remove sleep

* The background processor now only picks up processes that were last updated more than a minute ago to avoid conflicting with the interstitial page.  With the understanding that we can rmeove this limitation when we can refactor to allow the backend processes to provide updates on what they are doing.

* pyl w/ burnettk

* cache last milestone on instances

* pyl

* added test for last milestone and added it to the proces instance show page w/ burnettk

* fixed broken test w/ burnettk

* fixed last milestone header

* removed duplicated column

* fixed broken test

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: Kevin Burnett <18027+burnettk@users.noreply.github.com>
Co-authored-by: danfunk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-07 10:10:44 -04:00
jasquat d58391e21d added permissions for authentication endpoints 2023-08-24 08:37:01 -04:00
jasquat 29620cb17f Feature/add support permissions (#445)
* added support perm macro which removes secrets perms w/ burnettk

* support perm macro inherits from basic now and updated docs on permissions to be more accurate w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-22 08:53:26 +05:30
jasquat 17309fb7fd Feature/allow markdown in extension results (#435)
* allow markdown in extensions results w/ burnettk

* fixed tests

* moved our rjsf form to component so extensions can also use it w/ burnettk

* added ability to create extensions that can download files w/ burnettk

* added test for extensions-get-data endpoint w/ burnettk

* make user optional when getting process instance reports

* added extensions-get-data to elevated perm macro and raise an error if user is not specified when needed when running a report

* fixed typeguard test

* push extensions branch

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-08-21 21:59:25 +05:30
Dan Funk a0202c6a5b Feature/onboarding ephemeral (#442)
* The onboarding controller should not save the process model to the database.  It creates a pile of pointless noise.  So just cleaning that up.

* run_pyl

* assure we can handle user tasks if they happen during on-boarding, while keeping the list of processes clean.
Lots of weird stuff getting run_pyl going.

* pyl

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-08-16 12:11:38 -04:00
Kevin Burnett 9313a9f73a Feature/event payloads part 2 (#401)
* Revert "Revert "Feature/event payloads (#393)""

This reverts commit 95fafb7af1.

* Revert "Revert "poet not available in container""

This reverts commit 140220498c.

* Revert "Revert "Run event payloads data migration from background processor (#399)""

This reverts commit 2afced3a51.

* Revert "Revert "using new spiff api to get info about events. w/ elizabeth""

This reverts commit af857fee22.

* Revert "Revert "fix tests for waiting_event_can_be_skipped""

This reverts commit 886e6bd42a.

* push image for preview env

* default scripts to localhost w/ burnettk

* use the bugfix/update-split-task-inputs spiffworkflow branch w/ burnettk

* removed debug json files

* use main for spiffworkflow

* do not attempt to highlight non-diagram boundary items w/ burnettk

* updated SpiffWorkflow to fix multiple signal event issue w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-10 18:24:49 +05:30