Commit Graph

1931 Commits

Author SHA1 Message Date
jbirddog d4e0fc69fd Test params (#789) 2023-12-07 08:09:44 -05:00
jasquat 40b02ae488 check that connector-proxy-demo can run when releasing new tags w/ burnettk 2023-12-06 11:21:32 -05:00
burnettk 57cb064025 upgrade ruff and add a few rules 2023-12-05 19:43:41 -05:00
jasquat 18600189c8 Feature/background proc with celery (#788)
* WIP: some initial test code to test out celery w/ burnettk

* some cleanup for celery and added base model to put tasks waiting on timers

* removed dup bpmn file

* some more cleanup and added strategy to queue instructions

* some minor code changes w/ burnettk

* remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine

* added migration for future tasks and added test to make sure we are inserting into it w/ burnettk essweine

* ensure future task run at time can be updated w/ burnettk

* added table to queue instructions for end user in w/ burnettk

* added test to ensure we are storing instructions for end users w/ burnettk

* added progress page to display new instructions to user

* ignore dup instructions on db insert w/ burnettk

* some more updates for celery w/ burnettk

* some pyl and test fixes w/ burnettk

* fixed tests w/ burnettk

* WIP: added in page to show instructions on pi show page w/ burnettk

* pi show page is fully using not interstitial now w/ burnettk

* fixed broken test w/ burnettk

* moved background processing items to own module w/ burnettk

* fixed apscheduler start script

* updated celery task queue to handle future tasks and upgraded black and set its line-length to match ruff w/ burnettk

* added support to run future tasks using countdown w/ burnettk

* build image for celery branch w/ burnettk

* poet does not exist in the image w/ burnettk

* start blocking scheduler should always start the scheduler w/ burnettk

* add init and stuff for this branch

* make this work not just on my mac

* send other args to only

* added running status for process instance and use that on fe to go to show page and added additional identifier to locking system to isolate celery workers better w/ burnettk

* fixed typing error that typeguard found, not sure why mypy did not w/ burnettk

* do not check for no instructions on interstitial page for cypress tests on frontend w/ burnettk

* do not queue process instances twice w/ burnettk

* removed bad file w/ burnettk

* queue tasks using strings to avoid circular imports when attmepting to queue w/ burnettk

* only queue imminent new timer events and mock celery

* some keyboard shortcut support on frontend and added ability to force run a process instance over the api w/ burnettk

* some styles added for the shortcut menu w/ burnettk

* pyl w/ burnettk

* fixed test w/ burnettk

* removed temporary celery script and added support for celery worker in run server locally w/ burnettk

* cleaned up migrations w/ burnettk

* created new migration to clean up old migrations

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-05 11:41:59 -05:00
jbirddog 8d72ef5cfd Feature flag support (#787) 2023-12-05 10:29:54 -05:00
burnettk 04acca883d fix comment 2023-12-04 18:30:59 -05:00
Kevin Burnett 353ba4cd31 avoid print statements, generally (#784)
* avoid print statements, generally

* remove test for empty main script

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-01 06:24:11 -08:00
jasquat 1627f5dd07 bugfix/guest-login-multiple-auths (#782)
* This fixes guest login with using multiple auths, removes empty items from ApiError, and raises if redirect_url given to login does not match expected frontend host w/ burnettk

* get body for debug

* try to get the logs from the correct place to upload w/ burnettk

* mock the openid call instead of actually calling it w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-30 13:51:01 -05:00
burnettk c814e255e4 argument to switch to left right mode 2023-11-29 22:15:52 -05:00
burnettk f676fab480 command to run pydeps 2023-11-29 21:59:17 -05:00
burnettk af5402c6e4 add init in services 2023-11-29 21:43:27 -05:00
jasquat 74de2063d6 added spiffworkflow-2 realm based on spiffworkflow realm w/ burnettk 2023-11-28 12:41:16 -05:00
burnettk c0a0606913 remove accidentally added keycloak dir ignore w/ jasquat 2023-11-28 12:37:30 -05:00
burnettk 6c7fad9b80 removed bad dir w/ jasquat 2023-11-28 12:20:39 -05:00
Kevin Burnett cfe00a45d4 reset to page 1 when status changes to fix #765 (#769)
* reset to page 1 when status changes to fix #765 w/ jasquat

* upgrade connexion and werkzeug to fix snyk w/ jasquat

* fix all security issues like a boss w/ jasquat

* whoops, still no resolution for cryptography w/ jasquat

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-28 08:58:54 -08:00
burnettk 126ba9512c tweak 2023-11-28 09:54:24 -05:00
jbirddog abe865c19b Data Store CRUD interface (#689) 2023-11-24 13:27:46 -05:00
jasquat d737d40ab7 disabled no task data on completed tasks when rehydrating (#706)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-21 10:49:16 -05:00
jasquat 762f945e62 updated connector-proxy-demo dockerfile to be more like backend w/ burnettk 2023-11-20 16:49:30 -05:00
jasquat 91ee554543 ensure we do not delete user group assignments from regexes w/ burnettk (#690)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-20 08:27:28 -08:00
burnettk f524b78368 fixes from fixit 2 2023-11-20 11:04:33 -05:00
jasquat b031b0f940 remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine (#688)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-17 13:34:04 -08:00
jasquat 5e95849923 sort nested process groups as well (#685)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-17 10:12:56 -05:00
Kevin Burnett 8d85e5ac26 added some updates to help with logging deployments w/ burnettk (#683)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-16 13:15:49 -08:00
jasquat 9f58c7405d avoid serializing the bpmn process instance just to save data w/ burnettk 2023-11-16 12:32:38 -05:00
Kevin Burnett d9203f4d37 curl and procps in container for debugging (#680)
* curl and procps in container for debugging

* added some spacing between from lines in dockerfiles w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-16 07:18:11 -08:00
jasquat fc503ec76b feature/secrets-encryption-cleaning (#673)
* removed simple-crypt and cleaned up usage of keys for encryption w/ burnettk

* renamed var to SPIFFWORKFLOW_BACKEND_ENCRYPTION_KEY w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-16 10:12:47 -05:00
jasquat 1eed6e2444 feature/full-breadcrumb-on-task-show (#675)
* show the full breadcrumb on task show page w/ burnettk

* check read permission of process model before displaying in breadcrumb on task show page

* in the breadcrumb if the api returns 401 then just ignore the breadcrumb

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-16 10:05:08 -05:00
burnettk 166f7e9873 ignore this for a month and hope that we get a resolution on the internet during this time 2023-11-15 14:54:03 -05:00
jasquat 449fe87d43 Feature/deny macro on permission (#672)
* look for deny in the permission action rather than in the target uri and updated the documentation w/ burnettk

* added depecation warning if allowed_permissions is being used intead of actions w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-15 11:46:56 -05:00
jasquat 34ce785aea let spiff serialize the bpmn process data to let it handle datetimes before we give it to the db w/ burnettk 2023-11-15 10:43:56 -05:00
jasquat 981cfcb347 Merge branch 'main' of github.com:sartography/spiff-arena 2023-11-15 09:47:23 -05:00
jasquat 6d99fd0ac3 only log out the user when we return a 401, not raise an error 2023-11-15 09:47:20 -05:00
Kevin Burnett 8dcdae2590 Feature/fix filters (#670)
* work in progress

* respect multiple filters for the same field

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-15 06:20:00 -08:00
jasquat 1d37001727 Feature/no data for finished spiff tasks (#661)
* do not add data to spiff tasks if that task is finished w/ burnettk

* build docker image for this branch w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-14 12:47:20 -08:00
Dan Funk 12c8b899f0 Copy and Paste Error (#663)
* We were getting copy and paste errors that an id already existed.  This fixes that problem by assuring we always call the importXML method on the diagram modeller (and don't bypass it with a call to the fromXML of the protected _moddle.
we have to correct for the loop characteristics getting removed in a different way.

* run_pyl.

* eslint fixes

* Cypress caught some errors - I hate it, but it was right, and it caught something critical.
2023-11-14 15:14:07 -05:00
jasquat 18207bcfbd bugfix/fix-refresh-token-error (#668)
* when backend returns 401 also remove cookies and redirect in frontend if cookies are not set w/ burnettk

* added a copule helpful comments w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-14 14:54:16 -05:00
jasquat 1197a1a51a do not tell users to add a process model/group if they do not have access to do so w/ burnettk 2023-11-13 16:56:41 -05:00
jbirddog c122926e01 Support creating unique top level keys for KKV data store (#613) 2023-11-13 11:37:24 -05:00
jasquat 1596076a15 updated auth identifier header name to include spiffworkflow w/ burnettk 2023-11-09 15:45:52 -05:00
jasquat 07895d3965 get the auth identifier from the cookie or the header or return default 2023-11-09 15:39:06 -05:00
jasquat 45bcba172d default other open id configs is url is specified w/ burnettk 2023-11-09 14:31:30 -05:00
jasquat 04e763116e authentication_identifier comes from cookies, not headers 2023-11-09 13:50:45 -05:00
jasquat b8cfa4fd3c fixed backwards compatibility with multiple auths 2023-11-09 11:09:11 -05:00
jasquat d5b0330609 Feature/support multiple auth (#602)
* added some support for configs to have mutliple auths

* multiple openids services are mostly working - still needs some cleanup

* some cleanup for pyl and fixed login_return for internal openid server w/ burnettk

* if only one auth is returned from backend then just do that w/ burnettk

* login page has been formatted w/ burnettk

* some extra formatting on the login page w/ burnettk

* relabel test openid providers and add user

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-09 10:34:07 -05:00
jasquat a48bc8a885 do not attempt to migrate instance from interstitial page if the serializer version has not been set yet 2023-11-09 09:13:18 -05:00
jasquat 7b904387fc updated example permissions to use macros and prefer perm absolute path over filename w/ burnettk 2023-11-08 16:57:19 -05:00
jasquat 57c8112eea run the data migrator from the insterstitial page code when loading the pi show page w/ burnettk (#607)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-07 16:20:47 -05:00
Kevin Burnett eb13ab843b fix standard loop task serialization (#606)
* fix standard loop task serialization

* lint

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-06 12:18:33 -08:00
Kevin Burnett 6014f2940d fix serialization issue with custom service task, w/ essweine (#605)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-03 15:00:30 -07:00
burnettk c57cc0ed0b ignore another werkzeug issue until we can upgrade 2023-11-01 18:52:49 -04:00
jasquat 11512a888e disable element units for now w/ burnettk 2023-10-27 15:59:28 -04:00
jasquat 234c9646e1 feature/skip-user-input-required-tasks (#596)
* only check for timer events in ready_user_task_has_associated_timer so we can skip user_input_required instances w/ burnettk

* removed test.py file w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-27 15:01:28 -04:00
jasquat 566f383918 adds basic support to use certain components in extensions w/ burnettk (#597)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-27 14:54:45 -04:00
jasquat c2ce27e961 Feature/drop id column on json data (#587)
* added migration to drop the id column from json_data and make hash the primary key

* removed id column from task_draft_data as well

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-27 11:02:52 -04:00
jbirddog f8112641cc Reenable element units (#593) 2023-10-26 18:20:55 -04:00
jasquat 24741b29de use last_state_change to figure out if a cancelled task needs an event in the task service instead of with get_tasks and check when manually completing tasks w/ burnettk (#595)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-26 16:11:16 -04:00
jbirddog eb3fd9e2af Bump SpiffWorkflow (#590)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-26 12:27:35 -04:00
jasquat 7e128c5a55 Feature/completed tasks on pi show (#591)
* added api to get all completed tasks for an instance and display it in a table w/ burnettk

* moved completed tasks table on pi show page to sub tabs

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-26 11:37:55 -04:00
jasquat a025aaa017 upgraded cryptography to satisfy snyk and added ignore for werkzeug issue since we cannot do anything about it now w/ burnettk 2023-10-26 11:28:37 -04:00
jasquat 83094c66ce process all cancelled tasks to see if that fixes error boundary events 2023-10-26 10:35:52 -04:00
jasquat 39edd60fd6 Feature/hide private data objects (#581)
* added test and some additional support for deny permissions w/ burnettk

* added support for deny through permissions-check api w/ burnettk

* support DENY at the beginning of a permission target marcro

* do not look up permissions using grant type, only use the uniqueness key

* added support in frontend to display a nice error if user does not have access to a data object value w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-25 16:31:06 -04:00
jbirddog 36b7b2462e Remove backend references to Box (#579) 2023-10-25 16:09:25 -04:00
jasquat c18c5a8d9c Feature/do not save cancelled task events twice (#575)
* do not save cancelled task events again

* actually only process cancelled events that were cancelled during the current run

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-25 13:45:20 -04:00
jbirddog 16ae1d9ccd Add support for KKV data stores to the data stores tab (#576) 2023-10-25 12:59:09 -04:00
jasquat 54b7c5c3ec feature/formatting-in-extension-md (#559)
* support formatting data client side in markdown and support greater than and less than for metadata column filters w/ burnettk

* moved spiff conversion functions to FormattingService and use it in InstructionsForEndUser w/ burnettk

* added tests for greater than and less than metadata operators and added negative tests w/ burnettk

* removed unneeded useEffect w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-20 15:19:26 -04:00
jasquat 6664b52985 install vim-tiny instead of vim for security w/ burnettk 2023-10-19 15:01:39 -04:00
jasquat fe4dc14b8d Feature/docker CVE issues (#558)
* updated Dockerfile to try to remove security vulnerabilities w/ burnettk

* we require curl for health checks w/ burnettk

* try to scan docker image in ci

* use Dockerfile from backend w/ burnettk

* continue-on-error w/ burnettk

* attempt to elevate permissions of snyk w/ burnettk

* added snyk security github workflow w/ burnettk

* fixed location of constraints w/ burnettk

* add in or true for snyk tests w/ burnettk

* sent the snyk token w/ burnettk

* specify the directory for the sarif file w/ burnettk

* updated spiffworkflow-connector-command for snyk issue w/ burnettk

* updated sql statements sanitize input

* ignore issues for debug_controller and check frontend with snyk w/ burnettk

* updated babel and electron for snyk w/ burnettk

* some more updates to fix vulnerabilities w/ burnettk

* prune repeated deps for frontend builds since

* uncomment ci code so it runs again and use node for frontend base image w/ burnettk

* fixed backend image name w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-19 14:22:52 -04:00
jasquat 2f9aa12906 Feature/error boundary (#552)
* WIP for error boundary support w/ burnettk

* updates to catch error boundary events and send them back out w/ burnettk

* fixed broken test w/ burnettk

* use connector-http and spiffworkflow-proxy from main w/ burnettk

* updated smtp and slack connectors in connector-proxy-demo w/ burnettk

* added more tests for the service task service call connector to test errors w/ burnettk

* added spiffworkflow-connector-command and some code clean up

* updated the connectors in the proxy demo

* use SpiffWorkflow from git instead of locally

* some cleanup while code reviewing

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-18 14:00:12 -04:00
jasquat b6a4d40f74 updated urllib3 for snyk 2023-10-18 11:21:53 -04:00
jbirddog a80abc4c47 Keyed key/value data store implementation (#548) 2023-10-17 10:59:01 -04:00
jasquat 4d7347c131 do not save events to db when running data migrations w/ burnettk 2023-10-16 15:02:19 -04:00
jasquat abf87a5aa3 corrected typename for tasks in data migration 3 w/ burnettk 2023-10-16 14:51:50 -04:00
jasquat d22e5b2550 check if form contents is None and not truthy when deciding to return back to the user w/ burnettk 2023-10-16 11:42:09 -04:00
burnettk 1eadeb0b29 add cris 2023-10-16 11:24:48 -04:00
burnettk 53def3f3c8 update README 2023-10-13 23:32:01 -04:00
jasquat c2dc4a738e Feature/data migrator tests (#546)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* attempting to add tests for data migrator and fix 1.3 for postgres

* run version_1_3 migration differently from postgres versus mysql and sqlite

* look up the task model again to make sure it is fresh w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:29:51 -04:00
jasquat 4d842e8dbf Improvement/better serialization (#540)
* using new spiffworkflow locally and the db can be recreated w/ burnettk

* tests are passing w/ burnettk

* added version 3 data migration for typenames on tasks and bpmn processes w/ burnettk

* pyl w/ burnettk

* switch SpiffWorkflow back to main

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-12 14:14:02 -04:00
jasquat db4e9292f6 remove colons as well when checking wildcard permissions and updated test to ensure it works as expected 2023-10-11 10:25:02 -04:00
jasquat ceb06cc227 Hotfix/user task with timer cancel (#533)
* cherry picked changes from b12af9f3bc to pin form json files

* use the class name to determine what a task type is w/ burnettk

* initial thoughts to fix cancel timer issue w/ burnettk

* added migration to run predict on all open instances w/ burnettk

* remove debug, refactor data migrations, add benchmark_log_func

* log progress of script

* only process predicted tasks and their parents in the version 2 data miagration w/ burnettk

* added data migrator and using that to run version 2 migrations when needed w/ burnettk

* removed some unwanted code

* fix issue, but tests still need updating

* fix tests by returning code to closer to what it was

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-10-10 11:17:09 -04:00
burnettk dd6dcdcec9 add dinithi 2023-10-09 10:01:20 -04:00
jasquat 8bf92f7a39 Feature/regex support in permissions (#530)
* support wildcards when adding users to groups from waiting table

* moved the user route to authentication_controller to avoid having so many user routes and this controller was all about login

* added test to ensure regexes work for permissions - still need to remove old ones on refresh

* moved token related code out of authorization service and into authentication service w/ burnettk

* remove old user group assignment waiting entries when refreshing permissions w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-06 13:47:40 -04:00
jasquat 01ef4e6eaa pyl w/ burnettk 2023-10-06 13:36:24 -04:00
danfunk a83e11c504 overriding more of the login screen - can't do it all with CSS anymore so overriding one of the ftl files (Whatever they are, not sure, but works) 2023-10-06 11:02:11 -04:00
jasquat 94a01fcc80 Merge branch 'main' of github.com:sartography/spiff-arena 2023-10-05 14:01:09 -04:00
jasquat aeded99114 keycloak output on import seems to have changed 2023-10-05 14:01:06 -04:00
jasquat 15041734a7 use the class name to determine what a task type is w/ burnettk (#527)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-05 13:28:41 -04:00
jasquat cba4a19d3b Feature/remove group service (#529)
* removed group service in favor of user service and messing around with ruff and pre-commeit w/ burnettk

* pyl can succeed now w/ burnettk

* removed debug code w/ burnettk

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-05 13:27:38 -04:00
burnettk d4984a0269 do the joinedload so we actually eliminate the n plus 1 query 2023-10-05 09:30:02 -04:00
danfunk 77845d38ce copy was not creating the spifftheme directory. 2023-10-04 15:48:57 -04:00
jasquat b12af9f3bc Feature/pin form schemas to revision (#526)
* task show loads the correct revision of the json schema form w/ burnettk

* display error if form cannot be found at revision w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-04 15:23:42 -04:00
jasquat dbbf14fe58 fixed copy of theme to keycloak docker image w/ burnettk 2023-10-04 15:08:55 -04:00
danfunk 3ce513a29b Enabled copy in the docker file
Added a spiffworkflow logo, colors, fonts, styling to login page based off the styles used on our website.
Maybe someday these can bleed through to the app itself.
2023-10-04 14:31:08 -04:00
jasquat b23bdbbebc upgrade keycloak to 22.0.4 and added a base custom theme for login screen w/ burnettk 2023-10-04 11:32:53 -04:00
jbirddog 34a0323c4d Upsearch for data stores (#520) 2023-10-04 09:42:25 -04:00
jasquat 22570ce8d3 fix git command with c option 2023-10-04 08:37:41 -04:00
burnettk d7d3858a7a return error if no token instead of blowing up 2023-10-03 22:07:20 -04:00
burnettk d4104f4d12 tell sentry about release info if we have it 2023-10-03 21:23:25 -04:00
burnettk 9a59d37b53 remove debug print statement 2023-10-03 16:03:52 -04:00
Kevin Burnett fa5109ff57 update permission check to hopefully improve performance (#523)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-10-03 13:00:13 -07:00