Commit Graph

129 Commits

Author SHA1 Message Date
jasquat 18600189c8 Feature/background proc with celery (#788)
* WIP: some initial test code to test out celery w/ burnettk

* some cleanup for celery and added base model to put tasks waiting on timers

* removed dup bpmn file

* some more cleanup and added strategy to queue instructions

* some minor code changes w/ burnettk

* remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine

* added migration for future tasks and added test to make sure we are inserting into it w/ burnettk essweine

* ensure future task run at time can be updated w/ burnettk

* added table to queue instructions for end user in w/ burnettk

* added test to ensure we are storing instructions for end users w/ burnettk

* added progress page to display new instructions to user

* ignore dup instructions on db insert w/ burnettk

* some more updates for celery w/ burnettk

* some pyl and test fixes w/ burnettk

* fixed tests w/ burnettk

* WIP: added in page to show instructions on pi show page w/ burnettk

* pi show page is fully using not interstitial now w/ burnettk

* fixed broken test w/ burnettk

* moved background processing items to own module w/ burnettk

* fixed apscheduler start script

* updated celery task queue to handle future tasks and upgraded black and set its line-length to match ruff w/ burnettk

* added support to run future tasks using countdown w/ burnettk

* build image for celery branch w/ burnettk

* poet does not exist in the image w/ burnettk

* start blocking scheduler should always start the scheduler w/ burnettk

* add init and stuff for this branch

* make this work not just on my mac

* send other args to only

* added running status for process instance and use that on fe to go to show page and added additional identifier to locking system to isolate celery workers better w/ burnettk

* fixed typing error that typeguard found, not sure why mypy did not w/ burnettk

* do not check for no instructions on interstitial page for cypress tests on frontend w/ burnettk

* do not queue process instances twice w/ burnettk

* removed bad file w/ burnettk

* queue tasks using strings to avoid circular imports when attmepting to queue w/ burnettk

* only queue imminent new timer events and mock celery

* some keyboard shortcut support on frontend and added ability to force run a process instance over the api w/ burnettk

* some styles added for the shortcut menu w/ burnettk

* pyl w/ burnettk

* fixed test w/ burnettk

* removed temporary celery script and added support for celery worker in run server locally w/ burnettk

* cleaned up migrations w/ burnettk

* created new migration to clean up old migrations

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-05 11:41:59 -05:00
jbirddog 8d72ef5cfd Feature flag support (#787) 2023-12-05 10:29:54 -05:00
jasquat c2ce27e961 Feature/drop id column on json data (#587)
* added migration to drop the id column from json_data and make hash the primary key

* removed id column from task_draft_data as well

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-27 11:02:52 -04:00
jbirddog a80abc4c47 Keyed key/value data store implementation (#548) 2023-10-17 10:59:01 -04:00
jasquat ff558388ec updated files table on pm show page to force single line and added migration to increase column size of process-model identifier in message triggerable table w/ burnettk (#515)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-27 09:00:09 -07:00
jasquat 8bf38aaa1c Feature/new reference cache (#511)
* added basic model for new reference cache w/ burnettk

* switched out SpecReferenceCache for ReferenceCacheModel w/ burnettk jbirddog

* pyl w/ burnettk jbirddog

* save items to the db using the new cache with generation table w/ burnettk

* bulk save for performance

* tests are passing

* actually use the new generation table - we still need a test to ensure we are using it

* added test to ensure using new cache generation

* corrected reference interface on frontend w/ burnettk

* do not perform git pull in webhook if the revision is the same as the current w/ burnettk jbirddog

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-22 11:28:59 -04:00
jasquat f6d3bc8e73 Feature/api keys (#489)
* some initial work to support user api keys w/ burnettk

* some updates to store and use service accounts - migrations do not work in sqlite atm

* pyl

* minor tweak to the migration

* refactored user route

* this is working if returning user that created the service account

* put back migrations from main w/ burnettk

* tests pass with new migration w/ burnettk

* do not remove service account permissions on refresh_permissions w/ burnettk

* added new component to make some api calls to populate child components and routes w/ burnettk

* allow displaying extensions in configuration tab w/ burnettk

* removed service accounts controller in favor of extension and encrypt the api keys

* add fuzz to username to make deleting and recreating service accounts easier

* allow specifying the process id to use when running an extension w/ burnettk

* allow extensions to navigate to each other on form submit w/ burnettk

* removed commented out debug code

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-15 10:10:57 -04:00
jbirddog 45f85a7f77 Table based generic json data store (#486) 2023-09-13 11:57:26 -04:00
Dan Funk 948c633b2c Feature/better form nav (#474)
* Detect focus loss/return in the Diagram Editor - so that we can reload the process model and assure that we know about any changed files.

Allow for looking specifically for json SCHEMA files (those files that are named -schema.json or .schema.json (as is the convention).  Only show these in the dropdown for the form.

* * Run descriptions through the markdown processor so you can use bold/italic etc... in your description fields within a form.
* Move ExampleTable into it's own view component to keep the size of the form builder sane.
* Assure markdown within jrsf forms have reasonable styling that follows the containers style, rather than setting to some other default.
* Add a couple of example forms so people can get a sense of what is possible.
* Connect up the new Json Schema Editor Component to the process model edit diagram.
* Just select the schema file - not the ui file when selecting the form for a component - we may revert this to just a text box.
*

* Cleanup the formatting of arrays, so that they are sligtly intended, do not contain an awkward unneeded heading, and have some tighter css.

* Connect the form editing in the modal back to the BPMN-JS editor

Auto-Save edits in the Form Builder

Lots and lots of tweaks to the react form builder ui

* various fixes.

* test for prepare_schema

* minor fix for run_pyl

* css cleanup
less issues with reloading and jumping about when in the editor
Don't sort keys when returning the json.
More intelligent "ready"

* bump package to point to branch of bpmn-js-spiffworkflow so others can check it out.

* Assure that json keys are not sorted during serialization by default.
Allow adding example fields to an existing schema

Create a set of examples.

* db complaints in migration change.

* removed items from interface file that had been moved elsewhere w/ burnettk

* rename prepare_form to prepare-form

* rename prepare_form to prepare-form

* Remove commented out code.

* typo

* add a comment about the empty column

* move back to the main branch

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-08 11:07:43 -04:00
jasquat 4cf33b62fc Feature/user preference extension (#472)
* added ability to display navigation items in user profile toggle

* updated naming of some extension elements

* added user property table and updates for extensions to use it w/ burnettk

* moved extension ui interfaces to own file and linting issues

* some updates to render markdown results on load w/ burnettk

* added migration merge file w/ burnettk

* moved code to fix linting issues w/ burnettk

* resolved db migration conflict

* removed unnecessary migrations and added just one w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-07 11:22:40 -07:00
Elizabeth Esswein 75e6007ef3 multiinstance ui (#469)
* multiinstance ui

* fix black

* added migration merge file

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-07 14:00:09 -04:00
jasquat 9bb9ce47f8 Feature/business end states (#333)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

* translated mysql last milestone query to sqlalchemy w/ burnettk

* fixed last milestone query so instances still return if no milestone found and moved some code from the main report method to own methods

* added some comments

* added last milestone column to process instances table

* display last milestone in instance list table w/ burnettk

* remove 3 characters when truncating last milestone for ellipsis

* make sure we have a current processor so we don't return null

* remove sleep

* The background processor now only picks up processes that were last updated more than a minute ago to avoid conflicting with the interstitial page.  With the understanding that we can rmeove this limitation when we can refactor to allow the backend processes to provide updates on what they are doing.

* pyl w/ burnettk

* cache last milestone on instances

* pyl

* added test for last milestone and added it to the proces instance show page w/ burnettk

* fixed broken test w/ burnettk

* fixed last milestone header

* removed duplicated column

* fixed broken test

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: Kevin Burnett <18027+burnettk@users.noreply.github.com>
Co-authored-by: danfunk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-09-07 10:10:44 -04:00
jasquat dc2d163785 added db migration that merges branching heads together 2023-08-21 13:24:50 -04:00
jbirddog 2b363f4783 Oauth admin (#434)
* Adding dependencies

* Disconnect from /v1/auths for auth list. hardcoded for now.

* Revert changes

* WIP

* Getting hardcoded v2 auths into the frontend

* Better url for v2 oauth

* Pass the auth token from the frontend, don't verify token to start the auth process

* Manually verify the token from the querystring

* WIP

* WIP

* WIP, refactor SPIFF_SECRET handling, move dependencies

* Construct remote_app

* WIP

* WIP

* WIP

* WIP

* Ugly but getting the grant screen

* WIP

* WIP

* Github oauth ok

* Verify token, save access token

* Let secret name work with regex

* Getting bin_pyl to pass

* New component

* Load up the current config in an editor

* Getting bin_pyl to pass

* End point to update auth config

* Linting

* Adding configuration model

* Adding configuration model

* Prep to read config from db

* Read config from the db

* Save/reload poor man's styling

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Better handling of invalid json

* Getting bin_pyl to pass
2023-08-21 21:58:55 +05:30
jasquat 198f898f14 created migration to incrase the human task file names to 255 in the db w/ burnettk (#429)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-21 21:58:10 +05:30
Kevin Burnett 9313a9f73a Feature/event payloads part 2 (#401)
* Revert "Revert "Feature/event payloads (#393)""

This reverts commit 95fafb7af1.

* Revert "Revert "poet not available in container""

This reverts commit 140220498c.

* Revert "Revert "Run event payloads data migration from background processor (#399)""

This reverts commit 2afced3a51.

* Revert "Revert "using new spiff api to get info about events. w/ elizabeth""

This reverts commit af857fee22.

* Revert "Revert "fix tests for waiting_event_can_be_skipped""

This reverts commit 886e6bd42a.

* push image for preview env

* default scripts to localhost w/ burnettk

* use the bugfix/update-split-task-inputs spiffworkflow branch w/ burnettk

* removed debug json files

* use main for spiffworkflow

* do not attempt to highlight non-diagram boundary items w/ burnettk

* updated SpiffWorkflow to fix multiple signal event issue w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-10 18:24:49 +05:30
burnettk 95fafb7af1 Revert "Feature/event payloads (#393)"
This reverts commit f1c5f1ea21.
2023-07-19 11:44:06 -04:00
jasquat f1c5f1ea21 Feature/event payloads (#393)
* some initial changes for event payload changes in spiff

* fixed tests for new spiffworkflow with event payloads w/ burnettk essweine

* pyl w/ burnettk essweine

* updated SpiffWorkflow from branch

* switched SpiffWorkflow back to main w/ burnettk

* added base for migration script to upgrade db w/ burnettk essweine

* some updates to script w/ burnettk

* script has been written, needs to be tested

* pyl w/ burnettk

* updates to migration script so it can work w/ burnettk

* pyl w/ burnettk

* added comment to data migration file

* run the version 1 3 migration on app boot w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-18 06:56:24 -07:00
jasquat c2f9da9773 Revert "added uniqueness constraint to human task so it cannot have multiple human tasks for a process instance and task id w/ burnettk (#396)"
This reverts commit 9b199cad71.
2023-07-17 13:42:45 -04:00
jasquat 9b199cad71 added uniqueness constraint to human task so it cannot have multiple human tasks for a process instance and task id w/ burnettk (#396)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-17 22:14:32 +05:30
jasquat 15b2947107 Feature/draft data in join table (#355)
* added a new model to store task draft data in a join table

* cleaned up using the join table for draft table w/ burnettk

* created new single migration for changes w/ burnettk

* added hidden form which autosaves without validations w/ burnettk

* change close button name since it does indeed save on close now

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-28 22:23:39 +05:30
jasquat 51cfcfa2a5 Feature/autosave and null form data (#346)
* autosave form data on change w/ burnettk

* tests are now passing and pyl

* do not do anything on autosave success w/ burnettk

* merged autosave and null form data branches together w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-26 18:26:56 +05:30
jbirddog 5eff6d4ead Typeahead datastore (#321)
* Wiring up the datastore

* Writes into the data store

* Bulk save needs the timestamps

* Prep to do the local query

* Local typeahead working

* Pre pr cleanup

* ignore migrations dir in pre-commit for ruff w/ burnettk

* Getting ./bin/pyl to pass

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-09 12:28:59 -07:00
jbirddog 0fc32f82d4 Cycle Timer Start Event Support (#285) 2023-05-30 13:51:37 -04:00
burnettk 2b2875cb6f let ruff sort imports and ditch duplicative pre-commit linters 2023-05-26 20:01:08 -04:00
jasquat 699c3eb619 updated active user db model w/ burnettk 2023-05-04 12:51:15 -04:00
jasquat a219d8efd4 added ability to display users showing on the process model edit diagram page w/ burnettk 2023-05-03 17:36:43 -04:00
jasquat 07b4616d45 fixed tests w/ burnettk 2023-05-01 12:51:58 -04:00
jasquat 3247955db5 merged in main and resolved conflicts w/ burnettk 2023-05-01 12:35:10 -04:00
jbirddog e9827d0d5e Stable last updated time for human tasks with timers (#231) 2023-05-01 10:32:19 -04:00
jasquat b86ddf8a96 reports seem to be working again w/ burnettk 2023-04-27 15:19:02 -04:00
jasquat 8cc2d56dbd merged in main and resolved conflicts w/ burnettk 2023-04-20 14:10:23 -04:00
jasquat 08271da363 store stacktrace as a json array so we can reverse it when displaying and fixed up display of errors 2023-04-20 08:31:55 -04:00
jbirddog ff0202f4d4 Called elements (#218) 2023-04-19 18:07:15 -04:00
jasquat 9254d19b74 display event errors in the frontend using errorDisplay w/ burnettk 2023-04-19 15:20:19 -04:00
jasquat 320d1b4083 store process instance errors in the db w/ burnettk 2023-04-19 11:24:54 -04:00
jasquat 8fe98812fc try to make single process hash unique within full process model hash 2023-04-17 15:42:58 -04:00
jasquat fecbb3f040 added new column to bpmn process definition to store the full bpmn process hash digest w/ burnettk 2023-04-17 12:01:36 -04:00
jasquat 0ae74f8f35 removed spiff step details w/ burnettk 2023-03-23 16:33:30 -04:00
jasquat bc58de809e use task table for process instance show page. spiff steps are not working yet and neither is data w/ burnettk 2023-03-20 16:51:29 -04:00
jasquat 50acac8160 some more index updates and a new migration for it w/ burnettk 2023-03-17 16:52:05 -04:00
jasquat 3ca165bb94 recreated migrations to remove spiff logging w/ burnettk 2023-03-17 15:50:05 -04:00
jasquat 88df3bd5c3 added process instance event table 2023-03-17 13:20:06 -04:00
jasquat 363428df09 added bpmn_name columns to definition tables and added test for simple logs as well 2023-03-17 10:00:59 -04:00
jasquat d3393fc6d0 added bpmn process definition to bpmn process w/ burnettk 2023-03-16 16:59:37 -04:00
jasquat bbdac3c586 some initial code to use tasks for logs 2023-03-16 09:30:25 -04:00
jasquat bc619af0a0 check data when running main spiff test w/ burnettk 2023-03-15 15:38:58 -04:00
jasquat ceca4d1333 unit tests are passing w/ burnettk 2023-03-15 11:25:15 -04:00
jbirddog 764eb35d1b Move process instance locking to new queue table (#177) 2023-03-14 13:12:01 -04:00
jasquat 87bacbdaaf added some timestamp fields to bpmn process tables w/ burnettk 2023-03-07 10:41:54 -05:00