Commit Graph

118 Commits

Author SHA1 Message Date
Dan Funk 137e8a8232
Docs/pycharm setup (#1169)
* Just some notes from getting my dev environnent set up again.

* fix readme link

* fixing black configuration.

* Black config settings in pre-commit.

* Update README.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update README.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update spiffworkflow-backend/docs/pycharm.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update spiffworkflow-backend/docs/pycharm.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update spiffworkflow-backend/docs/pycharm.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update spiffworkflow-backend/docs/pycharm.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-03-07 14:40:24 -05:00
Kevin Burnett 3416843329
spiff lib upgrade (#1139)
* wip spiff lib upgrade

* unit and script tests are now passing w/ burnettk

* tests and pre-commit are passing

* pass None in as external_context as named param instead of creating a variable w/ burnettk

* some extra debugging in case bpmn_xml_file_contents is empty w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-03-04 12:29:37 -08:00
jasquat a71af6e40f
Tcoz openai assist (#1138)
* Update README.md

Some notes for people that might want to run full-on native, with detail about how Mac hijacks port 7000 and how to get around it.

* Revert "Update README.md"

This reverts commit 096887c26d.

* README update and native code patch

Some details for user that might be running Python3, Mac, and want to run everything locally/natively.

* Implement basic tooltips

Uses MUI tooltip, as it behaves more predictably with existing styling, and enables top-level theme config.

Top-level configuration for all MUI components can be controlled via overriding the existing theme. See index.tsx. This could be done per user for customization, etc.

Enabling JSON module imports in tsconfig.json seemed to fix the error in ReactDiagramEditor

* Naive AI code editor implementation

A working starting point.

* Implement API to return if script assist should be enabled

Along with route and function, api config, etc.

* UI calls backend to see if script assist is enabled.

If it is, loads the related UI, otherwise it doesn't appear.

* Moving forward with service for message processing.

* Services scaffolded

* Open API called, prompt-engineered to get script only.

* Little cleanup work

* Enabled + process message working.

Had to find all the places permissions are enabled, etc.

* Cleanup, comments, etc.

* Env vars, styling, error cases, conditional display of script assist

Finishing touches for the most part.

REQUIRES TWO ENV VARS BE SET.

SPIFFWORKFLOW_SCRIPT_ASSIST_ENABLED=["True" | "true" | 1]  (anything else is false)
SECRET_KEY_OPENAI_API=[thekey]

The are retrieved in default.py. I run the app locally, so I just set them in the terminal.

NEW INSTALL: @carbon/colors (so we consistently use carbon palette etc.)

* Fix tooltips, clean up some styling.

Finishing it off.

* Add loader and error message

Complete UX stuff

* Update useScriptAssistEnabled.tsx

Remove log

* Update script_assist_controller.py

Add this tweak to avoid TMI.

* Some reasonable changes suggested by the build process

* Comments from PR.

* Update ProcessModelEditDiagram.tsx

Should (but I don't know how to tell yet) call the change handler that wasn't firing before.

* updated the permissions setting in authorization service w/ burnettk

* precommit now passes. tests are failing w/ burnettk

* pinned SpiffWorkflow to known working version and fixed tests. we will update spiff in a later pr w/ burnettk

* made changes based on coderabbi suggestions

* updated the error handling to be more inline with how we have handled other errors and some ui tweaks

* removed pymysql package w/ burnettk

* forgot to remove pymysql from lock file w/ burnettk

---------

Co-authored-by: Tim Consolazio <tcoz@tcoz.com>
Co-authored-by: Kevin Burnett <18027+burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-03-04 10:42:27 -05:00
jasquat b0dbdfb192
commit the repo when a data store is added (#1115)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2024-02-27 11:50:17 -05:00
dependabot[bot] b13078802e
Bump coverage from 6.5.0 to 7.4.1 in /spiffworkflow-backend (#976)
Bumps [coverage](https://github.com/nedbat/coveragepy) from 6.5.0 to 7.4.1.
- [Release notes](https://github.com/nedbat/coveragepy/releases)
- [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst)
- [Commits](https://github.com/nedbat/coveragepy/compare/6.5.0...7.4.1)

---
updated-dependencies:
- dependency-name: coverage
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-06 16:08:56 +00:00
dependabot[bot] 4bd920bed5
Bump marshmallow-sqlalchemy in /spiffworkflow-backend (#972)
Bumps [marshmallow-sqlalchemy](https://github.com/marshmallow-code/marshmallow-sqlalchemy) from 0.29.0 to 1.0.0.
- [Changelog](https://github.com/marshmallow-code/marshmallow-sqlalchemy/blob/dev/CHANGELOG.rst)
- [Commits](https://github.com/marshmallow-code/marshmallow-sqlalchemy/compare/0.29.0...1.0.0)

---
updated-dependencies:
- dependency-name: marshmallow-sqlalchemy
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-06 07:07:39 -08:00
dependabot[bot] 6a0249955a
Bump bandit from 1.7.2 to 1.7.7 in /spiffworkflow-backend (#945)
Bumps [bandit](https://github.com/PyCQA/bandit) from 1.7.2 to 1.7.7.
- [Release notes](https://github.com/PyCQA/bandit/releases)
- [Commits](https://github.com/PyCQA/bandit/compare/1.7.2...1.7.7)

---
updated-dependencies:
- dependency-name: bandit
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-02 10:43:52 -08:00
dependabot[bot] 6567112615
Bump xdoctest from 1.1.2 to 1.1.3 in /spiffworkflow-backend (#943)
Bumps [xdoctest](https://github.com/Erotemic/xdoctest) from 1.1.2 to 1.1.3.
- [Release notes](https://github.com/Erotemic/xdoctest/releases)
- [Changelog](https://github.com/Erotemic/xdoctest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Erotemic/xdoctest/compare/v1.1.2...v1.1.3)

---
updated-dependencies:
- dependency-name: xdoctest
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-01 13:44:12 -08:00
burnettk b3b1f44d51
update cryptography 2024-02-01 14:02:49 -05:00
jbirddog 6c209fe2ff
Bump docker (#918)
* Bump docker base image to python 3.12.1

* Add libffi-dev

* Add chardet

* Dumping logs in CI

* Remove tmp CI step, bump gunicorn
2024-01-29 11:09:36 -08:00
jbirddog 858e8eaec4
Get backend working with Python 3.12 (#876)
Co-authored-by: Kevin Burnett <18027+burnettk@users.noreply.github.com>
2024-01-24 08:00:10 -05:00
Kevin Burnett b02f505bea
move snyk stuff to snyk file (#895)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2024-01-11 06:14:25 -08:00
jbirddog 1c0570c147
Bump Restricted Python to 7.x, needed for Python 3.12 support (#872) 2024-01-10 10:46:28 -05:00
jbirddog ab39569cac
Add direct dependency on jsonschema (#868) 2024-01-10 08:18:34 -05:00
Kevin Burnett e1fefa9fc7
nplusone (#824)
* n plus one checks

* tests passing with nplusone

* satisfy probably-wrong json schema

* remove useless comment and consolidate poetry dev deps

* not actually going to add this dependency, but leave docs for next guy

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-20 07:18:20 -08:00
burnettk 57cb064025 upgrade ruff and add a few rules 2023-12-05 19:43:41 -05:00
jasquat 18600189c8 Feature/background proc with celery (#788)
* WIP: some initial test code to test out celery w/ burnettk

* some cleanup for celery and added base model to put tasks waiting on timers

* removed dup bpmn file

* some more cleanup and added strategy to queue instructions

* some minor code changes w/ burnettk

* remove the unused next_task key from api calls since nobody uses it w/ burnettk essweine

* added migration for future tasks and added test to make sure we are inserting into it w/ burnettk essweine

* ensure future task run at time can be updated w/ burnettk

* added table to queue instructions for end user in w/ burnettk

* added test to ensure we are storing instructions for end users w/ burnettk

* added progress page to display new instructions to user

* ignore dup instructions on db insert w/ burnettk

* some more updates for celery w/ burnettk

* some pyl and test fixes w/ burnettk

* fixed tests w/ burnettk

* WIP: added in page to show instructions on pi show page w/ burnettk

* pi show page is fully using not interstitial now w/ burnettk

* fixed broken test w/ burnettk

* moved background processing items to own module w/ burnettk

* fixed apscheduler start script

* updated celery task queue to handle future tasks and upgraded black and set its line-length to match ruff w/ burnettk

* added support to run future tasks using countdown w/ burnettk

* build image for celery branch w/ burnettk

* poet does not exist in the image w/ burnettk

* start blocking scheduler should always start the scheduler w/ burnettk

* add init and stuff for this branch

* make this work not just on my mac

* send other args to only

* added running status for process instance and use that on fe to go to show page and added additional identifier to locking system to isolate celery workers better w/ burnettk

* fixed typing error that typeguard found, not sure why mypy did not w/ burnettk

* do not check for no instructions on interstitial page for cypress tests on frontend w/ burnettk

* do not queue process instances twice w/ burnettk

* removed bad file w/ burnettk

* queue tasks using strings to avoid circular imports when attmepting to queue w/ burnettk

* only queue imminent new timer events and mock celery

* some keyboard shortcut support on frontend and added ability to force run a process instance over the api w/ burnettk

* some styles added for the shortcut menu w/ burnettk

* pyl w/ burnettk

* fixed test w/ burnettk

* removed temporary celery script and added support for celery worker in run server locally w/ burnettk

* cleaned up migrations w/ burnettk

* created new migration to clean up old migrations

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-05 11:41:59 -05:00
Kevin Burnett 353ba4cd31 avoid print statements, generally (#784)
* avoid print statements, generally

* remove test for empty main script

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-12-01 06:24:11 -08:00
Kevin Burnett cfe00a45d4 reset to page 1 when status changes to fix #765 (#769)
* reset to page 1 when status changes to fix #765 w/ jasquat

* upgrade connexion and werkzeug to fix snyk w/ jasquat

* fix all security issues like a boss w/ jasquat

* whoops, still no resolution for cryptography w/ jasquat

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-11-28 08:58:54 -08:00
jbirddog abe865c19b Data Store CRUD interface (#689) 2023-11-24 13:27:46 -05:00
burnettk f524b78368 fixes from fixit 2 2023-11-20 11:04:33 -05:00
jasquat fc503ec76b feature/secrets-encryption-cleaning (#673)
* removed simple-crypt and cleaned up usage of keys for encryption w/ burnettk

* renamed var to SPIFFWORKFLOW_BACKEND_ENCRYPTION_KEY w/ burnettk

* pyl w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-11-16 10:12:47 -05:00
jasquat a025aaa017 upgraded cryptography to satisfy snyk and added ignore for werkzeug issue since we cannot do anything about it now w/ burnettk 2023-10-26 11:28:37 -04:00
jasquat 2f9aa12906 Feature/error boundary (#552)
* WIP for error boundary support w/ burnettk

* updates to catch error boundary events and send them back out w/ burnettk

* fixed broken test w/ burnettk

* use connector-http and spiffworkflow-proxy from main w/ burnettk

* updated smtp and slack connectors in connector-proxy-demo w/ burnettk

* added more tests for the service task service call connector to test errors w/ burnettk

* added spiffworkflow-connector-command and some code clean up

* updated the connectors in the proxy demo

* use SpiffWorkflow from git instead of locally

* some cleanup while code reviewing

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-18 14:00:12 -04:00
jasquat cba4a19d3b Feature/remove group service (#529)
* removed group service in favor of user service and messing around with ruff and pre-commeit w/ burnettk

* pyl can succeed now w/ burnettk

* removed debug code w/ burnettk

* pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-10-05 13:27:38 -04:00
jasquat a84750741e updated SpiffWorkflow for depth fix (#519)
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-28 12:51:48 -07:00
jasquat 47f94dccbb Feature/nonetype debug stuff (#513)
* some debug items w/ burnettk

* removed some db commits from workflow execution service so the passed in save is the only thing that actually commits the transaction

* set max depth to 50000 for now w/ burnettk

* pyl w/ burnettk

* use temp maxdepth fix in SpiffWorkflow w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-09-27 17:04:47 -04:00
jbirddog 9925105a5e Support lazy loading call activities with human tasks (#450)
* Switch to local wheel

* Fix to load the call activity by spec

* Added log

* Logging

* WIP - getting the right tasks it seems from the db, need to get into the workflow

* Clean up, revert things

* Working thanks dan

* Getting bin_pyl to pass

* run_pyl.

* Re-poet install

* Revert file

* Remove logging lines

* Getting bin_pyl to pass

* Clean up, revert things

* Bump wheel

* Use version off pypi

---------

Co-authored-by: danfunk <daniel.h.funk@gmail.com>
2023-09-07 10:33:38 -04:00
jbirddog 2b363f4783 Oauth admin (#434)
* Adding dependencies

* Disconnect from /v1/auths for auth list. hardcoded for now.

* Revert changes

* WIP

* Getting hardcoded v2 auths into the frontend

* Better url for v2 oauth

* Pass the auth token from the frontend, don't verify token to start the auth process

* Manually verify the token from the querystring

* WIP

* WIP

* WIP, refactor SPIFF_SECRET handling, move dependencies

* Construct remote_app

* WIP

* WIP

* WIP

* WIP

* Ugly but getting the grant screen

* WIP

* WIP

* Github oauth ok

* Verify token, save access token

* Let secret name work with regex

* Getting bin_pyl to pass

* New component

* Load up the current config in an editor

* Getting bin_pyl to pass

* End point to update auth config

* Linting

* Adding configuration model

* Adding configuration model

* Prep to read config from db

* Read config from the db

* Save/reload poor man's styling

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Getting bin_pyl to pass

* Better handling of invalid json

* Getting bin_pyl to pass
2023-08-21 21:58:55 +05:30
jasquat 3f6fb4d9f5 Feature/pin spiff in ci (#428)
* attempt to use the locked version of SpiffWorkflow in ci w/ burnettk

* fixed regex for spiffworkflow revision replacement

* make sure we also update the lockfile when getting new spiff

* install poetry before we attempt to update hte pyproject

* hardcoding spiffworkflow revision as a test w/ burnettk

* try running tests in ci from bash script w/ burnettk

* print working dir in ci w/ burnettk

* fixed location of instance dir w/ burnettk

* run with mysql in ci

* run typeguard with bash script as well w/ burnettk

* fixed postgres test w/ burnettk

* clean up github action file w/ burnettk

* fixed postgres test again w/ burnettk

* pyl

* attempt to remove nox from ci completely

* omit safety for now to test coverage

* fixed how coverage is being called from not nox

* allow running safety and macos again

* renamed run_not_nox to run_ci_session w/ burnettk

* attempt to only upload if matrix says to w/ burnettk

* attempt to install mysqlclient prereqs for mac and remove noxfile stuff w/ burnettk

* added back the constraints file w/ burnettk

* moved the contributing file to the root of arena w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-08-03 11:49:30 -04:00
jasquat 84369a1b20 return the process instance early from the interstitial if it is susp… (#407)
* return the process instance early from the interstitial if it is suspended or terminated

* added a test to make sure the interstitial page returns the process instance if suspended or termianted w/ burnettk

* randomize tests and cleaned up the pyproject file a little bit w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-07-25 22:57:53 +05:30
jasquat c936c02db0 use mysqlclient instead of mysql-connect w/ burnettk (#358)
* use mysqlclient instead of mysql-connect w/ burnettk

* updated some comments w/ burnettk

* runs tests in parallel in run_pyl

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-12 19:44:49 +05:30
jasquat e4bf22f5cf Feature/interstitial summary part 3 (#367)
* throw an error if backend gives a 500 from interstitial page w/ burnettk

* turn unexpected errors for the interstitial page into event stream responses to avoid error misdirection

* pyl

* raise the api_error from the original and make the render_data method private

* Feature/interstitial summary (#337)

* * Process instance logs and messages are now components rather than pages, and are included within tabs on the process instance page, along with the diagram.
* Removed the Zoom and Move modules when showing the readonly
  Diagram.  Assured this readonly view is resized to fit the space when possible.
* Checkbox Widget no longer displays a duplicate label.
* CSS Tweaks
  * All pages are limited to a max display width of 1440, with auto margins to center the main content on the page.
  * "Show" pages, like ProcessInstanceShow, TaskShow have the primary content limited to 1000, also with auto-margins.
  * Paragraphs, headings, blockquotes, list items are limited to a width of 640.
  * Reduced margin bottom on all breadcrumbs.
  * Slightly reduced the width and margin of tiles
  * ordered lists and unordered lists show numbers and bullets now.
* End user Instructions component can, optionally, auto-collapse, so that only a portion is displayed, along with a toggle.  This is how it is set up for the ProcessInstanceShow page.
* Greatly reduced the lag in the interstitial page when doing a re-direct.

* run_pyl

* kill console

* wait for permissionsLoaded too since we are using ability.can

* Previous change removed the top level Messages page - this re-adds it.

* I am always, ALWAYS shocked at how I can not wrap my head around when and where to use "useEffect".

This should cause the show/hide Instructions toggle to only show when useful.

* Minor cleanup on the process instance properties display.

* linting

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>

* Fix a linting error.

* minor permissions related fixes for the new ui.

* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf.

* do not execute tasks from the process instance show page w/ burnettk

* pyl w/ burnettk

* Very minor UI tweak to resolve a number of issues mentioned in SPIFF-316 Notion Ticket:

* Markdown links now open in a new window
* Tables on the home pages are now contained within Grids to better align them with the other content that is in a grid.
* Right aligned the "process instance list link" button to it is flush right with the table underneath
* Gave a little more breathing room the content on the info/metadata in the process instance view .

* updated docker image build action to contain the appropriate version info w/ burnettk

* properly resize to the available columns depending on s/m/l

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
Co-authored-by: Dan Funk <daniel.h.funk@gmail.com>
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-07-03 10:54:54 +05:30
Kevin Burnett b13b716424 parallel tests with xdist (#335)
* parallel tests with xdist

* add pytest-xdist as dev dep

* put back spiff

* update messaging

* get more in line with main

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-26 22:37:41 +05:30
Kevin Burnett 7de0d97303 Feature/better subworkflow management part 2 (#341)
* Revert "Revert "Feature/better subworkflow management (#331)""

This reverts commit 48dcde8faf.

* updated SpiffWorkflow to fix infinite loop in task trace w/ burnettk

---------

Co-authored-by: burnettk <burnettk@users.noreply.github.com>
Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-20 10:37:49 -04:00
burnettk 48dcde8faf Revert "Feature/better subworkflow management (#331)"
This reverts commit 23af5cbbe4.
2023-06-15 15:10:48 -04:00
jasquat 23af5cbbe4 Feature/better subworkflow management (#331)
* WIP: some updates to support new spiff w/ burnettk

* unit tests are passing

* all tests except message tests are passing

* fixed usage of catch message event w/ burnettk

* messages are working again w/ burnettk

* uncommented remaining message tests w/ burnettk

* fixed cypress tests w/ burnettk

* use main for spiffworkflow

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-15 10:31:42 -04:00
Kevin Burnett 35543cc362 add mccabe linter and auto fixes (#302)
Co-authored-by: burnettk <burnettk@users.noreply.github.com>
2023-06-08 14:26:33 -07:00
jbirddog 122d1efbda Bump flask for safety (#304)
* Bump flask for safety

* let snyk check flask again w/ burnettk

* attempt to use the same revision for front w/ burnettk

---------

Co-authored-by: jasquat <jasquat@users.noreply.github.com>
2023-06-06 17:33:48 -07:00
jasquat 5dfd96ca9e upgrade cryptography for security w/ burnettk 2023-06-02 12:09:24 -04:00
burnettk 397d99b5c0 enable flake8-bandit, cut off all http requests at 15 seconds to avoid hanging process, ignore xml lib spiff uses 2023-05-30 19:53:26 -04:00
burnettk aae043e0a4 add pylint 2023-05-30 19:39:15 -04:00
burnettk a567b990e6 enable pep8-naming and use ruff for autofixer 2023-05-30 07:15:49 -04:00
burnettk 6bbabd15f4 instead of actually adding pyright configs that do not work, document 2023-05-29 11:52:35 -04:00
burnettk 4b21f02355 get rid of some pyright diagnostics 2023-05-27 22:33:53 -04:00
burnettk 6a328cdb43 actually stop running tests against 3.9 2023-05-27 18:34:04 -04:00
burnettk be9859752e drop python 3.9 and remove unused libs 2023-05-27 18:17:51 -04:00
burnettk f8d4a6382b get compatible with ruff pyupgrade 2023-05-27 14:59:52 -04:00
burnettk 2b2875cb6f let ruff sort imports and ditch duplicative pre-commit linters 2023-05-26 20:01:08 -04:00
burnettk cf40546a09 get compatible with default ruff ruleset 2023-05-26 13:07:18 -04:00