7 Data Stores in 5 mins

jbirddog edited this page 2024-01-22 19:02:32 -05:00

Table of Contents

• Data Stores
• Current Data Stores
• JSON Data Store
• JSON File Data Store
• Keyed-Key-Value (KKV) Data Store
• Type Ahead Data Store
• Interacting with Data Stores

Data Stores

1. Allow BPMN Architects to define custom data models that can be shared across process instances
2. ClientA wants to deploy a website that will find a recipe based on selected ingredients, or allow the user to add an ingredient
   1. Not adding an ingredients table to the backend
   2. A Service Task could be used (postgres or custom recipe backend)
   3. Or a Data Store could be added with reads and writes modeled in the diagram
3. There can be many types of Data Stores, each handle the structure and persistence of the stored data
4. Data Stores are implemented in the backend and must be registered with SpiffWorkflow
5. When a Data Store is added to a process model, the type of Data Store and its identifier is defined
6. To/From arrows are then drawn which cause data to be put it/taken from task data
7. They work a lot like data objects, main difference is the data persists outside of the process instance

Current Data Stores

JSON Data Store

1. Allows storing any valid JSON blob
2. The entire blob is read/written when accessed
3. Very flexible but not optimal for large datasets
4. Stored in the backend's database, one row per instance

JSON File Data Store

1. Allows storing any valid JSON blob
2. The entire blob is read/written when accessed
3. Very flexible but not optimal for large datasets
4. Stored in a file that is committed to git along with process models
5. Good for small static global data that should be included in PRs/promoted to other environments

Keyed-Key-Value (KKV) Data Store

1. Allows storing any valid JSON blob
2. Each blob is segmented by two levels of keys
   1. An example use case, could be used to manage car lots for a large chain of dealerships
   2. Top level key could be the car lot id
   3. Second level keys could be "inventory", "employees", "weekly-deals", etc
3. Data is lazy loaded via a function pointer placed in task data
4. A subset of data is read/written when accessed
5. Stored in the backend's database, one row per instance/top level key/secondary key
6. Good for larger datasets

Type Ahead Data Store

1. Requires a structured JSON blob that details records to be used by the Type Ahead Widget
2. Allows records to be indexed with different search terms (Dan, dan@, Mr. Funk, His Funkness -> Dan Funk)
3. Is write-only from within BPMN diagrams
4. Requires the entire data to be loaded in one shot
5. Stored in the backend's database, one row per instance/search term
6. Good only if you are using the Type Ahead Widget

Interacting with Data Stores

1. From the UI, still some work to be done
2. Requires manually editing the XML to tweak what the modeler generates
3. Work in progress
4. By default Data Stores are global and can be used from any process model
5. As an alternative, an "upsearch" mechanism is currently being added to allow only using Data Stores defined within the "scope" of a given process model
6. Branch being worked on to allow JSON Data Stores to be added to a process group
   1. Only process models in or below that process group can see that Data Store
   2. Spiff Arena's permission model is then used to decide who can create Data Stores at a given location