Dagster IFT RAG 📄

This repository is used to extract all of the necessary IFT data for the Logos RAG such as project blogs and Notion documentation.

Structure 👷

Structured has been borrowed from Dagster University.

├── Dagster-Template
├── ├── ift_rag
│       ├── assets
│       ├── configs
│       ├── jobs
│       ├── partitions
│       ├── resources
│       ├── schedules
│       ├── resources
│       ├── sensors
├── ├── ift_rag_tests
├── ├── pyproject.toml
├── ├── setup.cfg
├── ├── .gitignore
└── 

To get the default Dagster project structure run:

dagster project scaffold --name your_name_here

How to run locally 🧰

To see the all dagster commands click here.

pip install -r requirements.txt
dagster dev