mirror of
https://github.com/logos-storage/logos-storage-docs-obsidian.git
synced 2026-01-07 15:53:07 +00:00
improve requirements section
This commit is contained in:
parent
f77b547ae1
commit
3b893905bd
@ -57,66 +57,79 @@ The current implementation has several shortcomings:
|
|||||||
|
|
||||||
## 4. Functional Requirements
|
## 4. Functional Requirements
|
||||||
|
|
||||||
**Atomic Block Operations**
|
### Available Today
|
||||||
|
|
||||||
- Store, retrieve, and delete operations must be atomic
|
- **Atomic Block Operations**
|
||||||
- Support retrieval via:
|
- Store, retrieve, and delete operations must be atomic.
|
||||||
|
- Support retrieval via:
|
||||||
- Direct CID
|
- Direct CID
|
||||||
- Tree-based addressing (`treeCid` + index)
|
- Tree-based addressing (`treeCid + index`)
|
||||||
- Unified block address
|
- Unified block address
|
||||||
|
|
||||||
**Metadata Management**
|
- **Metadata Management**
|
||||||
|
- Store protocol-level metadata (e.g., storage proofs, quota usage).
|
||||||
|
- Store block-level metadata (e.g., reference counts, total block count).
|
||||||
|
|
||||||
- Store protocol-level metadata (e.g., sales data, storage proofs, quota usage)
|
- **Multi-Datastore Support**
|
||||||
- Store block-level metadata (e.g., reference counts, total block count)
|
- Pluggable datastore interface supporting various backends.
|
||||||
|
- Typed datastore operations for metadata type safety.
|
||||||
|
|
||||||
**Multi-Datastore Support**
|
- **Lifecycle & Maintenance**
|
||||||
|
- BlockMaintainer service for removing expired data.
|
||||||
|
- Configurable maintenance intervals (default: 10 min).
|
||||||
|
- Batch processing (default: 1000 blocks/cycle).
|
||||||
|
|
||||||
- Pluggable datastore interface supporting various backends
|
### Future Requirements
|
||||||
- Typed datastore operations for metadata type safety
|
|
||||||
- Uniform API across implementations
|
|
||||||
|
|
||||||
**Lifecycle & Maintenance**
|
- **Transaction Rollback & Error Recovery**
|
||||||
|
- Rollback support for failed multi-step operations.
|
||||||
|
- Consistent state restoration after failures.
|
||||||
|
|
||||||
- **BlockMaintainer** service for removing expired data
|
- **Dataset-Level Operations**
|
||||||
- Configurable maintenance intervals (default: 10 min)
|
- Handle Dataset level meta data.
|
||||||
- Batch processing (default: 1000 blocks/cycle)
|
- Batch operations for dataset block groups.
|
||||||
- Cooperative scheduling to avoid blocking
|
|
||||||
- State tracking for large datasets
|
|
||||||
|
|
||||||
**Transaction Rollback & Error Recovery**
|
- **Concurrency Control**
|
||||||
|
- Consistent locking and coordination mechanisms to prevent inconsistencies during crashes or long-running operations.
|
||||||
|
|
||||||
- Rollback for failed multi-step operations
|
- **Lifecycle & Maintenance**
|
||||||
- Consistent state restoration after failures
|
- Cooperative scheduling to avoid blocking.
|
||||||
|
- State tracking for large datasets.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 5. Non-Functional Requirements
|
## 5. Non-Functional Requirements
|
||||||
|
|
||||||
**Performance**
|
### Available Today
|
||||||
|
|
||||||
- Batch metadata updates
|
- **Security**
|
||||||
- Efficient key lookups with configurable prefix lengths
|
- Verify block content integrity upon retrieval.
|
||||||
- Lazy iteration and streaming for large datasets
|
- Enforce quotas to prevent disk exhaustion.
|
||||||
- Support for both fast and slower storage tiers
|
- Safe orphaned data cleanup.
|
||||||
|
|
||||||
**Security**
|
- **Scalability**
|
||||||
|
- Configurable storage quotas (default: 20 GiB).
|
||||||
|
- Pagination for metadata queries.
|
||||||
|
- Reference counting–based garbage collection.
|
||||||
|
|
||||||
- Verify block content integrity upon retrieval
|
- **Reliability**
|
||||||
- Enforce quotas to prevent disk exhaustion
|
- Metrics collection (`codex_repostore_*`).
|
||||||
- Safe orphaned data cleanup
|
- Graceful shutdown with resource cleanup.
|
||||||
|
|
||||||
**Scalability**
|
### Future Requirements
|
||||||
|
|
||||||
- Configurable storage quotas (default: 20 GiB)
|
- **Performance**
|
||||||
- Pagination for metadata queries
|
- Batch metadata updates.
|
||||||
- Reference counting–based garbage collection
|
- Efficient key lookups with configurable prefix lengths.
|
||||||
|
- Support for both fast and slower storage tiers.
|
||||||
|
- Streaming APIs optimized for extremely large datasets.
|
||||||
|
|
||||||
**Reliability**
|
- **Security**
|
||||||
|
- Finer-grained quota enforcement across tenants/namespaces.
|
||||||
|
|
||||||
- Metrics collection (`codex_repostore_*`)
|
- **Reliability**
|
||||||
- Auto-recovery from inconsistent states
|
- Stronger rollback semantics for multi-node consistency.
|
||||||
- Graceful shutdown with resource cleanup
|
- Auto-recovery from inconsistent states.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@ -222,7 +235,7 @@ This store maintains **two separate LRU caches**:
|
|||||||
- Acts as the primary cache for block content
|
- Acts as the primary cache for block content
|
||||||
2. **CID/Proof Cache** — `LruCache[(Cid, Natural), (Cid, CodexProof)]`
|
2. **CID/Proof Cache** — `LruCache[(Cid, Natural), (Cid, CodexProof)]`
|
||||||
- Maps `(treeCid, index)` to `(blockCid, proof)`
|
- Maps `(treeCid, index)` to `(blockCid, proof)`
|
||||||
- Enables efficient tree-based block access without re-fetching proofs
|
- Supports direct access to block proofs keyed by `treeCid` and index
|
||||||
|
|
||||||
**Characteristics**:
|
**Characteristics**:
|
||||||
|
|
||||||
Loading…
x
Reference in New Issue
Block a user