# Codex Monthly Report - August 2023

## Client

### Milestone: Merkelizing block data
- Initial design writeup completed: https://github.com/codex-storage/codex-research/blob/master/design/metadata-overhead.md
- Initial Merkle Tree implementation: https://github.com/codex-storage/nim-codex/pull/504
- Work on persisting/serializing Merkle Tree
- Storing and retrieving data using merkle trees: https://github.com/codex-storage/nim-codex/pull/541
- Coders for merkle trees: https://github.com/codex-storage/nim-codex/pull/519
- Refine merkle tree construction: https://github.com/codex-storage/nim-codex/pull/516

### Milestone: Block discovery and retrieval
- Initial work breakdown and milestones: https://docs.google.com/document/d/1hnYWLvFDgqIYN8Vf9Nf5MZw04L2Lxc9VxaCXmp9Jb3Y/edit
- Initial analysis of block discovery: https://rpubs.com/giuliano_mega/1067876
- Initial block discovery simulator: https://gmega.shinyapps.io/block-discovery-sim/
- Continued analysis and review of peer sampling papers
- Started work on simulations based on the analysis

### Milestone: Distributed Client Testing
- Extensive work on log collection/analysis and monitoring
- Testing and debugging Codex in continuous testing environment
- Major effort to stabilize the Codex client through continuous automated testing

### Milestone: Async Disc Access & Threading support
- Tests on sharing thread data with refc
- Addressed errorVariable and thread safety issues
- Prototype proxy IO threadpool in progress

## Marketplace

### Milestone: L2
- Taiko L2 integration mostly completed, waiting on related fixes

### Milestone: Reservations and slot management
- Extensive work on slot reservation and queuing

### Milestone: Marketplace Sales
- Cleanup and refactoring of state machine
- Added support for loading node's slots during Sale's module start

## Remote auditing

### Milestone: Implement Poseidon2
- First pass at an implementation by Balazs

### Milestone: Refine proving system
- Extensive thinking around storage proofs and proving systems

## DAS (Data Availability Sampling)

### Milestone: DHT simulations
- Implementing a DHT in Python for the DAS simulator
- Implemented logical error-rates and delays to interactions between DHT clients

## Infra

### Milestone: Kubernetes Configuration and Management
- Moved Dist-Tests cluster to OVH and defined naming conventions
- Configured Ingress Controller for Kibana/Grafana
- Created documentation for Kubernetes management
- Configured Dist/Continuous-Tests Pods logs shipping

### Milestone: Continuous Testing and Labeling
- Implemented and configured Dist-Tests labeling
- Set up logs shipping based on labels
- Improved Docker workflows and added 'latest' tag

### Milestone: CI/CD and Synchronization
- Set up synchronization by codex-storage
- Configured Codex Storage and Demo CI/CD environments

### Milestone: Monitoring and Metrics
- Installed Node exporter and Prometheus in Dist-Tests cluster
- Updated Grafana Dashboard
- Implemented automated metrics scraping

## Notable Achievements
- Significant progress in Merkelizing block data, improving the foundation for the proving system
- Advancements in block discovery and retrieval, including simulation work
- Major improvements in distributed client testing and stability
- Successful integration with Taiko L2, expanding the project's capabilities
- Continued development of DHT simulations for Data Availability Sampling

## Challenges and Next Steps
- Continue refining the Merkle tree implementation and integration
- Further develop and optimize the block discovery and retrieval processes
- Enhance the continuous testing environment for improved client stability
- Complete the Taiko L2 integration and address any remaining issues
- Advance the DHT simulations and DAS implementation
- Continue improving infrastructure, monitoring, and CI/CD processes