adds some more docs

.obsidian/core-plugins.json

@@ -7,6 +7,7 @@
   "canvas": true,
   "outgoing-link": true,
   "tag-pane": true,
+  "footnotes": false,
   "properties": true,
   "page-preview": true,
   "daily-notes": true,
@@ -27,7 +28,6 @@
   "file-recovery": true,
   "publish": false,
   "sync": true,
-  "webviewer": false,
-  "footnotes": false,
-  "bases": true
+  "bases": true,
+  "webviewer": false
 }

10 Notes/Databse migration (AI) to accomodate Codex extensions.md

@@ -0,0 +1,410 @@
+**Q: Where does `protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK` come from?**
+
+Summarized conversation historyLooking at the code, `protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK` appears to be a constant from the protobuf package. Let me search for where this is defined. 
+
+Searched text for `COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK`, 2 results
+
+Now let me look at the protobuf file to see the full context: 
+
+Searched text for `enum Type` (`**/protocol/protobuf/application_metadata_message.proto`), 1 result
+
+Searched text for `COMMUNITY_MESSAGE_ARCHIVE` (`**/protocol/protobuf/application_metadata_message.proto`), 1 result
+
+Let me see more context around this enum definition: 
+
+Searched text for `(COMMUNITY_MESSAGE_ARCHIVE|= 4[0-9];|= 5[0-9];)` (`**/protocol/protobuf/application_metadata_message.proto`), 20 results
+
+Now let me check if there's a similar constant for INDEX_CID: 
+
+Searched text for `COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID`, 1 result
+
+Let me check if this constant exists in the protobuf file: 
+
+Searched text for `INDEX_CID` (`**/protocol/protobuf/application_metadata_message.proto`), no results
+
+It seems like `COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID` is being used but not defined in the protobuf file. Let me check the broader context around line 60 in the protobuf file: 
+
+Symbol `COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID` not found
+
+Let me look at the entire protobuf file to see the complete enum: 
+
+Searched codebase for "ApplicationMetadataMessage Type enum protobuf", 22 results
+
+Based on my search, I can now answer your question about where `protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK` comes from:
+
+## Answer
+
+`protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK` comes from the protobuf enum definition in application_metadata_message.proto.
+
+Specifically, it's defined at line 60 of the proto file as:
+
+```protobuf
+COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK = 43;
+```
+
+This is part of the `Type` enum within the `ApplicationMetadataMessage` protobuf definition. When the protobuf file is compiled to Go code (using the `//go:generate protoc` directive found in `protocol/protobuf/service.go`), it generates the Go constant `protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK` with the value `43`.
+
+The protobuf compilation process converts the proto enum value `COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK = 43` into the Go constant that you see being used throughout the codebase for message type identification.
+
+**Note:** I also notice that in the code there's a reference to `COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID` which doesn't appear to be defined in the protobuf file yet. This suggests it may need to be added to the enum if the Codex functionality requires it.
+
+
+**Q: in persistance - current file - there are a couple of attributes operating on the communities_archive_info (table?). Is it hard to add additional attributes? For my index CIDs I need the following:**
+
+- **last_cid which corresponds to `last_magnetlink_uri`**
+- **indexcid_clock => `magnetlink_clock`**
+
+**What is this clock used for?**
+
+Looking at the persistence.go file, I can see that the `communities_archive_info` table is used to store archive-related information for communities. Let me analyze the existing structure and answer your questions.
+
+## Current `communities_archive_info` table structure
+
+Based on the methods in the file, the table appears to have these columns:
+- `community_id` (string)
+- `magnetlink_clock` (uint64)
+- `last_magnetlink_uri` (string)
+- `last_message_archive_end_date` (uint64)
+
+## Adding new attributes
+
+It should **not be hard** to add your new attributes. You would need to:
+
+1. **Add the new columns to the table schema** (likely in a migration file)
+2. **Add corresponding methods** similar to the existing ones
+
+For your requirements:
+- `last_cid` (corresponds to `last_magnetlink_uri`)
+- `indexcid_clock` (corresponds to `magnetlink_clock`)
+
+You could add methods like:
+
+```go
+// Get methods
+func (p *Persistence) GetLastSeenIndexCid(communityID types.HexBytes) (string, error) {
+    var indexCid string
+    err := p.db.QueryRow(`SELECT last_cid FROM communities_archive_info WHERE community_id = ?`, communityID.String()).Scan(&indexCid)
+    if err == sql.ErrNoRows {
+        return "", nil
+    }
+    return indexCid, err
+}
+
+func (p *Persistence) GetIndexCidMessageClock(communityID types.HexBytes) (uint64, error) {
+    var indexCidClock uint64
+    err := p.db.QueryRow(`SELECT indexcid_clock FROM communities_archive_info WHERE community_id = ?`, communityID.String()).Scan(&indexCidClock)
+    if err == sql.ErrNoRows {
+        return 0, nil
+    }
+    return indexCidClock, err
+}
+
+// Update methods
+func (p *Persistence) UpdateLastSeenIndexCid(communityID types.HexBytes, indexCid string) error {
+    _, err := p.db.Exec(`UPDATE communities_archive_info SET last_cid = ? WHERE community_id = ?`,
+        indexCid, communityID.String())
+    return err
+}
+
+func (p *Persistence) UpdateIndexCidMessageClock(communityID types.HexBytes, clock uint64) error {
+    _, err := p.db.Exec(`UPDATE communities_archive_info SET indexcid_clock = ? WHERE community_id = ?`,
+        clock, communityID.String())
+    return err
+}
+```
+
+## What is the clock used for?
+
+The **clock** is a **logical timestamp** used for:
+
+1. **Message ordering** - Ensures messages are processed in the correct order
+2. **Conflict resolution** - When multiple nodes update the same data, the higher clock value wins
+3. **Synchronization** - Helps determine which updates are newer when syncing between devices/nodes
+4. **Deduplication** - Prevents processing the same update multiple times
+
+Looking at the existing code:
+
+```go
+func (p *Persistence) UpdateMagnetlinkMessageClock(communityID types.HexBytes, clock uint64) error {
+```
+
+This suggests that when a new magnetlink message is received/processed, the clock value is updated to track the latest state. This is common in distributed systems to maintain consistency without requiring synchronized system clocks.
+
+## Next steps
+
+You'll need to:
+1. **Create a database migration** to add the new columns
+2. **Update the `SaveCommunityArchiveInfo` method** to include the new fields
+3. **Add the getter/setter methods** for your new attributes
+4. **Update any initialization code** that creates archive info records
+
+The pattern is already established in the codebase, so adding your index CID attributes should be straightforward following the same structure as the magnetlink ones.
+
+
+Absolutely! I can see you've already added the four new methods for handling Index CID operations. Let me help you complete the remaining steps to fully integrate your Index CID functionality.
+
+## Step 1: Database Migration
+
+First, you'll need to create a database migration to add the new columns to the `communities_archive_info` table. Let me search for existing migration files to follow the pattern: 
+
+Searched for files matching `**/migrations/**/*.sql`, 20 matches
+
+Let me look for community-related migrations specifically: 
+
+Searched for files matching `**/migrations/**/*communit*.sql`, 20 matches
+
+Let me look for the communities_archive_info table creation: 
+
+Searched text for `communities_archive_info` (`**/migrations/**/*.sql`), 2 results
+
+Great! Let me check the structure of the existing table and the most recent migration: 
+
+Searched codebase for "communities_archive_info table CREATE TABLE migration", 21 results
+
+Now let me look at the actual migration files: 
+
+Read [](file:///home/mc2/code/status-im/status-go/appdatabase/migrations/sql)
+
+Let me check the latest migration number and the specific migration that adds the archive_info table: 
+
+Searched codebase for "1650616788_add_communities_archives_info_table CREATE TABLE communities_archive_info", 18 results
+
+Perfect! Now I can see the migration pattern. Let me create the migration file. Based on the existing files, I need to create a new migration with a timestamp.
+
+## Step 1: Create Database Migration
+
+````sql
+CREATE TABLE communities_archive_info_add_index_cid_columns.sql
+
+```sql
+ALTER TABLE communities_archive_info ADD COLUMN last_cid TEXT DEFAULT "";
+ALTER TABLE communities_archive_info ADD COLUMN indexcid_clock INTEGER DEFAULT 0;
+```
+````
+
+Let me check the current latest migration to get the right timestamp: 
+
+Searched text for `1757511667`, no results
+
+Based on the file listing, the latest migration is `1757511667_add_messages-backup_enabled.up.sql`. So your new migration should have a timestamp after that. Here's what you need:
+
+**Create the migration file:**
+`/home/mc2/code/status-im/status-go/appdatabase/migrations/sql/1757511668_add_index_cid_columns_to_communities_archive_info.up.sql`
+
+````sql
+ALTER TABLE communities_archive_info ADD COLUMN last_cid TEXT DEFAULT "";
+ALTER TABLE communities_archive_info ADD COLUMN indexcid_clock INTEGER DEFAULT 0;
+````
+
+## Step 2: Update `SaveCommunityArchiveInfo` Method
+
+You'll need to modify the existing method to include the new fields. Looking at your existing code, I can see you've already added the getter/setter methods, but you need to update the initial save method:
+
+````go
+func (p *Persistence) SaveCommunityArchiveInfo(communityID types.HexBytes, clock uint64, lastArchiveEndDate uint64) error {
+    _, err := p.db.Exec(`INSERT INTO communities_archive_info (magnetlink_clock, last_message_archive_end_date, community_id, last_cid, indexcid_clock) VALUES (?, ?, ?, ?, ?)`,
+        clock,
+        lastArchiveEndDate,
+        communityID.String(),
+        "", // default empty last_cid
+        0,  // default indexcid_clock
+    )
+    return err
+}
+````
+
+## Step 3: Update the Method Calls in Manager
+
+You'll need to find where these new methods are called. Let me search for usage patterns: 
+
+Searched text for `UpdateMagnetlinkMessageClock`, 6 results
+
+Searched text for `UpdateLastSeenMagnetlink`, 4 results
+
+Great! Now I can see the pattern. You'll need to add corresponding manager methods and update the handlers.
+
+## Step 4: Add Manager Methods
+
+Add these methods to manager.go:
+
+````go
+// Add these methods after the existing magnetlink methods around line 3730
+
+func (m *Manager) UpdateIndexCidMessageClock(communityID types.HexBytes, clock uint64) error {
+	return m.persistence.UpdateIndexCidMessageClock(communityID, clock)
+}
+
+func (m *Manager) GetIndexCidMessageClock(communityID types.HexBytes) (uint64, error) {
+	return m.persistence.GetIndexCidMessageClock(communityID)
+}
+
+func (m *Manager) UpdateLastSeenIndexCid(communityID types.HexBytes, indexCid string) error {
+	return m.persistence.UpdateLastSeenIndexCid(communityID, indexCid)
+}
+
+func (m *Manager) GetLastSeenIndexCid(communityID types.HexBytes) (string, error) {
+	return m.persistence.GetLastSeenIndexCid(communityID)
+}
+````
+
+## Step 5: Update Community Description Structure (if needed)
+
+Looking at the protobuf structure, you might want to add an `archive_indexcid_clock` field similar to `archive_magnetlink_clock`. Let me search for that: 
+
+Searched text for `archive_magnetlink_clock`, 1 result
+
+You might want to add a similar field to the protobuf for consistency:
+
+````proto
+// In /home/mc2/code/status-im/status-go/protocol/protobuf/communities.proto
+// Add after line 116:
+uint64 archive_indexcid_clock = 20; // or the next available number
+````
+
+## Step 6: Add Message Handlers (similar to magnetlink)
+
+Looking at the existing pattern in messenger_handler.go, you'll need to add handlers for Index CID messages. Let me check where the magnetlink handler is: 
+
+Searched text for `COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK` (`**/protocol/messenger_handler.go`), no results
+
+Let me search for the magnetlink message handling: 
+
+Searched text for `CommunityMessageArchiveMagnetlink`, 4 results
+
+Perfect! Now I can see the complete pattern. Here's what you need to do:
+
+## Step 7: Create Index CID Protobuf Message
+
+Add to communities.proto:
+
+````proto
+// Add after CommunityMessageArchiveMagnetlink message
+message CommunityMessageArchiveIndexCid {
+  uint64 clock = 1;
+  string cid = 2;
+}
+````
+
+## Step 8: Add Index CID Message Type
+
+Add to application_metadata_message.proto:
+
+````proto
+// Add after COMMUNITY_MESSAGE_ARCHIVE_MAGNETLINK = 43;
+COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID = 76; // or next available number
+````
+
+## Step 9: Add Index CID Message Handler
+
+Add to messenger_handler.go (similar to the magnetlink handler):
+
+````go
+func (m *Messenger) HandleCommunityMessageArchiveIndexCid(state *ReceivedMessageState, message *protobuf.CommunityMessageArchiveIndexCid, statusMessage *messagingtypes.Message) error {
+	logger := m.logger.Named("HandleCommunityMessageArchiveIndexCid")
+	
+	haveArchiveInfo, err := m.communitiesManager.HasCommunityArchiveInfo(statusMessage.CommunityID)
+	if err != nil {
+		logger.Debug("failed to check if we have community archive info", zap.Error(err))
+		return err
+	}
+
+	if !haveArchiveInfo {
+		logger.Debug("ignoring index cid message for community with no archive info", zap.String("communityID", types.EncodeHex(statusMessage.CommunityID)))
+		return nil
+	}
+
+	currentIndexCidClock, err := m.communitiesManager.GetIndexCidMessageClock(statusMessage.CommunityID)
+	if err != nil {
+		logger.Debug("failed to get index cid message clock", zap.Error(err))
+		return err
+	}
+
+	if currentIndexCidClock >= message.Clock {
+		// ignore older index cid messages
+		return nil
+	}
+
+	id := statusMessage.CommunityID
+	clock := message.Clock
+	err = m.communitiesManager.UpdateIndexCidMessageClock(id, clock)
+	if err != nil {
+		logger.Debug("failed to update index cid message clock", zap.Error(err))
+		return err
+	}
+
+	if len(message.Cid) > 0 {
+		err = m.communitiesManager.UpdateLastSeenIndexCid(id, message.Cid)
+		if err != nil {
+			logger.Debug("failed to update last seen index cid", zap.Error(err))
+			return err
+		}
+	}
+
+	return nil
+}
+````
+
+## Step 10: Update `dispatchIndexCidMessage` Method
+
+Update the method in messenger_communities.go to use the correct message type:
+
+````go
+func (m *Messenger) dispatchIndexCidMessage(communityID string) error {
+	community, err := m.communitiesManager.GetByIDString(communityID)
+	if err != nil {
+		return err
+	}
+
+	indexCid, err := m.archiveManager.GetHistoryArchiveIndexCid(community.ID())
+	if err != nil {
+		return err
+	}
+
+	indexCidMessage := &protobuf.CommunityMessageArchiveIndexCid{
+		Clock: m.getTimesource().GetCurrentTime(),
+		Cid:   indexCid,
+	}
+
+	encodedMessage, err := proto.Marshal(indexCidMessage)
+	if err != nil {
+		return err
+	}
+
+	chatID := community.MagnetlinkMessageChannelID()
+	rawMessage := messagingtypes.RawMessage{
+		LocalChatID:          chatID,
+		Sender:               community.PrivateKey(),
+		Payload:              encodedMessage,
+		MessageType:          protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID,
+		SkipGroupMessageWrap: true,
+		PubsubTopic:          community.PubsubTopic(),
+		Priority:             &messagingtypes.LowPriority,
+	}
+
+	_, err = m.messaging.SendPublic(context.Background(), chatID, rawMessage)
+	if err != nil {
+		return err
+	}
+
+	err = m.communitiesManager.UpdateCommunityDescriptionIndexCidMessageClock(community.ID(), indexCidMessage.Clock)
+	if err != nil {
+		return err
+	}
+	return m.communitiesManager.UpdateIndexCidMessageClock(community.ID(), indexCidMessage.Clock)
+}
+````
+
+## Summary
+
+Here are all the files you need to modify:
+
+1. **Migration**: Create `1757511668_add_index_cid_columns_to_communities_archive_info.up.sql`
+2. **Persistence**: Update `SaveCommunityArchiveInfo` method (already done the getters/setters)
+3. **Manager**: Add the 4 new methods for Index CID operations
+4. **Protobuf**: Add `CommunityMessageArchiveIndexCid` message and `COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID` enum
+5. **Handler**: Add `HandleCommunityMessageArchiveIndexCid` method
+6. **Dispatch**: Update `dispatchIndexCidMessage` to use correct message type
+
+The pattern is exactly the same as the magnetlink implementation, just with "IndexCid" instead of "Magnetlink". This approach ensures consistency with the existing codebase and makes it easy to maintain.
+
+Would you like me to help you with any specific part of this implementation?

10 Notes/Team-NLBR Solution Proposal.md

@@ -5,6 +5,7 @@ related-to:
   - "[[status-go processing magnet links]]"
   - "[[status-go-codex integration - design notes]]"
   - "[[Creating History Archives - InitHistoryArchiveTasks]]"
+  - "[[testing codex-status-go integration]]"
 ---
 The `TorrentConfig` type provides the configuration for the BitTorrent-based History Archive management functionality:
 

10 Notes/status-go publishing magnet links.md

@@ -96,7 +96,7 @@ SeedHistoryArchiveTorrent()
     ↓
 publisher.publish(HistoryArchivesSeedingSignal) [internal event]
     ↓
-handleCommunitiesHistoryArchivesSubscription() [event handler]
+`handleCommunitiesHistoryArchivesSubscription`() [event handler]
     ↓
 IF community.IsControlNode():
     dispatchMagnetlinkMessage(communityID)

10 Notes/status-go-codex integration - design notes.md

@@ -6,7 +6,6 @@ related-to:
   - "[[status-go-codex integration - design notes]]"
   - "[[Creating History Archives - InitHistoryArchiveTasks]]"
 ---
-
 ## Codex for History Archives
 
 As indicated in the [[Team-NLBR Solution Proposal]], the central entry point to the history management is [InitHistoryArchiveTasks](https://github.com/status-im/status-go/blob/6322f22783585474803cfc8a6f0a914757d763b5/protocol/messenger_communities.go#L3783). `InitHistoryArchiveTasks` is called from **two main places**:
@@ -197,7 +196,102 @@ path.Join(m.torrentConfig.DataDir, "codex", communityID, "data")
 
 This data file is temporary and will be overwritten for each new archive created. With Codex, we do not have to append, thus, we do not need the previous data file anymore. We just use file now, because it may be easier to start it this way.
 
-This is done with `LoadHistoryArchiveIndexFromFile`
+Now, just for convenience, let's recall the original data structures involved:
+
+![[team-nl-br-design-1.svg]]
+
+The data structures using with BitTorrent are:
+
+```go
+wakuMessageArchiveIndexProto := &protobuf.WakuMessageArchiveIndex{}
+wakuMessageArchiveIndex := make(map[string]*protobuf.WakuMessageArchiveIndexMetadata)
+```
+
+The original BitTorrent index, stored in `wakuMessageArchiveIndexProto`, is initially populated using `LoadHistoryArchiveIndexFromFile` function. After that `wakuMessageArchiveIndex` is used as temporary storage so that we can conveniently extend it with new entries and serialize it to protobuf afterwords. We use the contents of `wakuMessageArchiveIndexProto` to set it up:
+
+```go
+for hash, metadata := range wakuMessageArchiveIndexProto.Archives {
+	offset = offset + metadata.Size
+	wakuMessageArchiveIndex[hash] = metadata
+}
+```
+
+For the codex extension we proceed in the analogous way:
+
+![[replacing-bittorrent-with-codex-in-status-go-1.svg]]
+
+![[replacing bittorrent with codex in status-go-2.svg]]
+
+```go
+codexWakuMessageArchiveIndexProto := &protobuf.CodexWakuMessageArchiveIndex{}
+	codexWakuMessageArchiveIndex := make(map[string]*protobuf.CodexWakuMessageArchiveIndexMetadata)
+```
+
+and then:
+
+```go
+for hash, metadata := range codexWakuMessageArchiveIndexProto.Archives {
+	codexWakuMessageArchiveIndex[hash] = metadata
+}
+```
+
+Having those variables in place and initialized correctly, we enter the loop and start creating archives one by one.
+
+Basically, we proceed in the same way as with BitTorrent - the `WakuMessageArchive` type does not change.
+
+At some point, we arrive at:
+
+```go
+wakuMessageArchiveIndexMetadata := &protobuf.WakuMessageArchiveIndexMetadata{
+	Metadata: wakuMessageArchive.Metadata,
+	Offset:   offset,
+	Size:     uint64(size),
+	Padding:  uint64(padding),
+}
+```
+
+For Codex extension, we do not have `offset`, `size`, and `padding` any more as this is something that Codex will take care - but this is the moment we need to call into Codex, to upload the archive and get the corresponding CID back so that we can properly initialize the corresponding index entry:
+
+```go
+client := NewCodexClient("localhost", "8080") // make this configurable
+cid, err := client.UploadArchive(encodedArchive)
+if err != nil {
+		m.logger.Error("failed to upload to codex", zap.Error(err))
+		return codexArchiveIDs, err
+}
+
+m.logger.Debug("uploaded to codex", zap.String("cid", cid))
+
+codexWakuMessageArchiveIndexMetadata := &protobuf.CodexWakuMessageArchiveIndexMetadata{
+		Metadata: wakuMessageArchive.Metadata,
+		Cid:      cid,
+}
+
+codexWakuMessageArchiveIndexMetadataBytes, err := proto.Marshal(codexWakuMessageArchiveIndexMetadata)
+if err != nil {
+	return codexArchiveIDs, err
+}
+
+codexArchiveID := crypto.Keccak256Hash(codexWakuMessageArchiveIndexMetadataBytes).String()
+codexArchiveIDs = append(codexArchiveIDs, codexArchiveID)
+codexWakuMessageArchiveIndex[codexArchiveID] = codexWakuMessageArchiveIndexMetadata
+```
+
+where `CodexClient` is a helper that encapsulates uploading arbitrary data to a Codex client via `/api/codex/v1/data` API. The corresponding `curl` call would be similar to:
+
+```bash
+curl -X POST \
+  http://localhost:${PORT}/api/codex/v1/data \
+  -H 'Content-Type: application/octet-stream' \
+  -H 'Content-Disposition: filename="archive-data.bin"' \
+  -w '\n' \
+  -T archive-data.bin
+zDvZRwzm22eSYNdLBuNHVi7jSTR2a4n48yy4Ur9qws4vHV6madiz
+```
+
+At this stage we have an individual archive uploaded to Codex (it should be save there now) It is already being advertised but nobody is looking for it yet as we did not finish building the Codex-aware index file, which contains CIDs for all the archives.
+
+
 ## Testing
 
 There will be a number of tests that will need to adjust or fix.

10 Notes/testing codex-status-go integration.md

@@ -0,0 +1,90 @@
+First some low level simple manual tests.
+
+We will be running codex client, and then use a small testing utility to check if the low level abstraction - CodexClient - correctly uploads and downloads the content.
+### Running CodexClient
+
+I often remove some logging noise, by slightly changing the build params in `build.nims`:
+
+```nim
+task codex, "build codex binary":
+  buildBinary "codex",
+    # params = "-d:chronicles_runtime_filtering -d:chronicles_log_level=TRACE"
+    params =
+      "-d:chronicles_runtime_filtering -d:chronicles_log_level=TRACE -d:chronicles_enabled_topics:restapi:TRACE,node:TRACE"
+```
+
+You see a slightly more selective `params` in the `codex` task.
+
+To run the client I use the following command:
+
+```bash
+./build/codex --data-dir=./data-1 --listen-addrs=/ip4/127.0.0.1/tcp/8081 --api-port=8001 --nat=none --disc-port=8091 --log-level=TRACE
+```
+
+### Building codex-upload and codex-download utilities
+
+Use the following command to build the `codex-upload` and `codex-download` utilities:
+
+```bash
+go build -o bin/codex-upload ./cmd/upload
+go build -o bin/codex-download ./cmd/download
+```
+### Uploading content to Codex
+
+Now, using the `codex-upload` utility, we can upload the content to Codex as follows:
+
+```bash
+~/code/local/go-codex-client
+❯ ./bin/codex-upload -file test-data.bin -host localhost -port 8001
+Uploading test-data.bin (43 bytes) to Codex at localhost:8001...
+✅ Upload successful!
+CID: zDvZRwzm8K7bcyPeBXcZzWD7AWc4VqNuseduDr3VsuYA1yXej49V
+```
+
+### Downloading content from Codex
+
+Now, having the content uploaded to Codex - let's get it back using the `codex-download` utility:
+
+```bash
+~/code/local/go-codex-client
+❯ ./bin/codex-download -cid zDvZRwzm8K7bcyPeBXcZzWD7AWc4VqNuseduDr3VsuYA1yXej49V -file output.bin -host localhost -port 8001
+Downloading CID zDvZRwzm8K7bcyPeBXcZzWD7AWc4VqNuseduDr3VsuYA1yXej49V from Codex at localhost:8001...
+✅ Download successful!
+Saved to: output.bin
+```
+
+You can easily compare that the downloaded content matches the original using:
+
+```bash
+~/code/local/go-codex-client
+❯ openssl sha256 test-data.bin
+SHA2-256(test-data.bin)= c74ce73165c288348b168baffc477b6db38af3c629b42a7725c35d99d400d992
+
+~/code/local/go-codex-client
+❯ openssl sha256 output.bin
+SHA2-256(output.bin)= c74ce73165c288348b168baffc477b6db38af3c629b42a7725c35d99d400d992
+```
+
+### Running tests
+
+There are a couple of basic tests, including one integration test.
+
+To run the unit tests:
+
+```bash
+❯ go test -v ./communities
+=== RUN   TestUpload_Success
+--- PASS: TestUpload_Success (0.00s)
+=== RUN   TestDownload_Success
+--- PASS: TestDownload_Success (0.00s)
+=== RUN   TestDownloadWithContext_Cancel
+--- PASS: TestDownloadWithContext_Cancel (0.04s)
+PASS
+ok  	go-codex-client/communities	0.044s
+```
+
+To run the integration test, use `integration` tag and narrow the scope using `-run Integration`:
+
+```bash
+go test -v -tags=integration ./communities -run Integration -timeout 15s
+```