latest updates to the docs

2026-01-04 22:33:08 +00:00 · 2025-10-30 04:12:08 +01:00 · 2025-10-30 04:12:08 +01:00 · 5f3b9aaa66
commit 5f3b9aaa66
parent dede37f473
4 changed files with 576 additions and 19 deletions
--- a/InitHistoryArchiveTasks.md
+++ b/InitHistoryArchiveTasks.md
@ -6,12 +6,6 @@ related-to:
  - "[[status-go-codex integration - design notes]]"
  - "[[Creating History Archives - InitHistoryArchiveTasks]]"
 ---
-Great question! Let me trace the connection between `StartHistoryArchiveTasksInterval` and `InitHistoryArchiveTasks` to show you how the periodic archive creation system is initialized and started. 
-
-Searched codebase for "InitHistoryArchiveTasks StartHistoryArchiveTasksInterval", 13 results
-
-Perfect! Now I can show you the complete connection between `InitHistoryArchiveTasks` and `StartHistoryArchiveTasksInterval`. Let me trace through this step by step:
-
 ## **Connection Flow: `InitHistoryArchiveTasks` → `StartHistoryArchiveTasksInterval`**

 ### **1. Where `InitHistoryArchiveTasks` is Called** 
--- a/Description.md
+++ b/Description.md
@ -0,0 +1,413 @@
+marcinczenko: can you remind me how `ArchiveIndexCidClock` from the community description is used? Can you summarize how community description messages are sent and received and how they relate to magent links/indexCid updates - I remembered that community description had some impact on processing magnet links. I just to remember the details...
+
+GitHub Copilot: I'll help you understand how `ArchiveIndexCidClock` is used and how it relates to community description messages and index CID updates.
+
+Let me search for relevant code to give you a complete picture. 
+
+Searched text for `ArchiveIndexCidClock`, 2 results
+
+Searched text for `ArchiveMagnetlinkClock`, 2 results
+
+Searched text for `HandleCommunityDescriptionMessage`, 12 results
+
+Now let me look at the specific section where the clock comparison happens: 
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/communities/manager.go#2310-2310), lines 2310 to 2360
+
+Now let me check how these clocks are updated from the persistence layer and how magnetlink/indexCid messages are handled: 
+
+Searched text for `UpdateCommunityDescriptionIndexCidMessageClock`, 2 results
+
+Searched text for `UpdateCommunityDescriptionMagnetlinkMessageClock`, 2 results
+
+Searched codebase for "handle archive magnetlink index CID message torrent community", 14 results
+
+Now let me look at the index CID related code: 
+
+Searched text for `HandleIndexCidMessage`, no results
+
+Searched text for `dispatchIndexCidMessage`, 2 results
+
+Searched text for `CommunityMessageArchiveIndexCid`, 9 results
+
+Now let me look at these handler functions: 
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_handler.go#3785-3785), lines 3785 to 3825
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_communities.go#4145-4145), lines 4145 to 4205
+
+Now let me check the HandleHistoryArchiveIndexCidMessage: 
+
+Searched text for `HandleHistoryArchiveIndexCidMessage`, 2 results
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_handler.go#1344-1344), lines 1344 to 1414
+
+Perfect! Now let me check how the persistence saves this and look at the protobuf definition: 
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/protobuf/communities.proto#220-220), lines 220 to 270
+
+Searched text for `SaveCommunityArchiveInfo`, 2 results
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/communities/persistence.go#1023-1023), lines 1023 to 1083
+
+Perfect! Now I have a complete picture. Let me provide you with a comprehensive summary:
+
+## Summary: How `ArchiveIndexCidClock` and Community Description Messages Work
+
+### **Community Description Clocks for Archives**
+
+The community description contains two clock fields for tracking archive distribution:
+1. **`ArchiveMagnetlinkClock`** - tracks the clock of the last magnetlink (torrent) message  
+2. **`ArchiveIndexCidClock`** - tracks the clock of the last index CID (Codex) message
+
+These clocks are stored in the **community description protobuf** itself and are propagated when the community description is updated and distributed.
+
+---
+
+### **How Community Description Messages Are Sent**
+
+1. **Control node creates/updates community** → Community description is updated with current state
+2. **Community description is marshaled** → Converted to protobuf format
+3. **Wrapped in ApplicationMetadataMessage** → Signed and wrapped
+4. **Published to community channels** → Sent to all members via waku
+
+The community description includes:
+- All community metadata (name, description, members, etc.)
+- Token permissions
+- **Archive clocks**: `ArchiveMagnetlinkClock` and `ArchiveIndexCidClock`
+
+---
+
+### **How Community Description Messages Are Received**
+
+When a node receives a community description message:
+
+**In `handleCommunityDescriptionMessageCommon()` (line 2295):**
+
+```go
+// 1. Update the community with the new description
+changes, err := community.UpdateCommunityDescription(description, payload, newControlNode)
+
+// 2. Check if archive info exists in local DB
+hasCommunityArchiveInfo, err := m.persistence.HasCommunityArchiveInfo(community.ID())
+
+// 3. Extract clocks from the received community description
+cdMagnetlinkClock := community.config.CommunityDescription.ArchiveMagnetlinkClock
+cdIndexCidClock := community.config.CommunityDescription.ArchiveIndexCidClock
+
+// 4. If no archive info exists, save it with the clocks from CD
+if !hasCommunityArchiveInfo {
+    err = m.persistence.SaveCommunityArchiveInfo(community.ID(), cdMagnetlinkClock, 0, cdIndexCidClock)
+} else {
+    // 5. If archive info exists, compare clocks and update if CD has newer clock
+    
+    // For magnetlink:
+    magnetlinkClock, err := m.persistence.GetMagnetlinkMessageClock(community.ID())
+    if cdMagnetlinkClock > magnetlinkClock {
+        err = m.persistence.UpdateMagnetlinkMessageClock(community.ID(), cdMagnetlinkClock)
+    }
+    
+    // For index CID:
+    indexCidClock, err := m.persistence.GetIndexCidMessageClock(community.ID())
+    if cdIndexCidClock > indexCidClock {
+        err = m.persistence.UpdateIndexCidMessageClock(community.ID(), cdIndexCidClock)
+    }
+}
+```
+
+---
+
+### **The Key Insight: Community Description Clock Comparison**
+
+The **community description acts as a checkpoint** for archive distribution state. When nodes receive a community description:
+
+- **They compare the CD's archive clocks** with their local archive info clocks
+- **If CD has newer clocks**, they update their local tracking
+- This ensures that even if a node **missed** the actual magnetlink/indexCid messages, they'll know there's a newer archive available when they receive the next community description
+
+---
+
+### **How Magnetlink/IndexCid Messages Work**
+
+**When control node dispatches archive messages:**
+
+1. **Control node seeds archive** → Creates torrent/uploads to Codex
+2. **Dispatches message** (magnetlink or indexCid) → Sent to community
+3. **Updates CD clock** → Updates `ArchiveMagnetlinkClock` or `ArchiveIndexCidClock` in community description
+4. **Updates local tracking** → Also updates local persistence tables
+
+**When member receives archive message:**
+
+```go
+// In HandleHistoryArchiveIndexCidMessage (line 1344):
+
+// 1. Get the last clock from LOCAL tracking table
+lastIndexCidClock, err := m.communitiesManager.GetIndexCidMessageClock(id)
+lastMagnetlinkClock, err := m.communitiesManager.GetMagnetlinkMessageClock(id)
+
+// 2. Compare with incoming message clock
+// 3. If incoming clock >= lastClock, process it (download archives)
+// 4. Update local tracking clocks
+```
+
+---
+
+### **Why This Design?**
+
+This dual-tracking system (CD clocks + local tracking) provides:
+
+1. **Resilience**: If a node misses the actual archive message, the next community description will let them know there's a newer archive
+2. **Synchronization**: New members get the latest archive clock immediately when they join (via CD)
+3. **Deduplication**: Prevents re-downloading the same archive multiple times
+4. **Preference support**: Can compare clocks from both methods (torrent vs Codex) to support distribution preferences
+
+---
+
+### **Database Storage**
+
+All tracked in `communities_archive_info` table:
+- `magnetlink_clock` - last seen magnetlink message clock
+- `index_cid_clock` - last seen index CID message clock  
+- `last_magnetlink_uri` - actual magnetlink string
+- `last_index_cid` - actual CID string
+
+The community description clocks act as a "high water mark" that gets propagated with every CD update, while the local tracking tables record what archive messages were actually processed.
+
+marcinczenko: ok, so it looks like the clocks in community description prevent from downloading and processing outdated archives?
+
+Can you also show me:
+
+(1) where the control node opdates ArchiveMagnetlinkClock or ArchiveIndexCidClock in community description
+(2) when and where community description are sent (show me where it is in code as well)
+
+GitHub Copilot: Great question! Yes, exactly - the clocks in the community description act as a "high water mark" to prevent processing outdated archives. Let me show you the code for both parts:
+
+## (1) Where Control Node Updates Archive Clocks in Community Description 
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_communities.go#4081-4081), lines 4081 to 4131
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_communities.go#250-250), lines 250 to 290
+
+Now let me find where the community description clock is actually updated: 
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_communities.go#4130-4130), lines 4130 to 4160
+
+Perfect! Now let me see where `UpdateCommunityDescriptionMagnetlinkMessageClock` and `UpdateCommunityDescriptionIndexCidMessageClock` actually update the CD: 
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/communities/manager.go#3696-3696), lines 3696 to 3726
+
+Excellent! Now let me find where `SaveCommunity` triggers the community description to be published:
+
+## (2) When and Where Community Descriptions Are Sent 
+
+Searched text for `func (m *Manager) SaveCommunity`, 4 results
+
+Searched text for `func (m *Manager) saveAndPublish`, 2 results
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/communities/manager.go#5102-5102), lines 5102 to 5162
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/communities/manager.go#4604-4604), lines 4604 to 4644
+
+Now let me find the `publish` function to see how it actually sends the community description: 
+
+Searched text for `func (m *Manager) publish(`, 1 result
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/communities/manager.go#747-747), lines 747 to 767
+
+Now let me find where the subscription is consumed and the actual message is sent: 
+
+Searched text for `handleCommunitiesSubscription`, 3 results
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_communities.go#306-306), lines 306 to 406
+
+Perfect! Now let me find the `publishOrg` function: 
+
+Searched text for `func (m *Messenger) publishOrg`, 1 result
+
+Read [](file:///home/mc2/code/status-im/status-go/protocol/messenger_communities.go#115-115), lines 115 to 195
+
+Perfect! Now I have all the information. Let me provide you with a comprehensive answer:
+
+---
+
+## Summary: How Archive Clocks Are Updated and Community Descriptions Are Sent
+
+### (1) Where Control Node Updates Archive Clocks in Community Description
+
+**The Flow:**
+
+1. **Archive is seeded** → Archive manager detects seeding is complete
+2. **Signal is published** → `HistoryArchivesSeedingSignal` is sent via subscription
+3. **Control node dispatches archive messages** → Sends magnetlink/indexCid messages
+4. **Control node updates CD clocks** → Updates `ArchiveMagnetlinkClock` or `ArchiveIndexCidClock`
+
+**The Code Path:**
+
+```
+protocol/messenger_communities.go:250-270 (handleCommunitiesHistoryArchivesSubscription)
+└─> When HistoryArchivesSeedingSignal received and IsControlNode():
+    ├─> dispatchMagnetlinkMessage() (line 4099)
+    │   └─> UpdateCommunityDescriptionMagnetlinkMessageClock() (line 4138)
+    │       └─> manager.go:3696 - Sets community.config.CommunityDescription.ArchiveMagnetlinkClock
+    │           └─> SaveCommunity() - Saves to DB
+    │
+    └─> dispatchIndexCidMessage() (line 4145) 
+        └─> UpdateCommunityDescriptionIndexCidMessageClock() (line 4183)
+            └─> manager.go:3708 - Sets community.config.CommunityDescription.ArchiveIndexCidClock
+                └─> SaveCommunity() - Saves to DB
+```
+
+**Key Code Locations:**
+
+**In `messenger_communities.go:250-270`:**
+```go
+if c.IsControlNode() {
+    if sub.HistoryArchivesSeedingSignal.MagnetLink {
+        err := m.dispatchMagnetlinkMessage(sub.HistoryArchivesSeedingSignal.CommunityID)
+    }
+    if sub.HistoryArchivesSeedingSignal.IndexCid {
+        err := m.dispatchIndexCidMessage(sub.HistoryArchivesSeedingSignal.CommunityID)
+    }
+}
+```
+
+**In `messenger_communities.go:4138-4142`:**
+```go
+// After sending magnetlink message:
+err = m.communitiesManager.UpdateCommunityDescriptionMagnetlinkMessageClock(community.ID(), magnetLinkMessage.Clock)
+```
+
+**In `manager.go:3696-3705`:**
+```go
+func (m *Manager) UpdateCommunityDescriptionMagnetlinkMessageClock(communityID types.HexBytes, clock uint64) error {
+    community, err := m.GetByIDString(communityID.String())
+    community.config.CommunityDescription.ArchiveMagnetlinkClock = clock
+    return m.SaveCommunity(community)  // ← Saves to DB
+}
+```
+
+**Important:** Note that `SaveCommunity()` **only saves to the database** - it does NOT automatically publish the community description! The CD will be published later through the periodic check or when other changes trigger a publish.
+
+---
+
+### (2) When and Where Community Descriptions Are Sent
+
+**There are TWO main ways community descriptions are published:**
+
+#### **A. Event-Driven Publishing (Immediate)**
+
+When changes happen (like updating clocks, editing community, accepting members), the code calls:
+
+**`saveAndPublish()` → `manager.go:4608-4638`**
+```go
+func (m *Manager) saveAndPublish(community *Community) error {
+    err := m.SaveCommunity(community)  // Save to DB
+    if err != nil {
+        return err
+    }
+    
+    if community.IsControlNode() {
+        m.publish(&Subscription{Community: community})  // ← Publish event
+        return nil
+    }
+    // ... handle admin events
+}
+```
+
+**`publish()` → `manager.go:747-759`**
+```go
+func (m *Manager) publish(subscription *Subscription) {
+    for _, s := range m.subscriptions {
+        s <- subscription  // ← Send to subscription channel
+    }
+}
+```
+
+#### **B. Subscription Handler (Actual Network Send)**
+
+**`handleCommunitiesSubscription()` → `messenger_communities.go:307-406`**
+
+This goroutine listens to the subscription channel:
+
+```go
+func (m *Messenger) handleCommunitiesSubscription(c chan *communities.Subscription) {
+    // Checks every 5 minutes
+    ticker := time.NewTicker(5 * time.Minute)
+    
+    go func() {
+        for {
+            select {
+            case sub := <-c:  // ← Receives from subscription channel
+                if sub.Community != nil {
+                    publishOrgAndDistributeEncryptionKeys(community)
+                }
+            case <-ticker.C:  // ← Periodic check every 5 minutes
+                // ... republish if needed
+            }
+        }
+    }()
+}
+```
+
+The `publishOrgAndDistributeEncryptionKeys` function then calls:
+
+**`publishOrg()` → `messenger_communities.go:115-159`**
+```go
+func (m *Messenger) publishOrg(org *communities.Community, shouldRekey bool) error {
+    payload, err := org.MarshaledDescription()  // ← Marshal community description
+    
+    rawMessage := messagingtypes.RawMessage{
+        Payload:             payload,
+        Sender:              org.PrivateKey(),
+        SkipEncryptionLayer: true,
+        MessageType:         protobuf.ApplicationMetadataMessage_COMMUNITY_DESCRIPTION,
+        PubsubTopic:         org.PubsubTopic(),
+        Priority:            &messagingtypes.HighPriority,
+    }
+    
+    messageID, err := m.messaging.SendPublic(context.Background(), org.IDString(), rawMessage)
+    // ← Actually sends to the network via Waku
+    return err
+}
+```
+
+---
+
+### **The Complete Flow:**
+
+```
+1. Control Node Seeds Archive
+   └─> HistoryArchivesSeedingSignal published
+
+2. dispatchMagnetlinkMessage() or dispatchIndexCidMessage()
+   ├─> Sends actual magnetlink/indexCid message to network
+   └─> UpdateCommunityDescriptionMagnetlinkMessageClock()
+       └─> Updates community.config.CommunityDescription.ArchiveMagnetlinkClock
+           └─> SaveCommunity() - saves to DB only
+
+3. Later (when CD is published for any reason):
+   └─> saveAndPublish() called OR periodic check triggers
+       └─> publish(&Subscription{Community: community})
+           └─> handleCommunitiesSubscription receives event
+               └─> publishOrg()
+                   └─> MarshaledDescription() - includes updated clocks
+                       └─> SendPublic() - sends to Waku network
+
+4. Members receive Community Description
+   └─> HandleCommunityDescriptionMessage()
+       └─> handleCommunityDescriptionMessageCommon()
+           └─> Compares CD clocks with local tracking
+               └─> Updates local tracking if CD has newer clocks
+```
+
+---
+
+### **Key Insights:**
+
+1. **Archive clocks are updated in CD immediately** when magnetlink/indexCid messages are dispatched
+2. **CD is NOT immediately published** - it waits for the next publish event or periodic check
+3. **CD includes the updated clocks** whenever it's eventually published
+4. **This design ensures** that even if members miss the actual magnetlink/indexCid message, they'll get the updated clock in the next CD update
+5. **The clocks prevent reprocessing** - members won't download the same archive twice because they compare incoming message clocks against both the CD clocks and their local tracking
+
+This is why you can miss an archive message but still know there's a newer archive available - the CD acts as a "checkpoint" that gets propagated with every community update!
--- a/Notes/status-go
+++ b/Notes/status-go
@ -547,18 +547,18 @@ func (m *Messenger) handleRetrievedMessages(chatWithMessages map[messagingtypes.
 └─────────────────────────────────────────────────────────────────┘
                                ↓
 ┌─────────────────────────────────────────────────────────────────┐
-│                 MESSAGE COLLECTION                             │
-│  RetrieveAll() → messaging.RetrieveRawAll()                    │
-│       ↓                                                        │
-│  transport.RetrieveRawAll()                                    │
-│       ↓                                                        │
-│  For each filter: api.GetFilterMessages()                     │
-│       ↓                                                        │
-│  filter.Retrieve() → store.Pop()                              │
-│                                                                │
-│  Returns: map[ChatFilter][]*ReceivedMessage                   │
+│                 MESSAGE COLLECTION                              │
+│  RetrieveAll() → messaging.RetrieveRawAll()                     │
+│                       ↓                                         │
+│                  transport.RetrieveRawAll()                     │
+│                       ↓                                         │
+│                  For each filter: api.GetFilterMessages()       │
+│                       ↓                                         │
+│                  filter.Retrieve() → store.Pop()                │
+│                                                                 │
+│                  Returns: map[ChatFilter][]*ReceivedMessage     │
 └─────────────────────────────────────────────────────────────────┘
-                                ↓
+        ↓
 ┌─────────────────────────────────────────────────────────────────┐
 │                MESSAGE PROCESSING                              │
 │  handleRetrievedMessages(messages, true, false)               │
@ -575,6 +575,7 @@ func (m *Messenger) handleRetrievedMessages(chatWithMessages map[messagingtypes.
 └─────────────────────────────────────────────────────────────────┘
 ```

+And here is also a link to a diagram in excalidraw: [https://link.excalidraw.com/readonly/vSon9uiUhYJWrwXiKAsi](https://link.excalidraw.com/readonly/vSon9uiUhYJWrwXiKAsi).
 ### **6. Key Points About the Flow**

 1. **Asynchronous Message Queue**: Messages from network are queued and processed asynchronously
--- a/integration.md
+++ b/integration.md
@ -4,6 +4,13 @@ related-to:
 ---
 In [[Running Unit Tests for status-go]] we provide general notes on running unit tests in the status-go project. And then we have a similar note about functional tests in [[Running functional tests in status-go]].

+Also, to learn the history archive creation/upload and download/processing flows (recorded AI conversation with some edits), please check:
+
+- archive creation/upload: [[When are magnetlink messages sent]]
+- archive download/processing: [[status-go processing magnet links]]
+
+In that last document I am including a link to an excalidraw diagram that can be helpful - for convenience: [https://link.excalidraw.com/readonly/vSon9uiUhYJWrwXiKAsi](https://link.excalidraw.com/readonly/vSon9uiUhYJWrwXiKAsi)
+
 In this document, we focus on our Codex extension to status-go and here we focus on the related unit and integration tests.

 There is one existing test in status-go that has slightly more end-to-end nature. It is from the `protocol` package:
@ -275,7 +282,32 @@ I prefer to be more selective when running integration tests.

 This is about step 3 above: "Codex" version of `protocol/communities_messenger_token_permissions_test.go`.

-The test we are particularly interested in - `TestImportDecryptedArchiveMessages` - first creates a community and sets up the corresponding permissions. Then the community owner sends a message to the community and then immediately retrieves it so that it is now recorded in the DB.
+The test we are particularly interested in is `TestImportDecryptedArchiveMessages`.
+
+This test produces lots of output - with lot's warnings and errors - so looking at the log to judge the success would be a challenge. Yet, the test passes:
+
+```bash
+❯ gotestsum --packages="./protocol" -f testname -- -run "TestMessengerCommunitiesTokenPermissionsSuite/TestImportDecryptedArchiveMessages" -count 1 -tags "gowaku_no_rln gowaku_skip_migrations"
+PASS protocol.TestMessengerCommunitiesTokenPermissionsSuite/TestImportDecryptedArchiveMessages (1.88s)
+PASS protocol.TestMessengerCommunitiesTokenPermissionsSuite (1.88s)
+PASS protocol
+
+DONE 2 tests in 1.900s
+```
+
+If you want to take a look at the logs you can use the more verbose version of the above command:
+
+```bash
+❯ gotestsum --packages="./protocol" -f standard-verbose -- -run "TestMessengerCommunitiesTokenPermissionsSuite/TestImportDecryptedArchiveMessages" -v -count 1 -tags "gowaku_no_rln gowaku_skip_migrations"
+```
+
+and you can use `tee` to redirect all the output to a file:
+
+```bash
+❯ gotestsum --packages="./protocol" -f standard-verbose -- -run "TestMessengerCommunitiesTokenPermissionsSuite/TestImportDecryptedArchiveMessages" -v -count 1 -tags "gowaku_no_rln gowaku_skip_migrations" | tee "test_1.log"
+```
+
+The test first creates a community and sets up the corresponding permissions. Then the community owner sends a message to the community and then immediately retrieves it so that it is now recorded in the DB.

 After that it prepares archive parameters: `startDate`, `endDate`, `partition`, and community `topics`. All those will be passed to `CreateHistoryArchiveTorrentFromDB` - our entry point to creating history archive torrent.

@ -328,4 +360,121 @@ Notice, there is one archive expected.

 The `CreateHistoryArchiveTorrentFromDB` is called directly here, in a way bypassing the torrent seeding: in normal flow `CreateHistoryArchiveTorrentFromDB` is called in `CreateAndSeedHistoryArchive` which immediately after creating the archive, calls `SeedHistoryArchiveTorrent`.  `CreateHistoryArchiveTorrentFromDB` calls `createHistoryArchiveTorrent` - which is central to the archive creating.

-TBC...
+The "Codex" version of the `CreateHistoryArchiveTorrentFromDB` is `CreateHistoryArchiveCodexFromDB` which will call `createHistoryArchiveCodex` - this is where archives are created and uploaded to Codex.
+
+Another function that this test "touches" is `LoadHistoryArchiveIndexFromFile`, for which the "Codex" version is `CodexLoadHistoryArchiveIndexFromFile`.
+
+Thus, this test focuses on an important cut of the whole end-to-end flow and in our case, where we use Codex, it also indirectly tests seeding and retrieving the archives from the network.
+
+### Other places we can consider to test
+
+The integration test described above does not cover actual publishing of the generated archives over waku channel. This normally happens in `CreateAndSeedHistoryArchive`:
+
+```
+CreateAndSeedHistoryArchive
+ |- CreateHistoryArchiveTorrentFromDB
+ |- SeedHistoryArchiveTorrent
+ |- CreateHistoryArchiveCodexFromDB
+ |- publish: HistoryArchiveSeedingSignal
+```
+
+We see that we seed the index file and the archives for both torrent and Codex and when at least one publishing method succeeds, we do:
+
+```go
+m.publisher.publish(&Subscription{
+	HistoryArchivesSeedingSignal: &signal.HistoryArchivesSeedingSignal{
+		CommunityID: communityID.String(),
+		MagnetLink:  archiveTorrentCreatedSuccessfully, // true if torrent created successfully
+		IndexCid:    archiveCodexCreatedSuccessfully,   // true if codex created successfully
+	},
+})
+```
+
+This signal is subsequently received in `handleCommunitiesHistoryArchivesSubscription`, where we find:
+
+```go
+if c.IsControlNode() {
+	if sub.HistoryArchivesSeedingSignal.MagnetLink {
+		err := m.dispatchMagnetlinkMessage(sub.HistoryArchivesSeedingSignal.CommunityID)
+		if err != nil {
+			m.logger.Debug("failed to dispatch magnetlink message", zap.Error(err))
+		}
+	}
+	if sub.HistoryArchivesSeedingSignal.IndexCid {
+		err := m.dispatchIndexCidMessage(sub.HistoryArchivesSeedingSignal.CommunityID)
+		if err != nil {
+			m.logger.Debug("failed to dispatch index cid message", zap.Error(err))
+		}
+	}
+}
+```
+
+The `dispatchMagnetlinkMessage` and `dispatchIndexCidMessage` is where dispatching is happening. For Codex the MessageType used is `protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID`.
+
+The message is sent as follows:
+
+```go
+chatID := community.MagnetlinkMessageChannelID()
+rawMessage := messagingtypes.RawMessage{
+	LocalChatID:          chatID,
+	Sender:               community.PrivateKey(),
+	Payload:              encodedMessage,
+	MessageType:          protobuf.ApplicationMetadataMessage_COMMUNITY_MESSAGE_ARCHIVE_INDEX_CID,
+	SkipGroupMessageWrap: true,
+	PubsubTopic:          community.PubsubTopic(),
+	Priority:             &messagingtypes.LowPriority,
+}
+
+_, err = m.messaging.SendPublic(context.Background(), chatID, rawMessage)
+if err != nil {
+	return err
+}
+
+err = m.communitiesManager.UpdateCommunityDescriptionIndexCidMessageClock(community.ID(), indexCidMessage.Clock)
+if err != nil {
+	return err
+}
+return m.communitiesManager.UpdateIndexCidMessageClock(community.ID(), indexCidMessage.Clock)
+```
+
+Here notice the call to `UpdateCommunityDescriptionIndexCidMessageClock`. As you can see, the clocks are also recorded in the community description. Community description is sent over waku to other community members periodically (every 5mins) or on some events. This how it is sent in `publishOrg` in `protocol/messenger_communities.go`:
+
+```go
+rawMessage := messagingtypes.RawMessage{
+	Payload: payload,
+	Sender:  org.PrivateKey(),
+	// we don't want to wrap in an encryption layer message
+	SkipEncryptionLayer: true,
+	CommunityID:         org.ID(),
+	MessageType:         protobuf.ApplicationMetadataMessage_COMMUNITY_DESCRIPTION,
+	PubsubTopic:         org.PubsubTopic(), // TODO: confirm if it should be sent in community pubsub topic
+	Priority:            &messagingtypes.HighPriority,
+}
+if org.Encrypted() {
+	members := org.GetMemberPubkeys()
+	if err != nil {
+		return err
+	}
+	rawMessage.CommunityKeyExMsgType = messagingtypes.KeyExMsgRekey
+	// This should be the one that it was used to encrypt this community
+	rawMessage.HashRatchetGroupID = org.ID()
+	rawMessage.Recipients = members
+}
+messageID, err := m.messaging.SendPublic(context.Background(), org.IDString(), rawMessage)
+```
+
+When community members receive the community description, they compare the clocks in the community description with the most recent clocks of the most recent magnet link or index Cid. IF the clocks in the community description are newer (more recent), they update the local copies, so that when a new magnet link or index Cid message arrives, they know to ignore outdated messages.
+
+To read more, check: [[History Archives and Community Description]].
+
+This naturally brought us to the reception of the magnet links/index Cid (see also [[status-go processing magnet links]] and [[status-go publishing magnet links]]).
+
+Going from Waku and layers of status-go, the accumulated messages are periodically collected using `RetrieveAll` functions, which passes the message to `handleRetrievedMessages` for processing.
+
+For each message, `dispatchToHandler` is called (in it in generated file `message_handler.go` generated with `go generate ./cmd/generae_handlers/`) where are magnet links/index Cid message are forwarded to their respective handlers: `handleCommunityMessageArchiveMagnetlinkProtobuf` and `handleCommunityMessageArchiveIndexCidProtobuf`. In the end they end up in `HandleHistoryArchiveMagnetlinkMessage` and `HandleHistoryArchiveIndexCidMessage` respectively. At the end, they trigger async processing in go routines calling `downloadAndImportHistoryArchives` and `downloadAndImportCodexHistoryArchives`. Just after the respective go routines are created, the clock are updated with `m.communitiesManager.UpdateMagnetlinkMessageClock(id, clock)` and `m.communitiesManager.UpdateIndexCidMessageClock(id, clock)`.
+
+But before even starting go routine, we check the distribution preference with `GetArchiveDistributionPreference` and the last seen index Cid `m.communitiesManager.GetLastSeenIndexCid(id)` (it will be updated after downloading the archives but before processing them in `downloadAndImportCodexHistoryArchives`) and the corresponding clock `GetIndexCidMessageClock`, to make sure we are not processing some outdated archives.
+
+We can see that the above "integration test", in case of Codex, by calling `RetrieveAll` it effectively triggers archive download from Codex, yet it does not cover the dissemination of the indexCid - the test basically assumes that the index file has been stored, thus, if we want to cover more, we need to go beyond that test.
+
+Perhaps, to cover the whole flow, it is best to build status-desktop with our status-go library and test it from there.