feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
import { ContentTopic, IDecodedMessage } from "@waku/interfaces";
|
|
|
|
|
import { isAutoShardingRoutingInfo, Logger, RoutingInfo } from "@waku/utils";
|
2024-01-24 18:24:03 +05:30
|
|
|
import { expect } from "chai";
|
|
|
|
|
|
2024-09-13 14:18:29 +05:30
|
|
|
import { Args, MessageRpcQuery, MessageRpcResponse } from "../types.js";
|
2024-01-24 18:24:03 +05:30
|
|
|
import { delay, makeLogFileName } from "../utils/index.js";
|
|
|
|
|
|
|
|
|
|
import { MessageCollector } from "./message_collector.js";
|
2024-04-28 11:15:17 +02:00
|
|
|
import { runNodes } from "./runNodes.js";
|
2024-01-24 18:24:03 +05:30
|
|
|
import { defaultArgs, ServiceNode } from "./service_node.js";
|
|
|
|
|
|
|
|
|
|
export { ServiceNode, MessageCollector, defaultArgs };
|
2024-04-28 11:15:17 +02:00
|
|
|
export { runNodes };
|
2024-01-24 18:24:03 +05:30
|
|
|
|
|
|
|
|
const log = new Logger("test:message-collector");
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* This class is a wrapper over the ServiceNode & MessageCollector class
|
|
|
|
|
* that allows for the creation & handling of multiple ServiceNodes
|
|
|
|
|
*/
|
|
|
|
|
export class ServiceNodesFleet {
|
2024-07-19 15:58:17 +05:30
|
|
|
public static async createAndRun(
|
2024-01-24 18:24:03 +05:30
|
|
|
mochaContext: Mocha.Context,
|
|
|
|
|
nodesToCreate: number = 3,
|
|
|
|
|
strictChecking: boolean = false,
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
routingInfo: RoutingInfo,
|
2024-01-24 18:24:03 +05:30
|
|
|
_args?: Args,
|
|
|
|
|
withoutFilter = false
|
|
|
|
|
): Promise<ServiceNodesFleet> {
|
2025-04-11 01:34:11 +02:00
|
|
|
const nodes: ServiceNode[] = [];
|
|
|
|
|
|
|
|
|
|
for (let i = 0; i < nodesToCreate; i++) {
|
|
|
|
|
const node = new ServiceNode(
|
|
|
|
|
makeLogFileName(mochaContext) + Math.random().toString(36).substring(7)
|
|
|
|
|
);
|
2024-01-24 18:24:03 +05:30
|
|
|
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
const args = applyDefaultArgs(routingInfo, _args);
|
2024-01-24 18:24:03 +05:30
|
|
|
|
2025-04-11 01:34:11 +02:00
|
|
|
if (nodes[0]) {
|
|
|
|
|
const addr = await nodes[0].getExternalMultiaddr();
|
|
|
|
|
args.staticnode = addr ?? args.staticnode;
|
2024-01-24 18:24:03 +05:30
|
|
|
}
|
|
|
|
|
|
2025-04-11 01:34:11 +02:00
|
|
|
await node.start(args, {
|
|
|
|
|
retries: 3
|
|
|
|
|
});
|
|
|
|
|
|
|
|
|
|
nodes.push(node);
|
|
|
|
|
}
|
|
|
|
|
|
2024-01-24 18:24:03 +05:30
|
|
|
return new ServiceNodesFleet(nodes, withoutFilter, strictChecking);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Convert a [[WakuMessage]] to a [[WakuRelayMessage]]. The latter is used
|
|
|
|
|
* by the nwaku JSON-RPC API.
|
|
|
|
|
*/
|
2024-07-19 15:58:17 +05:30
|
|
|
public static toMessageRpcQuery(message: {
|
2024-01-24 18:24:03 +05:30
|
|
|
payload: Uint8Array;
|
|
|
|
|
contentTopic: string;
|
|
|
|
|
timestamp?: Date;
|
|
|
|
|
}): MessageRpcQuery {
|
|
|
|
|
return ServiceNode.toMessageRpcQuery(message);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
public messageCollector: MultipleNodesMessageCollector;
|
|
|
|
|
private constructor(
|
|
|
|
|
public nodes: ServiceNode[],
|
|
|
|
|
relay: boolean,
|
|
|
|
|
private strictChecking: boolean
|
|
|
|
|
) {
|
|
|
|
|
const _messageCollectors: MessageCollector[] = [];
|
|
|
|
|
this.nodes.forEach((node) => {
|
|
|
|
|
_messageCollectors.push(new MessageCollector(node));
|
|
|
|
|
});
|
|
|
|
|
this.messageCollector = new MultipleNodesMessageCollector(
|
|
|
|
|
_messageCollectors,
|
|
|
|
|
relay ? this.nodes : undefined,
|
|
|
|
|
strictChecking
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
|
2024-07-19 15:58:17 +05:30
|
|
|
public async start(): Promise<void> {
|
2024-01-24 18:24:03 +05:30
|
|
|
const startPromises = this.nodes.map((node) => node.start());
|
|
|
|
|
await Promise.all(startPromises);
|
|
|
|
|
}
|
|
|
|
|
|
2024-07-19 15:58:17 +05:30
|
|
|
public async sendRelayMessage(
|
2024-01-24 18:24:03 +05:30
|
|
|
message: MessageRpcQuery,
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
routingInfo: RoutingInfo
|
2024-01-24 18:24:03 +05:30
|
|
|
): Promise<boolean> {
|
2024-02-15 17:53:51 -08:00
|
|
|
const relayMessagePromises: Promise<boolean>[] = this.nodes.map((node) =>
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
node.sendMessage(message, routingInfo)
|
2024-02-15 17:53:51 -08:00
|
|
|
);
|
2024-01-24 18:24:03 +05:30
|
|
|
const relayMessages = await Promise.all(relayMessagePromises);
|
|
|
|
|
return relayMessages.every((message) => message);
|
|
|
|
|
}
|
|
|
|
|
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
/**
|
|
|
|
|
* This is a dodgy things to do as it assumes the nwaku node did not flush
|
|
|
|
|
* any messages from its cache.
|
|
|
|
|
*/
|
2025-02-22 03:36:45 +05:30
|
|
|
public async confirmMessageLength(numMessages: number): Promise<void> {
|
2024-01-24 18:24:03 +05:30
|
|
|
if (this.strictChecking) {
|
|
|
|
|
await Promise.all(
|
|
|
|
|
this.nodes.map(async (node) =>
|
|
|
|
|
expect(await node.messages()).to.have.length(numMessages)
|
|
|
|
|
)
|
|
|
|
|
);
|
|
|
|
|
} else {
|
|
|
|
|
// Wait for all promises to resolve and check if any meets the condition
|
|
|
|
|
const results = await Promise.all(
|
|
|
|
|
this.nodes.map(async (node) => {
|
|
|
|
|
const msgs = await node.messages();
|
|
|
|
|
return msgs.length === numMessages;
|
|
|
|
|
})
|
|
|
|
|
);
|
|
|
|
|
|
|
|
|
|
// Check if at least one result meets the condition
|
|
|
|
|
const conditionMet = results.some((result) => result);
|
|
|
|
|
expect(conditionMet).to.be.true;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
class MultipleNodesMessageCollector {
|
2025-10-13 16:35:01 +11:00
|
|
|
public callback: (msg: Partial<IDecodedMessage>) => void = () => {};
|
|
|
|
|
protected messageList: Array<Partial<IDecodedMessage>> = [];
|
2024-07-19 15:58:17 +05:30
|
|
|
public constructor(
|
2024-01-24 18:24:03 +05:30
|
|
|
private messageCollectors: MessageCollector[],
|
|
|
|
|
private relayNodes?: ServiceNode[],
|
|
|
|
|
private strictChecking: boolean = false
|
|
|
|
|
) {
|
2025-10-13 16:35:01 +11:00
|
|
|
this.callback = (msg: Partial<IDecodedMessage>): void => {
|
2024-01-24 18:24:03 +05:30
|
|
|
log.info("Got a message");
|
|
|
|
|
this.messageList.push(msg);
|
|
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
2024-07-19 15:58:17 +05:30
|
|
|
public get count(): number {
|
2024-01-24 18:24:03 +05:30
|
|
|
return this.messageList.length;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
public hasMessage(topic: string, text: string): boolean {
|
|
|
|
|
if (this.strictChecking) {
|
|
|
|
|
return this.messageCollectors.every((collector) =>
|
|
|
|
|
collector.hasMessage(topic, text)
|
|
|
|
|
);
|
|
|
|
|
} else {
|
|
|
|
|
return this.messageCollectors.some((collector) =>
|
|
|
|
|
collector.hasMessage(topic, text)
|
|
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2025-10-13 16:35:01 +11:00
|
|
|
public getMessage(
|
|
|
|
|
index: number
|
|
|
|
|
): MessageRpcResponse | Partial<IDecodedMessage> {
|
2024-01-24 18:24:03 +05:30
|
|
|
return this.messageList[index];
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Verifies a received message against expected values on all nodes.
|
|
|
|
|
* Returns true if any node's collector verifies the message successfully.
|
|
|
|
|
*/
|
2024-07-19 15:58:17 +05:30
|
|
|
public verifyReceivedMessage(
|
2024-01-24 18:24:03 +05:30
|
|
|
index: number,
|
|
|
|
|
options: {
|
|
|
|
|
expectedMessageText: string | Uint8Array | undefined;
|
|
|
|
|
expectedContentTopic?: string;
|
|
|
|
|
expectedPubsubTopic?: string;
|
|
|
|
|
expectedVersion?: number;
|
|
|
|
|
expectedMeta?: Uint8Array;
|
|
|
|
|
expectedEphemeral?: boolean;
|
|
|
|
|
expectedTimestamp?: bigint | number;
|
|
|
|
|
checkTimestamp?: boolean;
|
|
|
|
|
}
|
|
|
|
|
): boolean {
|
|
|
|
|
if (this.strictChecking) {
|
2025-02-22 03:36:45 +05:30
|
|
|
return this.messageCollectors.every((collector) => {
|
2024-01-24 18:24:03 +05:30
|
|
|
try {
|
|
|
|
|
collector.verifyReceivedMessage(index, options);
|
2025-02-22 03:36:45 +05:30
|
|
|
return true; // Verification successful
|
2024-01-24 18:24:03 +05:30
|
|
|
} catch (error) {
|
2025-02-22 03:36:45 +05:30
|
|
|
return false; // Verification failed, continue with the next collector
|
2024-01-24 18:24:03 +05:30
|
|
|
}
|
|
|
|
|
});
|
|
|
|
|
} else {
|
2025-02-22 03:36:45 +05:30
|
|
|
return this.messageCollectors.some((collector) => {
|
2024-01-24 18:24:03 +05:30
|
|
|
try {
|
|
|
|
|
collector.verifyReceivedMessage(index, options);
|
2025-02-22 03:36:45 +05:30
|
|
|
return true; // Verification successful
|
2024-01-24 18:24:03 +05:30
|
|
|
} catch (error) {
|
2025-02-22 03:36:45 +05:30
|
|
|
return false; // Verification failed, continue with the next collector
|
2024-01-24 18:24:03 +05:30
|
|
|
}
|
|
|
|
|
});
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Waits for a total number of messages across all nodes.
|
|
|
|
|
*/
|
2024-07-19 15:58:17 +05:30
|
|
|
public async waitForMessages(
|
2024-01-24 18:24:03 +05:30
|
|
|
numMessages: number,
|
|
|
|
|
options?: {
|
|
|
|
|
timeoutDuration?: number;
|
|
|
|
|
exact?: boolean;
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
contentTopic?: ContentTopic;
|
2024-01-24 18:24:03 +05:30
|
|
|
}
|
2025-02-17 19:21:33 +05:30
|
|
|
): Promise<boolean> {
|
|
|
|
|
const startTime = Date.now();
|
|
|
|
|
const timeoutDuration = options?.timeoutDuration || 400;
|
2024-01-24 18:24:03 +05:30
|
|
|
const exact = options?.exact || false;
|
|
|
|
|
|
|
|
|
|
while (this.messageList.length < numMessages) {
|
|
|
|
|
if (this.relayNodes) {
|
|
|
|
|
if (this.strictChecking) {
|
|
|
|
|
const results = await Promise.all(
|
2025-02-22 03:36:45 +05:30
|
|
|
this.relayNodes.map(async (node) => {
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
const msgs = await node.messages(options?.contentTopic);
|
2025-02-22 03:36:45 +05:30
|
|
|
return msgs.length >= numMessages;
|
2024-01-24 18:24:03 +05:30
|
|
|
})
|
|
|
|
|
);
|
2025-02-22 03:36:45 +05:30
|
|
|
return results.every((result) => result);
|
2024-01-24 18:24:03 +05:30
|
|
|
} else {
|
|
|
|
|
const results = await Promise.all(
|
2025-02-22 03:36:45 +05:30
|
|
|
this.relayNodes.map(async (node) => {
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
const msgs = await node.messages(options?.contentTopic);
|
2025-02-22 03:36:45 +05:30
|
|
|
return msgs.length >= numMessages;
|
2024-01-24 18:24:03 +05:30
|
|
|
})
|
|
|
|
|
);
|
2025-02-22 03:36:45 +05:30
|
|
|
return results.some((result) => result);
|
2025-02-17 19:21:33 +05:30
|
|
|
}
|
2025-02-22 03:36:45 +05:30
|
|
|
}
|
2024-01-24 18:24:03 +05:30
|
|
|
|
2025-02-22 03:36:45 +05:30
|
|
|
if (Date.now() - startTime > timeoutDuration * numMessages) {
|
|
|
|
|
return false;
|
2025-02-17 19:21:33 +05:30
|
|
|
}
|
2025-02-22 03:36:45 +05:30
|
|
|
|
|
|
|
|
await delay(10);
|
2024-01-24 18:24:03 +05:30
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (exact) {
|
|
|
|
|
if (this.messageList.length == numMessages) {
|
|
|
|
|
return true;
|
|
|
|
|
} else {
|
|
|
|
|
log.warn(
|
|
|
|
|
`Was expecting exactly ${numMessages} messages. Received: ${this.messageList.length}`
|
|
|
|
|
);
|
2025-02-22 03:36:45 +05:30
|
|
|
|
2024-01-24 18:24:03 +05:30
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
function applyDefaultArgs(routingInfo: RoutingInfo, args?: Args): Args {
|
|
|
|
|
const defaultArgs: Args = {
|
2024-01-24 18:24:03 +05:30
|
|
|
lightpush: true,
|
|
|
|
|
filter: true,
|
|
|
|
|
discv5Discovery: true,
|
|
|
|
|
peerExchange: true,
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
relay: true
|
|
|
|
|
};
|
|
|
|
|
|
2025-07-19 14:45:06 +10:00
|
|
|
defaultArgs.clusterId = routingInfo.clusterId;
|
feat!: Introduce routing info concept
Concepts are being mixed up between the global network config (static vs auto sharding), that needs to be the same of all nodes in the network, individual node configuration (eg relay node subscribing to a given shard), and the routing characteristic of a specific message (eg pubsub topic, shard).
This stops proper configuration of nwaku post 0.36.0 because we know need to be deliberate on whether nwaku nodes are running with auto or static sharding.
It also included various back and forth conversions between shards, pubsub topics, etc.
With this change, we tidy up the network configuration, and make it explicit whether it is static or auto sharded.
We also introduce the concept of routing info, which is specific to a message, and tied to the overall network configuration.
Routing info abstract pubsub topic, shard, and autosharding needs. Which should lead to easier tidy up of the pubsub concept at a later stage.
# Conflicts:
# packages/core/src/lib/connection_manager/connection_manager.ts
# packages/core/src/lib/metadata/metadata.ts
# packages/interfaces/src/metadata.ts
# packages/interfaces/src/sharding.ts
# packages/relay/src/create.ts
# packages/sdk/src/filter/filter.ts
# packages/sdk/src/filter/types.ts
# packages/sdk/src/light_push/light_push.spec.ts
# packages/tests/tests/sharding/auto_sharding.spec.ts
# packages/tests/tests/sharding/static_sharding.spec.ts
# Conflicts:
# packages/sdk/src/store/store.ts
2025-07-11 13:33:45 +10:00
|
|
|
|
|
|
|
|
if (isAutoShardingRoutingInfo(routingInfo)) {
|
|
|
|
|
defaultArgs.numShardsInNetwork =
|
|
|
|
|
routingInfo.networkConfig.numShardsInCluster;
|
|
|
|
|
|
|
|
|
|
defaultArgs.contentTopic = [routingInfo.contentTopic];
|
|
|
|
|
} else {
|
|
|
|
|
defaultArgs.numShardsInNetwork = 0;
|
|
|
|
|
defaultArgs.shard = [routingInfo.shardId];
|
2025-04-11 01:34:11 +02:00
|
|
|
}
|
|
|
|
|
|
2024-01-24 18:24:03 +05:30
|
|
|
return { ...defaultArgs, ...args };
|
|
|
|
|
}
|