2023-07-11 17:52:38 +00:00
---
2023-08-03 05:40:57 +00:00
title: Retrieve Messages Using Store Protocol
2023-11-23 11:01:32 +00:00
hide_table_of_contents: true
2023-07-11 17:52:38 +00:00
---
2023-11-23 11:01:32 +00:00
This guide provides detailed steps to create a Light Node for retrieving and filtering historical messages using the [Store protocol ](/learn/concepts/protocols#store ).
2023-07-11 17:52:38 +00:00
2023-11-23 11:01:32 +00:00
## Create a light node
2023-07-11 17:52:38 +00:00
2023-07-15 21:54:13 +00:00
Use the `createLightNode()` function to create a Light Node and interact with the Waku Network:
2023-07-11 17:52:38 +00:00
```js
import { createLightNode } from "@waku/sdk";
2023-07-15 21:54:13 +00:00
// Create and start a Light Node
2023-07-11 17:52:38 +00:00
const node = await createLightNode({ defaultBootstrap: true });
await node.start();
```
2023-11-23 11:01:32 +00:00
## Connect to store peers
2023-07-11 17:52:38 +00:00
2023-07-15 21:54:13 +00:00
Use the `waitForRemotePeer()` function to wait for the node to connect with Store peers:
2023-07-11 17:52:38 +00:00
```js
import { waitForRemotePeer, Protocols } from "@waku/sdk";
// Wait for a successful peer connection
await waitForRemotePeer(node, [Protocols.Store]);
```
2023-11-23 11:01:32 +00:00
## Choose a content topic
2023-07-11 17:52:38 +00:00
2023-11-23 11:01:32 +00:00
[Choose a content topic ](/learn/concepts/content-topics ) for filtering the messages to retrieve and create a message `decoder` :
2023-07-11 17:52:38 +00:00
```js
import { createDecoder } from "@waku/sdk";
// Choose a content topic
const contentTopic = "/store-guide/1/message/proto";
// Create a message decoder
const decoder = createDecoder(contentTopic);
```
2023-11-23 11:01:32 +00:00
## Retrieve messages
2023-07-11 17:52:38 +00:00
2023-09-07 02:14:42 +00:00
`@waku/sdk` provides the `queryWithOrderedCallback()` and `queryGenerator()` functions for querying `Store` nodes and retrieving historical or missed messages. The responses from `Store` nodes are paginated and require you to process each page sequentially.
2023-07-11 17:52:38 +00:00
2023-09-07 02:14:42 +00:00
### `queryWithOrderedCallback`
2023-07-11 17:52:38 +00:00
2023-09-07 02:14:42 +00:00
The `store.queryWithOrderedCallback()` function provides a straightforward method for querying `Store` nodes and processing messages in chronological order through a callback function. It accepts these parameters:
2023-07-11 17:52:38 +00:00
2024-05-16 17:22:15 +00:00
- `decoders` : List of `decoders` that specify the `content topic` to query for and their [message decryption ](https://rfc.vac.dev/waku/standards/application/26/payload ) methods.
2023-07-11 17:52:38 +00:00
- `callback` : The callback function for processing the retrieved messages.
- `options` (optional): [Query options ](/guides/js-waku/store-retrieve-messages#store-query-options ) to filter the retrieved messages.
```js
// Create the callback function
const callback = (wakuMessage) => {
2024-05-16 17:22:15 +00:00
// Render the message/payload in your application
console.log(wakuMessage);
2023-07-11 17:52:38 +00:00
};
2023-07-15 21:54:13 +00:00
// Query the Store peer
2024-05-16 17:22:15 +00:00
await node.store.queryWithOrderedCallback([decoder], callback);
2023-07-11 17:52:38 +00:00
```
2023-07-27 12:50:30 +00:00
:::info
2023-09-07 02:14:42 +00:00
The `queryWithOrderedCallback()` function always returns the most recent messages in a page first.
2023-07-27 12:50:30 +00:00
:::
2023-07-11 17:52:38 +00:00
### `queryGenerator`
The `store.queryGenerator()` function provides more control and flexibility over processing messages retrieved from `Store` nodes through [Async Generators ](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/AsyncGenerator ). It accepts these parameters:
2024-05-16 17:22:15 +00:00
- `decoders` : List of `decoders` that specify the `content topic` to query for and their [message decryption ](https://rfc.vac.dev/waku/standards/application/26/payload ) methods.
2023-07-11 17:52:38 +00:00
- `options` (optional): [Query options ](/guides/js-waku/store-retrieve-messages#store-query-options ) to filter the retrieved messages.
```js
// Create the store query
2023-07-27 12:50:30 +00:00
const storeQuery = node.store.queryGenerator([decoder]);
2023-07-11 17:52:38 +00:00
// Process the messages
for await (const messagesPromises of storeQuery) {
2024-05-16 17:22:15 +00:00
// Fulfil the messages promises
const messages = await Promise.all(
messagesPromises.map(async (p) => {
const msg = await p;
// Render the message/payload in your application
console.log(msg);
})
);
2023-07-11 17:52:38 +00:00
}
```
2023-07-27 12:50:30 +00:00
:::info
The `queryGenerator()` function always returns the oldest messages in a page first.
:::
2023-11-23 11:01:32 +00:00
## Store query options
2023-07-11 17:52:38 +00:00
### `pageDirection`
The `pageDirection` option specifies the direction in which pages are retrieved:
- `BACKWARD` (default): Most recent page first.
- `FORWARD` : Oldest page first.
```js
import { PageDirection } from "@waku/sdk";
// Retrieve recent messages first
const queryOptions = {
2024-05-16 17:22:15 +00:00
pageDirection: PageDirection.BACKWARD,
2023-07-11 17:52:38 +00:00
};
// Retrieve oldest messages first
const queryOptions = {
2024-05-16 17:22:15 +00:00
pageDirection: PageDirection.FORWARD,
2023-07-11 17:52:38 +00:00
};
2023-07-27 12:50:30 +00:00
// Query the Store peer with options
2023-09-07 02:14:42 +00:00
await node.store.queryWithOrderedCallback([decoder], callback, options);
2023-07-27 12:50:30 +00:00
const storeQuery = node.store.queryGenerator([decoder, options]);
2023-07-11 17:52:38 +00:00
```
### `cursor`
2023-07-18 18:51:12 +00:00
The `cursor` option specifies the starting index for retrieving messages. For example, consider a query that retrieves the first page messages and then continues with the next page:
2023-07-11 17:52:38 +00:00
```js
import { waku } from "@waku/sdk";
// Create the callback function
const messages = [];
const callback = (wakuMessage) => {
2024-05-16 17:22:15 +00:00
messages.push(wakuMessage);
// Return "true" to stop retrieving pages
// Here, it retrieves only the first page
return true;
2023-07-11 17:52:38 +00:00
};
2023-07-18 18:51:12 +00:00
// Retrieve the first page of messages
2023-07-27 12:50:30 +00:00
// This retrieves all the messages if "return true" is not present
2024-05-16 17:22:15 +00:00
await node.store.queryWithOrderedCallback([decoder], callback);
2023-07-11 17:52:38 +00:00
// Create the cursor
const lastMessage = messages[messages.length - 1];
const cursor = await waku.createCursor(lastMessage);
2023-07-18 18:51:12 +00:00
// Retrieve the next page of messages
2023-07-11 17:52:38 +00:00
// The message at the cursor index is excluded from the result
2024-05-16 17:22:15 +00:00
await node.store.queryWithOrderedCallback([decoder], callback, {
cursor: cursor,
});
2023-07-11 17:52:38 +00:00
console.log(messages);
```
:::info
If you omit the `cursor` option, the query will start from the beginning or end of the history, depending on the [page direction ](#pagedirection ).
:::
2023-07-27 12:50:30 +00:00
### `timeFilter`
2023-07-11 17:52:38 +00:00
2023-07-27 12:50:30 +00:00
The `timeFilter` option specifies a time frame to retrieve messages from. For example, consider a query that retrieves messages from the previous week:
2023-07-11 17:52:38 +00:00
```js
2023-07-27 12:50:30 +00:00
// Get the time frame
const endTime = new Date();
const startTime = new Date();
startTime.setDate(endTime.getDate() - 7);
// Retrieve a week of messages
2023-07-11 17:52:38 +00:00
const queryOptions = {
2024-05-16 17:22:15 +00:00
timeFilter: {
startTime,
endTime,
},
2023-07-11 17:52:38 +00:00
};
2023-07-27 12:50:30 +00:00
// Query the Store peer with options
2023-09-07 02:14:42 +00:00
await node.store.queryWithOrderedCallback([decoder], callback, options);
2023-07-27 12:50:30 +00:00
const storeQuery = node.store.queryGenerator([decoder, options]);
2023-07-11 17:52:38 +00:00
```
2023-07-27 12:50:30 +00:00
:::info
2023-08-01 19:39:04 +00:00
The `timeFilter` option significantly reduces message retrieval performance. To optimise it, consider resuming message retrieval using a [cursor ](#cursor ) that starts from the last seen message.
2023-07-27 12:50:30 +00:00
:::
2023-07-11 17:52:38 +00:00
:::tip Congratulations!
2023-08-01 19:39:04 +00:00
You have successfully retrieved and filtered historical messages on a Light Node using the `Store` protocol. Have a look at the [store-js ](https://github.com/waku-org/js-waku-examples/tree/master/examples/store-js ) and [store-reactjs-chat ](https://github.com/waku-org/js-waku-examples/tree/master/examples/store-reactjs-chat ) examples for working demos.
2024-05-16 17:22:15 +00:00
:::