Merge pull request #286 from status-im/encryption-api

This commit is contained in:
Franck Royer 2021-09-02 15:45:53 +10:00 committed by GitHub
commit 524fbc9361
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 243 additions and 130 deletions

View File

@ -10,9 +10,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Examples (eth-pm): Encrypt Public Key Messages using symmetric encryption.
- Guides: Encrypt messages using Waku Message Version 1.
- Allow passing decryption keys in hex string format.
- Allow passing decryption keys to `WakuStore` instance to avoid having to pass them at every `queryHistory` call.
- Allow passing decryption keys to `Waku` instance to avoid having to pass them to both `WakuRelay` and `WakuStore`.
### Changed
- **Breaking**: Moved `startTime` and `endTime` for history queries to a `timeFilter` property as both or neither must be passed; passing only one parameter is not supported.
- Renamed and promote the usage of `generateSymmetricKey()` to generate random symmetric keys.
- Improved errors thrown by `WakuStore.queryHistory`.
### Fixed
- Buffer concat error when using symmetric encryption in the browser.

View File

@ -138,14 +138,14 @@ A quick note on the cryptographic libraries used as it is a not a straightforwar
Asymmetric private keys and symmetric keys are expected to be 32 bytes arrays.
```ts
import { generatePrivateKey, getPublicKey } from 'js-waku';
import { generatePrivateKey, generateSymmetricKey, getPublicKey } from 'js-waku';
// Asymmetric
const privateKey = generatePrivateKey();
const publicKey = getPublicKey(privateKey);
// Symmetric
const symKey = generatePrivateKey();
const symKey = generateSymmetricKey();
```
#### Encrypt Waku Messages

View File

@ -28,7 +28,7 @@ If key recovery is important for your dApp, then check out
An example to save and load a key pair in local storage, protected with a password, can be found in [Eth-PM](https://github.com/status-im/js-waku/blob/main/examples/eth-pm/src/key_pair_handling/key_pair_storage.ts).
## Which encryption method do I need?
## Which encryption method should I use?
Whether you should use symmetric or asymmetric encryption depends on your use case.
@ -59,12 +59,12 @@ When Alice sends an encrypted message for Bob, only Bob can decrypt it.
### Generate Key
To use symmetric encryption, you first need to generate a key.
You can simply use `generatePrivateKey` for secure key generation:
Use `generateSymmetricKey` for secure key generation:
```js
import { generatePrivateKey } from 'js-waku';
import { generateSymmetricKey } from 'js-waku';
const key = generatePrivateKey();
const symmetricKey = generateSymmetricKey();
```
### Encrypt Message
@ -80,7 +80,7 @@ See [Receive and Send Messages Using Waku Relay](relay-receive-send-messages.md)
import { WakuMessage } from 'js-waku';
const message = await WakuMessage.fromBytes(payload, contentTopic, {
symKey: key
symKey: symmetricKey
});
```
@ -92,56 +92,34 @@ await waku.lightPush.push(message);
### Decrypt Messages
#### Waku Relay
To decrypt messages received over Waku Relay, add the key as a decryption key to your Waku Relay instance.
To decrypt messages,
whether they are received over Waku Relay or using Waku Store,
add the symmetric key as a decryption key to your Waku instance.
```js
waku.relay.addDecryptionKey(key);
waku.addDecryptionKey(symmetricKey);
```
Alternatively, you can pass the key when creating the instance:
```js
import { Waku } from 'js-waku';
const waku = Waku.create({ decryptionKeys: [symmetricKey] });
```
`waku.relay` will attempt to decrypt any message it receives using the key, for both symmetric and asymmetric encryption.
If the message is successfully decrypted, then the decrypted messages will be passed to the observers you have registered.
It will attempt to decrypt any message it receives using the key, for both symmetric and asymmetric encryption.
You can call `addDecryptionKey` several times if you are using multiple keys,
symmetric key and asymmetric private keys can be used together.
Messages that are not successfully decrypted are dropped.
To learn more about Waku Relay, check out [Receive and Send Messages Using Waku Relay](relay-receive-send-messages.md).
#### Waku Store
To decrypt messages retrieved via a store query,
pass the `key` to the query in the `decryptionKeys` property.
`decryptionKeys` accepts an array, allowing you to pass several keys.
Symmetric keys or asymmetric private keys can be mixed, both decryption methods will be attempted.
```js
// Using await syntax
const messages = await waku.store.queryHistory([contentTopic], {
decryptionKeys: [key]
});
// Using callback syntax
waku.store.queryHistory([contentTopic], {
decryptionKeys: [key],
callback: (messages) => {
// Process decrypted messages
}
});
```
Messages that are not successfully decrypted are excluded from the result array.
## Asymmetric Encryption
### Generate Key Pair
To use symmetric encryption, you first need to generate a private key and calculate the corresponding public key.
You can simply use `generatePrivateKey` for secure key generation:
To use asymmetric encryption, you first need to generate a private key and calculate the corresponding public key.
Use `generatePrivateKey` for secure key generation:
```js
import { generatePrivateKey, getPublicKey } from 'js-waku';
@ -181,52 +159,30 @@ await waku.lightPush.push(message);
### Decrypt Messages
#### Waku Relay
The private key is needed to decrypt messages.
For messages received over Waku Relay, add the private key as a decryption key to your Waku Relay instance.
To decrypt messages,
whether they are received over Waku Relay or using Waku Store,
add the private key as a decryption key to your Waku instance.
```js
waku.relay.addDecryptionKey(privateKey);
waku.addDecryptionKey(privateKey);
```
Alternatively, you can pass the key when creating the instance:
```js
import { Waku } from 'js-waku';
const waku = Waku.create({ decryptionKeys: [privateKey] });
```
`waku.relay` will attempt to decrypt any message it receives using the key, for both symmetric and asymmetric encryption.
If the message is successfully decrypted, then the decrypted messages will be passed to the observers you have registered.
It will attempt to decrypt any message it receives using the key, for both symmetric and asymmetric encryption.
You can call `addDecryptionKey` several times if you are using multiple keys,
symmetric key and asymmetric private keys can be used together.
Messages that are not successfully decrypted are dropped.
To learn more about Waku Relay, check out [Receive and Send Messages Using Waku Relay](relay-receive-send-messages.md).
#### Waku Store
To decrypt messages retrieved via a store query,
pass the `key` to the query in the `decryptionKeys` property.
`decryptionKeys` accepts an array, allowing you to pass several keys.
Symmetric keys or asymmetric private keys can be mixed, both decryption methods will be attempted.
```js
// Using await syntax
const messages = await waku.store.queryHistory([contentTopic], {
decryptionKeys: [privateKey],
});
// Using callback syntax
waku.store.queryHistory([contentTopic], {
decryptionKeys: [privateKey],
callback: (messages) => {
// Process decrypted messages
},
});
```
Messages that are not successfully decrypted are excluded from the result array.
## Handling `WakuMessage` instances
When creating a Waku Message using `WakuMessage.fromBytes` with an encryption key (symmetric or asymmetric),
@ -251,16 +207,16 @@ If a message was not successfully decrypted, then it will be dropped from the re
Which means that `WakuMessage` instances returned by `WakuRelay` and `WakuStore` always have a clear payload (in regard to Waku Message version 1):
```js
const messages = await waku.store.queryHistory([contentTopic], {
decryptionKeys: [privateKey]
});
import { Waku } from 'js-waku';
const waku = Waku.create({ decryptionKeys: [privateKey] });
const messages = await waku.store.queryHistory([contentTopic]);
if (messages && messages[0]) {
console.log(messages[0].payload); // This payload is decrypted
}
waku.relay.addDecryptionKey(privateKey);
waku.relay.addObserver((message) => {
console.log(message.payload); // This payload is decrypted
}, [contentTopic]);

View File

@ -6,7 +6,11 @@ export { Waku, DefaultPubSubTopic } from './lib/waku';
export { WakuMessage } from './lib/waku_message';
export { generatePrivateKey, getPublicKey } from './lib/waku_message/version_1';
export {
generatePrivateKey,
generateSymmetricKey,
getPublicKey,
} from './lib/waku_message/version_1';
export {
WakuLightPush,

View File

@ -12,6 +12,10 @@ import {
} from '../test_utils/';
import { Waku } from './waku';
import { WakuMessage } from './waku_message';
import { generateSymmetricKey } from './waku_message/version_1';
const TestContentTopic = '/test/1/waku/utf8';
describe('Waku Dial', function () {
let waku: Waku;
@ -130,3 +134,76 @@ describe('Waku Dial', function () {
});
});
});
describe('Decryption Keys', () => {
afterEach(function () {
if (this.currentTest?.state === 'failed') {
console.log(`Test failed, log file name is ${makeLogFileName(this)}`);
}
});
let waku1: Waku;
let waku2: Waku;
beforeEach(async function () {
[waku1, waku2] = await Promise.all([
Waku.create({ staticNoiseKey: NOISE_KEY_1 }),
Waku.create({
staticNoiseKey: NOISE_KEY_2,
libp2p: { addresses: { listen: ['/ip4/0.0.0.0/tcp/0/ws'] } },
}),
]);
waku1.addPeerToAddressBook(waku2.libp2p.peerId, waku2.libp2p.multiaddrs);
await Promise.all([
new Promise((resolve) =>
waku1.libp2p.pubsub.once('pubsub:subscription-change', () =>
resolve(null)
)
),
new Promise((resolve) =>
waku2.libp2p.pubsub.once('pubsub:subscription-change', () =>
resolve(null)
)
),
]);
});
afterEach(async function () {
this.timeout(5000);
await waku1.stop();
await waku2.stop();
});
it('Used by Waku Relay', async function () {
this.timeout(10000);
const symKey = generateSymmetricKey();
waku2.addDecryptionKey(symKey);
const messageText = 'Message is encrypted';
const messageTimestamp = new Date('1995-12-17T03:24:00');
const message = await WakuMessage.fromUtf8String(
messageText,
TestContentTopic,
{
timestamp: messageTimestamp,
symKey,
}
);
const receivedMsgPromise: Promise<WakuMessage> = new Promise((resolve) => {
waku2.relay.addObserver(resolve);
});
await waku1.relay.send(message);
const receivedMsg = await receivedMsgPromise;
expect(receivedMsg.contentTopic).to.eq(message.contentTopic);
expect(receivedMsg.version).to.eq(message.version);
expect(receivedMsg.payloadAsUtf8).to.eq(messageText);
expect(receivedMsg.timestamp?.valueOf()).to.eq(messageTimestamp.valueOf());
});
});

View File

@ -93,6 +93,7 @@ export interface CreateOptions {
* {@link CreateOptions.libp2p}.
*/
bootstrap?: boolean | string[] | (() => string[] | Promise<string[]>);
decryptionKeys?: Array<Uint8Array | string>;
}
export class Waku {
@ -133,6 +134,8 @@ export class Waku {
libp2p.connectionManager.on('peer:disconnect', (connection: Connection) => {
this.stopKeepAlive(connection.remotePeer);
});
options?.decryptionKeys?.forEach(this.addDecryptionKey);
}
/**
@ -270,6 +273,29 @@ export class Waku {
return this.libp2p.stop();
}
/**
* Register a decryption key to attempt decryption of messages received via
* [[WakuRelay]] and [[WakuStore]]. This can either be a private key for
* asymmetric encryption or a symmetric key.
*
* Strings must be in hex format.
*/
addDecryptionKey(key: Uint8Array | string): void {
this.relay.addDecryptionKey(key);
this.store.addDecryptionKey(key);
}
/**
* Delete a decryption key that was used to attempt decryption of messages
* received via [[WakuRelay]] or [[WakuStore]].
*
* Strings must be in hex format.
*/
deleteDecryptionKey(key: Uint8Array | string): void {
this.relay.deleteDecryptionKey(key);
this.store.deleteDecryptionKey(key);
}
/**
* Return the local multiaddr with peer id on which libp2p is listening.
* @throws if libp2p is not listening on localhost

View File

@ -14,7 +14,11 @@ import { delay } from '../delay';
import { hexToBuf } from '../utils';
import { Waku } from '../waku';
import { generatePrivateKey, getPublicKey } from './version_1';
import {
generatePrivateKey,
generateSymmetricKey,
getPublicKey,
} from './version_1';
import { WakuMessage } from './index';
@ -122,7 +126,7 @@ describe('Waku Message: Node only', function () {
payload: Buffer.from(messageText, 'utf-8').toString('hex'),
};
const symKey = generatePrivateKey();
const symKey = generateSymmetricKey();
waku.relay.addDecryptionKey(symKey);

View File

@ -1,4 +1,4 @@
import { IvSize, SymmetricKeySize } from './index';
import { IvSize } from './index';
declare global {
interface Window {
@ -44,10 +44,6 @@ export async function decrypt(
.then(Buffer.from);
}
export function generateKeyForSymmetricEnc(): Buffer {
return crypto.getRandomValues(Buffer.alloc(SymmetricKeySize));
}
export function generateIv(): Uint8Array {
const iv = new Uint8Array(IvSize);
crypto.getRandomValues(iv);

View File

@ -15,10 +15,6 @@ export interface Symmetric {
* Proceed with symmetric decryption of `cipherText` value.
*/
decrypt: (iv: Buffer, key: Buffer, cipherText: Buffer) => Promise<Buffer>;
/**
* Generate a new private key for Symmetric encryption purposes.
*/
generateKeyForSymmetricEnc: () => Buffer;
/**
* Generate an Initialization Vector (iv) for for Symmetric encryption purposes.
*/

View File

@ -1,6 +1,6 @@
import { createCipheriv, createDecipheriv, randomBytes } from 'crypto';
import { IvSize, SymmetricKeySize, TagSize } from './index';
import { IvSize, TagSize } from './index';
const Algorithm = 'aes-256-gcm';
@ -31,10 +31,6 @@ export async function decrypt(
return Buffer.concat([a, b]);
}
export function generateKeyForSymmetricEnc(): Buffer {
return randomBytes(SymmetricKeySize);
}
export function generateIv(): Buffer {
return randomBytes(IvSize);
}

View File

@ -7,7 +7,7 @@ import * as secp256k1 from 'secp256k1';
import { hexToBuf } from '../utils';
import { IvSize, symmetric } from './symmetric';
import { IvSize, symmetric, SymmetricKeySize } from './symmetric';
const FlagsLength = 1;
const FlagMask = 3; // 0011
@ -15,6 +15,8 @@ const IsSignedMask = 4; // 0100
const PaddingTarget = 256;
const SignatureLength = 65;
export const PrivateKeySize = 32;
/**
* Encode the payload pre-encryption.
*
@ -172,14 +174,19 @@ export async function decryptSymmetric(
}
/**
* Generate a new key. Can be used as a private key for Asymmetric encryption
* or a key for symmetric encryption.
* Generate a new private key to be used for asymmetric encryption.
*
* If using Asymmetric encryption, use {@link getPublicKey} to get the
* corresponding Public Key.
* Use {@link getPublicKey} to get the corresponding Public Key.
*/
export function generatePrivateKey(): Uint8Array {
return randomBytes(32);
return randomBytes(PrivateKeySize);
}
/**
* Generate a new symmetric key to be used for symmetric encryption.
*/
export function generateSymmetricKey(): Uint8Array {
return randomBytes(SymmetricKeySize);
}
/**

View File

@ -29,6 +29,9 @@ describe('Waku Relay', () => {
let waku1: Waku;
let waku2: Waku;
beforeEach(async function () {
this.timeout(10000);
log('Starting JS Waku instances');
[waku1, waku2] = await Promise.all([
Waku.create({ staticNoiseKey: NOISE_KEY_1 }),
Waku.create({
@ -36,9 +39,10 @@ describe('Waku Relay', () => {
libp2p: { addresses: { listen: ['/ip4/0.0.0.0/tcp/0/ws'] } },
}),
]);
log("Instances started, adding waku2 to waku1's address book");
waku1.addPeerToAddressBook(waku2.libp2p.peerId, waku2.libp2p.multiaddrs);
log('Wait for mutual pubsub subscription');
await Promise.all([
new Promise((resolve) =>
waku1.libp2p.pubsub.once('pubsub:subscription-change', () =>
@ -51,6 +55,7 @@ describe('Waku Relay', () => {
)
),
]);
log('before each hook done');
});
afterEach(async function () {
@ -60,11 +65,13 @@ describe('Waku Relay', () => {
});
it('Subscribe', async function () {
log('Getting subscribers');
const subscribers1 =
waku1.libp2p.pubsub.getSubscribers(DefaultPubSubTopic);
const subscribers2 =
waku2.libp2p.pubsub.getSubscribers(DefaultPubSubTopic);
log('Asserting mutual subscription');
expect(subscribers1).to.contain(waku2.libp2p.peerId.toB58String());
expect(subscribers2).to.contain(waku1.libp2p.peerId.toB58String());
});

View File

@ -17,6 +17,7 @@ import { InMessage } from 'libp2p-interfaces/src/pubsub';
import { SignaturePolicy } from 'libp2p-interfaces/src/pubsub/signature-policy';
import PeerId from 'peer-id';
import { hexToBuf } from '../utils';
import { CreateOptions, DefaultPubSubTopic } from '../waku';
import { WakuMessage } from '../waku_message';
@ -64,9 +65,6 @@ export class WakuRelay extends Gossipsub {
heartbeat: RelayHeartbeat;
pubSubTopic: string;
/**
* Decryption private keys to use to attempt decryption of incoming messages.
*/
public decryptionKeys: Set<Uint8Array>;
/**
@ -124,21 +122,24 @@ export class WakuRelay extends Gossipsub {
}
/**
* Register a decryption private key or symmetric key to attempt decryption
* of messages received on the given content topic. This can either be a
* private key for asymmetric encryption or a symmetric key. Waku relay will
* attempt to decrypt messages using both methods.
* Register a decryption key to attempt decryption of received messages.
* This can either be a private key for asymmetric encryption or a symmetric
* key. `WakuRelay` will attempt to decrypt messages using both methods.
*
* Strings must be in hex format.
*/
addDecryptionKey(privateKey: Uint8Array): void {
this.decryptionKeys.add(privateKey);
addDecryptionKey(key: Uint8Array | string): void {
this.decryptionKeys.add(hexToBuf(key));
}
/**
* Delete a decryption key to attempt decryption of messages received on the
* given content topic.
* Delete a decryption key that was used to attempt decryption of received
* messages.
*
* Strings must be in hex format.
*/
deleteDecryptionKey(privateKey: Uint8Array): void {
this.decryptionKeys.delete(privateKey);
deleteDecryptionKey(key: Uint8Array | string): void {
this.decryptionKeys.delete(hexToBuf(key));
}
/**

View File

@ -13,7 +13,11 @@ import {
import { delay } from '../delay';
import { Waku } from '../waku';
import { WakuMessage } from '../waku_message';
import { generatePrivateKey, getPublicKey } from '../waku_message/version_1';
import {
generatePrivateKey,
generateSymmetricKey,
getPublicKey,
} from '../waku_message/version_1';
import { Direction } from './history_rpc';
@ -160,7 +164,7 @@ describe('Waku Store', () => {
'This message is not for and I must not be able to read it';
const privateKey = generatePrivateKey();
const symKey = generatePrivateKey();
const symKey = generateSymmetricKey();
const publicKey = getPublicKey(privateKey);
const [
@ -232,9 +236,11 @@ describe('Waku Store', () => {
storePeers = waku2.store.peers;
}
waku2.store.addDecryptionKey(symKey);
dbg('Retrieve messages from store');
const messages = await waku2.store.queryHistory([], {
decryptionKeys: [privateKey, symKey],
decryptionKeys: [privateKey],
});
expect(messages?.length).eq(3);

View File

@ -8,6 +8,7 @@ import PeerId from 'peer-id';
import { HistoryResponse_Error } from '../../proto/waku/v2/store';
import { getPeersForProtocol, selectRandomPeer } from '../select_peer';
import { hexToBuf } from '../utils';
import { DefaultPubSubTopic } from '../waku';
import { WakuMessage } from '../waku_message';
@ -43,7 +44,7 @@ export interface QueryOptions {
pageSize?: number;
timeFilter?: TimeFilter;
callback?: (messages: WakuMessage[]) => void;
decryptionKeys?: Uint8Array[];
decryptionKeys?: Array<Uint8Array | string>;
}
/**
@ -51,6 +52,7 @@ export interface QueryOptions {
*/
export class WakuStore {
pubSubTopic: string;
public decryptionKeys: Set<Uint8Array>;
constructor(public libp2p: Libp2p, options?: CreateOptions) {
if (options?.pubSubTopic) {
@ -58,6 +60,8 @@ export class WakuStore {
} else {
this.pubSubTopic = DefaultPubSubTopic;
}
this.decryptionKeys = new Set();
}
/**
@ -104,16 +108,25 @@ export class WakuStore {
let peer;
if (opts.peerId) {
peer = this.libp2p.peerStore.get(opts.peerId);
if (!peer) throw 'Peer is unknown';
if (!peer)
throw `Failed to retrieve connection details for provided peer in peer store: ${opts.peerId.toB58String()}`;
} else {
peer = this.randomPeer;
if (!peer)
throw 'Failed to find known peer that registers waku store protocol';
}
if (!peer) throw 'No peer available';
if (!peer.protocols.includes(StoreCodec))
throw 'Peer does not register waku store protocol';
throw `Peer does not register waku store protocol: ${peer.id.toB58String()}`;
const connection = this.libp2p.connectionManager.get(peer.id);
if (!connection) throw 'Failed to get a connection to the peer';
const decryptionKeys = Array.from(this.decryptionKeys.values());
if (opts.decryptionKeys) {
opts.decryptionKeys.forEach((key) => {
decryptionKeys.push(hexToBuf(key));
});
}
const messages: WakuMessage[] = [];
let cursor = undefined;
while (true) {
@ -154,10 +167,7 @@ export class WakuStore {
const pageMessages: WakuMessage[] = [];
await Promise.all(
response.messages.map(async (protoMsg) => {
const msg = await WakuMessage.decodeProto(
protoMsg,
opts.decryptionKeys
);
const msg = await WakuMessage.decodeProto(protoMsg, decryptionKeys);
if (msg) {
messages.push(msg);
@ -193,6 +203,28 @@ export class WakuStore {
}
}
/**
* Register a decryption key to attempt decryption of messages received in any
* subsequent [[queryHistory]] call. This can either be a private key for
* asymmetric encryption or a symmetric key. [[WakuStore]] will attempt to
* decrypt messages using both methods.
*
* Strings must be in hex format.
*/
addDecryptionKey(key: Uint8Array | string): void {
this.decryptionKeys.add(hexToBuf(key));
}
/**
* Delete a decryption key that was used to attempt decryption of messages
* received in subsequent [[queryHistory]] calls.
*
* Strings must be in hex format.
*/
deleteDecryptionKey(key: Uint8Array | string): void {
this.decryptionKeys.delete(hexToBuf(key));
}
/**
* Returns known peers from the address book (`libp2p.peerStore`) that support
* store protocol. Waku may or may not be currently connected to these peers.