validateWorker() reads from validateQ and invokes validate function
that performs validation of the message. Signature validation is performed
synchronously. The number of validate workers defaults to the number of CPUs
and can be updated with WithValidateWorkers function. With no additional user
validators, signature validation is the bottleneck when receiving new messages.
Increasing the number of validating workers does not help given the context
switching and bottleneck nature of this spot. As stated in WithValidateWorkers
documentation, this function should be used rather to limit the number of workers
to devote less CPU time for synchronous validation. On the other hand, with the
default size of `validateQ`, some applications built on a top of libp2p may
experience throttled validation and lost messages.
This problem is addressed by WithValidateQueueSize allowing to configure the buffer
size for synchronous validation. Application developers knowing the nature of their
protocols can set this value to minimise the possibility of throttled synchronous
validation and dropped messages. Configurable buffer size allows to gracefully
handle peaks of messages and, from the other side, the number of concurrent
synchronous workers is still limited by validateWorkers property so the receiver
should not get congested.
vyzo
Diederik Loerakker
Edgar Aroutiounian
Piotr Dyraga
Adin Schmahmann
Raúl Kripalani