nimbus-eth2/beacon_chain/rpc/rest_utils.nim

275 lines
9.5 KiB
Nim
Raw Normal View History

New validator client using REST API. (#2651) * Initial commit. * Exporting getConfig(). * Add beacon node checking procedures. * Post rebase fixes. * Use runSlotLoop() from nimbus_beacon_node. Fallback implementation. Fixes for ETH2 REST serialization. * Add beacon_clock.durationToNextSlot(). Move type declarations from beacon_rest_api to json_rest_serialization. Fix seq[ValidatorIndex] serialization. Refactor ValidatorPool and add some utility procedures. Create separate version of validator_client. * Post-rebase fixes. Remove CookedPubKey from validator_pool.nim. * Now we should be able to produce attestations and aggregate and proofs. But its not working yet. * Debugging attestation sending. * Add durationToNextAttestation. Optimize some debug logs. Fix aggregation_bits encoding. Bump chronos/presto. * Its alive. * Fixes for launch_local_testnet script. Bump chronos. * Switch client API to not use `/api` prefix. * Post-rebase adjustments. * Fix endpoint for publishBlock(). * Add CONFIG_NAME. Add more checks to ensure that beacon_node is compatible. * Add beacon committee subscription support to validator_client. * Fix stacktrace should be an array of strings. Fix committee subscriptions should not be `data` keyed. * Log duration to next block proposal. * Fix beacon_node_status import. * Use jsonMsgResponse() instead of jsonError(). * Fix graffityBytes usage. Remove unnecessary `await`. Adjust creation of SignedBlock instance. Remove legacy files. * Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`. * Fix race condition for block proposal and attestations for same slot. Fix local_testnet script to properly kill tasks on Windows. Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth). * Catch services errors. Improve performance of local_testnet.sh script on Windows. Fix race condition when attestation producing. * Post-rebase fixes. * Bump chronos and presto. * Calculate block publishing delay. Fix pkill in one more place. * Add error handling and timeouts to firstSuccess() template. Add onceToAll() template. Add checkNodes() procedure. Refactor firstSuccess() template. Add error checking to api.nim calls. * Deprecated usage onceToAll() for better stability. Address comment and send attestations asap. * Avoid unnecessary loop when calculating minimal duration.
2021-07-13 11:15:07 +00:00
import presto, presto/client as presto_client,
nimcrypto/utils as ncrutils,
../spec/[crypto, datatypes, digest, forks],
../spec/eth2_apis/[rest_types, eth2_rest_serialization],
../beacon_node_common,
../consensus_object_pools/[block_pools_types, blockchain_dag]
export
blockchain_dag, presto, presto_client, rest_types, eth2_rest_serialization
2021-03-17 18:46:45 +00:00
const
MaxEpoch* = compute_epoch_at_slot(not(0'u64))
2021-04-08 10:49:28 +00:00
BlockValidationError* =
"The block failed validation, but was successfully broadcast anyway. It " &
"was not integrated into the beacon node's database."
BlockValidationSuccess* =
"The block was validated successfully and has been broadcast"
BeaconNodeInSyncError* =
"Beacon node is currently syncing and not serving request on that endpoint"
BlockNotFoundError* =
"Block header/data has not been found"
BlockProduceError* =
"Could not produce the block"
2021-04-08 10:49:28 +00:00
EmptyRequestBodyError* =
"Empty request's body"
InvalidBlockObjectError* =
"Unable to decode block object(s)"
InvalidAttestationObjectError* =
"Unable to decode attestation object(s)"
AttestationValidationError* =
"Some errors happened while validating attestation(s)"
AttestationValidationSuccess* =
"Attestation object(s) was broadcasted"
InvalidAttesterSlashingObjectError* =
"Unable to decode attester slashing object(s)"
AttesterSlashingValidationError* =
"Invalid attester slashing, it will never pass validation so it's rejected"
AttesterSlashingValidationSuccess* =
"Attester slashing object was broadcasted"
InvalidProposerSlashingObjectError* =
"Unable to decode proposer slashing object(s)"
ProposerSlashingValidationError* =
"Invalid proposer slashing, it will never pass validation so it's rejected"
ProposerSlashingValidationSuccess* =
"Proposer slashing object was broadcasted"
InvalidVoluntaryExitObjectError* =
"Unable to decode voluntary exit object(s)"
VoluntaryExitValidationError* =
"Invalid voluntary exit, it will never pass validation so it's rejected"
VoluntaryExitValidationSuccess* =
"Voluntary exit object(s) was broadcasted"
InvalidAggregateAndProofObjectError* =
"Unable to decode aggregate and proof object(s)"
AggregateAndProofValidationError* =
"Invalid aggregate and proof, it will never pass validation so it's " &
"rejected"
AggregateAndProofValidationSuccess* =
"Aggregate and proof object(s) was broadcasted"
New validator client using REST API. (#2651) * Initial commit. * Exporting getConfig(). * Add beacon node checking procedures. * Post rebase fixes. * Use runSlotLoop() from nimbus_beacon_node. Fallback implementation. Fixes for ETH2 REST serialization. * Add beacon_clock.durationToNextSlot(). Move type declarations from beacon_rest_api to json_rest_serialization. Fix seq[ValidatorIndex] serialization. Refactor ValidatorPool and add some utility procedures. Create separate version of validator_client. * Post-rebase fixes. Remove CookedPubKey from validator_pool.nim. * Now we should be able to produce attestations and aggregate and proofs. But its not working yet. * Debugging attestation sending. * Add durationToNextAttestation. Optimize some debug logs. Fix aggregation_bits encoding. Bump chronos/presto. * Its alive. * Fixes for launch_local_testnet script. Bump chronos. * Switch client API to not use `/api` prefix. * Post-rebase adjustments. * Fix endpoint for publishBlock(). * Add CONFIG_NAME. Add more checks to ensure that beacon_node is compatible. * Add beacon committee subscription support to validator_client. * Fix stacktrace should be an array of strings. Fix committee subscriptions should not be `data` keyed. * Log duration to next block proposal. * Fix beacon_node_status import. * Use jsonMsgResponse() instead of jsonError(). * Fix graffityBytes usage. Remove unnecessary `await`. Adjust creation of SignedBlock instance. Remove legacy files. * Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`. * Fix race condition for block proposal and attestations for same slot. Fix local_testnet script to properly kill tasks on Windows. Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth). * Catch services errors. Improve performance of local_testnet.sh script on Windows. Fix race condition when attestation producing. * Post-rebase fixes. * Bump chronos and presto. * Calculate block publishing delay. Fix pkill in one more place. * Add error handling and timeouts to firstSuccess() template. Add onceToAll() template. Add checkNodes() procedure. Refactor firstSuccess() template. Add error checking to api.nim calls. * Deprecated usage onceToAll() for better stability. Address comment and send attestations asap. * Avoid unnecessary loop when calculating minimal duration.
2021-07-13 11:15:07 +00:00
BeaconCommitteeSubscriptionSuccess* =
"Beacon node processed committee subscription request"
2021-04-08 10:49:28 +00:00
InvalidParentRootValueError* =
"Invalid parent root value"
MissingSlotValueError* =
"Missing `slot` value"
InvalidSlotValueError* =
"Invalid slot value"
MissingCommitteeIndexValueError* =
"Missing `committee_index` value"
InvalidCommitteeIndexValueError* =
"Invalid committee index value"
MissingAttestationDataRootValueError* =
"Missing `attestation_data_root` value"
InvalidAttestationDataRootValueError* =
"Invalid attestation data root value"
UnableToGetAggregatedAttestationError* =
"Unable to retrieve an aggregated attestation"
MissingRandaoRevealValue* =
"Missing `randao_reveal` value"
InvalidRandaoRevealValue* =
"Invalid randao reveal value"
InvalidGraffitiBytesValye* =
"Invalid graffiti bytes value"
2021-04-08 10:49:28 +00:00
InvalidEpochValueError* =
"Invalid epoch value"
InvalidStateIdValueError* =
"Invalid state identifier value"
InvalidBlockIdValueError* =
"Invalid block identifier value"
InvalidValidatorIdValueError* =
"Invalid validator's identifier value(s)"
MaximumNumberOfValidatorIdsError* =
"Maximum number of validator identifier values exceeded"
InvalidValidatorStatusValueError* =
"Invalid validator's status value error"
InvalidValidatorIndexValueError* =
"Invalid validator's index value(s)"
EmptyValidatorIndexArrayError* =
"Empty validator's index array"
InvalidSubscriptionRequestValueError* =
"Invalid subscription request object(s)"
2021-04-08 10:49:28 +00:00
ValidatorNotFoundError* =
"Could not find validator"
ValidatorStatusNotFoundError* =
"Could not obtain validator's status"
UniqueValidatorKeyError* =
"Only unique validator's keys are allowed"
TooHighValidatorIndexValueError* =
"Validator index exceeds maximum number of validators allowed"
UnsupportedValidatorIndexValueError* =
"Validator index exceeds maximum supported number of validators"
UniqueValidatorIndexError* =
"Only unique validator's index are allowed"
StateNotFoundError* =
"State not found"
SlotNotFoundError* =
"Slot number is too far away"
SlotNotInNextWallSlotEpochError* =
"Requested slot not in next wall-slot epoch"
SlotFromThePastError* =
"Requested slot from the past"
ProposerNotFoundError* =
"Could not find proposer for the head and slot"
2021-04-08 10:49:28 +00:00
NoHeadForSlotError* =
"Cound not find head for slot"
EpochOverflowValueError* =
"Requesting epoch for which slot would overflow"
InvalidPeerStateValueError* =
"Invalid peer's state value(s) error"
InvalidPeerDirectionValueError* =
"Invalid peer's direction value(s) error"
InvalidPeerIdValueError* =
"Invalid peer's id value(s) error"
PeerNotFoundError* =
"Peer not found"
InvalidLogLevelValueError* =
"Invalid log level value error"
2021-04-08 10:49:28 +00:00
InternalServerError* =
"Internal server error"
NoImplementationError* =
"Not implemented yet"
2021-03-17 18:46:45 +00:00
type
ValidatorIndexError* {.pure.} = enum
UnsupportedValue, TooHighValue
2021-03-17 18:46:45 +00:00
func match(data: openarray[char], charset: set[char]): int =
for ch in data:
if ch notin charset:
return 1
0
proc validate(key: string, value: string): int =
## This is rough validation procedure which should be simple and fast,
## because it will be used for query routing.
case key
of "{epoch}":
2021-04-12 16:23:45 +00:00
0
2021-03-17 18:46:45 +00:00
of "{slot}":
2021-04-12 16:23:45 +00:00
0
2021-03-17 18:46:45 +00:00
of "{peer_id}":
2021-04-12 16:23:45 +00:00
0
2021-03-17 18:46:45 +00:00
of "{state_id}":
2021-04-12 16:23:45 +00:00
0
2021-03-17 18:46:45 +00:00
of "{block_id}":
2021-04-12 16:23:45 +00:00
0
2021-03-17 18:46:45 +00:00
of "{validator_id}":
2021-04-12 16:23:45 +00:00
0
2021-03-17 18:46:45 +00:00
else:
1
proc getCurrentHead*(node: BeaconNode,
slot: Slot): Result[BlockRef, cstring] =
let res = node.dag.head
2021-03-17 18:46:45 +00:00
# if not(node.isSynced(res)):
# return err("Cannot fulfill request until node is synced")
if res.slot + uint64(2 * SLOTS_PER_EPOCH) < slot:
return err("Requesting way ahead of the current head")
ok(res)
proc getCurrentHead*(node: BeaconNode,
epoch: Epoch): Result[BlockRef, cstring] =
if epoch > MaxEpoch:
2021-03-17 18:46:45 +00:00
return err("Requesting epoch for which slot would overflow")
node.getCurrentHead(compute_start_slot_at_epoch(epoch))
proc toBlockSlot*(blckRef: BlockRef): BlockSlot =
blckRef.atSlot(blckRef.slot)
proc getBlockSlot*(node: BeaconNode,
stateIdent: StateIdent): Result[BlockSlot, cstring] =
case stateIdent.kind
of StateQueryKind.Slot:
let head = ? getCurrentHead(node, stateIdent.slot)
let bslot = head.atSlot(stateIdent.slot)
if isNil(bslot.blck):
return err("Block not found")
ok(bslot)
of StateQueryKind.Root:
let blckRef = node.dag.getRef(stateIdent.root)
2021-03-17 18:46:45 +00:00
if isNil(blckRef):
return err("Block not found")
ok(blckRef.toBlockSlot())
of StateQueryKind.Named:
case stateIdent.value
of StateIdentType.Head:
ok(node.dag.head.toBlockSlot())
2021-03-17 18:46:45 +00:00
of StateIdentType.Genesis:
ok(node.dag.getGenesisBlockSlot())
2021-03-17 18:46:45 +00:00
of StateIdentType.Finalized:
ok(node.dag.finalizedHead)
2021-03-17 18:46:45 +00:00
of StateIdentType.Justified:
ok(node.dag.head.atEpochStart(getStateField(
node.dag.headState.data, current_justified_checkpoint).epoch))
2021-03-17 18:46:45 +00:00
proc getBlockDataFromBlockIdent*(node: BeaconNode,
id: BlockIdent): Result[BlockData, cstring] =
case id.kind
of BlockQueryKind.Named:
case id.value
of BlockIdentType.Head:
ok(node.dag.get(node.dag.head))
2021-03-17 18:46:45 +00:00
of BlockIdentType.Genesis:
ok(node.dag.getGenesisBlockData())
2021-03-17 18:46:45 +00:00
of BlockIdentType.Finalized:
ok(node.dag.get(node.dag.finalizedHead.blck))
2021-03-17 18:46:45 +00:00
of BlockQueryKind.Root:
let res = node.dag.get(id.root)
2021-03-17 18:46:45 +00:00
if res.isNone():
return err("Block not found")
ok(res.get())
of BlockQueryKind.Slot:
let head = ? node.getCurrentHead(id.slot)
let blockSlot = head.atSlot(id.slot)
if isNil(blockSlot.blck):
return err("Block not found")
ok(node.dag.get(blockSlot.blck))
2021-03-17 18:46:45 +00:00
template withStateForBlockSlot*(node: BeaconNode,
blockSlot: BlockSlot, body: untyped): untyped =
2021-03-17 20:42:55 +00:00
template isState(state: StateData): bool =
state.blck.atSlot(getStateField(state.data, slot)) == blockSlot
2021-03-17 20:42:55 +00:00
if isState(node.dag.headState):
withStateVars(node.dag.headState):
2021-03-17 20:42:55 +00:00
var cache {.inject.}: StateCache
body
else:
let rpcState = assignClone(node.dag.headState)
node.dag.withState(rpcState[], blockSlot):
2021-03-17 20:42:55 +00:00
body
proc toValidatorIndex*(value: RestValidatorIndex): Result[ValidatorIndex,
ValidatorIndexError] =
when sizeof(ValidatorIndex) == 4:
if uint64(value) < VALIDATOR_REGISTRY_LIMIT:
# On x86 platform Nim allows only `int32` indexes, so all the indexes in
# range `2^31 <= x < 2^32` are not supported.
if uint64(value) <= uint64(high(int32)):
ok(ValidatorIndex(value))
else:
err(ValidatorIndexError.UnsupportedValue)
else:
err(ValidatorIndexError.TooHighValue)
elif sizeof(ValidatorIndex) == 8:
if uint64(value) < VALIDATOR_REGISTRY_LIMIT:
ok(ValidatorIndex(value))
else:
err(ValidatorIndexError.TooHighValue)
else:
doAssert(false, "ValidatorIndex type size is incorrect")
New validator client using REST API. (#2651) * Initial commit. * Exporting getConfig(). * Add beacon node checking procedures. * Post rebase fixes. * Use runSlotLoop() from nimbus_beacon_node. Fallback implementation. Fixes for ETH2 REST serialization. * Add beacon_clock.durationToNextSlot(). Move type declarations from beacon_rest_api to json_rest_serialization. Fix seq[ValidatorIndex] serialization. Refactor ValidatorPool and add some utility procedures. Create separate version of validator_client. * Post-rebase fixes. Remove CookedPubKey from validator_pool.nim. * Now we should be able to produce attestations and aggregate and proofs. But its not working yet. * Debugging attestation sending. * Add durationToNextAttestation. Optimize some debug logs. Fix aggregation_bits encoding. Bump chronos/presto. * Its alive. * Fixes for launch_local_testnet script. Bump chronos. * Switch client API to not use `/api` prefix. * Post-rebase adjustments. * Fix endpoint for publishBlock(). * Add CONFIG_NAME. Add more checks to ensure that beacon_node is compatible. * Add beacon committee subscription support to validator_client. * Fix stacktrace should be an array of strings. Fix committee subscriptions should not be `data` keyed. * Log duration to next block proposal. * Fix beacon_node_status import. * Use jsonMsgResponse() instead of jsonError(). * Fix graffityBytes usage. Remove unnecessary `await`. Adjust creation of SignedBlock instance. Remove legacy files. * Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`. * Fix race condition for block proposal and attestations for same slot. Fix local_testnet script to properly kill tasks on Windows. Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth). * Catch services errors. Improve performance of local_testnet.sh script on Windows. Fix race condition when attestation producing. * Post-rebase fixes. * Bump chronos and presto. * Calculate block publishing delay. Fix pkill in one more place. * Add error handling and timeouts to firstSuccess() template. Add onceToAll() template. Add checkNodes() procedure. Refactor firstSuccess() template. Add error checking to api.nim calls. * Deprecated usage onceToAll() for better stability. Address comment and send attestations asap. * Avoid unnecessary loop when calculating minimal duration.
2021-07-13 11:15:07 +00:00
proc getRouter*(): RestRouter =
RestRouter.init(validate)