nim-codex/tests/codex/testvalidation.nim

152 lines
6.1 KiB
Nim
Raw Normal View History

Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
import pkg/chronos
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
import std/strformat
import std/random
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
import codex/validation
import codex/periods
import ../asynctest
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
import ./helpers/mockmarket
import ./helpers/mockclock
import ./examples
import ./helpers
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
asyncchecksuite "validation":
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
let period = 10
let timeout = 5
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
let maxSlots = MaxSlots(100)
let validationGroups = ValidationGroups(8).some
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
let slot = Slot.example
let proof = Groth16Proof.example
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
let collateral = slot.request.ask.collateral
var validation: Validation
var market: MockMarket
var clock: MockClock
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
var groupIndex: uint16
proc initValidationConfig(maxSlots: MaxSlots,
validationGroups: ?ValidationGroups,
groupIndex: uint16 = 0): ValidationConfig =
without validationConfig =? ValidationConfig.init(
maxSlots, groups=validationGroups, groupIndex), error:
raiseAssert fmt"Creating ValidationConfig failed! Error msg: {error.msg}"
validationConfig
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
setup:
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
groupIndex = groupIndexForSlotId(slot.id, !validationGroups)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
market = MockMarket.new()
clock = MockClock.new()
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
let validationConfig = initValidationConfig(
maxSlots, validationGroups, groupIndex)
validation = Validation.new(clock, market, validationConfig)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
market.config.proofs.period = period.u256
market.config.proofs.timeout = timeout.u256
await validation.start()
teardown:
await validation.stop()
proc advanceToNextPeriod =
let periodicity = Periodicity(seconds: period.u256)
let period = periodicity.periodOf(clock.now().u256)
let periodEnd = periodicity.periodEnd(period)
clock.set((periodEnd + 1).truncate(int))
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
test "the list of slots that it's monitoring is empty initially":
check validation.slots.len == 0
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
for (validationGroups, groupIndex) in [(100, 100'u16), (100, 101'u16)]:
test "initializing ValidationConfig fails when groupIndex is " &
"greater than or equal to validationGroups " &
fmt"(testing for {groupIndex = }, {validationGroups = })":
let groups = ValidationGroups(validationGroups).some
let validationConfig = ValidationConfig.init(
maxSlots, groups = groups, groupIndex = groupIndex)
check validationConfig.isFailure == true
check validationConfig.error.msg == "The value of the group index " &
"must be less than validation groups! " &
fmt"(got: {groupIndex = }, groups = {!groups})"
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
test "initializing ValidationConfig fails when maxSlots is negative":
let maxSlots = -1
let validationConfig = ValidationConfig.init(
maxSlots = maxSlots, groups = ValidationGroups.none)
check validationConfig.isFailure == true
check validationConfig.error.msg == "The value of maxSlots must " &
fmt"be greater than or equal to 0! (got: {maxSlots})"
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
test "initializing ValidationConfig fails when maxSlots is negative " &
"(validationGroups set)":
let maxSlots = -1
let validationConfig = ValidationConfig.init(
maxSlots = maxSlots, groups = validationGroups, groupIndex)
check validationConfig.isFailure == true
check validationConfig.error.msg == "The value of maxSlots must " &
fmt"be greater than or equal to 0! (got: {maxSlots})"
test "slot is not observed if it is not in the validation group":
let validationConfig = initValidationConfig(maxSlots, validationGroups,
(groupIndex + 1) mod uint16(!validationGroups))
let validation = Validation.new(clock, market, validationConfig)
await validation.start()
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
await validation.stop()
check validation.slots.len == 0
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
test "when a slot is filled on chain, it is added to the list":
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
check validation.slots == @[slot.id]
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
test "slot should be observed if maxSlots is set to 0":
let validationConfig = initValidationConfig(
maxSlots = 0, ValidationGroups.none)
let validation = Validation.new(clock, market, validationConfig)
await validation.start()
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
await validation.stop()
check validation.slots == @[slot.id]
test "slot should be observed if validation group is not set (and " &
"maxSlots is not 0)":
let validationConfig = initValidationConfig(
maxSlots, ValidationGroups.none)
let validation = Validation.new(clock, market, validationConfig)
await validation.start()
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
await validation.stop()
check validation.slots == @[slot.id]
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
for state in [SlotState.Finished, SlotState.Failed]:
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
test fmt"when slot state changes to {state}, it is removed from the list":
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
market.slotState[slot.id] = state
advanceToNextPeriod()
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
check eventually validation.slots.len == 0
test "when a proof is missed, it is marked as missing":
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
market.setCanProofBeMarkedAsMissing(slot.id, true)
advanceToNextPeriod()
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
await sleepAsync(100.millis) # allow validation loop to run
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
check market.markedAsMissingProofs.contains(slot.id)
test "when a proof can not be marked as missing, it will not be marked":
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
market.setCanProofBeMarkedAsMissing(slot.id, false)
advanceToNextPeriod()
Fix concurrency issues (#993) * Use http subscriptions instead of websocket for tests To work around this issue when subscriptions are inactive for more than 5 minutes: https://github.com/NomicFoundation/hardhat/issues/2053 Use 100 millisecond polling; default polling interval of 4 seconds is too close to the 5 second timeout for `check eventually`. * use .confirm(1) instead of confirm(0) confirm(0) doesn't wait at all, confirm(1) waits for the transaction to be mined * speed up partial payout integration test * update nim-ethers to version 0.10.0 includes fixes for http polling and .confirm() * fix timing of marketplace tests allow for a bit more time to withdraw funds * use .confirm(1) in marketplace tests to ensure that the transaction has been processed before continuing with the test * fix timing issue in validation unit test * fix proof integration test there were two logic errors in this test: - a slot is freed anyway at the end of the contract - when starting the request takes a long time, the first slot can already be freed because there were too many missing proofs * fix intermittent error in contract tests currentTime() doesn't always correctly reflect the time of the next transaction * reduce number of slots in integration test otherwise the windows runner in the CI won't be able to start the request before it expires * fix timing in purchasing test allow for a bit more time for a request to be submitted * fix timing of request submission in test windows ci is so slow, it can take up to 40 seconds just to submit a storage request to hardhat * increase proof period to 90 seconds * adjust timing of integration tests reason: with the increased period length of 90 seconds, it can take longer to wait for a stable challenge at the beginning of a period. * increase CI timeout to 2 hours * Fix slow builds on windows apparently it takes windows 2-3 seconds to resolve "localhost" to 127.0.0.1 for every json-rpc connection that we make 🤦
2024-11-25 11:23:04 +00:00
await sleepAsync(100.millis) # allow validation loop to run
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
check market.markedAsMissingProofs.len == 0
test "it does not monitor more than the maximum number of slots":
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
let validationGroups = ValidationGroups.none
let validationConfig = initValidationConfig(
maxSlots, validationGroups)
let validation = Validation.new(clock, market, validationConfig)
await validation.start()
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
for _ in 0..<maxSlots + 1:
let slot = Slot.example
await market.fillSlot(slot.request.id, slot.slotIndex, proof, collateral)
Validator - support partitioning of the slot id space (#890) * Adds validatorPartitionSize and validatorPartitionIndex config options * adds partitioning options to the validation type * adds partitioning logic to the validator * ignores partitionIndex when partitionSize is either 0 or 1 * clips the partition index to <<partitionIndex mod partitionSize>> * handles negative values for the validation partition index * updates long description of the new validator cli options * makes default partitionSize to be 0 for better backward compatibility * Improving formatting on validator CLI * reactors validation params into a separate type and simplifies validation of validation params * removes suspected duplication * fixes typo in validator CLI help * updates README * Applies review comments - using optionals and range types to handle validation params * Adds initializer to the configFactory for validatorMaxSlots * [Review] update validator CLI description and README * [Review]: renaming validationParams to validationConfig (config) * [Review]: move validationconfig.nim to a higher level (next to validation.nim) * changes backing type of MaxSlots to be int and makes sure slots are validated without limit when maxSlots is set to 0 * adds more end-to-end test for the validator and the groups * fixes typo in README and conf.nim * makes `maxSlotsConstraintRespected` and `shouldValidateSlot` private + updates the tests * fixes public address of the signer account in the marketplace tutorial * applies review comments - removes two tests
2024-10-02 22:00:40 +00:00
await validation.stop()
Validator (#387) * [contracts] Add SlotFreed event * [integration] allow test node to be stopped twice * [cli] add --validator option * [contracts] remove dead code * [contracts] instantiate OnChainMarket and OnChainClock only once * [contracts] add Validation * [sales] remove duplicate import * [market] add missing import * [market] subscribe to all SlotFilled events * [market] add freeSlot() * [sales] fix warnings * [market] subscribe to SlotFreed events * [contracts] fix warning * [validator] keep track of filled slots * [validation] remove slots that have ended * [proving] absorb Proofs into Market Both Proofs and Market are abstractions around the Marketplace contract, having them separately is more trouble than it's worth at the moment. * [market] add markProofAsMissing() * [clock] speed up waiting for clock in tests * [validator] mark proofs as missing * [timer] fix error on node shutdown * [cli] handle --persistence and --validator separately * [market] allow retrieval of proof timeout value * [validator] do not subscribe to SlotFreed events Freed slots are already handled in removeSlotsThatHaveEnded(), and onSlotsFreed() interfered with its iterator. * [validator] Start validation at the start of a new period To decrease the likelihood that we hit the validation timeout. * [validator] do not mark proofs as missing after timeout * [market] check whether proof can be marked as missing * [validator] simplify validation Simulate a transaction to mark proof as missing, instead of trying to keep track of all the conditions that may lead to a proof being marked as missing. * [build] use nim-ethers PR #40 Uses "pending" blocktag instead of "latest" blocktag for better simulation of transactions before sending them. https://github.com/status-im/nim-ethers/pull/40 * [integration] integration test for validator * [validator] monitor a maximum number of slots Adds cli parameter --validator-max-slots. * [market] fix missing collateral argument After rebasing, add the new argument to fillSlot calls. * [build] update to nim-ethers 0.2.5 * [validator] use Set instead of Table to keep track of slots * [validator] add logging * [validator] add test for slot failure * [market] use "pending" blocktag to use more up to date block time * [contracts] remove unused import * [validator] fix: wait until after period ends The smart contract checks that 'end < block.timestamp', so we need to wait until the block timestamp is greater than the period end.
2023-04-19 13:06:00 +00:00
check validation.slots.len == maxSlots