logos-co
/
nomos-specs
mirror of https://github.com/logos-co/nomos-specs.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Overlay based happy tests (#11) 

* Cleanup

* Leaf committee member vote.

* Leaf committee member vote, test.

* Leaf committee member vote, test.

* Description

* Description

* Description-refactoring

* Cleanup

* Fix leaf votes test

* Clean overlay

* Test single committee advances

* Remove unhappy path test file

* Update carnot description

* Refactor local_timeout for NewView

* Fix unhappy path conditions and added broadcasting

* Unhappy path tests description

* Commit all grandparents of a block from latest_committed view

* Cleanup docs

* Add unhappy path test vector

* Remove block content

---------

Co-authored-by: mjalalzai <33738574+MForensic@users.noreply.github.com>
This commit is contained in:
Daniel Sanchez 2023-04-10 10:28:48 +02:00 committed by GitHub
parent 541eb2f1b7
commit d8d22e7219
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 508 additions and 125 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

321
carnot/carnot.py
View File

@ -1,3 +1,38 @@
# The Carnot protocol is designed to be elastic, responsive, and provide fast finality
# Elastic scalability allows the protocol to operate effectively with both small and large networks
# All nodes in the Carnot network participate in the consensus of a block
# Optimistic responsiveness enables the protocol to operate quickly during periods of synchrony and honest leadership
# There is no block generation time in Carnot, allowing for fast finality
# Carnot avoids the chain reorg problem, making it compatible with PoS schemes
# This enhances the robustness of the protocol, making it a valuable addition to the ecosystem of consensus protocols
# The protocol in Carnot operates in two modes: the happy path and the unhappy path.
#
# In Carnot, nodes are arranged in a binary tree overlay committee structure. Moreover, Carnot is a
# pipelined consensus protocol where a block contains the proof of attestation of its parent. In happy path the
# leader proposes a block that contains a quorum certificate (QC) with votes from more than two-thirds of the root
# committee and its child committee/ committees. The voting process begins at the leaf committee where nodes verify
# the proposal and send their votes to the parent committee. Once a node in the parent committee receives more than
# two-thirds of the votes from its child committee members, it sends its votes to its parent. This process continues
# recursively until the root committee members collect votes from its child committee/ committees. The root committee
# member builds a QC from the votes and sends it to the next leader. The leader builds a QC and proposes the next block
# upon receiving more than two-thirds of votes.
# In the unhappy path, if a node does not receive a message within a timeout interval, it will timeout. Only nodes at
# the root committee and its child committee/ committees send their timeout messages to the root committee. The root
# committee builds a timeout QC from more than two-thirds of messages, recalculates the new overlay, and broadcasts it
# to the network. Similar to the happy path, the timeout message moves from leaves to the root. Each parent waits for
# more than two-thirds of timeout messages from its child committees and sends its timeout to the parent committee once
# the threshold is reached. A node in the root committee builds a QC from timeout messages received from its
# child committee/committees and forwards it to the next leader. Upon receiving more than two-thirds of timeout
# messages, the next leader builds an aggregated QC and proposes the next block containing the aggregated QC.
# It should be noted that while receiving timeout messages, each node also updates its high_qc (the most recent QC)
# and passes it to its parent through the timeout message. In this way, the aggregated QC will include the high_qc seen
# by the majority of honest nodes. Hence, after the view change, the protocol safety is preserved.
from dataclasses import dataclass
from typing import TypeAlias, List, Set, Self, Optional, Dict, FrozenSet
from abc import abstractmethod
@ -41,7 +76,7 @@ Qc: TypeAlias = StandardQc | AggregateQc
class Block:
view: View
qc: Qc
content: FrozenSet[Id]
_id: Id # this is an abstration over the block id, which should be the hash of the contents
def extends(self, ancestor: Self) -> bool:
"""
@ -54,7 +89,7 @@ class Block:
return self.qc.block
def id(self) -> Id:
return int_to_id(hash(self.content))
return self._id
@dataclass(unsafe_hash=True)
@ -74,14 +109,19 @@ class TimeoutQc:
sender: Id
# Timeout in the root or direct children committees
@dataclass
class Timeout:
"""
Local timeout field is only used by the root committee and its children when they timeout. The timeout_qc is built
from local_timeouts. Leaf nodes when receive timeout_qc build their timeout msg and includes the timeout_qc in it.
The timeout_qc is indicator that the root committee and its child committees (if exist) have failed to collect votes.
"""
view: View
high_qc: Qc
sender: Id
timeout_qc: TimeoutQc
# Timeout has been detected, nodes agree on it and gather high qc
@dataclass
class NewView:
@ -129,7 +169,7 @@ class Overlay:
pass
@abstractmethod
def member_of_leaf_committee(self, _id: Id) -> bool:
def is_member_of_leaf_committee(self, _id: Id) -> bool:
"""
:param _id: Node id to be checked
:return: true if the participant with Id _id is in the leaf committee of the committee overlay
@ -137,7 +177,7 @@ class Overlay:
pass
@abstractmethod
def member_of_root_committee(self, _id: Id) -> bool:
def is_member_of_root_committee(self, _id: Id) -> bool:
"""
:param _id:
:return: true if the participant with Id _id is member of the root committee withing the tree overlay
@ -145,23 +185,7 @@ class Overlay:
pass
@abstractmethod
def member_of_root_com(self, _id: Id) -> bool:
"""
:param _id:
:return: true if the participant with Id _id is member of the root committee withing the tree overlay
"""
pass
@abstractmethod
def member_of_internal_com(self, _id: Id) -> bool:
"""
:param _id:
:return: True if the participant with Id _id is member of internal committees within the committee tree overlay
"""
pass
@abstractmethod
def child_committee(self, parent: Id, child: Id) -> bool:
def is_member_of_child_committee(self, parent: Id, child: Id) -> bool:
"""
:param parent:
:param child:
@ -187,7 +211,7 @@ class Overlay:
"""
pass
def child_of_root_committee(self, _id: Id) -> Optional[Set[Committee]]:
def is_child_of_root_committee(self, _id: Id) -> bool:
"""
:return: returns child committee/s of root committee if present
"""
@ -202,10 +226,6 @@ class Overlay:
"""
pass
@abstractmethod
def root_super_majority_threshold(self, _id: Id) -> int:
pass
@abstractmethod
def super_majority_threshold(self, _id: Id) -> int:
pass
@ -230,12 +250,16 @@ class Carnot:
self.local_high_qc: Optional[Qc] = None
# The latest view committed by a node.
self.latest_committed_view: View = 0
#
# Validated blocks with their validated QCs are included here. If commit conditions is satisfied for
# each one of these blocks it will be committed.
self.safe_blocks: Dict[Id, Block] = dict()
# Block received for a specific view. Make sure the node doesn't receive duplicate blocks.
self.seen_view_blocks: Dict[View, bool] = dict()
# Last timeout QC and its view
self.last_timeout_view_qc: Optional[TimeoutQc] = None
self.last_timeout_view: Optional[View] = None
self.overlay: Overlay = Overlay() # TODO: integrate overlay
# Committed blocks are kept here.
self.committed_blocks: Dict[Id, Block] = dict()
def block_is_safe(self, block: Block) -> bool:
@ -290,47 +314,54 @@ class Carnot:
self.try_commit_grand_parent(block)
def receive_timeout_qc(self, timeout_qc: TimeoutQc):
# TODO: we should be more strict with views in the sense that we should not
# accept 'future' events
# TODO: we should be more strict with views in the sense that we should not accept 'future' events
assert timeout_qc.view >= self.current_view
self.rebuild_overlay_from_timeout_qc(timeout_qc)
def approve_block(self, block: Block, votes: Set[Vote]):
assert block.id() in self.safe_blocks
assert len(votes) == self.overlay.super_majority_threshold(self.id)
assert all(self.overlay.child_committee(self.id, vote.voter) for vote in votes)
assert all(self.overlay.is_member_of_child_committee(self.id, vote.voter) for vote in votes)
assert all(vote.block == block.id() for vote in votes)
assert block.view > self.highest_voted_view
if self.overlay.member_of_root_com(self.id):
if (
self.overlay.is_member_of_root_committee(self.id) and
not self.overlay.is_member_of_leaf_committee(self.id)
):
vote: Vote = Vote(
block=block.id(),
voter=self.id,
view=self.current_view,
qc=self.build_qc(votes)
view=block.view,
qc=self.build_qc(block.view, block)
)
self.send(vote, self.overlay.leader(self.current_view + 1))
else:
vote: Vote = Vote(
block=block.id(),
voter=self.id,
view=self.current_view,
view=block.view,
qc=None
)
self.send(vote, *self.overlay.parent_committee(self.id))
if self.overlay.is_member_of_root_committee(self.id):
self.send(vote, self.overlay.leader(block.view + 1))
else:
self.send(vote, *self.overlay.parent_committee(self.id))
self.increment_voted_view(block.view) # to avoid voting again for this view.
self.increment_view_qc(block.qc)
# This step is very similar to approving a block in the happy path
# A goal of this process is to guarantee that the high_qc gathered at the top
# (or a more recent one) has been seen by the supermajority of nodes in the network
# TODO: Check comment
def approve_new_view(self, timeouts: Set[NewView]):
"""
This step is very similar to approving a block in the happy path
A goal of this process is to guarantee that the high_qc gathered at the top
(or a more recent one) has been seen by the supermajority of nodes in the network
# TODO: Check comment
"""
assert len(set(timeout.view for timeout in timeouts)) == 1
assert all(timeout.view >= self.current_view for timeout in timeouts)
assert all(timeout.view == timeout.timeout_qc.view for timeout in timeouts)
assert len(timeouts) == self.overlay.super_majority_threshold(self.id)
assert all(self.overlay.child_committee(self.id, timeout.sender) for timeout in timeouts)
assert all(self.overlay.is_member_of_child_committee(self.id, timeout.sender) for timeout in timeouts)
timeouts = list(timeouts)
timeout_qc = timeouts[0].timeout_qc
@ -341,12 +372,12 @@ class Carnot:
self.update_timeout_qc(timeout_qc)
self.increment_view_timeout_qc(timeout_qc)
if self.overlay.member_of_root_com(self.id):
if self.overlay.is_member_of_root_committee(self.id):
new_view_msg = NewView(
view=self.current_view,
high_qc=self.local_high_qc,
sender=self.id,
timeout_qc=timeout_qc, # should we do some aggregation here?
timeout_qc=timeout_qc, # should we do some aggregation here?
)
self.send(new_view_msg, self.overlay.leader(self.current_view + 1))
else:
@ -355,52 +386,103 @@ class Carnot:
high_qc=self.local_high_qc,
sender=self.id,
timeout_qc=timeout_qc
)
)
self.send(new_view_msg, *self.overlay.parent_committee(self.id))
self.increment_view_timeout_qc(timeout_qc)
# This checks if a not has already incremented its voted view by local_timeout. If not then it should
# do it now to avoid voting in this view.
self.increment_voted_view(timeout_qc.view)
def forward_vote(self, msg: Vote):
def forward_vote(self, vote: Vote):
assert vote.block in self.safe_blocks
assert self.overlay.child_committee(self.id, vote.voter)
assert self.overlay.is_member_of_child_committee(self.id, vote.voter)
if self.overlay.member_of_root_com(self.id):
if self.overlay.is_member_of_root_committee(self.id):
self.send(vote, self.overlay.leader(self.current_view + 1))
def forward_new_view(self, msg: NewView):
assert msg.view == self.current_view
assert self.overlay.child_committee(self.id, vote.voter)
assert self.overlay.is_member_of_child_committee(self.id, msg.sender)
if self.overlay.member_of_root_com(self.id):
self.send(vote, self.overlay.leader(self.current_view + 1))
if self.overlay.is_member_of_root_committee(self.id):
self.send(msg, self.overlay.leader(self.current_view + 1))
def build_qc(self, quorum: Quorum) -> Qc:
pass
def build_qc(self, view: View, block: Block) -> Qc:
# TODO: implement unhappy path
# Maybe better do build aggregatedQC for unhappy path?
return StandardQc(
view=view,
block=block.id()
)
def propose_block(self, view: View, quorum: Quorum):
assert self.overlay.is_leader(self.id)
assert len(quorum) == self.overlay.leader_super_majority_threshold(self.id)
qc = self.build_qc(quorum)
block = Block(view=view, qc=qc)
assert len(quorum) >= self.overlay.leader_super_majority_threshold(self.id)
vote = list(quorum)[0]
qc = self.build_qc(vote.view, self.safe_blocks[vote.block])
block = Block(
view=view,
qc=qc,
# Dummy id for proposing next block
_id=int_to_id(hash(
(
bytes(f"{view}".encode(encoding="utf8")),
bytes(f"{qc.view}".encode(encoding="utf8"))
)
))
)
self.broadcast(block)
def local_timeout(self):
def is_safe_to_timeout(
self,
highest_voted_view: View,
local_high_qc: Qc,
last_timeout_view_qc: TimeoutQc,
current_view: View
):
"""
Local timeout is different for the root and its child committees. If other committees timeout, they only
stop taking part in consensus. If a member of root or its child committees timeout it sends its timeout message
to all members of root to build the timeout qc. Using this qc we assume that the new
overlay can be built. Hence, by building the new overlay members of root committee can send the timeout qc
to the leaf committee of the new overlay. Upon receipt of the timeout qc the leaf committee members update
their local_high_qc, last_timeout_view_qc and last_voted_view if the view of qcs
(local_high_qc, last_timeout_view_qc) received is higher than their local view. Similalry last_voted_view is
updated if it is greater than the current last_voted_view. When parent committee member receives more than two
third of timeout messages from its children it also updates its local_high_qc, last_timeout_view_qc and
last_voted_view if needed and then send its timeout message upward. In this way the latest qcs move upward
that makes it possible for the next leader to propose a block with the latest local_high_qcs in aggregated qc
from more than two third members of root committee and its children.
"""
# Make sure the node doesn't time out continuously without finishing the step to increment the current view.
# Make sure current view is always higher than the local_high_qc so that the node won't timeout unnecessary
# for a previous view.
assert self.current_view > max(highest_voted_view - 1, local_high_qc.view)
# This condition makes sure a node waits for timeout_qc from root committee to change increment its view with
# a view change.
# A node must change its view after making sure it has the high_Qc or last_timeout_view_qc
# from previous view.
assert (is_sequential_ascending(self.current_view, self.local_high_qc.view) or
is_sequential_ascending(self.current_view, self.last_timeout_view_qc.view))
return (
is_sequential_ascending(current_view, local_high_qc.view) or
is_sequential_ascending(current_view, last_timeout_view_qc.view) or
(current_view == last_timeout_view_qc.view)
)
def local_timeout(self):
assert self.is_safe_to_timeout(
self.highest_voted_view,
self.local_high_qc,
self.last_timeout_view_qc,
self.current_view
)
self.increment_voted_view(self.current_view)
if self.overlay.member_of_root_committee(self.id) or self.overlay.child_of_root_committee(self.id):
if self.overlay.is_member_of_root_committee(self.id) or self.overlay.is_child_of_root_committee(self.id):
timeout_msg: Timeout = Timeout(
view=self.current_view,
high_qc=self.local_high_qc,
local_timeout=True,
# local_timeout is only true for the root committee or members of its children
# root committee or its children can trigger the timeout.
timeout_qc=self.last_timeout_view_qc,
@ -408,49 +490,121 @@ class Carnot:
)
self.send(timeout_msg, *self.overlay.root_committee())
# Root committee detected that supermajority of root + its children has timed out
# The view has failed and this information is sent to all participants along with the information
# necessary to reconstruct the new overlay
def timeout_detected(self, msgs: Set[Timeout]):
"""
Root committee detected that supermajority of root + its children has timed out
The view has failed and this information is sent to all participants along with the information
necessary to reconstruct the new overlay
:param msgs:
:return:
"""
assert len(msgs) == self.overlay.leader_super_majority_threshold(self.id)
assert all(msg.view >= self.current_view for msg in msgs)
assert len(set(msg.view for msg in msgs)) == 1
assert all(msg.local_timeout for msg in msgs)
assert self.overlay.member_of_root_committee(self.id)
assert self.current_view > max(self.highest_voted_view - 1, self.local_high_qc.view)
assert self.overlay.is_member_of_root_committee(self.id)
timeout_qc = self.build_timeout_qc(msgs)
self.update_timeout_qc(timeout_qc)
self.update_high_qc(timeout_qc.high_qc)
self.rebuild_overlay_from_timeout_qc(timeout_qc)
self.broadcast(timeout_qc) # can be sent only to the leafs
self.broadcast(timeout_qc) # we broadcast so all nodes can get ready for voting on a new view
def gather_new_view(self, timeouts: Set[NewView]):
assert not self.overlay.is_member_of_leaf_committee(self.id)
assert len(set(timeout.view for timeout in timeouts)) == 1
assert all(timeout.view >= self.current_view for timeout in timeouts)
assert all(timeout.view == timeout.timeout_qc.view for timeout in timeouts)
assert len(timeouts) == self.overlay.super_majority_threshold(self.id)
assert all(self.overlay.is_member_of_child_committee(self.id, timeout.sender) for timeout in timeouts)
timeouts = list(timeouts)
timeout_qc = timeouts[0].timeout_qc
new_high_qc = timeout_qc.high_qc
self.rebuild_overlay_from_timeout_qc(timeout_qc)
if new_high_qc.view >= self.local_high_qc.view:
self.update_high_qc(new_high_qc)
self.update_timeout_qc(timeout_qc)
self.increment_view_timeout_qc(timeout_qc)
if self.overlay.is_member_of_root_committee(self.id):
timeout_msg = NewView(
view=self.current_view,
high_qc=self.local_high_qc,
sender=self.id,
timeout_qc=timeout_qc,
)
self.send(timeout_msg, self.overlay.leader(self.current_view + 1))
else:
timeout_msg = NewView(
view=self.current_view,
high_qc=self.local_high_qc,
sender=self.id,
timeout_qc=timeout_qc,
)
self.send(timeout_msg, *self.overlay.parent_committee(self.id))
self.increment_view_timeout_qc(timeout_qc)
# This checks if a node has already incremented its voted view by local_timeout. If not then it should
# do it now to avoid voting in this view.
if self.highest_voted_view < self.current_view:
self.increment_voted_view(timeout_qc.view)
def received_timeout_qc(self, timeout_qc: TimeoutQc):
assert timeout_qc.view >= self.current_view
self.rebuild_overlay_from_timeout_qc(timeout_qc)
if self.overlay.is_member_of_leaf_committee(self.id):
new_high_qc = timeout_qc.high_qc
if new_high_qc.view >= self.local_high_qc.view:
self.update_high_qc(new_high_qc)
self.update_timeout_qc(timeout_qc)
self.increment_view_timeout_qc(timeout_qc)
timeout_msg = NewView(
view=self.current_view,
high_qc=self.local_high_qc,
sender=self.id,
timeout_qc=timeout_qc,
)
self.send(timeout_msg, *self.overlay.parent_committee(self.id))
# This checks if a node has already incremented its voted view by local_timeout. If not then it should
# do it now to avoid voting in this view.
if self.highest_voted_view < self.current_view:
self.increment_voted_view(timeout_qc.view)
def rebuild_overlay_from_timeout_qc(self, timeout_qc: TimeoutQc):
assert timeout_qc.view >= self.current_view
self.overlay = Overlay()
def build_timeout_qc(self, msgs: Set[Timeout]) -> TimeoutQc:
pass
def send(self, vote: Vote | Timeout | TimeoutQc, *ids: Id):
def send(self, vote: Vote | Timeout | NewView | TimeoutQc, *ids: Id):
pass
def broadcast(self, block):
pass
# todo blocks from latest_committed_block to grand_parent must be committed.
def try_commit_grand_parent(self, block: Block):
parent = self.safe_blocks.get(block.parent())
grand_parent = self.safe_blocks.get(parent.parent())
# this case should just trigger on genesis_case,
# as the preconditions on outer calls should check on block validity
if not parent or not grand_parent:
return
can_commit = (
parent.view == (grand_parent.view + 1) and
isinstance(block.qc, (StandardQc,)) and
isinstance(parent.qc, (StandardQc,))
)
if can_commit:
self.committed_blocks[grand_parent.id()] = grand_parent
self.increment_latest_committed_view(grand_parent.view)
while grand_parent and grand_parent.view > self.latest_committed_view:
# this case should just trigger on genesis_case,
# as the preconditions on outer calls should check on block validity
if not parent or not grand_parent:
return
can_commit = (
parent.view == (grand_parent.view + 1) and
isinstance(block.qc, (StandardQc,)) and
isinstance(parent.qc, (StandardQc,))
)
if can_commit:
self.committed_blocks[grand_parent.id()] = grand_parent
self.increment_latest_committed_view(grand_parent.view)
grand_parent = self.safe_blocks.get(grand_parent.parent())
def increment_voted_view(self, view: View):
self.highest_voted_view = max(view, self.highest_voted_view)
@ -458,12 +612,11 @@ class Carnot:
def increment_latest_committed_view(self, view: View):
self.latest_committed_view = max(view, self.latest_committed_view)
def increment_view_qc(self, qc: Qc) -> bool:
def increment_view_qc(self, qc: Qc):
if qc.view < self.current_view:
return False
return
self.last_timeout_view_qc = None
self.current_view = qc.view + 1
return True
def increment_view_timeout_qc(self, timeout_qc: TimeoutQc):
if timeout_qc is None or timeout_qc.view < self.current_view:
@ -480,3 +633,5 @@ class Carnot:
if __name__ == "__main__":
pass

289
carnot/test_happy_path.py
View File

@ -1,11 +1,12 @@
from .carnot import *
from unittest import TestCase
from unittest import TestCase, mock
from unittest.mock import patch
class TestCarnotHappyPath(TestCase):
@staticmethod
def add_genesis_block(carnot: Carnot) -> Block:
genesis_block = Block(view=0, qc=StandardQc(block=b"", view=0), content=frozenset(b""))
genesis_block = Block(view=0, qc=StandardQc(block=b"", view=0), _id=b"")
carnot.safe_blocks[genesis_block.id()] = genesis_block
carnot.committed_blocks[genesis_block.id()] = genesis_block
return genesis_block
@ -13,33 +14,33 @@ class TestCarnotHappyPath(TestCase):
def test_receive_block(self):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
block = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block)
def test_receive_multiple_blocks_for_the_same_view(self):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
# 2
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), content=frozenset(b"2"))
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), _id=b"2")
carnot.receive_block(block2)
# 3
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"3"))
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), _id=b"3")
carnot.receive_block(block3)
# 4
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block4)
self.assertEqual(len(carnot.safe_blocks), 5)
# next block is duplicated and as it is already processed should be skipped
block5 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block5 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block5)
self.assertEqual(len(carnot.safe_blocks), 5)
# next block has a different view but is duplicated and as it is already processed should be skipped
block5 = Block(view=5, qc=StandardQc(block=block3.id(), view=4), content=frozenset(b"4"))
block5 = Block(view=5, qc=StandardQc(block=block3.id(), view=4), _id=b"4")
carnot.receive_block(block5)
self.assertEqual(len(carnot.safe_blocks), 5)
@ -47,25 +48,25 @@ class TestCarnotHappyPath(TestCase):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
# 2
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), content=frozenset(b"2"))
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), _id=b"2")
carnot.receive_block(block2)
# 3
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"3"))
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), _id=b"3")
carnot.receive_block(block3)
# 4
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block4)
self.assertEqual(len(carnot.safe_blocks), 5)
# This block should be rejected based on the condition below in block_is_safe().
# block.view >= self.latest_committed_view and block.view == (standard.view + 1)
# block_is_safe() should return false.
block5 = Block(view=3, qc=StandardQc(block=block4.id(), view=4), content=frozenset(b"5"))
block5 = Block(view=3, qc=StandardQc(block=block4.id(), view=4), _id=b"5")
self.assertFalse(carnot.block_is_safe(block5))
carnot.receive_block(block5)
self.assertEqual(len(carnot.safe_blocks), 5)
@ -74,18 +75,18 @@ class TestCarnotHappyPath(TestCase):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
# 2
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), content=frozenset(b"2"))
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), _id=b"2")
carnot.receive_block(block2)
# 3
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"3"))
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), _id=b"3")
carnot.receive_block(block3)
# 4
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block4)
self.assertEqual(len(carnot.safe_blocks), 5)
@ -93,7 +94,7 @@ class TestCarnotHappyPath(TestCase):
# This block should be rejected based on the condition below in block_is_safe().
# block.view >= self.latest_committed_view and block.view == (standard.view + 1)
# block_is_safe() should return false.
block5 = Block(view=5, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"5"))
block5 = Block(view=5, qc=StandardQc(block=block3.id(), view=3), _id=b"5")
self.assertFalse(carnot.block_is_safe(block5))
carnot.receive_block(block5)
self.assertEqual(len(carnot.safe_blocks), 5)
@ -105,21 +106,21 @@ class TestCarnotHappyPath(TestCase):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
# 2
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), content=frozenset(b"2"))
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), _id=b"2")
carnot.receive_block(block2)
# 3
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"3"))
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), _id=b"3")
carnot.receive_block(block3)
# 4
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block4)
block5 = Block(view=5, qc=StandardQc(block=block4.id(), view=4), content=frozenset(b"5"))
block5 = Block(view=5, qc=StandardQc(block=block4.id(), view=4), _id=b"5")
carnot.receive_block(block5)
for block in (block1, block2, block3):
@ -133,25 +134,25 @@ class TestCarnotHappyPath(TestCase):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
# 2
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), content=frozenset(b"2"))
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), _id=b"2")
carnot.receive_block(block2)
# 3
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"3"))
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), _id=b"3")
carnot.receive_block(block3)
# 4
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block4)
self.assertEqual(len(carnot.safe_blocks), 5)
# 5 This is the old standard qc of block number 2. By using the QC for block2, block5 tries to form a fork
# to avert block3 and block b4. Block3 is a committed block
# block_is_safe() should return false.
block5 = Block(view=5, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"5"))
block5 = Block(view=5, qc=StandardQc(block=block2.id(), view=2), _id=b"5")
self.assertFalse(carnot.block_is_safe(block5))
carnot.receive_block(block5)
self.assertEqual(len(carnot.safe_blocks), 5)
@ -160,22 +161,22 @@ class TestCarnotHappyPath(TestCase):
carnot = Carnot(int_to_id(0))
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
# 2
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), content=frozenset(b"2"))
block2 = Block(view=2, qc=StandardQc(block=block1.id(), view=1), _id=b"2")
carnot.receive_block(block2)
# 3
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), content=frozenset(b"3"))
block3 = Block(view=3, qc=StandardQc(block=block2.id(), view=2), _id=b"3")
carnot.receive_block(block3)
# 4
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), content=frozenset(b"4"))
block4 = Block(view=4, qc=StandardQc(block=block3.id(), view=3), _id=b"4")
carnot.receive_block(block4)
self.assertEqual(len(carnot.safe_blocks), 5)
block5 = Block(view=5, qc=StandardQc(block=block4.id(), view=4), content=frozenset(b"5"))
block5 = Block(view=5, qc=StandardQc(block=block4.id(), view=4), _id=b"5")
carnot.receive_block(block5)
self.assertEqual(carnot.latest_committed_view, 3)
self.assertEqual(carnot.local_high_qc.view, 4)
@ -186,11 +187,12 @@ class TestCarnotHappyPath(TestCase):
1: Votes received should increment highest_voted_view and current_view but should not change
latest_committed_view and last_timeout_view
"""
class MockOverlay(Overlay):
def member_of_root_com(self, _id: Id) -> bool:
def is_member_of_root_committee(self, _id: Id) -> bool:
return False
def child_committee(self, parent: Id, child: Id) -> bool:
def is_member_of_child_committee(self, parent: Id, child: Id) -> bool:
return True
def super_majority_threshold(self, _id: Id) -> int:
@ -203,7 +205,7 @@ class TestCarnotHappyPath(TestCase):
carnot.overlay = MockOverlay()
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
votes = set(
Vote(
@ -213,7 +215,7 @@ class TestCarnotHappyPath(TestCase):
qc=StandardQc(block=block1.id(), view=1)
) for i in range(10)
)
carnot.vote(block1, votes)
carnot.approve_block(block1, votes)
self.assertEqual(carnot.highest_voted_view, 1)
self.assertEqual(carnot.current_view, 1)
self.assertEqual(carnot.latest_committed_view, 0)
@ -223,11 +225,12 @@ class TestCarnotHappyPath(TestCase):
"""
2 If last_voted_view is incremented after calling vote with votes lower than.
"""
class MockOverlay(Overlay):
def member_of_root_com(self, _id: Id) -> bool:
def is_member_of_root_committee(self, _id: Id) -> bool:
return False
def child_committee(self, parent: Id, child: Id) -> bool:
def is_member_of_child_committee(self, parent: Id, child: Id) -> bool:
return True
def super_majority_threshold(self, _id: Id) -> int:
@ -240,7 +243,7 @@ class TestCarnotHappyPath(TestCase):
carnot.overlay = MockOverlay()
genesis_block = self.add_genesis_block(carnot)
# 1
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), content=frozenset(b"1"))
block1 = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
carnot.receive_block(block1)
votes = set(
@ -253,9 +256,211 @@ class TestCarnotHappyPath(TestCase):
)
with self.assertRaises((AssertionError, )):
carnot.vote(block1, votes)
carnot.approve_block(block1, votes)
# The test passes as asserting fails in len(votes) == self.overlay.super_majority_threshold(self.id)
# when number of votes are < 9
self.assertEqual(carnot.highest_voted_view, 0)
self.assertEqual(carnot.current_view, 0)
def test_initial_leader_proposes_and_advance(self):
class MockOverlay(Overlay):
def is_leader(self, _id: Id):
return True
def is_member_root(self, _id: Id):
return True
def is_member_leaf(self, _id: Id):
return True
def leader(self, view: View) -> Id:
return int_to_id(0)
def is_member_of_child_committee(self, parent: Id, child: Id) -> bool:
return True
def leader_super_majority_threshold(self, _id: Id) -> int:
return 10
def super_majority_threshold(self, _id: Id) -> int:
return 10
def parent_committee(self, _id: Id) -> Optional[Committee]:
return set()
class MockCarnot(Carnot):
def __init__(self, _id):
super(MockCarnot, self).__init__(_id)
self.proposed_block = None
def broadcast(self, block):
self.proposed_block = block
carnot = MockCarnot(int_to_id(0))
carnot.overlay = MockOverlay()
genesis_block = self.add_genesis_block(carnot)
# votes for genesis block
votes = set(
Vote(
block=genesis_block.id(),
view=0,
voter=int_to_id(i),
qc=StandardQc(
block=genesis_block.id(),
view=0
),
) for i in range(10)
)
# propose a new block
carnot.propose_block(view=1, quorum=votes)
proposed_block = carnot.proposed_block
# process the proposed block as member of a committee
carnot.receive_block(proposed_block)
child_votes = set(
Vote(
block=proposed_block.id(),
view=1,
voter=int_to_id(i),
qc=StandardQc(
block=genesis_block.id(),
view=0
),
) for i in range(10)
)
# vote with votes from child committee
carnot.approve_block(proposed_block, child_votes)
# check carnot state advanced
self.assertTrue(carnot.current_view, 1)
self.assertEqual(carnot.highest_voted_view, 1)
self.assertEqual(carnot.local_high_qc.view, 0)
self.assertIn(proposed_block.id(), carnot.safe_blocks)
def test_leaf_member_advance(self):
"""
Leaf committees do not collect votes as they don't have any child. Therefore, leaf committees in happy
path votes and updates state after receipt of a block
"""
class MockOverlay(Overlay):
def is_leader(self, _id: Id):
return False
def leader(self, view: View) -> Id:
return int_to_id(0)
def parent_committee(self, _id: Id) -> Optional[Committee]:
return set()
def is_member_of_leaf_committee(self, _id: Id) -> bool:
return True
def super_majority_threshold(self, _id: Id) -> int:
return 0
carnot = Carnot(int_to_id(0))
carnot.overlay = MockOverlay()
genesis_block = self.add_genesis_block(carnot)
proposed_block = Block(view=1, qc=StandardQc(block=genesis_block.id(), view=0), _id=b"1")
# Receive the proposed block as a member of the leaf committee
carnot.receive_block(proposed_block)
carnot.approve_block(proposed_block, set())
proposed_block = Block(view=2, qc=StandardQc(block=genesis_block.id(), view=1), _id=b"2")
carnot.receive_block(proposed_block)
carnot.approve_block(proposed_block, set())
# Assert that the current view, highest voted view, and local high QC have all been updated correctly
self.assertEqual(carnot.current_view, 2)
self.assertEqual(carnot.highest_voted_view, 2)
self.assertEqual(carnot.local_high_qc.view, 1)
# Assert that the proposed block has been added to the set of safe blocks
self.assertIn(proposed_block.id(), carnot.safe_blocks)
def test_single_committee_advance(self):
"""
Test that having a single committee (both root and leaf) and a leader is able to advance
"""
class MockCarnot(Carnot):
def __init__(self, id):
super(MockCarnot, self).__init__(id)
self.proposed_block = None
self.latest_vote = None
def broadcast(self, block):
self.proposed_block = block
def send(self, vote: Vote | Timeout | TimeoutQc, *ids: Id):
self.latest_vote = vote
nodes = [MockCarnot(int_to_id(i)) for i in range(4)]
leader = nodes[0]
class MockOverlay(Overlay):
def is_member_of_child_committee(self, parent: Id, child: Id) -> bool:
return False
def leader_super_majority_threshold(self, _id: Id) -> int:
return 3
def is_leader(self, _id: Id):
# Leader is the node with id 0, otherwise not
return {
int_to_id(0): True
}.get(_id, False)
def is_member_of_root_committee(self, _id: Id):
return True
def leader(self, view: View) -> Id:
return int_to_id(0)
def parent_committee(self, _id: Id) -> Optional[Committee]:
return None
def is_member_of_leaf_committee(self, _id: Id) -> bool:
return True
def super_majority_threshold(self, _id: Id) -> int:
return 0
overlay = MockOverlay()
# inject overlay
genesis_block = None
for node in nodes:
node.overlay = overlay
genesis_block = self.add_genesis_block(node)
# votes for genesis block
votes = set(
Vote(
block=genesis_block.id(),
view=0,
voter=int_to_id(i),
qc=StandardQc(
block=genesis_block.id(),
view=0
),
) for i in range(3)
)
leader.propose_block(1, votes)
proposed_block = leader.proposed_block
votes = []
for node in nodes:
node.receive_block(proposed_block)
node.approve_block(proposed_block, set())
votes.append(node.latest_vote)
leader.propose_block(2, set(votes))
next_proposed_block = leader.proposed_block
for node in nodes:
# A node receives the second proposed block
node.receive_block(next_proposed_block)
# it hasn't voted for the view 2, so its state is linked to view 1 still
self.assertEqual(node.highest_voted_view, 1)
self.assertEqual(node.current_view, 1)
# when the node approves the vote we update the current view
# and the local high qc, so they need to be increased
node.approve_block(next_proposed_block, set())
self.assertEqual(node.current_view, 2)
self.assertEqual(node.local_high_qc.view, 1)
self.assertEqual(node.highest_voted_view, 2)

23
carnot/test_unhappy_path.py Normal file
View File

@ -0,0 +1,23 @@
# Unhappy path tests
# 1: At the end of the timeout the highQC in the next leader's aggregatedQC should be the highestQC held by the
# majority of nodes or a qc higher than th highestQC held by the majority of nodes.
# Majority means more than two thirds of total number of nodes, randomly assigned to committees.
# 2: Have consecutive view changes and verify the following state variable:
# last_timeout_view_qc.view
# high_qc.view
# current_view
# last_voted_view
# 3: Due failure consecutive condition between parent and grand parent blocks might not meet. So whenever the
# Consecutive view condition in the try_to_commit fails, then all the blocks between the latest_committed_block and the
# grandparent (including the grandparent) must be committed in order.
# As far as I know current code only excutes the grandparent only. It should also address the case above.
# 4: Have consecutive success adding two blocks then a failure and two consecutive success + 1 failure+ 1 success
# S1 <- S2 <- F1 <- S3 <- S4 <-F2 <- S5
# At S3, S1 should be committed. At S5, S2 and S3 must be committed