status-im
/
nimbus-eth1
mirror of https://github.com/status-im/nimbus-eth1.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth1/nimbus/evm/code_bytes.nim

80 lines
2.6 KiB
Nim
Raw Normal View History

Cache code and invalid jump destination tables (fixes #2268) (#2404) It is common for many accounts to share the same code - at the database level, code is stored by hash meaning only one copy exists per unique program but when loaded in memory, a copy is made for each account. Further, every time we execute the code, it must be scanned for invalid jump destinations which slows down EVM exeuction. Finally, the extcodesize call causes code to be loaded even if only the size is needed. This PR improves on all these points by introducing a shared CodeBytesRef type whose code section is immutable and that can be shared between accounts. Further, a dedicated `len` API call is added so that the EXTCODESIZE opcode can operate without polluting the GC and code cache, for cases where only the size is requested - rocksdb will in this case cache the code itself in the row cache meaning that lookup of the code itself remains fast when length is asked for first. With 16k code entries, there's a 90% hit rate which goes up to 99% during the 2.3M attack - the cache significantly lowers memory consumption and execution time not only during this event but across the board.
2024-06-21 07:44:10 +00:00
# Nimbus
# Copyright (c) 2018-2024 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed except according to those terms.
import stew/byteutils, results, ./interpreter/op_codes
export results
type CodeBytesRef* = ref object
## Code buffer that caches invalid jump positions used for verifying jump
## destinations - `bytes` is immutable once instances is created while
## `invalidPositions` will be built up on demand
bytes: seq[byte]
invalidPositions: seq[byte] # bit seq of invalid jump positions
processed: int
avoid re-writing code (#2490) Avoids pointless rocksdb writes that cause write compaction / amplification, specially in the case where code is shared between multiple accounts
2024-07-15 13:02:23 +00:00
persisted*: bool ## This code stream has been persisted to the database
Cache code and invalid jump destination tables (fixes #2268) (#2404) It is common for many accounts to share the same code - at the database level, code is stored by hash meaning only one copy exists per unique program but when loaded in memory, a copy is made for each account. Further, every time we execute the code, it must be scanned for invalid jump destinations which slows down EVM exeuction. Finally, the extcodesize call causes code to be loaded even if only the size is needed. This PR improves on all these points by introducing a shared CodeBytesRef type whose code section is immutable and that can be shared between accounts. Further, a dedicated `len` API call is added so that the EXTCODESIZE opcode can operate without polluting the GC and code cache, for cases where only the size is requested - rocksdb will in this case cache the code itself in the row cache meaning that lookup of the code itself remains fast when length is asked for first. With 16k code entries, there's a 90% hit rate which goes up to 99% during the 2.3M attack - the cache significantly lowers memory consumption and execution time not only during this event but across the board.
2024-06-21 07:44:10 +00:00
template bitpos(pos: int): (int, byte) =
(pos shr 3, 1'u8 shl (pos and 0x07))
avoid re-writing code (#2490) Avoids pointless rocksdb writes that cause write compaction / amplification, specially in the case where code is shared between multiple accounts
2024-07-15 13:02:23 +00:00
func init*(
T: type CodeBytesRef, bytes: sink seq[byte], persisted = false
): CodeBytesRef =
Cache code and invalid jump destination tables (fixes #2268) (#2404) It is common for many accounts to share the same code - at the database level, code is stored by hash meaning only one copy exists per unique program but when loaded in memory, a copy is made for each account. Further, every time we execute the code, it must be scanned for invalid jump destinations which slows down EVM exeuction. Finally, the extcodesize call causes code to be loaded even if only the size is needed. This PR improves on all these points by introducing a shared CodeBytesRef type whose code section is immutable and that can be shared between accounts. Further, a dedicated `len` API call is added so that the EXTCODESIZE opcode can operate without polluting the GC and code cache, for cases where only the size is requested - rocksdb will in this case cache the code itself in the row cache meaning that lookup of the code itself remains fast when length is asked for first. With 16k code entries, there's a 90% hit rate which goes up to 99% during the 2.3M attack - the cache significantly lowers memory consumption and execution time not only during this event but across the board.
2024-06-21 07:44:10 +00:00
let ip = newSeq[byte]((bytes.len + 7) div 8)
avoid re-writing code (#2490) Avoids pointless rocksdb writes that cause write compaction / amplification, specially in the case where code is shared between multiple accounts
2024-07-15 13:02:23 +00:00
CodeBytesRef(bytes: move(bytes), invalidPositions: ip, persisted: persisted)
Cache code and invalid jump destination tables (fixes #2268) (#2404) It is common for many accounts to share the same code - at the database level, code is stored by hash meaning only one copy exists per unique program but when loaded in memory, a copy is made for each account. Further, every time we execute the code, it must be scanned for invalid jump destinations which slows down EVM exeuction. Finally, the extcodesize call causes code to be loaded even if only the size is needed. This PR improves on all these points by introducing a shared CodeBytesRef type whose code section is immutable and that can be shared between accounts. Further, a dedicated `len` API call is added so that the EXTCODESIZE opcode can operate without polluting the GC and code cache, for cases where only the size is requested - rocksdb will in this case cache the code itself in the row cache meaning that lookup of the code itself remains fast when length is asked for first. With 16k code entries, there's a 90% hit rate which goes up to 99% during the 2.3M attack - the cache significantly lowers memory consumption and execution time not only during this event but across the board.
2024-06-21 07:44:10 +00:00
avoid re-writing code (#2490) Avoids pointless rocksdb writes that cause write compaction / amplification, specially in the case where code is shared between multiple accounts
2024-07-15 13:02:23 +00:00
func init*(
T: type CodeBytesRef, bytes: openArray[byte], persisted = false
): CodeBytesRef =
CodeBytesRef.init(@bytes, persisted = persisted)
Cache code and invalid jump destination tables (fixes #2268) (#2404) It is common for many accounts to share the same code - at the database level, code is stored by hash meaning only one copy exists per unique program but when loaded in memory, a copy is made for each account. Further, every time we execute the code, it must be scanned for invalid jump destinations which slows down EVM exeuction. Finally, the extcodesize call causes code to be loaded even if only the size is needed. This PR improves on all these points by introducing a shared CodeBytesRef type whose code section is immutable and that can be shared between accounts. Further, a dedicated `len` API call is added so that the EXTCODESIZE opcode can operate without polluting the GC and code cache, for cases where only the size is requested - rocksdb will in this case cache the code itself in the row cache meaning that lookup of the code itself remains fast when length is asked for first. With 16k code entries, there's a 90% hit rate which goes up to 99% during the 2.3M attack - the cache significantly lowers memory consumption and execution time not only during this event but across the board.
2024-06-21 07:44:10 +00:00
func init*(T: type CodeBytesRef, bytes: openArray[char]): CodeBytesRef =
CodeBytesRef.init(bytes.toOpenArrayByte(0, bytes.high()))
func fromHex*(T: type CodeBytesRef, hex: string): Opt[CodeBytesRef] =
try:
Opt.some(CodeBytesRef.init(hexToSeqByte(hex)))
except ValueError:
Opt.none(CodeBytesRef)
func invalidPosition(c: CodeBytesRef, pos: int): bool =
let (bpos, bbit) = bitpos(pos)
(c.invalidPositions[bpos] and bbit) > 0
func bytes*(c: CodeBytesRef): lent seq[byte] =
c[].bytes
func len*(c: CodeBytesRef): int =
len(c.bytes)
func isValidOpcode*(c: CodeBytesRef, position: int): bool =
if position >= len(c):
false
elif c.invalidPosition(position):
false
elif position <= c.processed:
true
else:
var i = c.processed
while i <= position:
var opcode = Op(c.bytes[i])
if opcode >= Op.Push1 and opcode <= Op.Push32:
var leftBound = (i + 1)
Fix CodeBytes: invalidPositions out of bound crash (#2523)
2024-07-25 12:23:53 +00:00
var rightBound = min(leftBound + (opcode.int - 95), c.bytes.len)
Cache code and invalid jump destination tables (fixes #2268) (#2404) It is common for many accounts to share the same code - at the database level, code is stored by hash meaning only one copy exists per unique program but when loaded in memory, a copy is made for each account. Further, every time we execute the code, it must be scanned for invalid jump destinations which slows down EVM exeuction. Finally, the extcodesize call causes code to be loaded even if only the size is needed. This PR improves on all these points by introducing a shared CodeBytesRef type whose code section is immutable and that can be shared between accounts. Further, a dedicated `len` API call is added so that the EXTCODESIZE opcode can operate without polluting the GC and code cache, for cases where only the size is requested - rocksdb will in this case cache the code itself in the row cache meaning that lookup of the code itself remains fast when length is asked for first. With 16k code entries, there's a 90% hit rate which goes up to 99% during the 2.3M attack - the cache significantly lowers memory consumption and execution time not only during this event but across the board.
2024-06-21 07:44:10 +00:00
for z in leftBound ..< rightBound:
let (bpos, bbit) = bitpos(z)
c.invalidPositions[bpos] = c.invalidPositions[bpos] or bbit
i = rightBound
else:
i += 1
c.processed = i - 1
not c.invalidPosition(position)
func `==`*(a: CodeBytesRef, b: openArray[byte]): bool =
a.bytes == b