logos-storage/plonky2
1
0
Fork 0
mirror of https://github.com/logos-storage/plonky2.git synced 2026-01-14 11:43:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
plonky2/evm/src/cpu/kernel/aggregator.rs

132 lines
5.7 KiB
Rust
Raw Normal View History

Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
//! Loads each kernel assembly file and concatenates them.
use itertools::Itertools;
Continue work on bootstrapping The kernel is hashed using a Keccak based sponge for now. We could switch to Poseidon later if our kernel grows too large. Note that we use simple zero-padding (pad0*) instead of the standard pad10* rule. It's simpler, and we don't care that the prover can add extra 0s at the end of the code. The program counter can never reach those bytes, and even if it could, they'd be 0 anyway given the EVM's zero-initialization rule. In one CPU row, we can do a whole Keccak hash (via the CTL), absorbing 136 bytes. But we can't actually bootstrap that many bytes of kernel code in one row, because we're also limited by memory bandwidth. Currently we can write 4 bytes of the kernel to memory in one row. So we treat the `keccak_input_limbs` columns as a buffer. We gradually fill up this buffer, 4 bytes (one `u32` word) at a time. Every `136 / 4 = 34` rows, the buffer will be full, so at that point we activate the Keccak CTL to absorb the buffer.
2022-07-14 11:31:47 -07:00
use once_cell::sync::Lazy;
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
use super::assembler::{assemble, Kernel};
More constants for kernel ASM - `GlobalMetadata` - offsets for global kernel variables in memory - `ContextMetadata` - offsets for context-specific kernel variables in memory - `GAS_CONSTANTS`, based on the yellowpaper Also move constants to a separate module since `aggregator` was getting long.
2022-08-01 14:49:28 -07:00
use crate::cpu::kernel::constants::evm_constants;
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
use crate::cpu::kernel::parser::parse;
Continue work on bootstrapping The kernel is hashed using a Keccak based sponge for now. We could switch to Poseidon later if our kernel grows too large. Note that we use simple zero-padding (pad0*) instead of the standard pad10* rule. It's simpler, and we don't care that the prover can add extra 0s at the end of the code. The program counter can never reach those bytes, and even if it could, they'd be 0 anyway given the EVM's zero-initialization rule. In one CPU row, we can do a whole Keccak hash (via the CTL), absorbing 136 bytes. But we can't actually bootstrap that many bytes of kernel code in one row, because we're also limited by memory bandwidth. Currently we can write 4 bytes of the kernel to memory in one row. So we treat the `keccak_input_limbs` columns as a buffer. We gradually fill up this buffer, 4 bytes (one `u32` word) at a time. Every `136 / 4 = 34` rows, the buffer will be full, so at that point we activate the Keccak CTL to absorb the buffer.
2022-07-14 11:31:47 -07:00
pub static KERNEL: Lazy<Kernel> = Lazy::new(combined_kernel);
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
pub(crate) fn combined_kernel() -> Kernel {
let files = vec![
Core transaction processing logic With lots of TODOs to fill in afterward; this is just a start.
2022-07-28 15:46:36 -07:00
include_str!("asm/core/bootloader.asm"),
include_str!("asm/core/create.asm"),
include_str!("asm/core/create_addresses.asm"),
include_str!("asm/core/intrinsic_gas.asm"),
Save columns by verifying invalid opcodes in software (#701) * Save columns by verifying invalid opcodes in software * Autogenerate invalid opcode bitfield (Daniel comment) * Remove unnecessary panic label
2022-08-30 13:06:03 -07:00
include_str!("asm/core/invalid.asm"),
Kernel code to do jumpdest analysis
2022-11-20 12:46:15 -08:00
include_str!("asm/core/jumpdest_analysis.asm"),
Core transaction processing logic With lots of TODOs to fill in afterward; this is just a start.
2022-07-28 15:46:36 -07:00
include_str!("asm/core/nonce.asm"),
include_str!("asm/core/process_txn.asm"),
Combine all syscalls into one flag (#802) * Combine all syscalls into one flag * Minor: typo * Daniel PR comments
2022-11-07 12:29:28 -08:00
include_str!("asm/core/syscall.asm"),
include_str!("asm/core/syscall_stubs.asm"),
Core transaction processing logic With lots of TODOs to fill in afterward; this is just a start.
2022-07-28 15:46:36 -07:00
include_str!("asm/core/terminate.asm"),
include_str!("asm/core/transfer.asm"),
include_str!("asm/core/util.asm"),
Add a asm/curve/ directory
2022-07-30 22:24:11 -07:00
include_str!("asm/curve/bn254/curve_add.asm"),
Move ecrecover
2022-07-31 09:24:04 -07:00
include_str!("asm/curve/bn254/curve_mul.asm"),
Add a asm/curve/ directory
2022-07-30 22:24:11 -07:00
include_str!("asm/curve/bn254/moddiv.asm"),
include_str!("asm/curve/common.asm"),
include_str!("asm/curve/secp256k1/curve_mul.asm"),
include_str!("asm/curve/secp256k1/curve_add.asm"),
Move ecrecover
2022-07-31 09:24:04 -07:00
include_str!("asm/curve/secp256k1/ecrecover.asm"),
Add a asm/curve/ directory
2022-07-30 22:24:11 -07:00
include_str!("asm/curve/secp256k1/inverse_scalar.asm"),
Move ecrecover
2022-07-31 09:24:04 -07:00
include_str!("asm/curve/secp256k1/lift_x.asm"),
include_str!("asm/curve/secp256k1/moddiv.asm"),
alphabetical
2022-10-03 14:10:10 -07:00
include_str!("asm/exp.asm"),
fp6 mul
2022-10-20 16:18:41 -04:00
include_str!("asm/fields/fp6_macros.asm"),
include_str!("asm/fields/fp6_mul.asm"),
include_str!("asm/fields/fp12_mul.asm"),
Inter-row program counter constraints (#639) * Beginning of control flow support * Fixes to halt spin loop
2022-07-28 04:36:33 +10:00
include_str!("asm/halt.asm"),
FIX
2022-11-30 17:46:11 -08:00
include_str!("asm/hash/blake/addresses.asm"),
Blake progress
2022-11-09 10:29:13 -08:00
include_str!("asm/hash/blake/compression.asm"),
include_str!("asm/hash/blake/g_functions.asm"),
FIX
2022-11-30 17:46:11 -08:00
include_str!("asm/hash/blake/hash.asm"),
fixes
2022-11-09 13:48:56 -08:00
include_str!("asm/hash/blake/iv.asm"),
include_str!("asm/hash/blake/ops.asm"),
include_str!("asm/hash/blake/permutations.asm"),
include_str!("asm/hash/blake/store.asm"),
progress
2022-11-10 15:24:08 -08:00
include_str!("asm/hash/blake/util.asm"),
blake initial
2022-12-09 09:34:12 -08:00
include_str!("asm/hash/ripemd/box.asm"),
include_str!("asm/hash/ripemd/compression.asm"),
include_str!("asm/hash/ripemd/constants.asm"),
include_str!("asm/hash/ripemd/functions.asm"),
include_str!("asm/hash/ripemd/main.asm"),
include_str!("asm/hash/ripemd/memory.asm"),
include_str!("asm/hash/ripemd/update.asm"),
include_str!("asm/hash/sha2/compression.asm"),
include_str!("asm/hash/sha2/constants.asm"),
include_str!("asm/hash/sha2/message_schedule.asm"),
include_str!("asm/hash/sha2/ops.asm"),
include_str!("asm/hash/sha2/store_pad.asm"),
include_str!("asm/hash/sha2/temp_words.asm"),
include_str!("asm/hash/sha2/write_length.asm"),
Main function, txn processing loop
2022-09-29 23:09:32 -07:00
include_str!("asm/main.asm"),
Split up memory asm and add more helper functions
2022-08-03 22:09:36 -07:00
include_str!("asm/memory/core.asm"),
include_str!("asm/memory/memcpy.asm"),
Packing memory operations
2022-08-03 22:54:17 -07:00
include_str!("asm/memory/metadata.asm"),
include_str!("asm/memory/packing.asm"),
Fixes to get test_simple_transfer working
2022-12-06 23:05:47 -08:00
include_str!("asm/memory/syscalls.asm"),
Split up memory asm and add more helper functions
2022-08-03 22:09:36 -07:00
include_str!("asm/memory/txn_fields.asm"),
MPT storage logic
2022-10-17 11:31:08 -07:00
include_str!("asm/mpt/accounts.asm"),
More directories for MPT logic
2022-10-16 22:55:09 -07:00
include_str!("asm/mpt/delete/delete.asm"),
include_str!("asm/mpt/hash/hash.asm"),
include_str!("asm/mpt/hash/hash_trie_specific.asm"),
MPT hashing logic, part 1
2022-09-30 13:04:16 -07:00
include_str!("asm/mpt/hex_prefix.asm"),
More directories for MPT logic
2022-10-16 22:55:09 -07:00
include_str!("asm/mpt/insert/insert.asm"),
include_str!("asm/mpt/insert/insert_extension.asm"),
include_str!("asm/mpt/insert/insert_leaf.asm"),
include_str!("asm/mpt/insert/insert_trie_specific.asm"),
include_str!("asm/mpt/load/load.asm"),
include_str!("asm/mpt/load/load_trie_specific.asm"),
Basic MPT logic For now this contains most of the basic framework/structure. Logic for things like insertions will come later.
2022-09-18 09:45:31 -07:00
include_str!("asm/mpt/read.asm"),
MPT storage logic
2022-10-17 11:31:08 -07:00
include_str!("asm/mpt/storage/storage_read.asm"),
include_str!("asm/mpt/storage/storage_write.asm"),
Basic MPT logic For now this contains most of the basic framework/structure. Logic for things like insertions will come later.
2022-09-18 09:45:31 -07:00
include_str!("asm/mpt/util.asm"),
fix merge problem
2022-10-03 14:57:04 -07:00
include_str!("asm/ripemd/box.asm"),
include_str!("asm/ripemd/compression.asm"),
include_str!("asm/ripemd/constants.asm"),
include_str!("asm/ripemd/functions.asm"),
include_str!("asm/ripemd/main.asm"),
include_str!("asm/ripemd/memory.asm"),
include_str!("asm/ripemd/update.asm"),
blake initial
2022-12-09 09:34:12 -08:00
include_str!("asm/rlp/encode.asm"),
alphabetical
2022-10-03 14:10:10 -07:00
include_str!("asm/rlp/decode.asm"),
Fixes to get test_simple_transfer working
2022-12-06 23:05:47 -08:00
include_str!("asm/rlp/encode.asm"),
include_str!("asm/rlp/encode_rlp_string.asm"),
Unroll num_bytes Since it's in some code paths where speed is critical.
2022-10-03 14:40:49 -07:00
include_str!("asm/rlp/num_bytes.asm"),
alphabetical
2022-10-03 14:10:10 -07:00
include_str!("asm/rlp/read_to_memory.asm"),
EVM shift left/right operations (#801) * First parts of shift implementation. * Disable range check errors. * Tidy up ASM. * Update comments; fix some .sum() expressions. * First full draft of shift left/right. * Missed a +1. * Clippy. * Address Jacqui's comments. * Add comment. * Fix missing filter. * Address second round of comments from Jacqui.
2022-11-09 10:47:15 +11:00
include_str!("asm/shift.asm"),
Transaction (RLP) parsing Will add tests once we have the interpreter support for other segmnets.
2022-07-06 19:47:58 -07:00
include_str!("asm/transactions/router.asm"),
include_str!("asm/transactions/type_0.asm"),
include_str!("asm/transactions/type_1.asm"),
include_str!("asm/transactions/type_2.asm"),
Move couple asm files
2022-07-31 09:28:16 -07:00
include_str!("asm/util/assertions.asm"),
include_str!("asm/util/basic_macros.asm"),
MPT storage logic
2022-10-17 11:31:08 -07:00
include_str!("asm/util/keccak.asm"),
Finish extcodecopy
2022-10-21 18:11:27 +02:00
include_str!("asm/account_code.asm"),
Balance
2022-10-28 11:15:43 +02:00
include_str!("asm/balance.asm"),
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
];
let parsed_files = files.iter().map(|f| parse(f)).collect_vec();
Some simple optimization rules Depends on #647.
2022-07-25 09:36:26 -07:00
assemble(parsed_files, evm_constants(), true)
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
}
#[cfg(test)]
mod tests {
fmt
2022-08-08 11:37:35 -07:00
use env_logger::{try_init_from_env, Env, DEFAULT_FILTER_ENV};
Have `make_kernel` log the size of each (assembled) file For now it doesn't log filenames, but we can compare against the list of filenames in `combined_kernel`. Current output: ``` [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 0 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 49 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 387 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 27365 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 0 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 11 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 7 bytes [DEBUG plonky2_evm::cpu::kernel::aggregator::tests] Total kernel size: 27819 bytes ``` This shows that most of our kernel code is from `curve_add.asm`, which makes sense since it invovles a couple uses of the large `inverse` macro. Thankfully that will be replaced at some point.
2022-07-13 09:53:44 -07:00
use log::debug;
EC ops test
2022-07-07 18:06:24 +02:00
message schedule progress
2022-08-01 16:36:36 -07:00
use crate::cpu::kernel::aggregator::combined_kernel;
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
#[test]
fn make_kernel() {
fix
2022-07-31 09:22:13 -07:00
let _ = try_init_from_env(Env::default().filter_or(DEFAULT_FILTER_ENV, "debug"));
Have `make_kernel` log the size of each (assembled) file For now it doesn't log filenames, but we can compare against the list of filenames in `combined_kernel`. Current output: ``` [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 0 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 49 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 387 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 27365 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 0 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 11 bytes [DEBUG plonky2_evm::cpu::kernel::assembler] Assembled file size: 7 bytes [DEBUG plonky2_evm::cpu::kernel::aggregator::tests] Total kernel size: 27819 bytes ``` This shows that most of our kernel code is from `curve_add.asm`, which makes sense since it invovles a couple uses of the large `inverse` macro. Thankfully that will be replaced at some point.
2022-07-13 09:53:44 -07:00
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
// Make sure we can parse and assemble the entire kernel.
Macros with arguments (#595) * Macros with arguments See `basic_macros.rs` for an example. * rename
2022-07-07 08:59:53 -07:00
let kernel = combined_kernel();
fix
2022-07-30 22:31:07 -07:00
debug!("Total kernel size: {} bytes", kernel.code.len());
Parse and assemble kernel functions (#567) * Parse and assemble kernel functions Written in "EVM++" assembly. Later on we will add some priviledged opcodes (in unused opcode ordinals), making it an extension of EVM bytecode. I don't think there's much of a standard for EVM assembly, but I loosely based the syntax on this [proposal](https://gist.github.com/axic/17ddbbce4738ccf4040d30cbb5de484e). * PR feedback * tweaks for consistency * terminology tweaks * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> * Update evm/src/cpu/kernel/opcodes.rs Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org> Co-authored-by: Jacqueline Nabaglo <jakub@mirprotocol.org>
2022-06-20 20:32:29 -07:00
}
}
Reference in New Issue Copy Permalink