Merge pull request #99 from status-im/RunVMTestsDuringCI

Run VMTests during CI and drop VMTests.md
This commit is contained in:
coffeepots 2018-08-08 14:48:29 +01:00 committed by GitHub
commit 06b9471ef1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 2 additions and 663 deletions

View File

@ -1,659 +0,0 @@
VMTests
===
## vmArithmeticTest
```diff
+ add0.json OK
+ add1.json OK
+ add2.json OK
+ add3.json OK
+ add4.json OK
+ addmod0.json OK
+ addmod1.json OK
+ addmod1_overflow2.json OK
+ addmod1_overflow3.json OK
+ addmod1_overflow4.json OK
+ addmod1_overflowDiff.json OK
+ addmod2.json OK
+ addmod2_0.json OK
+ addmod2_1.json OK
+ addmod3.json OK
+ addmod3_0.json OK
+ addmodBigIntCast.json OK
+ addmodDivByZero.json OK
+ addmodDivByZero1.json OK
+ addmodDivByZero2.json OK
+ addmodDivByZero3.json OK
+ arith1.json OK
+ div1.json OK
+ divBoostBug.json OK
+ divByNonZero0.json OK
+ divByNonZero1.json OK
+ divByNonZero2.json OK
+ divByNonZero3.json OK
+ divByZero.json OK
+ divByZero_2.json OK
+ exp0.json OK
+ exp1.json OK
+ exp2.json OK
+ exp3.json OK
+ exp4.json OK
+ exp5.json OK
+ exp6.json OK
+ exp7.json OK
+ exp8.json OK
+ expPowerOf256Of256_0.json OK
+ expPowerOf256Of256_1.json OK
+ expPowerOf256Of256_10.json OK
+ expPowerOf256Of256_11.json OK
+ expPowerOf256Of256_12.json OK
+ expPowerOf256Of256_13.json OK
+ expPowerOf256Of256_14.json OK
+ expPowerOf256Of256_15.json OK
+ expPowerOf256Of256_16.json OK
+ expPowerOf256Of256_17.json OK
+ expPowerOf256Of256_18.json OK
+ expPowerOf256Of256_19.json OK
+ expPowerOf256Of256_2.json OK
+ expPowerOf256Of256_20.json OK
+ expPowerOf256Of256_21.json OK
+ expPowerOf256Of256_22.json OK
+ expPowerOf256Of256_23.json OK
+ expPowerOf256Of256_24.json OK
+ expPowerOf256Of256_25.json OK
+ expPowerOf256Of256_26.json OK
+ expPowerOf256Of256_27.json OK
+ expPowerOf256Of256_28.json OK
+ expPowerOf256Of256_29.json OK
+ expPowerOf256Of256_3.json OK
+ expPowerOf256Of256_30.json OK
+ expPowerOf256Of256_31.json OK
+ expPowerOf256Of256_32.json OK
+ expPowerOf256Of256_33.json OK
+ expPowerOf256Of256_4.json OK
+ expPowerOf256Of256_5.json OK
+ expPowerOf256Of256_6.json OK
+ expPowerOf256Of256_7.json OK
+ expPowerOf256Of256_8.json OK
+ expPowerOf256Of256_9.json OK
+ expPowerOf256_1.json OK
+ expPowerOf256_10.json OK
+ expPowerOf256_11.json OK
+ expPowerOf256_12.json OK
+ expPowerOf256_13.json OK
+ expPowerOf256_14.json OK
+ expPowerOf256_15.json OK
+ expPowerOf256_16.json OK
+ expPowerOf256_17.json OK
+ expPowerOf256_18.json OK
+ expPowerOf256_19.json OK
+ expPowerOf256_2.json OK
+ expPowerOf256_20.json OK
+ expPowerOf256_21.json OK
+ expPowerOf256_22.json OK
+ expPowerOf256_23.json OK
+ expPowerOf256_24.json OK
+ expPowerOf256_25.json OK
+ expPowerOf256_26.json OK
+ expPowerOf256_27.json OK
+ expPowerOf256_28.json OK
+ expPowerOf256_29.json OK
+ expPowerOf256_3.json OK
+ expPowerOf256_30.json OK
+ expPowerOf256_31.json OK
+ expPowerOf256_32.json OK
+ expPowerOf256_33.json OK
+ expPowerOf256_4.json OK
+ expPowerOf256_5.json OK
+ expPowerOf256_6.json OK
+ expPowerOf256_7.json OK
+ expPowerOf256_8.json OK
+ expPowerOf256_9.json OK
+ expPowerOf2_128.json OK
+ expPowerOf2_16.json OK
+ expPowerOf2_2.json OK
+ expPowerOf2_256.json OK
+ expPowerOf2_32.json OK
+ expPowerOf2_4.json OK
+ expPowerOf2_64.json OK
+ expPowerOf2_8.json OK
+ expXY.json OK
+ expXY_success.json OK
+ fibbonacci_unrolled.json OK
+ mod0.json OK
+ mod1.json OK
+ mod2.json OK
+ mod3.json OK
+ mod4.json OK
+ modByZero.json OK
+ mul0.json OK
+ mul1.json OK
+ mul2.json OK
+ mul3.json OK
+ mul4.json OK
+ mul5.json OK
+ mul6.json OK
+ mul7.json OK
+ mulUnderFlow.json OK
+ mulmod0.json OK
+ mulmod1.json OK
+ mulmod1_overflow.json OK
+ mulmod1_overflow2.json OK
+ mulmod1_overflow3.json OK
+ mulmod1_overflow4.json OK
+ mulmod2.json OK
+ mulmod2_0.json OK
+ mulmod2_1.json OK
+ mulmod3.json OK
+ mulmod3_0.json OK
+ mulmod4.json OK
+ mulmoddivByZero.json OK
+ mulmoddivByZero1.json OK
+ mulmoddivByZero2.json OK
+ mulmoddivByZero3.json OK
+ not1.json OK
+ sdiv0.json OK
+ sdiv1.json OK
+ sdiv2.json OK
+ sdiv3.json OK
+ sdiv4.json OK
+ sdiv5.json OK
+ sdiv6.json OK
+ sdiv7.json OK
+ sdiv8.json OK
+ sdiv9.json OK
+ sdivByZero0.json OK
+ sdivByZero1.json OK
+ sdivByZero2.json OK
+ sdiv_dejavu.json OK
+ sdiv_i256min.json OK
+ sdiv_i256min2.json OK
+ sdiv_i256min3.json OK
+ signextendInvalidByteNumber.json OK
+ signextend_00.json OK
+ signextend_0_BigByte.json OK
+ signextend_AlmostBiggestByte.json OK
+ signextend_BigByteBigByte.json OK
+ signextend_BigBytePlus1_2.json OK
+ signextend_BigByte_0.json OK
+ signextend_BitIsNotSet.json OK
+ signextend_BitIsNotSetInHigherByte.json OK
+ signextend_BitIsSetInHigherByte.json OK
+ signextend_Overflow_dj42.json OK
+ signextend_bigBytePlus1.json OK
+ signextend_bitIsSet.json OK
+ smod0.json OK
+ smod1.json OK
+ smod2.json OK
+ smod3.json OK
+ smod4.json OK
+ smod5.json OK
+ smod6.json OK
+ smod7.json OK
+ smod8_byZero.json OK
+ smod_i256min1.json OK
+ smod_i256min2.json OK
+ stop.json OK
+ sub0.json OK
+ sub1.json OK
+ sub2.json OK
+ sub3.json OK
+ sub4.json OK
```
OK: 196/196 Fail: 0/196 Skip: 0/196
## vmBitwiseLogicOperation
```diff
+ and0.json OK
+ and1.json OK
+ and2.json OK
+ and3.json OK
+ and4.json OK
+ and5.json OK
+ byte0.json OK
+ byte1.json OK
+ byte10.json OK
+ byte11.json OK
+ byte2.json OK
+ byte3.json OK
+ byte4.json OK
+ byte5.json OK
+ byte6.json OK
+ byte7.json OK
+ byte8.json OK
+ byte9.json OK
+ byteBN.json OK
+ eq0.json OK
+ eq1.json OK
+ eq2.json OK
+ gt0.json OK
+ gt1.json OK
+ gt2.json OK
+ gt3.json OK
+ iszeo2.json OK
+ iszero0.json OK
+ iszero1.json OK
+ lt0.json OK
+ lt1.json OK
+ lt2.json OK
+ lt3.json OK
+ not0.json OK
+ not1.json OK
+ not2.json OK
+ not3.json OK
+ not4.json OK
+ not5.json OK
+ or0.json OK
+ or1.json OK
+ or2.json OK
+ or3.json OK
+ or4.json OK
+ or5.json OK
+ sgt0.json OK
+ sgt1.json OK
+ sgt2.json OK
+ sgt3.json OK
+ sgt4.json OK
+ slt0.json OK
+ slt1.json OK
+ slt2.json OK
+ slt3.json OK
+ slt4.json OK
+ xor0.json OK
+ xor1.json OK
+ xor2.json OK
+ xor3.json OK
+ xor4.json OK
+ xor5.json OK
```
OK: 61/61 Fail: 0/61 Skip: 0/61
## vmBlockInfoTest
```diff
+ coinbase.json OK
+ difficulty.json OK
+ gaslimit.json OK
+ number.json OK
+ timestamp.json OK
```
OK: 5/5 Fail: 0/5 Skip: 0/5
## vmEnvironmentalInfo
```diff
+ address0.json OK
+ address1.json OK
+ calldatacopy0.json OK
+ calldatacopy0_return.json OK
+ calldatacopy1.json OK
+ calldatacopy1_return.json OK
+ calldatacopy2.json OK
+ calldatacopy2_return.json OK
+ calldatacopyUnderFlow.json OK
+ calldatacopyZeroMemExpansion.json OK
+ calldatacopyZeroMemExpansion_return.json OK
+ calldatacopy_DataIndexTooHigh.json OK
+ calldatacopy_DataIndexTooHigh2.json OK
+ calldatacopy_DataIndexTooHigh2_return.json OK
+ calldatacopy_DataIndexTooHigh_return.json OK
+ calldatacopy_sec.json OK
+ calldataload0.json OK
+ calldataload1.json OK
+ calldataload2.json OK
+ calldataloadSizeTooHigh.json OK
+ calldataloadSizeTooHighPartial.json OK
+ calldataload_BigOffset.json OK
+ calldatasize0.json OK
+ calldatasize1.json OK
+ calldatasize2.json OK
+ caller.json OK
+ callvalue.json OK
+ codecopy0.json OK
+ codecopyZeroMemExpansion.json OK
+ codecopy_DataIndexTooHigh.json OK
+ codesize.json OK
+ gasprice.json OK
+ origin.json OK
```
OK: 33/33 Fail: 0/33 Skip: 0/33
## vmIOandFlowOperations
```diff
+ BlockNumberDynamicJump0_AfterJumpdest.json OK
+ BlockNumberDynamicJump0_AfterJumpdest3.json OK
+ BlockNumberDynamicJump0_foreverOutOfGas.json OK
+ BlockNumberDynamicJump0_jumpdest0.json OK
+ BlockNumberDynamicJump0_jumpdest2.json OK
+ BlockNumberDynamicJump0_withoutJumpdest.json OK
+ BlockNumberDynamicJump1.json OK
+ BlockNumberDynamicJumpInsidePushWithJumpDest.json OK
+ BlockNumberDynamicJumpInsidePushWithoutJumpDest.json OK
+ BlockNumberDynamicJumpi0.json OK
+ BlockNumberDynamicJumpi1.json OK
+ BlockNumberDynamicJumpi1_jumpdest.json OK
+ BlockNumberDynamicJumpiAfterStop.json OK
+ BlockNumberDynamicJumpiOutsideBoundary.json OK
+ BlockNumberDynamicJumpifInsidePushWithJumpDest.json OK
+ BlockNumberDynamicJumpifInsidePushWithoutJumpDest.json OK
+ DyanmicJump0_outOfBoundary.json OK
+ DynamicJump0_AfterJumpdest.json OK
+ DynamicJump0_AfterJumpdest3.json OK
+ DynamicJump0_foreverOutOfGas.json OK
+ DynamicJump0_jumpdest0.json OK
+ DynamicJump0_jumpdest2.json OK
+ DynamicJump0_withoutJumpdest.json OK
+ DynamicJump1.json OK
+ DynamicJumpAfterStop.json OK
+ DynamicJumpInsidePushWithJumpDest.json OK
+ DynamicJumpInsidePushWithoutJumpDest.json OK
+ DynamicJumpJD_DependsOnJumps0.json OK
+ DynamicJumpJD_DependsOnJumps1.json OK
+ DynamicJumpPathologicalTest0.json OK
+ DynamicJumpPathologicalTest1.json OK
+ DynamicJumpPathologicalTest2.json OK
+ DynamicJumpPathologicalTest3.json OK
+ DynamicJumpStartWithJumpDest.json OK
+ DynamicJump_value1.json OK
+ DynamicJump_value2.json OK
+ DynamicJump_value3.json OK
+ DynamicJump_valueUnderflow.json OK
+ DynamicJumpi0.json OK
+ DynamicJumpi1.json OK
+ DynamicJumpi1_jumpdest.json OK
+ DynamicJumpiAfterStop.json OK
+ DynamicJumpiOutsideBoundary.json OK
+ DynamicJumpifInsidePushWithJumpDest.json OK
+ DynamicJumpifInsidePushWithoutJumpDest.json OK
+ JDfromStorageDynamicJump0_AfterJumpdest.json OK
+ JDfromStorageDynamicJump0_AfterJumpdest3.json OK
+ JDfromStorageDynamicJump0_foreverOutOfGas.json OK
+ JDfromStorageDynamicJump0_jumpdest0.json OK
+ JDfromStorageDynamicJump0_jumpdest2.json OK
+ JDfromStorageDynamicJump0_withoutJumpdest.json OK
+ JDfromStorageDynamicJump1.json OK
+ JDfromStorageDynamicJumpInsidePushWithJumpDest.json OK
+ JDfromStorageDynamicJumpInsidePushWithoutJumpDest.json OK
+ JDfromStorageDynamicJumpi0.json OK
+ JDfromStorageDynamicJumpi1.json OK
+ JDfromStorageDynamicJumpi1_jumpdest.json OK
+ JDfromStorageDynamicJumpiAfterStop.json OK
+ JDfromStorageDynamicJumpiOutsideBoundary.json OK
+ JDfromStorageDynamicJumpifInsidePushWithJumpDest.json OK
+ JDfromStorageDynamicJumpifInsidePushWithoutJumpDest.json OK
+ bad_indirect_jump1.json OK
+ bad_indirect_jump2.json OK
+ byte1.json OK
+ calldatacopyMemExp.json OK
+ codecopyMemExp.json OK
+ deadCode_1.json OK
+ dupAt51becameMload.json OK
+ for_loop1.json OK
+ for_loop2.json OK
+ gas0.json OK
+ gas1.json OK
+ gasOverFlow.json OK
+ indirect_jump1.json OK
+ indirect_jump2.json OK
+ indirect_jump3.json OK
+ indirect_jump4.json OK
+ jump0_AfterJumpdest.json OK
+ jump0_AfterJumpdest3.json OK
+ jump0_foreverOutOfGas.json OK
+ jump0_jumpdest0.json OK
+ jump0_jumpdest2.json OK
+ jump0_outOfBoundary.json OK
+ jump0_withoutJumpdest.json OK
+ jump1.json OK
+ jumpAfterStop.json OK
+ jumpDynamicJumpSameDest.json OK
+ jumpHigh.json OK
+ jumpInsidePushWithJumpDest.json OK
+ jumpInsidePushWithoutJumpDest.json OK
+ jumpOntoJump.json OK
+ jumpTo1InstructionafterJump.json OK
+ jumpTo1InstructionafterJump_jumpdestFirstInstruction.json OK
+ jumpTo1InstructionafterJump_noJumpDest.json OK
+ jumpToUint64maxPlus1.json OK
+ jumpToUintmaxPlus1.json OK
+ jumpdestBigList.json OK
+ jumpi0.json OK
+ jumpi1.json OK
+ jumpi1_jumpdest.json OK
+ jumpiAfterStop.json OK
+ jumpiOutsideBoundary.json OK
+ jumpiToUint64maxPlus1.json OK
+ jumpiToUintmaxPlus1.json OK
+ jumpi_at_the_end.json OK
+ jumpifInsidePushWithJumpDest.json OK
+ jumpifInsidePushWithoutJumpDest.json OK
+ kv1.json OK
+ log1MemExp.json OK
+ loop_stacklimit_1020.json OK
+ loop_stacklimit_1021.json OK
+ memory1.json OK
+ mloadError0.json OK
+ mloadError1.json OK
+ mloadMemExp.json OK
+ mloadOutOfGasError2.json OK
+ msize0.json OK
+ msize1.json OK
+ msize2.json OK
+ msize3.json OK
+ mstore0.json OK
+ mstore1.json OK
+ mstore8MemExp.json OK
+ mstore8WordToBigError.json OK
+ mstore8_0.json OK
+ mstore8_1.json OK
+ mstoreMemExp.json OK
+ mstoreWordToBigError.json OK
+ mstore_mload0.json OK
+ pc0.json OK
+ pc1.json OK
+ pop0.json OK
+ pop1.json OK
+ return1.json OK
+ return2.json OK
+ sha3MemExp.json OK
+ sstore_load_0.json OK
+ sstore_load_1.json OK
+ sstore_load_2.json OK
+ sstore_underflow.json OK
+ stack_loop.json OK
+ stackjump1.json OK
+ swapAt52becameMstore.json OK
+ when.json OK
```
OK: 144/144 Fail: 0/144 Skip: 0/144
## vmLogTest
```diff
+ log0_emptyMem.json OK
+ log0_logMemStartTooHigh.json OK
+ log0_logMemsizeTooHigh.json OK
+ log0_logMemsizeZero.json OK
+ log0_nonEmptyMem.json OK
+ log0_nonEmptyMem_logMemSize1.json OK
+ log0_nonEmptyMem_logMemSize1_logMemStart31.json OK
+ log1_Caller.json OK
+ log1_MaxTopic.json OK
+ log1_emptyMem.json OK
+ log1_logMemStartTooHigh.json OK
+ log1_logMemsizeTooHigh.json OK
+ log1_logMemsizeZero.json OK
+ log1_nonEmptyMem.json OK
+ log1_nonEmptyMem_logMemSize1.json OK
+ log1_nonEmptyMem_logMemSize1_logMemStart31.json OK
+ log2_Caller.json OK
+ log2_MaxTopic.json OK
+ log2_emptyMem.json OK
+ log2_logMemStartTooHigh.json OK
+ log2_logMemsizeTooHigh.json OK
+ log2_logMemsizeZero.json OK
+ log2_nonEmptyMem.json OK
+ log2_nonEmptyMem_logMemSize1.json OK
+ log2_nonEmptyMem_logMemSize1_logMemStart31.json OK
+ log3_Caller.json OK
+ log3_MaxTopic.json OK
+ log3_PC.json OK
+ log3_emptyMem.json OK
+ log3_logMemStartTooHigh.json OK
+ log3_logMemsizeTooHigh.json OK
+ log3_logMemsizeZero.json OK
+ log3_nonEmptyMem.json OK
+ log3_nonEmptyMem_logMemSize1.json OK
+ log3_nonEmptyMem_logMemSize1_logMemStart31.json OK
+ log4_Caller.json OK
+ log4_MaxTopic.json OK
+ log4_PC.json OK
+ log4_emptyMem.json OK
+ log4_logMemStartTooHigh.json OK
+ log4_logMemsizeTooHigh.json OK
+ log4_logMemsizeZero.json OK
+ log4_nonEmptyMem.json OK
+ log4_nonEmptyMem_logMemSize1.json OK
+ log4_nonEmptyMem_logMemSize1_logMemStart31.json OK
+ log_2logs.json OK
```
OK: 46/46 Fail: 0/46 Skip: 0/46
## vmPerformance
```diff
+ ackermann31.json OK
+ ackermann32.json OK
+ ackermann33.json OK
+ fibonacci10.json OK
+ fibonacci16.json OK
loop-add-10M.json Skip
loop-divadd-10M.json Skip
loop-divadd-unr100-10M.json Skip
loop-exp-16b-100k.json Skip
loop-exp-1b-1M.json Skip
loop-exp-2b-100k.json Skip
loop-exp-32b-100k.json Skip
loop-exp-4b-100k.json Skip
loop-exp-8b-100k.json Skip
loop-exp-nop-1M.json Skip
loop-mul.json Skip
loop-mulmod-2M.json Skip
+ manyFunctions100.json OK
```
OK: 6/18 Fail: 0/18 Skip: 12/18
## vmPushDupSwapTest
```diff
+ dup1.json OK
+ dup10.json OK
+ dup11.json OK
+ dup12.json OK
+ dup13.json OK
+ dup14.json OK
+ dup15.json OK
+ dup16.json OK
+ dup2.json OK
+ dup2error.json OK
+ dup3.json OK
+ dup4.json OK
+ dup5.json OK
+ dup6.json OK
+ dup7.json OK
+ dup8.json OK
+ dup9.json OK
+ push1.json OK
+ push10.json OK
+ push11.json OK
+ push12.json OK
+ push13.json OK
+ push14.json OK
+ push15.json OK
+ push16.json OK
+ push17.json OK
+ push18.json OK
+ push19.json OK
+ push1_missingStack.json OK
+ push2.json OK
+ push20.json OK
+ push21.json OK
+ push22.json OK
+ push23.json OK
+ push24.json OK
+ push25.json OK
+ push26.json OK
+ push27.json OK
+ push28.json OK
+ push29.json OK
+ push3.json OK
+ push30.json OK
+ push31.json OK
+ push32.json OK
+ push32AndSuicide.json OK
+ push32FillUpInputWithZerosAtTheEnd.json OK
+ push32Undefined.json OK
+ push32Undefined2.json OK
+ push32Undefined3.json OK
+ push33.json OK
+ push4.json OK
+ push5.json OK
+ push6.json OK
+ push7.json OK
+ push8.json OK
+ push9.json OK
+ swap1.json OK
+ swap10.json OK
+ swap11.json OK
+ swap12.json OK
+ swap13.json OK
+ swap14.json OK
+ swap15.json OK
+ swap16.json OK
+ swap2.json OK
+ swap2error.json OK
+ swap3.json OK
+ swap4.json OK
+ swap5.json OK
+ swap6.json OK
+ swap7.json OK
+ swap8.json OK
+ swap9.json OK
+ swapjump1.json OK
```
OK: 74/74 Fail: 0/74 Skip: 0/74
## vmRandomTest
```diff
+ 201503102320PYTHON.json OK
+ 201503110206PYTHON.json OK
+ 201503110219PYTHON.json OK
+ 201503110346PYTHON_PUSH24.json OK
+ 201503111844PYTHON.json OK
+ 201503112218PYTHON.json OK
```
OK: 6/6 Fail: 0/6 Skip: 0/6
## vmSha3Test
```diff
+ sha3_0.json OK
+ sha3_1.json OK
+ sha3_2.json OK
+ sha3_3.json OK
+ sha3_4.json OK
+ sha3_5.json OK
+ sha3_6.json OK
+ sha3_bigOffset.json OK
+ sha3_bigOffset2.json OK
+ sha3_bigSize.json OK
+ sha3_memSizeNoQuadraticCost31.json OK
+ sha3_memSizeQuadraticCost32.json OK
+ sha3_memSizeQuadraticCost32_zeroSize.json OK
+ sha3_memSizeQuadraticCost33.json OK
+ sha3_memSizeQuadraticCost63.json OK
+ sha3_memSizeQuadraticCost64.json OK
+ sha3_memSizeQuadraticCost64_2.json OK
+ sha3_memSizeQuadraticCost65.json OK
```
OK: 18/18 Fail: 0/18 Skip: 0/18
## vmSystemOperations
```diff
+ TestNameRegistrator.json OK
+ return0.json OK
+ return1.json OK
+ return2.json OK
+ suicide0.json OK
+ suicideNotExistingAccount.json OK
+ suicideSendEtherToMe.json OK
```
OK: 7/7 Fail: 0/7 Skip: 0/7
## vmTests
```diff
+ suicide.json OK
```
OK: 1/1 Fail: 0/1 Skip: 0/1

View File

@ -12,7 +12,5 @@ when true:
./test_stack,
./test_opcode,
./test_storage_backends,
./test_genesis
when false:
import ./test_vm_json
./test_genesis,
./test_vm_json