The Status module that consumes go-ethereum https://status.im
Go to file
Stefan 63efa6ae84 feat: cache to database retrieved balance history
Implement fetching of balance history from the database if it exists,
otherwise retrieves it using a RPC calls. Every balance is stored in DB
as an entry.

Also fetch regularly balance history for all enabled networks, accounts
and tokens. Fetching for a specific configuration will pause the
full update which will resume after the specific configuration is fetched.

The granularity of data points returned is defined by the constant
increment step define in `timeIntervalToHoursPerStep` for each interval.
The `timeIntervalToHoursPerStep` values have a common divisor to
have cache hit between requests.

To ensure cache hit the requested blocks are "snapped" to specific
time points defined by the `timeIntervalToSnapUnit` enumeration

The cached data is stored in `balance_history` table.
The "calibration" data, special case, used for bloc average timestamp is
stored with address `0x0` and balance is not fetched/used.

Uniqueness constrained is enforced by the `balance_history_identify_entry`
UNIQUE index.

Optimal DB fetching is ensured by the `balance_history_filter_entries`
index

Tests are implemented using a mock of `BlockInfoSource` interface used
to intercept the RPC calls.
`getBalanceHistoryFromBlocksSource` contains main functionality independent
of the `chainClientSource`, `BlockInfoSource`'s concrete implementation.

Notes:

- the timestamp for a block balance is approximate and based on a very
rough heuristic: blocks in between data points have a constant timestamp
(average of the fist and last block in the one year rage)
- this is a complete refactoring of the logic used to fetch due to the
nature of the heuristic chosen to ensure cache hit independent of the
time interval if the granularity matches the cache

Updates status-desktop #8226
2023-01-04 14:20:15 +04:00
.dependabot Patch/dependabot protocol (#1980) 2020-05-19 00:14:02 +01:00
.github fix documentation url in the checklist (#2770) 2022-07-22 22:12:29 +05:30
.vscode chore: fix vscode test run 2022-11-17 10:40:26 +01:00
_assets ci: use WORKSPACE_TMP for TMPDIR to not abuse /run 2023-01-03 16:54:22 +01:00
_docs rename status-react to status-mobile 2022-07-17 15:37:14 +02:00
_examples go-waku integration (#2247) 2021-06-16 16:19:45 -04:00
abi-spec replace web3-utils methods usage by status-go (#2840) 2022-10-18 21:36:54 +08:00
account replace deprecated ioutil.TempDir with os.MkdirTemp 2023-01-03 15:40:11 +01:00
api replace deprecated ioutil.TempDir with os.MkdirTemp 2023-01-03 15:40:11 +01:00
appdatabase feat: add balance_history DB table and update migration scripts 2023-01-04 12:15:41 +04:00
appmetrics feat: make kdf iterations configurable when creating a multiaccount (#2796) 2022-09-27 16:27:20 -04:00
build/bin add .gitkeep to build/bin so it always exists 2019-03-05 14:22:00 +01:00
cmd fix(RequestCommunityInfoFromMailserver): Add useDatabase flag to read/not read community info before requesting mailserver.. 2022-11-30 11:10:13 +01:00
common Move services to status-node 2021-07-20 10:57:38 +02:00
config chore: add status.prod fleet (#2583) 2022-03-11 15:48:51 -05:00
connection Address linter issues 2022-03-28 13:14:12 +01:00
contracts feat: cbridge integration 2023-01-04 08:23:18 +01:00
db Address linter issues 2022-03-28 13:14:12 +01:00
discovery chore: upgrade go-waku and log about messages sent/received 2022-11-09 17:39:17 +03:00
eth-node synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
exportlogs Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
extkeys Upgrade geth 2021-07-20 10:57:38 +02:00
images fix(images): ensure decode from URL emits error on HTTP error code 2022-09-19 12:55:17 +02:00
ipfs fix: add mode build param and change ipfs gateway accordingly 2022-08-26 18:57:55 +02:00
logutils Removed unnecessary dependency from logutil package 2021-08-19 11:35:40 +02:00
mailserver feat: add balance_history DB table and update migration scripts 2023-01-04 12:15:41 +04:00
metrics metrics: fix p2p_peers_count metrics labels 2021-08-24 11:07:39 +02:00
mobile converting desktop key to mobile readable (#3042) 2023-01-04 07:25:07 +05:30
multiaccounts feat: add balance_history DB table and update migration scripts 2023-01-04 12:15:41 +04:00
node replace deprecated ioutil.TempDir with os.MkdirTemp 2023-01-03 15:40:11 +01:00
nodecfg feat: use identify protocol 2022-11-15 11:00:36 -04:00
notifier Notifier package to interact with gorush notifications (#910) 2018-05-08 16:30:03 +02:00
params replace deprecated ioutil.TempDir with os.MkdirTemp 2023-01-03 15:40:11 +01:00
peers synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
postgres Anon Metrics Broadcast (#2198) 2021-09-01 13:02:18 +01:00
profiling replace deprecated ioutil.TempDir with os.MkdirTemp 2023-01-03 15:40:11 +01:00
protocol feat: add balance_history DB table and update migration scripts 2023-01-04 12:15:41 +04:00
rpc synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
rtt rename status-react to status-mobile 2022-07-17 15:37:14 +02:00
server replace deprecated ioutil.TempDir with os.MkdirTemp 2023-01-03 15:40:11 +01:00
services feat: cache to database retrieved balance history 2023-01-04 14:20:15 +04:00
signal synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
sqlite feat: make kdf iterations configurable when creating a multiaccount (#2796) 2022-09-27 16:27:20 -04:00
static feat: add balance_history DB table and update migration scripts 2023-01-04 12:15:41 +04:00
t feat: add balance_history DB table and update migration scripts 2023-01-04 12:15:41 +04:00
telemetry feat: Push messages to telemetry server when receiving messages 2021-11-10 09:34:14 +01:00
timesource Move services to status-node 2021-07-20 10:57:38 +02:00
transactions fix: lint (#2845) 2022-09-27 18:59:02 -04:00
vendor feat/currency rounding (#3041) 2023-01-03 10:24:55 -03:00
waku synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
wakuv2 Restart discovery when it fails 2022-12-14 20:54:48 +00:00
.codeclimate.yml feat: delete for me (#2866) 2022-09-28 19:42:17 +08:00
.dockerignore Ignore *.test files from all folders (#635) 2018-02-08 14:23:08 +02:00
.gitignore feat: add visual identity service (#2550) 2022-03-11 16:17:48 -05:00
.golangci.yml Remove protocol and eth-node submodules (#1835) 2020-02-10 12:22:37 +01:00
.travis.yml Remove Ropsten and Rinkeby test networks 2022-10-24 14:17:46 +02:00
BOOTNODE.md docs: clarify purpose and describe ports 2020-02-06 18:44:23 +01:00
CMakeLists.txt dev: allow option to force compiling for apple silicon 2022-06-30 09:42:19 +02:00
CONTRIBUTING.md rename status-react to status-mobile 2022-07-17 15:37:14 +02:00
LICENSE.md Create LICENSE.md 2016-09-21 10:00:29 +08:00
MAILSERVER.md rcp.sh: curl wrapper for easier RPC calls 2021-08-24 11:23:25 +02:00
Makefile ci: use Nix shell to provide Android SDK for builds 2022-12-05 15:48:27 +01:00
README.md Explain how to run Waku node in README (#1921) 2020-04-09 18:33:53 +02:00
RELEASING.md rename status-react to status-mobile 2022-07-17 15:37:14 +02:00
VERSION chore: code review comments applied 2022-12-23 15:07:14 +01:00
WAKU.md Create WAKU.md with spec support (#1855) 2020-02-21 17:56:15 +08:00
go.mod feat/currency rounding (#3041) 2023-01-03 10:24:55 -03:00
go.sum feat/currency rounding (#3041) 2023-01-03 10:24:55 -03:00
nix.conf ci: use Nix shell to provide Android SDK for builds 2022-12-05 15:48:27 +01:00
shell.nix nix: disable chroot on Darwin to avoid Xcode issues 2023-01-03 16:54:23 +01:00