status-im
/
infra-utils
mirror of https://github.com/status-im/infra-utils.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
219 Commits 5 Branches 0 Tags 569 KiB
Commit Graph

36 Commits

Author SHA1 Message Date
markoburcul 6cef788ca3 elasticsearch: Script for migrating shards in cluster 
This script was created as the result of an effort for this task:
https://github.com/status-im/infra-hq/issues/151
You can use it when adding/removing nodes from an ES cluster if you want
to migrate data in a more kosher way.

Signed-off-by: markoburcul <marko@status.im>
 2024-10-07 09:05:16 +02:00
Jakub Sokołowski b313825d25
elasticsearch/esclean.py: add propper logger usage 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-24 11:03:53 +02:00
Jakub Sokołowski 1f4f88d31f
elasticsearch/esclean.py: parametrize retries 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-24 10:25:38 +02:00
Jakub Sokołowski ce6071e292
elasticsearch/esclean.py: add ES timeout flag 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-24 10:23:26 +02:00
Jakub Sokołowski 42818e26f5
elasticsearch/esclean: fix deprecation warnings 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-07-15 11:49:19 +02:00
Jakub Sokołowski 5b0bf5d420
elasticsearch/esclean: add logsource flag 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2024-01-31 16:07:01 +01:00
Jakub Sokołowski 5dc408f866
elasticsearch/esclean.py: add missing schema argument 
Fixes:
```
TypeError: __init__() missing 1 required positional argument: 'scheme'
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-12-04 18:37:37 +01:00
Jakub Sokołowski 1e1228cd3a
elasticsearch/esclean.py: use match for fleet field 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2023-06-26 21:04:27 +02:00
Jakub Sokołowski 0f4dbde7f8
elasticsearch/esclean.py: allow use of wildcard with program 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2021-06-14 13:28:40 +02:00
Jakub Sokołowski d1e0426cc2
elasticsearch: hash peer_id's before putting them into CSV 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-06-30 09:56:45 +02:00
Jakub Sokołowski 0a5c14a854
elasticsearch: add unique_count.py for creating CSV of peers 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-06-30 09:52:31 +02:00
Jakub Sokołowski 3c142fffd8
reindex.sh: fix variable names 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-28 23:06:36 +02:00
Jakub Sokołowski 7baff3493d
esclean.py: add support for source IP and tags 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-27 13:02:10 +02:00
Jakub Sokołowski 55ba86a864
reindex.sh: fix paths for cloning 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-27 11:53:19 +02:00
Jakub Sokołowski 4dcc207cc4
reindex.sh: make index read-only before cloning 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-27 11:10:52 +02:00
Jakub Sokołowski 88c94f6857
reindex.sh: use _clone to re-name index back 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-27 09:23:12 +02:00
Jakub Sokołowski c1ff90f5e9
peer_id.py: fix query check 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-27 09:23:03 +02:00
Jakub Sokołowski 4f3ccd269e
esclean.py: fix query argument 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-27 09:22:35 +02:00
Jakub Sokołowski f0e4fb0846
es: add utility for removing fields 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-26 20:50:41 +02:00
Jakub Sokołowski 62cbb5d7a9
fix parameters for peer_id.py 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-05-26 20:35:17 +02:00
Jakub Sokołowski 4b28eae979
fix help 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-02-20 22:45:32 +01:00
Jakub Sokołowski adf52e441f
lower the backoff and delay 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-02-20 16:28:57 +01:00
Jakub Sokołowski a647b307e9
esclean: fix retry 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-02-19 23:01:36 +01:00
Jakub Sokołowski ab457106a6
esclean: add --older-than flag 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-02-07 13:16:11 +01:00
Jakub Sokołowski bdb06a90bd
add retries to esclean.py 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2020-01-13 17:27:46 +01:00
Jakub Sokołowski a76ed2ad96
adjust formatting 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-10-28 13:49:06 +01:00
Jakub Sokołowski 0b674b83d2
add script for re-indexing ES indices 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-10-28 13:17:49 +01:00
Jakub Sokołowski 07f1c4297d
if there's nothing to delete dont delete 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-06-13 00:14:32 -04:00
Jakub Sokołowski bcc5793486
add --severity option 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-06-13 00:10:57 -04:00
Jakub Sokołowski 9ad8ef644b
don't update if count is 0 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-06-12 20:00:36 -04:00
Jakub Sokołowski 5f7c448c18
don't run for docs that already have peer_id 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-06-12 19:55:47 -04:00
Jakub Sokołowski e0e06ea36d
add script for extracting statusd peer_id from logs 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-06-10 12:40:18 -04:00
Jakub Sokołowski 9174a62108
increase timeout and add retries 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2019-04-24 12:58:33 +02:00
Jakub Sokołowski 67f794df1c
add --fleet search term 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2018-11-30 11:27:49 +01:00
Jakub Sokołowski 563f840e4a
add timeout, fix when deleting/querying is ran 2018-11-28 12:18:31 +01:00
Jakub Sokołowski 1b8754c424
add script for quietying ES and deleting logs 
Signed-off-by: Jakub Sokołowski <jakub@status.im>
 2018-11-26 23:49:16 +01:00