status-im
/
go-watchdog
mirror of https://github.com/status-im/go-watchdog.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
a library to curb OOMs by running Go GC according to a user-defined policy.
5 Commits 3 Branches 6 Tags 94 KiB
Go 97%
Makefile 3%
Go to file
Raúl Kripalani ffbfd5e37a improve logging. 2020-12-02 16:33:00 +00:00
LICENSE-APACHE initial commit. 2020-12-02 00:03:20 +00:00
LICENSE-MIT initial commit. 2020-12-02 00:03:20 +00:00
README.md add godocs link to README. 2020-12-02 00:14:25 +00:00
adaptive.go misc fixes to docs; import path. 2020-12-02 00:10:21 +00:00
go.mod misc fixes to docs; import path. 2020-12-02 00:10:21 +00:00
go.sum initial commit. 2020-12-02 00:03:20 +00:00
log.go improve logging. 2020-12-02 16:33:00 +00:00
watchdog.go improve logging. 2020-12-02 16:33:00 +00:00
watchdog_test.go initial commit. 2020-12-02 00:03:20 +00:00
watermarks.go improve logging. 2020-12-02 16:33:00 +00:00
watermarks_test.go improve logging. 2020-12-02 16:33:00 +00:00

README.md

Go memory watchdog

🐺 A library to curb OOMs by running Go GC according to a user-defined policy.

GoDoc

go-watchdog runs a singleton memory watchdog. It takes system and heap memory readings periodically, and feeds them to a user-defined policy to determine whether GC needs to run immediately.

This library ships with two policies out of the box:

  • watermarks policy: runs GC at configured watermarks of system or heap memory utilisation.
  • adaptive policy: runs GC when the current usage surpasses a dynamically-set threshold.

You can easily build a custom policy tailored to the allocation patterns of your program.

It is recommended that you set both (a) a memory limit and (b) a scope of application of that limit (system or heap) when you start the watchdog. Otherwise, go-watchdog will use the system scope, and will default to the total system memory as the limit. elastic/go-sigar is used to make the discovery.

Why is this even needed?

The garbage collector that ships with the go runtime is pretty good in some regards (low-latency, negligible no stop-the-world), but it's insatisfactory in a number of situations that yield ill-fated outcomes:

  1. it is incapable of dealing with bursty/spiky allocations efficiently; depending on the workload, the program may OOM as a consequence of not scheduling GC in a timely manner.
  2. part of the above is due to the fact that go doesn't concern itself with any limits. To date, it is not possible to set a maximum heap size.
  3. its default policy of scheduling GC when the heap doubles, coupled with its ignorance of system or process limits, can easily cause it to OOM.

For more information, check out these GitHub issues:

License

Dual-licensed: MIT, Apache Software License v2, by way of the Permissive License Stack.