status-im
/
go-watchdog
mirror of https://github.com/status-im/go-watchdog.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
go-watchdog/README.md

55 lines
2.3 KiB
Markdown
Raw Normal View History

initial commit.
2020-12-02 00:03:20 +00:00
# Go memory watchdog
> 🐺 A library to curb OOMs by running Go GC according to a user-defined policy.
add godocs link to README.
2020-12-02 00:14:25 +00:00
[![GoDoc](https://img.shields.io/badge/godoc-reference-5272B4.svg?style=flat-square)](https://godoc.org/github.com/raulk/go-watchdog)
major rewrite of go-watchdog. This commit introduces a major rewrite of go-watchdog. * HeapDriven and SystemDriven are now distinct run modes. * WIP ProcessDriven that uses cgroups. * Policies are now stateless, pure and greatly simplified. * Policies now return the next utilization at which GC should run. The watchdog enforces that value differently depending on the run mode. * The heap-driven run mode adjusts GOGC dynamically. This places the responsibility on the Go runtime to honour the trigger point, and results in more robust logic that is not vulnerable to very quick bursts within sampling periods. * The heap-driven run mode is no longer polling (interval-driven). Instead, it relies entirely on GC signals. * The Silence and Emergency features of the watermark policy have been removed. If utilization is above the last watermark, the policy will request immediate GC. * Races removed.
2020-12-04 13:08:00 +00:00
go-watchdog runs a singleton memory watchdog in the process, which watches
memory utilization and forces Go GC in accordance with a user-defined policy.
There are two kinds of watchdog so far:
* **heap-driven:** applies a limit to the heap, and obtains current usage through
`runtime.ReadMemStats()`.
* **system-driven:** applies a limit to the total system memory used, and obtains
current usage through [`elastic/go-sigar`](https://github.com/elastic/gosigar).
A third process-driven watchdog that uses cgroups is underway.
initial commit.
2020-12-02 00:03:20 +00:00
This library ships with two policies out of the box:
* watermarks policy: runs GC at configured watermarks of system or heap memory
utilisation.
add godocs link to README.
2020-12-02 00:14:25 +00:00
* adaptive policy: runs GC when the current usage surpasses a dynamically-set
initial commit.
2020-12-02 00:03:20 +00:00
threshold.
You can easily build a custom policy tailored to the allocation patterns of your
program.
## Why is this even needed?
The garbage collector that ships with the go runtime is pretty good in some
regards (low-latency, negligible no stop-the-world), but it's insatisfactory in
a number of situations that yield ill-fated outcomes:
1. it is incapable of dealing with bursty/spiky allocations efficiently;
depending on the workload, the program may OOM as a consequence of not
scheduling GC in a timely manner.
2. part of the above is due to the fact that go doesn't concern itself with any
limits. To date, it is not possible to set a maximum heap size.
2. its default policy of scheduling GC when the heap doubles, coupled with its
ignorance of system or process limits, can easily cause it to OOM.
For more information, check out these GitHub issues:
* https://github.com/golang/go/issues/42805
* https://github.com/golang/go/issues/42430
* https://github.com/golang/go/issues/14735
* https://github.com/golang/go/issues/16843
* https://github.com/golang/go/issues/10064
* https://github.com/golang/go/issues/9849
## License
Dual-licensed: [MIT](./LICENSE-MIT), [Apache Software License v2](./LICENSE-APACHE), by way of the
fix README link.
2020-12-02 00:12:08 +00:00
[Permissive License Stack](https://protocol.ai/blog/announcing-the-permissive-license-stack/).