151 lines
6.9 KiB
Markdown
151 lines
6.9 KiB
Markdown
|
# BigCache [](https://travis-ci.org/allegro/bigcache) [](https://coveralls.io/github/allegro/bigcache?branch=master) [](https://godoc.org/github.com/allegro/bigcache) [](https://goreportcard.com/report/github.com/allegro/bigcache)
|
|||
|
|
|
|||
|
|
Fast, concurrent, evicting in-memory cache written to keep big number of entries without impact on performance.
|
|||
|
|
BigCache keeps entries on heap but omits GC for them. To achieve that operations on bytes arrays take place,
|
|||
|
|
therefore entries (de)serialization in front of the cache will be needed in most use cases.
|
|||
|
|
|
|||
|
|
## Usage
|
|||
|
|
|
|||
|
|
### Simple initialization
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
import "github.com/allegro/bigcache"
|
|||
|
|
|
|||
|
|
cache, _ := bigcache.NewBigCache(bigcache.DefaultConfig(10 * time.Minute))
|
|||
|
|
|
|||
|
|
cache.Set("my-unique-key", []byte("value"))
|
|||
|
|
|
|||
|
|
entry, _ := cache.Get("my-unique-key")
|
|||
|
|
fmt.Println(string(entry))
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Custom initialization
|
|||
|
|
|
|||
|
|
When cache load can be predicted in advance then it is better to use custom initialization because additional memory
|
|||
|
|
allocation can be avoided in that way.
|
|||
|
|
|
|||
|
|
```go
|
|||
|
|
import (
|
|||
|
|
"log"
|
|||
|
|
|
|||
|
|
"github.com/allegro/bigcache"
|
|||
|
|
)
|
|||
|
|
|
|||
|
|
config := bigcache.Config {
|
|||
|
|
// number of shards (must be a power of 2)
|
|||
|
|
Shards: 1024,
|
|||
|
|
// time after which entry can be evicted
|
|||
|
|
LifeWindow: 10 * time.Minute,
|
|||
|
|
// rps * lifeWindow, used only in initial memory allocation
|
|||
|
|
MaxEntriesInWindow: 1000 * 10 * 60,
|
|||
|
|
// max entry size in bytes, used only in initial memory allocation
|
|||
|
|
MaxEntrySize: 500,
|
|||
|
|
// prints information about additional memory allocation
|
|||
|
|
Verbose: true,
|
|||
|
|
// cache will not allocate more memory than this limit, value in MB
|
|||
|
|
// if value is reached then the oldest entries can be overridden for the new ones
|
|||
|
|
// 0 value means no size limit
|
|||
|
|
HardMaxCacheSize: 8192,
|
|||
|
|
// callback fired when the oldest entry is removed because of its expiration time or no space left
|
|||
|
|
// for the new entry, or because delete was called. A bitmask representing the reason will be returned.
|
|||
|
|
// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
|
|||
|
|
OnRemove: nil,
|
|||
|
|
// OnRemoveWithReason is a callback fired when the oldest entry is removed because of its expiration time or no space left
|
|||
|
|
// for the new entry, or because delete was called. A constant representing the reason will be passed through.
|
|||
|
|
// Default value is nil which means no callback and it prevents from unwrapping the oldest entry.
|
|||
|
|
// Ignored if OnRemove is specified.
|
|||
|
|
OnRemoveWithReason: nil,
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
cache, initErr := bigcache.NewBigCache(config)
|
|||
|
|
if initErr != nil {
|
|||
|
|
log.Fatal(initErr)
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
cache.Set("my-unique-key", []byte("value"))
|
|||
|
|
|
|||
|
|
if entry, err := cache.Get("my-unique-key"); err == nil {
|
|||
|
|
fmt.Println(string(entry))
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Benchmarks
|
|||
|
|
|
|||
|
|
Three caches were compared: bigcache, [freecache](https://github.com/coocood/freecache) and map.
|
|||
|
|
Benchmark tests were made using an i7-6700K with 32GB of RAM on Windows 10.
|
|||
|
|
|
|||
|
|
### Writes and reads
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd caches_bench; go test -bench=. -benchtime=10s ./... -timeout 30m
|
|||
|
|
|
|||
|
|
BenchmarkMapSet-8 3000000 569 ns/op 202 B/op 3 allocs/op
|
|||
|
|
BenchmarkConcurrentMapSet-8 1000000 1592 ns/op 347 B/op 8 allocs/op
|
|||
|
|
BenchmarkFreeCacheSet-8 3000000 775 ns/op 355 B/op 2 allocs/op
|
|||
|
|
BenchmarkBigCacheSet-8 3000000 640 ns/op 303 B/op 2 allocs/op
|
|||
|
|
BenchmarkMapGet-8 5000000 407 ns/op 24 B/op 1 allocs/op
|
|||
|
|
BenchmarkConcurrentMapGet-8 3000000 558 ns/op 24 B/op 2 allocs/op
|
|||
|
|
BenchmarkFreeCacheGet-8 2000000 682 ns/op 136 B/op 2 allocs/op
|
|||
|
|
BenchmarkBigCacheGet-8 3000000 512 ns/op 152 B/op 4 allocs/op
|
|||
|
|
BenchmarkBigCacheSetParallel-8 10000000 225 ns/op 313 B/op 3 allocs/op
|
|||
|
|
BenchmarkFreeCacheSetParallel-8 10000000 218 ns/op 341 B/op 3 allocs/op
|
|||
|
|
BenchmarkConcurrentMapSetParallel-8 5000000 318 ns/op 200 B/op 6 allocs/op
|
|||
|
|
BenchmarkBigCacheGetParallel-8 20000000 178 ns/op 152 B/op 4 allocs/op
|
|||
|
|
BenchmarkFreeCacheGetParallel-8 20000000 295 ns/op 136 B/op 3 allocs/op
|
|||
|
|
BenchmarkConcurrentMapGetParallel-8 10000000 237 ns/op 24 B/op 2 allocs/op
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Writes and reads in bigcache are faster than in freecache.
|
|||
|
|
Writes to map are the slowest.
|
|||
|
|
|
|||
|
|
### GC pause time
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd caches_bench; go run caches_gc_overhead_comparison.go
|
|||
|
|
|
|||
|
|
Number of entries: 20000000
|
|||
|
|
GC pause for bigcache: 5.8658ms
|
|||
|
|
GC pause for freecache: 32.4341ms
|
|||
|
|
GC pause for map: 52.9661ms
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Test shows how long are the GC pauses for caches filled with 20mln of entries.
|
|||
|
|
Bigcache and freecache have very similar GC pause time.
|
|||
|
|
It is clear that both reduce GC overhead in contrast to map
|
|||
|
|
which GC pause time took more than 10 seconds.
|
|||
|
|
|
|||
|
|
## How it works
|
|||
|
|
|
|||
|
|
BigCache relies on optimization presented in 1.5 version of Go ([issue-9477](https://github.com/golang/go/issues/9477)).
|
|||
|
|
This optimization states that if map without pointers in keys and values is used then GC will omit its content.
|
|||
|
|
Therefore BigCache uses `map[uint64]uint32` where keys are hashed and values are offsets of entries.
|
|||
|
|
|
|||
|
|
Entries are kept in bytes array, to omit GC again.
|
|||
|
|
Bytes array size can grow to gigabytes without impact on performance
|
|||
|
|
because GC will only see single pointer to it.
|
|||
|
|
|
|||
|
|
## Bigcache vs Freecache
|
|||
|
|
|
|||
|
|
Both caches provide the same core features but they reduce GC overhead in different ways.
|
|||
|
|
Bigcache relies on `map[uint64]uint32`, freecache implements its own mapping built on
|
|||
|
|
slices to reduce number of pointers.
|
|||
|
|
|
|||
|
|
Results from benchmark tests are presented above.
|
|||
|
|
One of the advantage of bigcache over freecache is that you don’t need to know
|
|||
|
|
the size of the cache in advance, because when bigcache is full,
|
|||
|
|
it can allocate additional memory for new entries instead of
|
|||
|
|
overwriting existing ones as freecache does currently.
|
|||
|
|
However hard max size in bigcache also can be set, check [HardMaxCacheSize](https://godoc.org/github.com/allegro/bigcache#Config).
|
|||
|
|
|
|||
|
|
## HTTP Server
|
|||
|
|
|
|||
|
|
This package also includes an easily deployable HTTP implementation of BigCache, which can be found in the [server](/server) package.
|
|||
|
|
|
|||
|
|
## More
|
|||
|
|
|
|||
|
|
Bigcache genesis is described in allegro.tech blog post: [writing a very fast cache service in Go](http://allegro.tech/2016/03/writing-fast-cache-service-in-go.html)
|
|||
|
|
|
|||
|
|
## License
|
|||
|
|
|
|||
|
|
BigCache is released under the Apache 2.0 license (see [LICENSE](LICENSE))
|