This is a more or less complete revamp of the snappy library aiming to:
* clear out a lot of the duplicate code
* remove some of the redundant API
* unify the codebase behind a single, optimized "inner" encoder/decoder
* unify the public API for in-memory and stream
compression/decompression
* improve performance
As such, only the documented API remains backwards-compatible - the rest
has been refactored, moved around and rewritten:
* `import snappy` now exposes only in-memory encoders / decoders
* framed format moved to `snappy` module, `snappy/framing` removed
* faststreams integration moved to `snappy/faststreams`
* minimal `std/streams` integration started in `snappy/streams`
Other changes include:
* up-to-date documentation
* allocation- and exception-free API (uses some amount of stack memory)
* a 2-3x improvement to both compression and decompression performance,
putting the library mostly on par with the C++ implementation (see
README)
* the implementation was heavily inspired by the `C++`, `C` and `go`
implementations, but somewhat simplified
* nonetheless, the code uses a significant amount of unsafe code to
work around inefficiencies in the safe subset of Nim
With bulk operations in place, the cost of range checks falls
significantly - we can reintroduce them without any significant loss in
performance by carefully ordering operations such that optimizers can
elide most.
The snappy codebase is a mess with competing implementations,
nonsensical code duplication and no real direction due to a partially
implemented faststreams migration.
This PR makes it slightly less of a mess, but make no mistake, it's
still a mess - the difference being that there are a few more signposts
along the way in terms of module organisation, and a little less mess as
the line count of the PR discloses.
Performance remains poor - ~3x slower than C++ - but at least there's
less code to look at :)
The `appendSnappyBytes` implementation of `snappy` computes indices
incorrectly, resulting in wrong data being produced. The implementation
was fixed and the test suite extended accordingly. Note that this issue
is not reachable because `appendSnappyBytes` is only used in test code.
When using `framingFormatCompress`, the given payload is compressed to
determine whether its compressed form is shorter than its raw form.
For short payloads the Snappy compression will never be shorter, so it
is not necessary to compress such payloads. Instead, short payloads can
always be treated as uncompressable data. This patch optimizes for that.
Jacek Sieka
Etan Kissling
jangko
Zahary Karadjov