fix: f007: avoid decompression bombs

This commit is contained in:
Iuri Matias
2026-06-10 12:47:14 -04:00
parent a2eec36945
commit 6fe6559bd4
5 changed files with 393 additions and 38 deletions
+21 -1
View File
@@ -109,12 +109,32 @@ logos-package/
- No original filename
- OS byte = 0xFF (unknown)
**Decompression-bomb protection:** Both decompression paths enforce a hard cap on
total decompressed output (`DEFAULT_MAX_DECOMPRESSED_SIZE` = 1 GiB by default). A
gzip stream that would inflate past the cap is rejected before the excess bytes
are materialized, so a tiny but highly-compressible archive cannot exhaust host
memory. `Package::load()` decompresses untrusted `.lgx` data through this guard,
which also bounds the buffer handed to `TarReader`.
The limit is configurable two ways:
- **Library-wide:** `setDefaultMaxDecompressedSize(bytes)` changes the default
for every subsequent call that does not pass an explicit cap (including the
`.lgx` load path). `getDefaultMaxDecompressedSize()` reads the current value.
It is thread-safe; passing `0` is rejected so bomb protection cannot be
silently disabled.
- **Per call:** pass `maxOutputSize` to `decompress` / `decompressStream` for
callers with larger legitimate payloads. The default argument
(`USE_DEFAULT_MAX`) means "use the library-wide value".
**API:**
| Method | Description |
|--------|-------------|
| `compress(data) → vector<uint8_t>` | Compress data with deterministic settings |
| `decompress(data) → vector<uint8_t>` | Decompress gzip data |
| `decompress(data, maxOutputSize=USE_DEFAULT_MAX) → vector<uint8_t>` | Decompress gzip data, rejecting streams that exceed the output cap |
| `decompressStream(data, writeCallback, maxOutputSize=USE_DEFAULT_MAX) → bool` | Stream-decompress with the same running-total output cap |
| `setDefaultMaxDecompressedSize(bytes)` | Set the library-wide default output cap (thread-safe; `0` ignored) |
| `getDefaultMaxDecompressedSize() → size_t` | Read the current library-wide default output cap |
| `isGzipData(data) → bool` | Check if data has gzip magic bytes |
| `getLastError() → string` | Get last error message |
+26
View File
@@ -231,6 +231,32 @@ prevents zip-slip / path-traversal arbitrary file writes from an untrusted
These are forbidden for portability and security: links can escape variant roots or behave differently on extract; special files are unsafe/meaningless for plugins.
### Decompression Limits
A `.lgx` is a gzip-compressed tar archive, and DEFLATE can reach compression
ratios on the order of 1000:1. Left unbounded, a small crafted archive could
inflate to gigabytes when loaded and exhaust the host's memory — a
"decompression bomb" that OOM-kills or hangs the process loading it (basecamp
and every in-process module).
To prevent this, decompression enforces a **hard cap on total decompressed
output** (1 GiB by default). The gzip reader tracks a running total as it
inflates and rejects the stream the moment the output would exceed the cap,
before the excess bytes are allocated — so the cost of an oversized archive is
bounded regardless of how small the compressed input is. Loading an untrusted
`.lgx` (`lgx verify`, `lgpm install`, signature inspection, the `lgx_*` C API)
runs through this guard, which also bounds the buffer subsequently handed to the
tar reader. A package whose contents exceed the cap is rejected with an error
and no oversized buffer is ever materialized.
The cap applies to the **whole archive** — the total size of the decompressed
tar (every entry plus tar overhead), not any single file within it. It is
configurable by embedders of the library: globally via
`GzipHandler::setDefaultMaxDecompressedSize(bytes)` (affects every load that
does not specify its own limit) or per call via the `maxOutputSize` argument to
`decompress` / `decompressStream`. There is no "unlimited" setting — a `0`
value is rejected — so the protection cannot be turned off by misconfiguration.
### Package Creation Workflow
```
+78 -25
View File
@@ -8,6 +8,24 @@ namespace lgx {
thread_local std::string GzipHandler::lastError_;
std::atomic<size_t> GzipHandler::defaultMaxDecompressedSize_{
GzipHandler::DEFAULT_MAX_DECOMPRESSED_SIZE
};
void GzipHandler::setDefaultMaxDecompressedSize(size_t maxBytes) {
// Reject 0: there is no "unlimited" mode, so a bad config can't silently
// disable bomb protection. USE_DEFAULT_MAX (0) is reserved as the per-call
// "use library default" sentinel and must never become the stored value.
if (maxBytes == 0) {
return;
}
defaultMaxDecompressedSize_.store(maxBytes, std::memory_order_relaxed);
}
size_t GzipHandler::getDefaultMaxDecompressedSize() {
return defaultMaxDecompressedSize_.load(std::memory_order_relaxed);
}
std::vector<uint8_t> GzipHandler::compress(const std::vector<uint8_t>& data) {
if (data.empty()) {
// Return valid empty gzip for empty input
@@ -142,45 +160,63 @@ std::vector<uint8_t> GzipHandler::compressStream(
return compress(data);
}
std::vector<uint8_t> GzipHandler::decompress(const std::vector<uint8_t>& data) {
std::vector<uint8_t> GzipHandler::decompress(
const std::vector<uint8_t>& data,
size_t maxOutputSize
) {
if (maxOutputSize == USE_DEFAULT_MAX) {
maxOutputSize = getDefaultMaxDecompressedSize();
}
if (!isGzipData(data)) {
lastError_ = "Not valid gzip data";
return {};
}
std::vector<uint8_t> result;
// Initialize inflate with gzip detection
z_stream strm;
std::memset(&strm, 0, sizeof(strm));
int ret = inflateInit2(&strm, 16 + MAX_WBITS); // 16 = gzip decoding
if (ret != Z_OK) {
lastError_ = "Failed to initialize inflate: " + std::to_string(ret);
return {};
}
strm.next_in = const_cast<Bytef*>(data.data());
strm.avail_in = static_cast<uInt>(data.size());
std::array<uint8_t, 32768> outBuf;
do {
strm.next_out = outBuf.data();
strm.avail_out = outBuf.size();
ret = inflate(&strm, Z_NO_FLUSH);
if (ret == Z_STREAM_ERROR || ret == Z_NEED_DICT ||
if (ret == Z_STREAM_ERROR || ret == Z_NEED_DICT ||
ret == Z_DATA_ERROR || ret == Z_MEM_ERROR) {
inflateEnd(&strm);
lastError_ = "Inflate error: " + std::to_string(ret);
return {};
}
size_t have = outBuf.size() - strm.avail_out;
// Decompression-bomb guard: reject the stream before appending any
// chunk that would push total output past the cap. This bounds memory
// even for a tiny archive that inflates to gigabytes (see F-007).
if (have > maxOutputSize - result.size()) {
inflateEnd(&strm);
lastError_ = "Decompressed size exceeds limit of " +
std::to_string(maxOutputSize) + " bytes";
return {};
}
result.insert(result.end(), outBuf.begin(), outBuf.begin() + have);
// Check for truncated data: input exhausted but stream not ended
if (strm.avail_in == 0 && ret != Z_STREAM_END) {
inflateEnd(&strm);
@@ -188,48 +224,65 @@ std::vector<uint8_t> GzipHandler::decompress(const std::vector<uint8_t>& data) {
return {};
}
} while (ret != Z_STREAM_END);
inflateEnd(&strm);
return result;
}
bool GzipHandler::decompressStream(
const std::vector<uint8_t>& data,
std::function<bool(const uint8_t* buffer, size_t size)> writeCallback
std::function<bool(const uint8_t* buffer, size_t size)> writeCallback,
size_t maxOutputSize
) {
if (maxOutputSize == USE_DEFAULT_MAX) {
maxOutputSize = getDefaultMaxDecompressedSize();
}
if (!isGzipData(data)) {
lastError_ = "Not valid gzip data";
return false;
}
z_stream strm;
std::memset(&strm, 0, sizeof(strm));
int ret = inflateInit2(&strm, 16 + MAX_WBITS);
if (ret != Z_OK) {
lastError_ = "Failed to initialize inflate: " + std::to_string(ret);
return false;
}
strm.next_in = const_cast<Bytef*>(data.data());
strm.avail_in = static_cast<uInt>(data.size());
std::array<uint8_t, 32768> outBuf;
size_t totalOut = 0;
do {
strm.next_out = outBuf.data();
strm.avail_out = outBuf.size();
ret = inflate(&strm, Z_NO_FLUSH);
if (ret == Z_STREAM_ERROR || ret == Z_NEED_DICT ||
if (ret == Z_STREAM_ERROR || ret == Z_NEED_DICT ||
ret == Z_DATA_ERROR || ret == Z_MEM_ERROR) {
inflateEnd(&strm);
lastError_ = "Inflate error: " + std::to_string(ret);
return false;
}
size_t have = outBuf.size() - strm.avail_out;
// Decompression-bomb guard: reject before forwarding any chunk that
// would push the running total past the cap (see F-007).
if (have > maxOutputSize - totalOut) {
inflateEnd(&strm);
lastError_ = "Decompressed size exceeds limit of " +
std::to_string(maxOutputSize) + " bytes";
return false;
}
totalOut += have;
if (have > 0) {
if (!writeCallback(outBuf.data(), have)) {
inflateEnd(&strm);
@@ -237,7 +290,7 @@ bool GzipHandler::decompressStream(
return false;
}
}
// Check for truncated data: input exhausted but stream not ended
if (strm.avail_in == 0 && ret != Z_STREAM_END) {
inflateEnd(&strm);
@@ -245,7 +298,7 @@ bool GzipHandler::decompressStream(
return false;
}
} while (ret != Z_STREAM_END);
inflateEnd(&strm);
return true;
}
+77 -9
View File
@@ -3,8 +3,10 @@
#include <string>
#include <vector>
#include <cstdint>
#include <cstddef>
#include <optional>
#include <functional>
#include <atomic>
namespace lgx {
@@ -18,9 +20,51 @@ namespace lgx {
*/
class GzipHandler {
public:
/**
* Factory default hard cap on decompressed output size (1 GiB).
*
* Decompression refuses to produce more than this many bytes from a single
* gzip stream. This bounds the damage from a "decompression bomb" — a small,
* highly-compressible archive (DEFLATE can reach ~1000:1) that would
* otherwise inflate to gigabytes and exhaust host memory.
*
* This is the built-in default; the effective limit can be changed at
* runtime for the whole library via setDefaultMaxDecompressedSize(), or
* per call via the maxOutputSize argument to decompress() /
* decompressStream().
*/
static constexpr size_t DEFAULT_MAX_DECOMPRESSED_SIZE = 1024ull * 1024 * 1024;
/**
* Sentinel for the maxOutputSize arguments below: use the library-wide
* configured default (see getDefaultMaxDecompressedSize()).
*/
static constexpr size_t USE_DEFAULT_MAX = 0;
/**
* Set the library-wide default cap on decompressed output size, in bytes.
*
* Applies to every subsequent decompress() / decompressStream() call that
* does not pass an explicit maxOutputSize (including the untrusted .lgx
* load path in Package::load()). Thread-safe. Passing 0 is rejected and
* leaves the current value unchanged — there is no "unlimited" setting, by
* design, so a misconfiguration cannot silently disable bomb protection.
*
* @param maxBytes New default limit in bytes (must be > 0)
*/
static void setDefaultMaxDecompressedSize(size_t maxBytes);
/**
* Get the current library-wide default cap on decompressed output size.
*
* Returns DEFAULT_MAX_DECOMPRESSED_SIZE unless changed via
* setDefaultMaxDecompressedSize().
*/
static size_t getDefaultMaxDecompressedSize();
/**
* Compress data using deterministic gzip settings.
*
*
* @param data Input data to compress
* @return Compressed data in gzip format, or empty vector on failure
*/
@@ -38,22 +82,43 @@ public:
/**
* Decompress gzip data.
*
*
* Enforces a hard cap on total decompressed output to defend against
* decompression bombs: if the inflated stream would exceed maxOutputSize,
* decompression aborts and an empty vector is returned (getLastError()
* reports the limit). This guards the untrusted .lgx load path, where a
* tiny archive could otherwise inflate to gigabytes and OOM the host.
*
* @param data Gzip compressed data
* @return Decompressed data, or empty vector on failure
* @param maxOutputSize Maximum decompressed bytes to produce before
* rejecting the stream. Defaults to USE_DEFAULT_MAX, meaning the
* library-wide limit from getDefaultMaxDecompressedSize().
* @return Decompressed data, or empty vector on failure / cap exceeded
*/
static std::vector<uint8_t> decompress(const std::vector<uint8_t>& data);
static std::vector<uint8_t> decompress(
const std::vector<uint8_t>& data,
size_t maxOutputSize = USE_DEFAULT_MAX
);
/**
* Decompress gzip data with streaming output.
*
*
* Like decompress(), this enforces maxOutputSize as a running total across
* all chunks handed to writeCallback; the stream is rejected (returns false)
* once the cap is exceeded, bounding memory even when the caller streams
* output to disk.
*
* @param data Gzip compressed data
* @param writeCallback Function that receives decompressed chunks
* @return true on success, false on failure
* @param maxOutputSize Maximum total decompressed bytes before rejecting
* the stream. Defaults to USE_DEFAULT_MAX, meaning the library-wide
* limit from getDefaultMaxDecompressedSize().
* @return true on success, false on failure / cap exceeded
*/
static bool decompressStream(
const std::vector<uint8_t>& data,
std::function<bool(const uint8_t* buffer, size_t size)> writeCallback
std::function<bool(const uint8_t* buffer, size_t size)> writeCallback,
size_t maxOutputSize = USE_DEFAULT_MAX
);
/**
@@ -68,7 +133,10 @@ public:
private:
static thread_local std::string lastError_;
// Library-wide configurable cap, initialized to the factory default.
static std::atomic<size_t> defaultMaxDecompressedSize_;
// Gzip header constants for determinism
static constexpr uint8_t GZIP_MAGIC1 = 0x1f;
static constexpr uint8_t GZIP_MAGIC2 = 0x8b;
+191 -3
View File
@@ -172,14 +172,202 @@ TEST(GzipHandlerTest, Decompress_TruncatedData) {
TEST(GzipHandlerTest, DecompressStream) {
std::vector<uint8_t> original = {'S', 't', 'r', 'e', 'a', 'm', ' ', 't', 'e', 's', 't'};
auto compressed = GzipHandler::compress(original);
std::vector<uint8_t> result;
bool success = GzipHandler::decompressStream(compressed,
bool success = GzipHandler::decompressStream(compressed,
[&result](const uint8_t* buffer, size_t size) {
result.insert(result.end(), buffer, buffer + size);
return true;
});
EXPECT_TRUE(success);
EXPECT_EQ(result, original);
}
// =============================================================================
// Decompression Bomb Protection (F-007)
//
// A small, highly-compressible gzip stream can inflate to gigabytes. The
// decompress paths must enforce a hard cap on total output and reject the
// stream once it is exceeded, instead of growing an unbounded in-memory buffer
// and OOM-killing the host process.
// =============================================================================
// Build a "bomb": a small gzip stream that decompresses to `decompressedSize`
// bytes of zeros (DEFLATE achieves ~1000:1 on a run of identical bytes).
static std::vector<uint8_t> makeZeroBomb(size_t decompressedSize) {
std::vector<uint8_t> zeros(decompressedSize, 0);
return GzipHandler::compress(zeros);
}
TEST(GzipHandlerTest, Decompress_RejectsBombExceedingCap) {
// 64 MiB of zeros compresses to a handful of KiB on disk...
const size_t bombSize = 64 * 1024 * 1024;
auto bomb = makeZeroBomb(bombSize);
ASSERT_FALSE(bomb.empty());
// ...the on-disk archive is tiny relative to its decompressed size.
EXPECT_LT(bomb.size(), bombSize / 100);
// Decompress with a 1 MiB cap: the loop must bail out long before
// materializing the full 64 MiB, and must not return the payload.
const size_t cap = 1 * 1024 * 1024;
auto result = GzipHandler::decompress(bomb, cap);
EXPECT_TRUE(result.empty());
EXPECT_LE(result.size(), cap);
EXPECT_FALSE(GzipHandler::getLastError().empty());
}
TEST(GzipHandlerTest, Decompress_AllowsOutputUpToCap) {
// Data that decompresses to just under the cap must still succeed.
std::vector<uint8_t> original(64 * 1024);
for (size_t i = 0; i < original.size(); ++i) {
original[i] = static_cast<uint8_t>((i * 31 + 7) % 256);
}
auto compressed = GzipHandler::compress(original);
auto result = GzipHandler::decompress(compressed, 1 * 1024 * 1024);
EXPECT_EQ(result, original);
}
TEST(GzipHandlerTest, Decompress_DefaultCapAllowsNormalData) {
// The default cap must not interfere with ordinary, legitimately-sized data.
std::vector<uint8_t> original(2 * 1024 * 1024);
for (size_t i = 0; i < original.size(); ++i) {
original[i] = static_cast<uint8_t>(i % 256);
}
auto compressed = GzipHandler::compress(original);
auto result = GzipHandler::decompress(compressed); // default cap
EXPECT_EQ(result, original);
}
TEST(GzipHandlerTest, DecompressStream_RejectsBombExceedingCap) {
const size_t bombSize = 64 * 1024 * 1024;
auto bomb = makeZeroBomb(bombSize);
ASSERT_FALSE(bomb.empty());
const size_t cap = 1 * 1024 * 1024;
size_t written = 0;
bool success = GzipHandler::decompressStream(bomb,
[&written](const uint8_t*, size_t size) {
written += size;
return true;
},
cap);
EXPECT_FALSE(success);
// The cap bounds how much is ever handed to the write callback.
EXPECT_LE(written, cap);
EXPECT_FALSE(GzipHandler::getLastError().empty());
}
TEST(GzipHandlerTest, DecompressStream_AllowsOutputUpToCap) {
std::vector<uint8_t> original(64 * 1024);
for (size_t i = 0; i < original.size(); ++i) {
original[i] = static_cast<uint8_t>((i * 17 + 3) % 256);
}
auto compressed = GzipHandler::compress(original);
std::vector<uint8_t> result;
bool success = GzipHandler::decompressStream(compressed,
[&result](const uint8_t* buffer, size_t size) {
result.insert(result.end(), buffer, buffer + size);
return true;
},
1 * 1024 * 1024);
EXPECT_TRUE(success);
EXPECT_EQ(result, original);
}
// =============================================================================
// Configurable Library-Wide Default Cap
//
// The decompression limit is also configurable for the whole library via
// GzipHandler::setDefaultMaxDecompressedSize(). Calls that pass no explicit
// maxOutputSize (including the .lgx load path) must honor the configured value.
//
// These tests mutate process-global state, so the fixture restores the factory
// default after each one to avoid leaking the setting into other tests.
// =============================================================================
class GzipDefaultCapTest : public ::testing::Test {
protected:
void TearDown() override {
GzipHandler::setDefaultMaxDecompressedSize(
GzipHandler::DEFAULT_MAX_DECOMPRESSED_SIZE);
}
};
TEST_F(GzipDefaultCapTest, FactoryDefaultIsOneGiB) {
EXPECT_EQ(GzipHandler::getDefaultMaxDecompressedSize(),
GzipHandler::DEFAULT_MAX_DECOMPRESSED_SIZE);
EXPECT_EQ(GzipHandler::DEFAULT_MAX_DECOMPRESSED_SIZE,
1024ull * 1024 * 1024);
}
TEST_F(GzipDefaultCapTest, SetAndGetRoundtrip) {
GzipHandler::setDefaultMaxDecompressedSize(4 * 1024 * 1024);
EXPECT_EQ(GzipHandler::getDefaultMaxDecompressedSize(), 4u * 1024 * 1024);
}
TEST_F(GzipDefaultCapTest, ConfiguredDefaultIsEnforced) {
// Lower the library-wide default below the bomb's decompressed size...
GzipHandler::setDefaultMaxDecompressedSize(1 * 1024 * 1024);
auto bomb = makeZeroBomb(64 * 1024 * 1024);
ASSERT_FALSE(bomb.empty());
// ...and decompress WITHOUT an explicit cap: the configured default applies.
auto result = GzipHandler::decompress(bomb);
EXPECT_TRUE(result.empty());
EXPECT_FALSE(GzipHandler::getLastError().empty());
}
TEST_F(GzipDefaultCapTest, RaisedDefaultAllowsLargerData) {
// Data larger than a deliberately-low base, allowed once the default is raised.
std::vector<uint8_t> original(4 * 1024 * 1024);
for (size_t i = 0; i < original.size(); ++i) {
original[i] = static_cast<uint8_t>((i * 13 + 5) % 256);
}
auto compressed = GzipHandler::compress(original);
// Default too low: rejected.
GzipHandler::setDefaultMaxDecompressedSize(1 * 1024 * 1024);
EXPECT_TRUE(GzipHandler::decompress(compressed).empty());
// Raise the default above the payload: now accepted.
GzipHandler::setDefaultMaxDecompressedSize(16 * 1024 * 1024);
EXPECT_EQ(GzipHandler::decompress(compressed), original);
}
TEST_F(GzipDefaultCapTest, ExplicitArgOverridesConfiguredDefault) {
// A low library default must not override a deliberately-higher per-call cap.
GzipHandler::setDefaultMaxDecompressedSize(1024); // 1 KiB
std::vector<uint8_t> original(256 * 1024);
for (size_t i = 0; i < original.size(); ++i) {
original[i] = static_cast<uint8_t>((i * 7 + 1) % 256);
}
auto compressed = GzipHandler::compress(original);
// Explicit cap is honored regardless of the (lower) configured default.
auto result = GzipHandler::decompress(compressed, 1 * 1024 * 1024);
EXPECT_EQ(result, original);
}
TEST_F(GzipDefaultCapTest, ZeroIsRejectedAndLeavesProtectionIntact) {
// 0 is the USE_DEFAULT_MAX sentinel and must never become the stored cap,
// otherwise it would silently disable bomb protection.
GzipHandler::setDefaultMaxDecompressedSize(2 * 1024 * 1024);
GzipHandler::setDefaultMaxDecompressedSize(0); // must be ignored
EXPECT_EQ(GzipHandler::getDefaultMaxDecompressedSize(), 2u * 1024 * 1024);
// And the bomb is still rejected under the still-effective limit.
auto bomb = makeZeroBomb(64 * 1024 * 1024);
ASSERT_FALSE(bomb.empty());
EXPECT_TRUE(GzipHandler::decompress(bomb).empty());
}