mirror of
https://github.com/status-im/sqlcipher.git
synced 2025-02-23 09:18:11 +00:00
429 lines
20 KiB
Markdown
429 lines
20 KiB
Markdown
## SQLCipher
|
|
|
|
SQLCipher extends the [SQLite](https://www.sqlite.org) database library to add security enhancements that make it more suitable for encrypted local data storage such as on-the-fly encryption, tamper evidence, and key derivation. Based on SQLite, SQLCipher closely tracks SQLite and periodically integrates stable SQLite release features.
|
|
|
|
SQLCipher is maintained by Zetetic, LLC, the official site can be found [here](https://www.zetetic.net/sqlcipher/).
|
|
|
|
## Features
|
|
|
|
- Fast performance with as little as 5-15% overhead for encryption on many operations
|
|
- 100% of data in the database file is encrypted
|
|
- Good security practices (CBC mode, HMAC, key derivation)
|
|
- Zero-configuration and application level cryptography
|
|
- Algorithms provided by the peer reviewed OpenSSL crypto library.
|
|
- Configurable crypto providers
|
|
|
|
## Contributions
|
|
|
|
We welcome contributions, to contribute to SQLCipher, a [contributor agreement](https://www.zetetic.net/contributions/) needs to be submitted. All submissions should be based on the `prerelease` branch.
|
|
|
|
## Compiling
|
|
|
|
Building SQLCipher is almost the same as compiling a regular version of
|
|
SQLite with two small exceptions:
|
|
|
|
1. You *must* define `SQLITE_HAS_CODEC` and `SQLITE_TEMP_STORE=2` when building sqlcipher.
|
|
2. If compiling against the default OpenSSL crypto provider, you will need to link libcrypto
|
|
|
|
Example Static linking (replace /opt/local/lib with the path to libcrypto.a). Note in this
|
|
example, `--enable-tempstore=yes` is setting `SQLITE_TEMP_STORE=2` for the build.
|
|
|
|
$ ./configure --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC" \
|
|
LDFLAGS="/opt/local/lib/libcrypto.a"
|
|
$ make
|
|
|
|
Example Dynamic linking
|
|
|
|
$ ./configure --enable-tempstore=yes CFLAGS="-DSQLITE_HAS_CODEC" \
|
|
LDFLAGS="-lcrypto"
|
|
$ make
|
|
|
|
## Encrypting a database
|
|
|
|
To specify an encryption passphrase for the database via the SQL interface you
|
|
use a pragma. The passphrase you enter is passed through PBKDF2 key derivation to
|
|
obtain the encryption key for the database
|
|
|
|
PRAGMA key = 'passphrase';
|
|
|
|
Alternately, you can specify an exact byte sequence using a blob literal. If you
|
|
use this method it is your responsibility to ensure that the data you provide is a
|
|
64 character hex string, which will be converted directly to 32 bytes (256 bits) of
|
|
key data without key derivation.
|
|
|
|
PRAGMA key = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'";
|
|
|
|
To encrypt a database programatically you can use the `sqlite3_key` function.
|
|
The data provided in `pKey` is converted to an encryption key according to the
|
|
same rules as `PRAGMA key`.
|
|
|
|
int sqlite3_key(sqlite3 *db, const void *pKey, int nKey);
|
|
|
|
`PRAGMA key` or `sqlite3_key` should be called as the first operation when a database is open.
|
|
|
|
## Changing a database key
|
|
|
|
To change the encryption passphrase for an existing database you may use the rekey pragma
|
|
after you've supplied the correct database password;
|
|
|
|
PRAGMA key = 'passphrase'; -- start with the existing database passphrase
|
|
PRAGMA rekey = 'new-passphrase'; -- rekey will reencrypt with the new passphrase
|
|
|
|
The hex rekey pragma may be used to rekey to a specific binary value
|
|
|
|
PRAGMA rekey = "x'2DD29CA851E7B56E4697B0E1F08507293D761A05CE4D1B628663F411A8086D99'";
|
|
|
|
This can be accomplished programtically by using sqlite3_rekey;
|
|
|
|
sqlite3_rekey(sqlite3 *db, const void *pKey, int nKey)
|
|
|
|
## Support
|
|
|
|
The primary avenue for support and discussions is the SQLCipher discuss site:
|
|
|
|
https://discuss.zetetic.net/c/sqlcipher
|
|
|
|
Issues or support questions on using SQLCipher should be entered into the
|
|
GitHub Issue tracker:
|
|
|
|
https://github.com/sqlcipher/sqlcipher/issues
|
|
|
|
Please DO NOT post issues, support questions, or other problems to blog
|
|
posts about SQLCipher as we do not monitor them frequently.
|
|
|
|
If you are using SQLCipher in your own software please let us know at
|
|
support@zetetic.net!
|
|
|
|
## License
|
|
|
|
Copyright (c) 2016, ZETETIC LLC
|
|
All rights reserved.
|
|
|
|
Redistribution and use in source and binary forms, with or without
|
|
modification, are permitted provided that the following conditions are met:
|
|
* Redistributions of source code must retain the above copyright
|
|
notice, this list of conditions and the following disclaimer.
|
|
* Redistributions in binary form must reproduce the above copyright
|
|
notice, this list of conditions and the following disclaimer in the
|
|
documentation and/or other materials provided with the distribution.
|
|
* Neither the name of the ZETETIC LLC nor the
|
|
names of its contributors may be used to endorse or promote products
|
|
derived from this software without specific prior written permission.
|
|
|
|
THIS SOFTWARE IS PROVIDED BY ZETETIC LLC ''AS IS'' AND ANY
|
|
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
|
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
DISCLAIMED. IN NO EVENT SHALL ZETETIC LLC BE LIABLE FOR ANY
|
|
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
|
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
|
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
|
|
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
|
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
|
|
# Begin SQLite README.md
|
|
|
|
<h1 align="center">SQLite Source Repository</h1>
|
|
|
|
This repository contains the complete source code for the
|
|
[SQLite database engine](https://sqlite.org/). Some test scripts
|
|
are also included. However, many other test scripts
|
|
and most of the documentation are managed separately.
|
|
|
|
SQLite [does not use Git](https://sqlite.org/whynotgit.html).
|
|
If you are reading this on GitHub, then you are looking at an
|
|
unofficial mirror. See <https://sqlite.org/src> for the official
|
|
repository.
|
|
|
|
## Obtaining The Code
|
|
|
|
SQLite sources are managed using the
|
|
[Fossil](https://www.fossil-scm.org/), a distributed version control system
|
|
that was specifically designed to support SQLite development.
|
|
If you do not want to use Fossil, you can download tarballs or ZIP
|
|
archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows:
|
|
|
|
* Lastest trunk check-in as
|
|
[Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz),
|
|
[ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip), or
|
|
[SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar).
|
|
|
|
* Latest release as
|
|
[Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release),
|
|
[ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip?r=release), or
|
|
[SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar?r=release).
|
|
|
|
* For other check-ins, substitute an appropriate branch name or
|
|
tag or hash prefix for "release" in the URLs of the previous
|
|
bullet. Or browse the [timeline](https://www.sqlite.org/src/timeline)
|
|
to locate the check-in desired, click on its information page link,
|
|
then click on the "Tarball" or "ZIP Archive" links on the information
|
|
page.
|
|
|
|
If you do want to use Fossil to check out the source tree,
|
|
first install Fossil version 2.0 or later.
|
|
(Source tarballs and precompiled binaries available
|
|
[here](https://www.fossil-scm.org/fossil/uv/download.html). Fossil is
|
|
a stand-alone program. To install, simply download or build the single
|
|
executable file and put that file someplace on your $PATH.)
|
|
Then run commands like this:
|
|
|
|
mkdir ~/sqlite
|
|
cd ~/sqlite
|
|
fossil clone https://www.sqlite.org/src sqlite.fossil
|
|
fossil open sqlite.fossil
|
|
|
|
After setting up a repository using the steps above, you can always
|
|
update to the lastest version using:
|
|
|
|
fossil update trunk ;# latest trunk check-in
|
|
fossil update release ;# latest official release
|
|
|
|
Or type "fossil ui" to get a web-based user interface.
|
|
|
|
## Compiling
|
|
|
|
First create a directory in which to place
|
|
the build products. It is recommended, but not required, that the
|
|
build directory be separate from the source directory. Cd into the
|
|
build directory and then from the build directory run the configure
|
|
script found at the root of the source tree. Then run "make".
|
|
|
|
For example:
|
|
|
|
tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite"
|
|
mkdir bld ;# Build will occur in a sibling directory
|
|
cd bld ;# Change to the build directory
|
|
../sqlite/configure ;# Run the configure script
|
|
make ;# Run the makefile.
|
|
make sqlite3.c ;# Build the "amalgamation" source file
|
|
make test ;# Run some tests (requires Tcl)
|
|
|
|
See the makefile for additional targets.
|
|
|
|
The configure script uses autoconf 2.61 and libtool. If the configure
|
|
script does not work out for you, there is a generic makefile named
|
|
"Makefile.linux-gcc" in the top directory of the source tree that you
|
|
can copy and edit to suit your needs. Comments on the generic makefile
|
|
show what changes are needed.
|
|
|
|
## Using MSVC
|
|
|
|
On Windows, all applicable build products can be compiled with MSVC.
|
|
First open the command prompt window associated with the desired compiler
|
|
version (e.g. "Developer Command Prompt for VS2013"). Next, use NMAKE
|
|
with the provided "Makefile.msc" to build one of the supported targets.
|
|
|
|
For example:
|
|
|
|
mkdir bld
|
|
cd bld
|
|
nmake /f Makefile.msc TOP=..\sqlite
|
|
nmake /f Makefile.msc sqlite3.c TOP=..\sqlite
|
|
nmake /f Makefile.msc sqlite3.dll TOP=..\sqlite
|
|
nmake /f Makefile.msc sqlite3.exe TOP=..\sqlite
|
|
nmake /f Makefile.msc test TOP=..\sqlite
|
|
|
|
There are several build options that can be set via the NMAKE command
|
|
line. For example, to build for WinRT, simply add "FOR_WINRT=1" argument
|
|
to the "sqlite3.dll" command line above. When debugging into the SQLite
|
|
code, adding the "DEBUG=1" argument to one of the above command lines is
|
|
recommended.
|
|
|
|
SQLite does not require [Tcl](http://www.tcl.tk/) to run, but a Tcl installation
|
|
is required by the makefiles (including those for MSVC). SQLite contains
|
|
a lot of generated code and Tcl is used to do much of that code generation.
|
|
|
|
## Source Code Tour
|
|
|
|
Most of the core source files are in the **src/** subdirectory. The
|
|
**src/** folder also contains files used to build the "testfixture" test
|
|
harness. The names of the source files used by "testfixture" all begin
|
|
with "test".
|
|
The **src/** also contains the "shell.c" file
|
|
which is the main program for the "sqlite3.exe"
|
|
[command-line shell](https://sqlite.org/cli.html) and
|
|
the "tclsqlite.c" file which implements the
|
|
[Tcl bindings](https://sqlite.org/tclsqlite.html) for SQLite.
|
|
(Historical note: SQLite began as a Tcl
|
|
extension and only later escaped to the wild as an independent library.)
|
|
|
|
Test scripts and programs are found in the **test/** subdirectory.
|
|
Addtional test code is found in other source repositories.
|
|
See [How SQLite Is Tested](http://www.sqlite.org/testing.html) for
|
|
additional information.
|
|
|
|
The **ext/** subdirectory contains code for extensions. The
|
|
Full-text search engine is in **ext/fts3**. The R-Tree engine is in
|
|
**ext/rtree**. The **ext/misc** subdirectory contains a number of
|
|
smaller, single-file extensions, such as a REGEXP operator.
|
|
|
|
The **tool/** subdirectory contains various scripts and programs used
|
|
for building generated source code files or for testing or for generating
|
|
accessory programs such as "sqlite3_analyzer(.exe)".
|
|
|
|
### Generated Source Code Files
|
|
|
|
Several of the C-language source files used by SQLite are generated from
|
|
other sources rather than being typed in manually by a programmer. This
|
|
section will summarize those automatically-generated files. To create all
|
|
of the automatically-generated files, simply run "make target_source".
|
|
The "target_source" make target will create a subdirectory "tsrc/" and
|
|
fill it with all the source files needed to build SQLite, both
|
|
manually-edited files and automatically-generated files.
|
|
|
|
The SQLite interface is defined by the **sqlite3.h** header file, which is
|
|
generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The
|
|
[Tcl script](http://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion.
|
|
The manifest.uuid file contains the SHA3 hash of the particular check-in
|
|
and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file
|
|
contains the current SQLite version number. The sqlite3.h header is really
|
|
just a copy of src/sqlite.h.in with the source-id and version number inserted
|
|
at just the right spots. Note that comment text in the sqlite3.h file is
|
|
used to generate much of the SQLite API documentation. The Tcl scripts
|
|
used to generate that documentation are in a separate source repository.
|
|
|
|
The SQL language parser is **parse.c** which is generate from a grammar in
|
|
the src/parse.y file. The conversion of "parse.y" into "parse.c" is done
|
|
by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code
|
|
for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a
|
|
template for generating its parser.
|
|
|
|
Lemon also generates the **parse.h** header file, at the same time it
|
|
generates parse.c. But the parse.h header file is
|
|
modified further (to add additional symbols) using the ./addopcodes.tcl
|
|
Tcl script.
|
|
|
|
The **opcodes.h** header file contains macros that define the numbers
|
|
corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h
|
|
file is generated by the scanning the src/vdbe.c source file. The
|
|
Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h.
|
|
A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate
|
|
the **opcodes.c** source file, which contains a reverse mapping from
|
|
opcode-number to opcode-name that is used for EXPLAIN output.
|
|
|
|
The **keywordhash.h** header file contains the definition of a hash table
|
|
that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into
|
|
the numeric codes used by the parse.c parser. The keywordhash.h file is
|
|
generated by a C-language program at tool mkkeywordhash.c.
|
|
|
|
The **pragma.h** header file contains various definitions used to parse
|
|
and implement the PRAGMA statements. The header is generated by a
|
|
script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit
|
|
the **tool/mkpragmatab.tcl** file to insert the information needed by the
|
|
parser for your new PRAGMA, then run the script to regenerate the
|
|
**pragma.h** header file.
|
|
|
|
### The Amalgamation
|
|
|
|
All of the individual C source code and header files (both manually-edited
|
|
and automatically-generated) can be combined into a single big source file
|
|
**sqlite3.c** called "the amalgamation". The amalgamation is the recommended
|
|
way of using SQLite in a larger application. Combining all individual
|
|
source code files into a single big source code file allows the C compiler
|
|
to perform more cross-procedure analysis and generate better code. SQLite
|
|
runs about 5% faster when compiled from the amalgamation versus when compiled
|
|
from individual source files.
|
|
|
|
The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script.
|
|
First, all of the individual source files must be gathered into the tsrc/
|
|
subdirectory (using the equivalent of "make target_source") then the
|
|
tool/mksqlite3c.tcl script is run to copy them all together in just the
|
|
right order while resolving internal "#include" references.
|
|
|
|
The amalgamation source file is more than 200K lines long. Some symbolic
|
|
debuggers (most notably MSVC) are unable to deal with files longer than 64K
|
|
lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl,
|
|
can be run on the amalgamation to break it up into a single small C file
|
|
called **sqlite3-all.c** that does #include on about seven other files
|
|
named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-7.c**. In this way,
|
|
all of the source code is contained within a single translation unit so
|
|
that the compiler can do extra cross-procedure optimization, but no
|
|
individual source file exceeds 32K lines in length.
|
|
|
|
## How It All Fits Together
|
|
|
|
SQLite is modular in design.
|
|
See the [architectural description](http://www.sqlite.org/arch.html)
|
|
for details. Other documents that are useful in
|
|
(helping to understand how SQLite works include the
|
|
[file format](http://www.sqlite.org/fileformat2.html) description,
|
|
the [virtual machine](http://www.sqlite.org/opcode.html) that runs
|
|
prepared statements, the description of
|
|
[how transactions work](http://www.sqlite.org/atomiccommit.html), and
|
|
the [overview of the query planner](http://www.sqlite.org/optoverview.html).
|
|
|
|
Years of effort have gone into optimizating SQLite, both
|
|
for small size and high performance. And optimizations tend to result in
|
|
complex code. So there is a lot of complexity in the current SQLite
|
|
implementation. It will not be the easiest library in the world to hack.
|
|
|
|
Key files:
|
|
|
|
* **sqlite.h.in** - This file defines the public interface to the SQLite
|
|
library. Readers will need to be familiar with this interface before
|
|
trying to understand how the library works internally.
|
|
|
|
* **sqliteInt.h** - this header file defines many of the data objects
|
|
used internally by SQLite. In addition to "sqliteInt.h", some
|
|
subsystems have their own header files.
|
|
|
|
* **parse.y** - This file describes the LALR(1) grammar that SQLite uses
|
|
to parse SQL statements, and the actions that are taken at each step
|
|
in the parsing process.
|
|
|
|
* **vdbe.c** - This file implements the virtual machine that runs
|
|
prepared statements. There are various helper files whose names
|
|
begin with "vdbe". The VDBE has access to the vdbeInt.h header file
|
|
which defines internal data objects. The rest of SQLite interacts
|
|
with the VDBE through an interface defined by vdbe.h.
|
|
|
|
* **where.c** - This file (together with its helper files named
|
|
by "where*.c") analyzes the WHERE clause and generates
|
|
virtual machine code to run queries efficiently. This file is
|
|
sometimes called the "query optimizer". It has its own private
|
|
header file, whereInt.h, that defines data objects used internally.
|
|
|
|
* **btree.c** - This file contains the implementation of the B-Tree
|
|
storage engine used by SQLite. The interface to the rest of the system
|
|
is defined by "btree.h". The "btreeInt.h" header defines objects
|
|
used internally by btree.c and not published to the rest of the system.
|
|
|
|
* **pager.c** - This file contains the "pager" implementation, the
|
|
module that implements transactions. The "pager.h" header file
|
|
defines the interface between pager.c and the rest of the system.
|
|
|
|
* **os_unix.c** and **os_win.c** - These two files implement the interface
|
|
between SQLite and the underlying operating system using the run-time
|
|
pluggable VFS interface.
|
|
|
|
* **shell.c.in** - This file is not part of the core SQLite library. This
|
|
is the file that, when linked against sqlite3.a, generates the
|
|
"sqlite3.exe" command-line shell. The "shell.c.in" file is transformed
|
|
into "shell.c" as part of the build process.
|
|
|
|
* **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It
|
|
is not part of the core SQLite library. But as most of the tests in this
|
|
repository are written in Tcl, the Tcl language bindings are important.
|
|
|
|
* **test*.c** - Files in the src/ folder that begin with "test" go into
|
|
building the "testfixture.exe" program. The testfixture.exe program is
|
|
an enhanced Tcl shell. The testfixture.exe program runs scripts in the
|
|
test/ folder to validate the core SQLite code. The testfixture program
|
|
(and some other test programs too) is build and run when you type
|
|
"make test".
|
|
|
|
* **ext/misc/json1.c** - This file implements the various JSON functions
|
|
that are build into SQLite.
|
|
|
|
There are many other source files. Each has a succinct header comment that
|
|
describes its purpose and role within the larger system.
|
|
|
|
|
|
## Contacts
|
|
|
|
The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/)
|
|
with geographically distributed backups at
|
|
[http://www2.sqlite.org/](http://www2.sqlite.org) and
|
|
[http://www3.sqlite.org/](http://www3.sqlite.org).
|