1
- ** sqleet** is a public domain encryption extension for
2
- [ SQLite3] ( https://www.sqlite.org/ ) .
1
+ ** sqleet** is an encryption extension for [ SQLite3] ( https://www.sqlite.org/ ) .
2
+ The encryption is transparent (* on-the-fly* ) and based on modern cryptographic
3
+ algorithms designed for high performance in software and robust side-channel
4
+ resistance. The compilation of sqleet is easy because there are no external
5
+ dependencies, which simplifies cross-compiling and cross-platform development.
6
+
7
+ In the spirit of SQLite3, the sqleet source code is in the public domain.
3
8
4
9
- [ Compiling] ( #compiling )
5
- - [ Cryptography buzzwords] ( #cryptography-buzzwords )
6
10
- [ Example] ( #example )
7
- - [ Library API] ( #library-api )
11
+ - [ Cryptography] ( #cryptography )
12
+ - [ sqleet API] ( #sqleet-api )
8
13
- [ C programming interface] ( #c-programming-interface )
9
14
- [ URI configuration interface] ( #uri-configuration-interface )
10
15
- [ Android/iOS support] ( #androidios-support )
@@ -25,35 +30,32 @@ SQLite3 shell with sqleet encryption support can be compiled as follows:
25
30
% gcc sqleet.c shell.c -o sqleet
26
31
```
27
32
28
- [ Example] ( #example ) illustrates the sqleet encryption using the compiled shell.
29
- [ Library API] ( #library-api ) consists of [ C programming
30
- interface] ( #c-programming-interface ) and [ URI-based configuration
31
- interface] ( #uri-configuration-interface ) .
33
+ [ Example] ( #example ) demonstrates the use of the sqleet encryption extension with
34
+ the compiled shell. For application programmers, [ sqleet API] ( #sqleet-api )
35
+ offers [ C programming interface] ( #c-programming-interface ) and language-agnostic
36
+ [ URI-based configuration interface] ( #uri-configuration-interface ) for run-time
37
+ management of the encryption settings.
32
38
33
39
To use sqleet as a library, the recommended way is to download a preconfigured
34
40
[ release package] ( https://github.com/resilar/sqleet/releases/latest ) instead of
35
- cloning the source repository. Contained ` sqleet.c ` and ` sqleet.h ` files are
36
- drop-in replacements for the original ` sqlite3.c ` amalgamation and ` sqlite3.h `
37
- header. Alternatively, ` sqleet.c ` and ` sqleet.h ` of the source repository can
38
- be used directly if all sqleet source files are available at compile time.
41
+ cloning the git repository. Release package contains ` sqleet.c ` and ` sqleet.h `
42
+ amalgamations that are drop-in replacements for the official ` sqlite3.c ` and
43
+ ` sqlite3.h ` amalgamations. Non-amalgamated ` sqleet.c ` and ` sqleet.h ` files from
44
+ master branch can be used as drop-in replacements similarly, assuming all
45
+ necessary sqleet source files are available during compilation. However, sqleet
46
+ development mainly happens in the master branch, so release are considered to be
47
+ more stable and a better choice for the average user.
39
48
40
- To produce a custom release version of sqleet, run
41
- ` ./script/amalgamate.sh <sqleet.c >release.c `
42
- to create an amalgamation of SQLite3 with sqleet encryption support. Similarly,
43
- run
44
- ` ./script/amalgamate.sh <sqleet.h >release.h `
45
- to amalgamate the header.
49
+ Building a custom release version of sqleet is a straightforward task.
46
50
51
+ * Clone or fork sqleet and patch it as you wish
52
+ * Create source and header amalgamations for release
53
+ * ` ./script/amalgamate.sh <sqleet.c >sqlame.c `
54
+ * ` ./script/amalgamate.sh <sqleet.h >sqlame.h `
55
+ * Package the amalgamations with other release files
47
56
48
- Cryptography buzzwords
49
- ----------------------
50
-
51
- - PBKDF2-HMAC-SHA256 key derivation with a 16-byte salt and 12345 iterations.
52
- - ChaCha20 stream cipher with one-time keys.
53
- - Poly1305 authentication tags.
54
-
55
- A low-level description of the database encryption scheme is available in
56
- [ sqleet.c:265] ( sqleet.c#L265 ) .
57
+ [ script/release.sh] ( script/release.sh ) shows the exact release procedure of
58
+ sqleet.
57
59
58
60
59
61
Example
@@ -86,7 +88,8 @@ sqlite> .quit
86
88
00001ff0 e5 ad 4a cd 75 af 8e 8a e2 79 f3 d9 2e 21 e8 4b |..J.u....y...!.K|
87
89
```
88
90
89
- The resulting encrypted database is accessible only with the correct key.
91
+ Notice that the data of the encrypted database is indistinguishable from random.
92
+ After encryption, the unencrypted data is accessible only with the correct key.
90
93
91
94
```
92
95
[sqleet]% ./sqleet hello.db
@@ -106,38 +109,28 @@ INSERT INTO hello VALUES('Hello, world!');
106
109
COMMIT;
107
110
```
108
111
109
- The key can also be provided via [ SQLite3 URI
110
- filenames ] ( https://www.sqlite.org/uri.html ) instead of ` PRAGMA ` .
112
+ Instead of ` PRAGMA ` commands, the key can also be provided in [ SQLite3 URI
113
+ filename ] ( https://www.sqlite.org/uri.html ) using ` key ` parameter .
111
114
112
115
```
113
116
[sqleet]% ./sqleet 'file:hello.db?key=swordfish' 'SELECT * FROM hello'
114
117
Hello, world!
115
118
```
116
119
117
- ** Note** : \
118
- The contents of an encrypted database file are indistinguishable from random
119
- data of the same length. This is a conscious design decision made in sqleet,
120
- but as a drawback, database settings cannot be read directly from the database
121
- file. Thus, it is the user's responsibility to guarantee that the settings are
122
- initialized properly before accessing the database. Most importantly, if the
123
- database page size differs from the default 4096, then opening the database
124
- will fail regardless of correct key unless the user explicitly sets ` page_size `
125
- to the proper value using ` PRAGMA ` command or [ URI
126
- API] ( #uri-configuration-interface ) .
127
-
128
- In contrast, the official [ SQLite Encryption Extension
129
- (SEE)] ( https://www.sqlite.org/see ) leaves bytes 16..23 of the database header
130
- unencrypted so that certain information, including the page size, can be read
131
- from encrypted databases - with the obvious cost of making database files
132
- distinguishable from random. sqleet can optionally be compiled with
133
- ` -DSKIP_HEADER_BYTES=24 ` flag to get the same default behavior (bytes 0..15
134
- contain the KDF salt so only the bytes 16..23 are actually skipped). URI
135
- parameter ` skip=n ` overrides the compile-time value of ` SKIP_HEADER_BYTES ` with
136
- ` n ` .
137
-
138
-
139
- Library API
140
- -----------
120
+
121
+ Cryptography
122
+ ------------
123
+
124
+ - PBKDF2-HMAC-SHA256 key derivation with a 16-byte salt and 12345 iterations.
125
+ - ChaCha20 stream cipher with one-time keys.
126
+ - Poly1305 authentication tags.
127
+
128
+ A low-level description of the database encryption scheme is available in
129
+ [ sqleet.c:265] ( sqleet.c#L265 ) .
130
+
131
+
132
+ sqleet API
133
+ ----------
141
134
142
135
The public sqleet API consists of [ C programming
143
136
interface] ( #c-programming-interface ) and [ URI configuration
@@ -185,15 +178,38 @@ Return value is `SQLITE_OK` on success and an SQLite3 error code on failure.
185
178
In addition, there are ` sqlite3_key_v2() ` and ` sqlite3_rekey_v2() ` functions
186
179
that accept name of the target database as the second parameter.
187
180
181
+ ---
182
+
183
+ ** Note** : \
184
+ In sqleet, the contents of an encrypted database file are indistinguishable from
185
+ random data (of the same length). This is a conscious design decision, but as a
186
+ drawback, database settings cannot be read from the database file. Therefore, it
187
+ is the user's responsibility to properly initialize database settings before
188
+ accessing the database. The most common issue is that opening a database fails
189
+ regardless of valid key because the page size of the database differs from the
190
+ default 4096 and ` page_size ` has not been set to the correct value with ` PRAGMA `
191
+ or [ URI API] ( #uri-configuration-interface ) .
192
+
193
+ The official [ SQLite Encryption Extension (SEE)] ( https://www.sqlite.org/see )
194
+ leaves bytes 16..23 of the database header unencrypted so that page size and
195
+ other settings can be directly read from encrypted databases, which obviously
196
+ makes SEE-encrypted databases distinguishable from random data. In sqleet, this
197
+ behavior can be optionally enabled with ` -DSKIP_HEADER_BYTES=24 ` compile-time
198
+ flag (bytes 0..15 contain the KDF salt so only the bytes 16..23 are actually
199
+ skipped and left unencrypted). At run-time, the compile-time default can be
200
+ overridden with URI parameter ` skip=n ` where ` n ` is the skip amount.
201
+
202
+ ---
203
+
188
204
189
205
### URI configuration interface
190
206
191
207
** Disclaimer** : URI interface is experimental and subject to changes in future
192
208
versions. Use at your own risk!
193
209
194
210
Run-time configuration of sqleet encryption is implemented based on [ SQLite3
195
- URI filenames] ( https://www.sqlite.org/uri.html ) which allow defining parameters
196
- when opening a database . List of URI parameters supported by sqleet:
211
+ URI filenames] ( https://www.sqlite.org/uri.html ) which contain configuration
212
+ parameters for databases . List of URI parameters supported by sqleet:
197
213
198
214
| Parameter | Description |
199
215
| :---------- | :------------------------------------------------------------- |
@@ -204,31 +220,49 @@ when opening a database. List of URI parameters supported by sqleet:
204
220
| ` skip ` | Run-time setting overriding compile-time SKIP_HEADER_BYTES |
205
221
| ` page_size ` | Equivalent to ` page_size ` PRAGMA |
206
222
207
- Hex-prefixed versions accepting a hex input string are available for parameters
208
- ` key ` , ` salt ` and ` header ` .
209
-
210
- Parameters ` salt ` and ` header ` expect 16-byte strings as inputs (shorter
211
- strings are zero-padded to 16 bytes). The ` header ` string is stored in the
212
- first 16 bytes of the database file (` header ` defaults to ` salt ` value if
213
- undefined). If ` kdf=none ` , then the default PBKDF2-HMAC-SHA256 is disabled and
214
- ` key ` accepts accepts a 32-byte string that becomes the * master* encryption key
215
- which is normally derived from the key by the KDF. This ultimately allows the
216
- user of the library to take full control of the key derivation process.
217
-
218
- Parameters ` skip ` and ` page_size ` override compile-time ` SKIP_HEADER_BYTES `
219
- value and ` PRAGMA page_size ` value, respectively.
223
+ Parameters ` key ` , ` salt ` and ` header ` have additional ` hex ` -prefixed versions
224
+ that accept hex input strings such as ` '73716c656574' ` .
225
+
226
+ Parameters ` salt ` and ` header ` expect 16-byte input strings. Shorter strings are
227
+ zero-padded to 16-bytes, while longer inputs get automatically rejected.
228
+
229
+ Parameter ` header ` represents the first 16 bytes of the database file, that is,
230
+ SQLite3 magic header string for unencrypted databases. For encrypted databases,
231
+ ` header ` defaults to the value of ` salt ` unless explicitly set to other value.
232
+ Remember that ` salt ` is a parameter for the key derivation function (KDF) which
233
+ is stored in the beginning of the database file * by default* , in which case both
234
+ ` salt ` and ` header ` contain the same value (KDF salt). Sometimes, however, the
235
+ user may want to keep the salt secret, or control the first 16 bytes of the
236
+ database file for some purpose. In such cases, the user stores the salt and then
237
+ overwrites the beginning of the database file with any 16-byte ` header ` value.
238
+ (If this explanation was too abstract or nonsensical to fully grasp, see the iOS
239
+ workaround in the end of [ Android/iOS support] ( #androidios-support ) for a
240
+ practical real-world use-case of ` header ` feature).
241
+
242
+ URI parameter ` kdf=none ` disables the default PBKDF2-HMAC-SHA256 key derivation.
243
+ If KDF is disabled, ` key ` and ` hexkey ` accept a 32-byte * raw key* that becomes
244
+ the * master* encryption key which otherwise would be derived by the KDF from the
245
+ key and salt. Disabling KDF is a powerful feature for * advanced users* who need
246
+ full control of the key derivation process.
247
+
248
+ Parameters ` skip ` and ` page_size ` override the compile-time ` SKIP_HEADER_BYTES `
249
+ value and the database ` PRAGMA page_size ` configuration.
220
250
221
251
Changing URI settings of an existing database can be accomplished with `VACUUM
222
252
INTO` (introduced in SQLite 3.27.0) by giving new URI parameter values in the
223
253
` INTO ` filename. For example, ` VACUUM INTO 'file:skipped.db?skip=24' ` vacuums
224
254
the current main database to file ` skipped.db ` with ` skip ` set to 24. Other URI
225
255
settings, including the encryption key, are inherited from the main database
226
- (unless ` key ` parameter is specified, in which case default values are used for
227
- any unspecified parameters).
256
+ unless ` key ` parameter is specified, in which case any undefined parameters are
257
+ initialized to default values. Database settings update with ` VACUUM INTO ` is
258
+ complex operation with many special cases and important details (omitted here).
259
+ So be prepared for some undocumented behavior, but please open an issue if
260
+ encountering obviously broken on wrong behavior.
228
261
229
- Erroneus parameters (e.g., unsupported value or otherwise bad input) cause the
230
- opening (or vacuuming) of the database to fail with a non-zero SQLite3 error
231
- code.
262
+ Erroneus parameters, such as unsupported parameter value or otherwise bad input,
263
+ returns a non-zero SQLite3 error code when opening (or vacuuming) a database.
264
+ The current version returns ` SQLITE_MISUSE ` error, in most cases, if URI parsing
265
+ fails or the resulting configuration is invalid.
232
266
233
267
234
268
Android/iOS support
@@ -279,11 +313,10 @@ Versioning scheme
279
313
[ sqleet releases] ( https://github.com/resilar/sqleet/releases/ ) follow a
280
314
perverse form of semantic versioning which requires some explanation. Major
281
315
version number increments indicate compatibility breaks as usual, but the minor
282
- & patch version numbers match the targeted SQLite3 version. For instance,
316
+ and patch version numbers match the targeted SQLite3 version. For instance,
283
317
sqleet v0.25.1 corresponds to SQLite v3.25.1. Although the target SQLite3
284
318
version is the primarily supported, sqleet is typically forward and backward
285
- compatible across different SQLite3 versions without any changes to the source
286
- code.
319
+ compatible across different SQLite3 versions without any changes to the code.
287
320
288
321
As a corollary, sqleet releases are published whenever a new SQLite3 version is
289
322
released. A new sqleet release thus does not necessarily include bug fixes or
0 commit comments