|
| 1 | +# SQL Cache |
| 2 | + |
| 3 | +## Sections |
| 4 | +- [ListOptions Informer](#listoptions-informer) |
| 5 | + - [List Options](#list-options) |
| 6 | + - [ListOption Indexer](#listoptions-indexer) |
| 7 | + - [SQL Store](#sql-store) |
| 8 | + - [Partitions](#partitions) |
| 9 | +- [How to Use](#how-to-use) |
| 10 | +- [Technical Information](#technical-information) |
| 11 | + - [SQL Tables](#sql-tables) |
| 12 | + - [SQLite Driver](#sqlite-driver) |
| 13 | + - [Connection Pooling](#connection-pooling) |
| 14 | + - [Encryption Defaults](#encryption-defaults) |
| 15 | + - [Indexed Fields](#indexed-fields) |
| 16 | + - [ListOptions Behavior](#listoptions-behavior) |
| 17 | + - [Troubleshooting Sqlite](#troubleshooting-sqlite) |
| 18 | + |
| 19 | + |
| 20 | + |
| 21 | +## ListOptions Informer |
| 22 | +The main usable feature from the SQL cache is the ListOptions Informer. The ListOptionsInformer provides listing functionality, |
| 23 | +like any other informer, but with a wider array of options. The options are configured by informer.ListOptions. |
| 24 | + |
| 25 | +### List Options |
| 26 | +ListOptions includes the following: |
| 27 | +* Match filters for indexed fields. Filters are for specifying the value a given field in an object should be in order to |
| 28 | +be included in the list. Filters can be set to equals or not equals. Filters can be set to look for partial matches or |
| 29 | +exact (strict) matches. Filters can be OR'd and AND'd with one another. Filters only work on fields that have been indexed. |
| 30 | +* Primary field and secondary field sorting order. Can choose up to two fields to sort on. Sort order can be ascending |
| 31 | +or descending. Default sorting is to sort on metadata.namespace in ascending first and then sort on metadata.name. |
| 32 | +* Page size to specify how many items to include in a response. |
| 33 | +* Page number to specify offset. For example, a page size of 50 and a page number of 2, will return items starting at |
| 34 | +index 50. Index will be dependent on sort. Page numbers start at 1. |
| 35 | + |
| 36 | +### ListOptions Factory |
| 37 | +The ListOptions Factory helps manage multiple ListOption Informers. A user can call Factory.InformerFor(), to create new |
| 38 | +ListOptions informers if they do not exist and retrieve existing ones. |
| 39 | + |
| 40 | +### ListOptions Indexer |
| 41 | +Like all other informers, the ListOptions informer uses an indexer to cache objects of the informer's type. A few features |
| 42 | +set the ListOptions Indexer apart from others indexers: |
| 43 | +* an on-disk store instead of an in-memory store. |
| 44 | +* accepts list options backed by SQL queries for extended search/filter/sorting capability. |
| 45 | +* AES GCM encryption using key hierarchy. |
| 46 | + |
| 47 | +### SQL Store |
| 48 | +The SQL store is the main interface for interacting with the database. This store backs the indexer, and provides all |
| 49 | +functionality required by the cache.Store interface. |
| 50 | + |
| 51 | +### Partitions |
| 52 | +Partitions are constraints for ListOptionsInform ListByOptions() method that are separate from ListOptions. Partitions |
| 53 | +are strict conditions that dictate which namespaces or names can be searched from. These overrule ListOptions and are |
| 54 | +intended to be used as a way of enforcing RBAC. |
| 55 | + |
| 56 | +## How to Use |
| 57 | +```go |
| 58 | + package main |
| 59 | + import( |
| 60 | + "k8s.io/client-go/dynamic" |
| 61 | + "github.com/rancher/steve/pkg/sqlcache/informer" |
| 62 | + "github.com/rancher/steve/pkg/sqlcache/informer/factory" |
| 63 | + ) |
| 64 | + |
| 65 | + func main() { |
| 66 | + cacheFactory, err := factory.NewCacheFactory() |
| 67 | + if err != nil { |
| 68 | + panic(err) |
| 69 | + } |
| 70 | + // config should be some rest config created from kubeconfig |
| 71 | + // there are other ways to create a config and any client that conforms to k8s.io/client-go/dynamic.ResourceInterface |
| 72 | + // will work. |
| 73 | + client, err := dynamic.NewForConfig(config) |
| 74 | + if err != nil { |
| 75 | + panic(err) |
| 76 | + } |
| 77 | + |
| 78 | + fields := [][]string{{"metadata", "name"}, {"metadata", "namespace"}} |
| 79 | + opts := &informer.ListOptions{} |
| 80 | + // gvk should be of type k8s.io/apimachinery/pkg/runtime/schema.GroupVersionKind |
| 81 | + c, err := cacheFactory.CacheFor(fields, client, gvk) |
| 82 | + if err != nil { |
| 83 | + panic(err) |
| 84 | + } |
| 85 | + |
| 86 | + // continueToken will just be an offset that can be used in Resume on a subsequent request to continue |
| 87 | + // to next page |
| 88 | + list, continueToken, err := c.ListByOptions(apiOp.Context(), opts, partitions, namespace) |
| 89 | + if err != nil { |
| 90 | + panic(err) |
| 91 | + } |
| 92 | + } |
| 93 | +``` |
| 94 | + |
| 95 | +## Technical Information |
| 96 | + |
| 97 | +### SQL Tables |
| 98 | +There are three tables that are created for the ListOption informer: |
| 99 | +* object table - this contains objects, including all their fields, as blobs. These blobs may be encrypted. |
| 100 | +* fields table - this contains specific fields of value for objects. These are specified on informer create and are fields |
| 101 | +that it is desired to filter or order on. |
| 102 | +* indices table - the indices table stores indexes created and objects' values for each index. This backs the generic indexer |
| 103 | +that contains the functionality needed to conform to cache.Indexer. |
| 104 | + |
| 105 | +### SQLite Driver |
| 106 | +There are multiple SQLite drivers that this package could have used. One of the most, if not the most, popular SQLite golang |
| 107 | +drivers is [mattn/go-sqlite3](https://github.com/mattn/go-sqlite3). This driver is not being used because it requires enabling |
| 108 | +the cgo option when compiling and at the moment steve's main consumer, rancher, does not compile with cgo. We did not want |
| 109 | +the SQL informer to be the sole driver in switching to using cgo. Instead, modernc's driver which is in pure golang. Side-by-side |
| 110 | +comparisons can be found indicating the cgo version is, as expected, more performant. If in the future it is deemed worthwhile |
| 111 | +then the driver can be easily switched by replacing the empty import in `pkg/cache/sql/store` from `_ "modernc.org/sqlite"` to `_ "github.com/mattn/go-sqlite3"`. |
| 112 | + |
| 113 | +### Connection Pooling |
| 114 | +While working with the `database/sql` package for go, it is important to understand how sql.Open() and other methods manage |
| 115 | +connections. Open starts a connection pool; that is to say after calling open once, there may be anywhere from zero to many |
| 116 | +connections attached to a sql.Connection. `database/sql` manages this connection pool under the hood. In most cases, an |
| 117 | +application only need one sql.Connection, although sometimes application use two: one for writes, the other for reads. To |
| 118 | +read more about the `sql` package's connection pooling read [Managing connections](https://go.dev/doc/database/manage-connections). |
| 119 | + |
| 120 | +The use of connection pooling and the fact that steve potentially has many go routines accessing the same connection pool, |
| 121 | +means we have to be careful with writes. Exclusively using sql transaction to write helps ensure safety. To read more about |
| 122 | +sql transactions read SQLite's [Transaction docs](https://www.sqlite.org/lang_transaction.html). |
| 123 | + |
| 124 | +### Encryption Defaults |
| 125 | +By default only specified types are encrypted. These types are hard-coded and defined by defaultEncryptedResourceTypes |
| 126 | +in `pkg/cache/sql/informer/factory/informer_factory.go`. To enabled encryption for all types, set the ENV variable |
| 127 | +`CATTLE_ENCRYPT_CACHE_ALL` to "true". |
| 128 | + |
| 129 | +The key size used is 256 bits. Data-encryption-keys are stored in the object table and are rotated every 150,000 writes. |
| 130 | + |
| 131 | +### Indexed Fields |
| 132 | +Filtering and sorting only work on indexed fields. These fields are defined when using `CacheFor`. Objects will |
| 133 | +have the following indexes by default: |
| 134 | +* Fields in informer.defaultIndexedFields |
| 135 | +* Fields passed to InformerFor() |
| 136 | + |
| 137 | +### ListOptions Behavior |
| 138 | +Defaults: |
| 139 | +* Sort.PrimaryField: `metadata.namespace` |
| 140 | +* Sort.SecondaryField: `metadata.name` |
| 141 | +* Sort.PrimaryOrder: `ASC` (ascending) |
| 142 | +* Sort.SecondaryOrder: `ASC` (ascending) |
| 143 | +* All filters have partial matching set to false by default |
| 144 | + |
| 145 | +There are some uncommon ways someone could use ListOptions where it would be difficult to predict what the result would be. |
| 146 | +Below is a non-exhaustive list of some of these cases and what the behavior is: |
| 147 | +* Setting Pagination.Page but not Pagination.PageSize will cause Page to be ignored |
| 148 | +* Setting Sort.SecondaryField only will sort as though it was Sort.PrimaryField. Sort.SecondaryOrder will still be applied |
| 149 | +and Sort.PrimaryOrder will be ignored |
| 150 | + |
| 151 | +### Writing Secure Queries |
| 152 | +Values should be supplied to SQL queries using placeholders, read [Avoiding SQL Injection Risk](https://go.dev/doc/database/sql-injection). Any other portions |
| 153 | +of a query that may be user supplied, such as columns, should be carefully validated against a fixed set of acceptable values. |
| 154 | + |
| 155 | +### Troubleshooting SQLite |
| 156 | +A useful tool for troubleshooting the database files is the sqlite command line tool. Another useful tool is the goland |
| 157 | +sqlite plugin. Both of these tools can be used with the database files. |
0 commit comments