|
1 | 1 | # boot-bucket
|
2 | 2 |
|
3 |
| -Boot task `spew` for spewing output files into an s3 bucket. |
| 3 | +a boot task for spewing files into an S3 bucket. |
4 | 4 |
|
5 | 5 | [](dependency)
|
6 | 6 | ```clojure
|
7 | 7 | [tailrecursion/boot-bucket "1.1.0"] ;; latest release
|
8 | 8 | ```
|
9 | 9 | [](/dependency)
|
10 | 10 |
|
11 |
| -## Overview |
| 11 | +## overview |
12 | 12 |
|
13 |
| -This task uploads any files whose filenames or hashes differ from those in the |
14 |
| -targeted s3 bucket, then decorates them with boot metadata so they may be |
| 13 | +the `spew` task uploads any files whose filenames or hashes differ from those in |
| 14 | +the targeted S3 bucket, then decorates them with boot metadata so they may be |
15 | 15 | identified by subsequent tasks (see [boot-front](https://github.com/tailrecursion/boot-front)).
|
16 | 16 |
|
17 |
| -Also optionally allows S3 metadata to be set so that HTTP headers can be |
18 |
| -configured on a per-file basis for files served by S3/Cloudfront. |
| 17 | +the uploaded files may also themselves be adorned with AWS metadata on a per- |
| 18 | +file basis to configure HTTP headers in S3 and cloundfront. |
19 | 19 |
|
20 |
| -## Usage |
| 20 | +## canned access control lists (ACLs) |
21 | 21 |
|
22 |
| -Excerpt of a build.boot file using boot-bucket with boot-front for deployment. |
| 22 | +in a departure from previous releases, as of `2.0.0`, boot-bucket defaults to |
| 23 | +the `:private` canned ACL, which should be changed to `:public-read` when |
| 24 | +serving files from an S3 bucket. alternately, any of the following canned ACL |
| 25 | +keywords may be specified to the `access-control` parameter: |
| 26 | + |
| 27 | +``` |
| 28 | +:private (default) |
| 29 | +:log-delivery-write |
| 30 | +:bucket-owner-read |
| 31 | +:bucket-owner-full-control |
| 32 | +:authenticated-read |
| 33 | +:public-read |
| 34 | +:public-read-write |
| 35 | +``` |
| 36 | + |
| 37 | +custom acls are currently unsupported, but pull requests are most welcome. |
| 38 | + |
| 39 | +## metadata |
| 40 | +both user and system metadata may be passed to the `metadata` parameter in the |
| 41 | +same map to modify the http headers served up by S3. these are supported on a |
| 42 | +per-file basis of the form `{"<filename>" {:<meta-key> "meta-value"}}`. an |
| 43 | +example is shown below: |
| 44 | + |
| 45 | +``` |
| 46 | +{"index.html.js" {:content-encoding "gzip"}} |
| 47 | +``` |
| 48 | + |
| 49 | +## usage |
| 50 | + |
| 51 | +the following is an excerpt of a `build.boot' file using boot-bucket together |
| 52 | +with [boot-front][1] for deployments: |
23 | 53 |
|
24 | 54 | ```clojure
|
25 | 55 | (require
|
@@ -62,68 +92,42 @@ Excerpt of a build.boot file using boot-bucket with boot-front for deployment.
|
62 | 92 | (task-options!
|
63 | 93 | serve {:port 3001}
|
64 | 94 | sift {:include #{#"index.html.out/" #"<app-ns>/"} :invert true}
|
65 |
| - spew {:access-key (System/getenv "<AWS_ACCESS_KEY_ENV_VAR>") |
66 |
| - :secret-key (System/getenv "<AWS_SECRET_KEY_ENV_VAR>")} |
| 95 | + spew {:access-control :public-read |
| 96 | + :access-key (System/getenv "<AWS_ACCESS_KEY_ENV_VAR>") |
| 97 | + :secret-key (System/getenv "<AWS_SECRET_KEY_ENV_VAR>")} |
67 | 98 | burst {:access-key (System/getenv "<AWS_ACCESS_KEY_ENV_VAR>")
|
68 | 99 | :secret-key (System/getenv "<AWS_SECRET_KEY_ENV_VAR>")})
|
69 | 100 | ```
|
70 | 101 |
|
71 |
| -## Public read ACL on all files |
72 |
| - |
73 |
| -**All files are set to public read upon upload.** |
| 102 | +# aot gzipping |
74 | 103 |
|
75 |
| -This is so that the files can be served over HTTP either directly from S3 or |
76 |
| -through Cloudfront. |
77 |
| - |
78 |
| -This suits the primary use-case - compiling an SPA through CLJS. |
79 |
| - |
80 |
| -If you have a different need that requires private uploads, you'll need to |
81 |
| -submit a pull request with some conditional logic around |
82 |
| -`tailrecursion.boot-bucket.client/with-public-read!` |
83 |
| - |
84 |
| -## s3 metadata |
85 |
| - |
86 |
| -S3 supports two kinds of metadata, system and user. These are served by S3 as |
87 |
| -headers over HTTP. |
88 |
| - |
89 |
| -`spew` supports providing a map of path/metadata where metadata is a map of k/v |
90 |
| -pairs. |
91 |
| - |
92 |
| -Internally the boot task normalizes user/system metadata so that the correct fns |
93 |
| -in the AWS SDK are called, based on the metadata key name. |
94 |
| - |
95 |
| -### Example: AOT gzipping |
96 |
| - |
97 |
| -We can set the `Content-Encoding` header to `gzip` for a large file such as |
98 |
| -`index.html.js` that we [already compressed in an earlier task](https://github.com/martinklepsch/boot-gzip). |
| 104 | +the `Content-Encoding` header may be set to `"gzip"` for a large file such as |
| 105 | +an `index.html.js` artifact that was compressed upstream by another task like |
| 106 | +[boot-gzip][2]: |
99 | 107 |
|
100 | 108 | ```clojure
|
101 | 109 | (spew
|
102 |
| - :access-key ; ... |
103 |
| - :secret-key ; ... |
104 |
| - :metadata {"index.html.js" {:content-encoding "gzip"}}) |
| 110 | + :access-key "<aws-access-key>" |
| 111 | + :secret-key "<aws-secret-key>" |
| 112 | + :access-control :public-read |
| 113 | + :metadata {"index.html.js" {:content-encoding "gzip"}}) |
105 | 114 | ```
|
106 | 115 |
|
107 |
| -Gzipping compiled CLJS output is pretty important as core/goog can weigh in at |
108 |
| -multiple MB and compression can drop this file size by 80%+. |
109 |
| - |
110 |
| -In fact, it's possible for compiled CLJS to weigh in at 10MB+ and compress to |
111 |
| -under 2MB - but at this point Cloudfront will no longer apply compression due to |
112 |
| -platform limitations. |
| 116 | +note that it's often important to zip CLJS output as core/goog can weigh in at |
| 117 | +multiple MB and compression can drop this file size by 80%+. in fact, it's |
| 118 | +possible for compiled CLJS to weigh in at 10MB+ and compress to under 2MB - but |
| 119 | +at this point Cloudfront will no longer apply compression due to platform |
| 120 | +limitations. pre-gzipping files and setting the correct metadata headers in this |
| 121 | +way has several benefits: |
113 | 122 |
|
114 |
| -Pre-gzipping files and setting the correct metadata headers in this way has |
115 |
| -several benefits: |
| 123 | +- achieve compressed files without cloudfront (e.g. serving site from S3) |
| 124 | +- avoid the 1MB min/10MB max limits for on-the-fly compression in cloudfront |
| 125 | +- reduce S3 resources needed in upload and storage |
| 126 | +- minor performance benefits as cloudfront does not need to apply compression |
116 | 127 |
|
117 |
| -- Achieve compressed files without Cloudfront (e.g. serving site from S3) |
118 |
| -- Avoid the 1MB min/10MB max limits for on-the-fly compression in Cloudfront |
119 |
| -- Reduce S3 resources needed in upload and storage |
120 |
| -- Minor performance benefits as Cloudfront does not need to apply compression |
121 |
| - |
122 |
| -Note that Cloudfront will not apply on-the-fly compression to any file with the |
| 128 | +note that cloudfront will not apply on-the-fly compression to any file with the |
123 | 129 | `Content-Encoding` header set, so be careful to only set it on files that have
|
124 | 130 | been gzipped during deployment.
|
125 | 131 |
|
126 |
| -## Parallel file uploading |
127 |
| - |
128 |
| -By default `spew` will upload all files sequentially, but can be configured to |
129 |
| -parallel uploads through the `-p` flag or `:parallel?` task config. |
| 132 | +[1]: http://github.com/tailrecursion/boot-front |
| 133 | +[2]: https://github.com/martinklepsch/boot-gzip |
0 commit comments