Merge pull request #88 from cybozu-go/doc

Add documents and changelog

CHANGELOG.md

@@ -0,0 +1,18 @@
+# Change Log
+
+All notable changes to this project will be documented in this file.
+This project adheres to [Semantic Versioning](http://semver.org/).
+
+## [Unreleased]
+
+### Added
+- Add integration tests using [placemat][] VMs.
+
+## [0.14] - 2018-07-18
+
+### Changed
+- Fix but in image deletion (#86).
+
+[placemat]: https://github.com/cybozu-go/placemat
+[Unreleased]: https://github.com/cybozu-go/sabakan/compare/v0.14...HEAD
+[0.14]: https://github.com/cybozu-go/sabakan/compare/v0.13...v0.14

README.md

@@ -37,20 +37,20 @@ Features
 
     Sabakan serves OS images to machines via HTTP.
 
+* Distributed asset management
+
+    In order to help initialization of client servers, sabakan can work
+    as a file server from which clients can download assets via HTTP.
+    Assets are automatically synchronized between sabakan servers.
+
 * Encryption key store
 
     Sabakan provides REST API to store and retrieve encryption keys
     to help automated disk encryption/decryption.
 
 * Life-cycle management
 
-    Sabakan provides API to change server status for life-cycle management.
-
-* Distributed asset management
-
-    In order to help initialization of client servers, sabakan can work
-    as a file server from which clients can download assets via HTTP.
-    Assets are automatically synchronized between sabakan servers.
+    Sabakan provides API to change server status for [life-cycle management](docs/lifecycle.md).
 
 * Audit logs
 
@@ -60,10 +60,11 @@ Features
 Programs
 --------
 
-This repository contains two programs:
+This repository contains these programs:
 
 * `sabakan`: the network service to manage servers.
 * `sabactl`: CLI tool for `sabakan`.
+* `sabakan-cryptsetup`: a utility to encrypt a block device using [dm-crypt][].
 
 To see their usage, run them with `-h` option.
 
@@ -72,6 +73,8 @@ Documentation
 
 [docs](docs/) directory contains tutorials and specifications.
 
+Read [getting started](docs/getting_started.md) first.
+
 License
 -------
 
@@ -82,3 +85,4 @@ Sabakan is licensed under MIT license.
 [etcd]: https://coreos.com/etcd/
 [HTTPBoot]: https://github.com/tianocore/tianocore.github.io/wiki/HTTP-Boot
 [iPXE]: https://ipxe.org/
+[dm-crypt]: https://gitlab.com/cryptsetup/cryptsetup/wikis/DMCrypt

cmd/sabakan/sabakan.service

@@ -0,0 +1,15 @@
+[Unit]
+Description=sabakan
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+Type=simple
+Restart=on-failure
+# RestartForceExitStatus=SIGPIPE is critically important for Go programs w/
+# journald logging.  See https://github.com/cybozu-go/cmd/issues/13
+RestartForceExitStatus=SIGPIPE
+ExecStart=/usr/local/sbin/sabakan -config-file /usr/local/etc/sabakan.yml
+
+[Install]
+WantedBy=multi-user.target

docs/getting_started.md

@@ -1,50 +1,75 @@
 Getting started
 ===============
 
-* [Setup sabakan](#setupsabakan)
-  * [Prepare etcd](#prepareetcd)
-  * [Install sabakan and sabactl](#installsabakan)
+This document quickly guides you to configure sabakan and netboot
+your servers with CoreOS Container Linux.
+
+* [Setup sabakan](#setup)
+  * [Prepare etcd](#etcd)
   * [Prepare data directory](#datadir)
-  * [Run sabakan](#runsabakan)
-  * [Configure sabakan](#configuresabakan)
+  * [Prepare sabakan.yml](#configure)
+  * [Run sabakan](#run)
 * [Netboot](#netboot)
-* [Test](#test)
+  * [Configure IPAM](#ipam)
+  * [Configure DHCP](#dhcp)
+  * [Upload CoreOS Container Linux](#upload)
+  * [Register machines](#register)
 * [What's next](#whatsnext)
 
-## <a name="setupsabakan" /> Setup sabakan
-
-### <a name="prepareetcd" /> Prepare etcd
+## <a name="setup" />Setup sabakan
 
-Sabakan requires [etcd][].  Install and run it at somewhere.
+### <a name="etcd" />Prepare etcd
 
-### <a name="installsabakan" /> Install sabakan and sabactl
-
-Install `sabakan` and `sabactl`:
+Sabakan requires [etcd][].  Install and run it at `localhsot`.
 
+You may use docker to run etcd as follows:
 ```console
-$ go get -u github.com/cybozu-go/sabakan/cmd/sabakan
-$ go get -u github.com/cybozu-go/sabakan/cmd/sabactl
+$ docker pull quay.io/cybozu/etcd:3.3
+$ docker run -d --rm --name etcd --network=host --uts=host quay.io/cybozu/etcd:3.3
 ```
 
-`sabakan` Docker image is so available at [quay.io/cybozu/sabakan](https://quay.io/cybozu/sabakan)
-
 ### <a name="datadir" />Prepare data directory
 
 ```console
 $ sudo mkdir -p /var/lib/sabakan
 ```
 
-### <a name="runsabakan" /> Run sabakan
+### <a name="configure" />Prepare sabakan.yml
+
+Save the following contents as `/usr/local/etc/sabakan.yml`:
+
+```yaml
+etcd-servers:
+  - http://localhost:2379
+dhcp-bind: 0.0.0.0:67
+```
+
+For other options, read [sabakan.md](sabakan.md).
+
+### <a name="run" />Run sabakan
+
+Compile and run sabakan as follows:
 
 ```console
-$ sabakan -etcd-servers http://etcd-host:2379
+$ GOPATH=$HOME/go
+$ mkdir -p $GOPATH/src
+$ export GOPATH
+$ go get -u github.com/cybozu-go/sabakan/...
+$ sudo $GOPATH/bin/sabakan -config-file /usr/local/etc/sabakan.yml
 ```
 
-### <a name="configuresabakan" /> Configure sabakan
+A sample systemd service file is available at
+[cmd/sabakan/sabakan.service](../cmd/sabakan/sabakan.service).
+
+Alternatively, you may use docker to run sabakan:
+* Repository: [quay.io/cybozu/sabakan](https://quay.io/cybozu/sabakan)
+* Usage: https://github.com/cybozu/neco-containers/blob/master/sabakan/README.md
+
+## <a name="netboot" />Netboot
 
-First of all, prepare JSON files
+### <a name="ipam" />Configure IPAM
 
-- ipam.json
+Prepare `ipam.json` as follows:
 ```json
 {
    "max-nodes-in-rack": 28,
@@ -60,42 +85,83 @@ First of all, prepare JSON files
 }
 ```
 
-Read [ipam](ipam.md) if you want to know meaning of each parameter.
+Then put the JSON to sabakan:
+```console
+$ sabactl ipam set -f ipam.json
+```
+
+Read [ipam.md](ipam.md) for details.
+
+### <a name="dhcp" />Configure DHCP
 
-- dhcp.json
+Prepare `dhcp.json` as follows:
 ```json
 {
-   "gateway-offset": 100,
-   "lease-minutes": 120
+   "gateway-offset": 1,
 }
 ```
 
-Read [dhcp](dhcp.md) if you want to know meaning of each parameter.
-
-Use `sabactl` to configure `sabakan`.
-
+Then put the JSON to sabakan:
 ```console
-$ sabactl ipam set -f ipam.json
 $ sabactl dhcp set -f dhcp.json
 ```
 
-Make sure current configuration.
+Read [dhcp.md](dhcp.md) for details.
+
+### <a name="upload" />Upload CoreOS Container Linux
 
+Download CoreOS PXE boot images:
 ```console
-$ sabactl ipam get
-$ sabactl dhcp get
+$ curl -o kernel -Lf http://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe.vmlinuz
+$ curl -o initrd.gz -Lf http://stable.release.core-os.net/amd64-usr/current/coreos_production_pxe_image.cpio.gz
 ```
 
-Each output will be the same as [above JSON](#configuresabakan)
+Upload them to sabakan as follows:
+```console
+$ sabactl images upload ID kernel initrd.gz
+```
 
-## <a name="netboot" /> Netboot
+### <a name="machines" />Register machines
 
-**ToDo**
+Prepare `machines.json` as follows:
+```json
+[
+  {
+    "serial": "1234abcd",
+    "product": "Dell R640",
+    "datacenter": "tokyo1",
+    "rack": 0,
+    "role": "boot",
+    "bmc": {
+      "type": "IPMI-2.0",
+      "ipv4": "10.72.17.37"
+    }
+  },
+  {
+    // another machine
+  }
+]
+```
 
-## <a name="test" /> Test
+Then put the JSON to sabakan:
+```console
+$ sabactl machines create -f machines.json
+```
 
-**ToDo**
+Each object in the array is [`MachineSpec`](machine.md#machinespec-struct).
+Sabakan identifies physical servers by `serial`.
+
+Once machines are properly registered with sabakan, they can netboot
+CoreOS Container Linux using [UEFI HTTP Boot][HTTPBoot].
+
+CoreOS can be initialized at first boot by [ignition][].
+Sabakan can generate ignition configuration from templates.
+Read [ignition.md](ignition.md) for details.
 
 ## <a name="whatsnext" /> What's next
 
 Learn sabakan [concepts](concepts.md), then read other specifications.
+
+[etcd]: https://github.com/coreos/etcd
+[HTTPBoot]: https://github.com/tianocore/tianocore.github.io/wiki/HTTP-Boot
+[ignition]: https://coreos.com/ignition/docs/latest/

docs/machine.md

@@ -12,7 +12,7 @@ MachineSpec struct
 
 Field           | Type     | Description
 --------------- | -------- | -----------
-`serial`        | string   | Serial number of the machine.
+`serial`        | string   | SMBIOS serial number of the machine.
 `product`       | string   | Product name of the machine
 `datacenter`    | string   | Data center name where the machine exists.
 `rack`          | int      | Logical rack number (LRN) where the machine exists.
@@ -48,7 +48,6 @@ A JSON representation of `Machine` looks like:
     "role": "boot",
     "ipv4": ["10.69.0.69", "10.69.0.133"],
     "ipv6": [],
-    "state": "healthy",
     "bmc": {
       "type": "iDRAC-9",
       "ipv4": "10.72.17.37"