|
1 | 1 | # OCI Cloud Controller Manager (CCM)
|
2 | 2 |
|
3 |
| -This project is an Kubernetes Cloud Controller Manager (or out-of-tree |
4 |
| -cloud-provider) for [Oracle Cloud Infrastucture][1] (OCI). |
| 3 | +`oci-cloud-controller-manager` is a Kubernetes Cloud Controller Manager |
| 4 | +implementation (or out-of-tree cloud-provider) for [Oracle Cloud |
| 5 | +Infrastucture][1] (OCI). |
5 | 6 |
|
6 | 7 | [](https://app.wercker.com/project/byKey/17a52304e0309d138ad41f7ae9f9ea49)
|
7 | 8 |
|
| 9 | +**WARNING**: this project is under active development and may not be production |
| 10 | +ready. |
| 11 | + |
8 | 12 | ## Introduction
|
9 | 13 |
|
10 | 14 | External cloud providers were introduced as an _Alpha_ feature in Kubernetes
|
11 |
| -1.6. External cloud providers are Kubernetes (master) controllers that implement |
12 |
| -the cloud-provider specific control loops required for Kubernetes to function. |
| 15 | +1.6 with the addition of the [Cloud Controller Manager][2] binary. External |
| 16 | +cloud providers are Kubernetes (master) controllers that implement the |
| 17 | +cloud-provider specific control loops required for Kubernetes to function. |
13 | 18 |
|
14 | 19 | This functionality is implemented in-tree in the `kube-controller-manger` binary
|
15 | 20 | for _existing_ cloud-providers (e.g. AWS, GCE, etc.), however, in-tree
|
16 | 21 | cloud-providers have entered maintenance mode and _no additional providers will
|
17 |
| -be accepted_. |
| 22 | +be accepted_. Furthermore, there is an ongoing effort to remove all existing |
| 23 | +cloud-provider specific code out of the Kubernetes codebase. |
18 | 24 |
|
19 | 25 | ## Compatibility matrix
|
20 | 26 |
|
21 |
| -| | Kubernetes < 1.7.2 | Kubernetes 1.7.{2..5} | Kubernetes 1.7.{6..} | Kubernetes 1.8.0 | |
| 27 | +| | Kubernetes < 1.7.2 | Kubernetes 1.7.{2..5} | Kubernetes 1.7.{6..} | Kubernetes 1.8.* | |
22 | 28 | |-------|-----------------------|-----------------------|----------------------|------------------|
|
23 | 29 | | v 0.1 | ✗ | † | ✓ | ✓ |
|
24 | 30 |
|
25 | 31 | Key:
|
26 | 32 |
|
27 | 33 | * `✓` oci-cloud-controller-manager is fully compatible.
|
28 | 34 | * `†` oci-cloud-controller-manager is compatible but requires the
|
29 |
| - `--provider-id` flag to be set on the kubelet of all nodes in the |
| 35 | + `--provider-id` flag to be set on the Kubelet of all nodes in the |
30 | 36 | cluster.
|
31 | 37 | * `✗` oci-cloud-controller-manager is not compatible.
|
32 | 38 |
|
| 39 | +## Implementation |
| 40 | + Currently `oci-cloud-controller-manager` implements: |
| 41 | + |
| 42 | + - NodeController - updates nodes with cloud provider specific labels and |
| 43 | + addresses, also deletes kubernetes nodes when deleted from the |
| 44 | + cloud-provider. |
| 45 | + - ServiceController - responsible for creating load balancers when a service |
| 46 | + of `type: LoadBalancer` is created in Kubernetes. |
| 47 | + |
33 | 48 | ## Setup and Installation
|
34 | 49 |
|
35 |
| -See [INSTALL.md](docs/install.md). |
| 50 | +To get the CCM running in your Kubernetes cluster you will need to do the |
| 51 | +following: |
36 | 52 |
|
37 |
| -## Development |
| 53 | + 1. Prepare your Kubernetes cluster for running an external cloud provider. |
| 54 | + 2. Create a Kubernetes secret containing the configuration for the CCM. |
| 55 | + 3. Deploy the CCM as a [DaemonSet][4]. |
38 | 56 |
|
39 |
| -See [DEVELOPMENT.md](docs/development.md). |
| 57 | +### Preparing Your Cluster |
40 | 58 |
|
41 |
| -## Support |
| 59 | +To deploy the Cloud Controller Manager (CCM) your cluster must be configured to |
| 60 | +use an external cloud-provider. |
| 61 | + |
| 62 | +This involves: |
| 63 | + - Setting the `--cloud-provider=external` flag on the `kubelet` on **all |
| 64 | + nodes** in your cluster. |
| 65 | + - Setting the `--cloud-provider=external` flag on the `kube-controller-manager` |
| 66 | + in your Kubernetes control plane. |
| 67 | + |
| 68 | +**Depending on how kube-proxy is run you _may_ need the following:** |
| 69 | + |
| 70 | +- Ensuring that `kube-proxy` tolerates the uninitialised cloud taint. The |
| 71 | + following should appear in the `kube-proxy` pod yaml: |
| 72 | + |
| 73 | +```yaml |
| 74 | +- effect: NoSchedule |
| 75 | + key: node.cloudprovider.kubernetes.io/uninitialized |
| 76 | + value: "true" |
| 77 | +``` |
| 78 | +
|
| 79 | +If your cluster was created using `kubeadm` >= v1.7.2 this toleration will |
| 80 | +already be applied. See [kubernetes/kubernetes#49017][5] for details. |
| 81 | + |
| 82 | +Remember to restart any components that you have reconfigured before continuing. |
| 83 | + |
| 84 | +### Authentication and Configuration |
| 85 | + |
| 86 | +We publish the `oci-cloud-controller-mananger` to a private Docker registry. You |
| 87 | +will need a [Docker registry secret][6] to pull images from it. |
42 | 88 |
|
43 |
| -If you think you've found a bug, please [raise an issue][2]. |
| 89 | +```bash |
| 90 | +$ kubectl -n kube-system create secret docker-registry wcr-docker-pull-secret \ |
| 91 | + --docker-server="wcr.io" \ |
| 92 | + --docker-username="$DOCKER_REGISTRY_USERNAME" \ |
| 93 | + --docker-password="$DOCKER_REGISTRY_PASSWORD" \ |
| 94 | + --docker-email="[email protected]" |
| 95 | +``` |
44 | 96 |
|
45 |
| -## Cloud Controller Manager |
| 97 | +An example configuration file can be found [here][7]. Download this file and |
| 98 | +populate it with values specific to your chosen OCI identity and tenancy. |
| 99 | +Then create the Kubernetes secret with the following command: |
46 | 100 |
|
47 |
| -The cloud-controller-manager allows cloud vendors code and the Kubernetes core |
48 |
| -to evolve independent of each other. In prior releases, the core Kubernetes code |
49 |
| -was dependent upon cloud-provider-specific code for functionality. In future |
50 |
| -releases, code specific to cloud vendors should be maintained by the cloud |
51 |
| -vendor themselves, and linked to cloud-controller-manager while running |
52 |
| -Kubernetes. |
| 101 | +```bash |
| 102 | +$ kubectl create secret generic oci-cloud-controller-manager \ |
| 103 | + -n kube-system \ |
| 104 | + --from-file=cloud-provider.yaml=cloud-provider-example.yaml |
| 105 | +``` |
53 | 106 |
|
54 |
| -The following controllers have cloud provider dependencies: |
| 107 | +Note that you must ensure the secret contains the key `cloud-provider.yaml` |
| 108 | +rather than the name of the file on disk. |
| 109 | + |
| 110 | +### Deployment |
| 111 | + |
| 112 | +Lastly deploy the controller manager and associated RBAC rules if your cluster |
| 113 | +is configured to use RBAC: |
| 114 | + |
| 115 | +```bash |
| 116 | +$ kubectl apply -f https://raw.githubusercontent.com/oracle/oci-cloud-controller-manager/master/manifests/oci-cloud-controller-manager.yaml |
| 117 | +$ kubectl apply -f https://raw.githubusercontent.com/oracle/oci-cloud-controller-manager/master/manifests/oci-cloud-controller-manager-rbac.yaml |
| 118 | +``` |
| 119 | + |
| 120 | +Check the CCM logs to ensure it's running correctly: |
| 121 | + |
| 122 | +```bash |
| 123 | +$ kubectl -n kube-system get po | grep oci |
| 124 | +oci-cloud-controller-manager-ds-k2txq 1/1 Running 0 19s |
| 125 | +
|
| 126 | +$ kubectl -n kube-system logs oci-cloud-controller-manager-ds-k2txq |
| 127 | +I0905 13:44:51.785964 7 flags.go:52] FLAG: --address="0.0.0.0" |
| 128 | +I0905 13:44:51.786063 7 flags.go:52] FLAG: --allocate-node-cidrs="false" |
| 129 | +I0905 13:44:51.786074 7 flags.go:52] FLAG: --alsologtostderr="false" |
| 130 | +I0905 13:44:51.786078 7 flags.go:52] FLAG: --cloud-config="/etc/oci/cloud-config.cfg" |
| 131 | +I0905 13:44:51.786083 7 flags.go:52] FLAG: --cloud-provider="oci" |
| 132 | +``` |
| 133 | + |
| 134 | +## Examples |
| 135 | + |
| 136 | + - [Service `type: LoadBalancer` basic NGINX example][8] |
| 137 | + - [Service `type: LoadBalancer` NGINX SSL example][9] |
| 138 | + |
| 139 | +## Development |
| 140 | + |
| 141 | +See [DEVELOPMENT.md](docs/development.md). |
| 142 | + |
| 143 | +## Support |
55 | 144 |
|
56 |
| -- **Node Controller**: For checking the cloud provider to determine if a node |
57 |
| - has been deleted in the cloud after it stops responding. |
58 |
| -- **Route Controller**: For setting up routes in the underlying cloud |
59 |
| - infrastructure. |
60 |
| -- **Service Controller**: For creating, updating and deleting cloud provider |
61 |
| - load balancers. |
| 145 | +If you think you've found a bug, please [raise an issue][3]. |
62 | 146 |
|
63 | 147 | [1]: https://cloud.oracle.com/iaas
|
64 |
| -[2]: https://github.com/oracle/oci-cloud-controller-manager/issues/new |
| 148 | +[2]: https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/ |
| 149 | +[3]: https://github.com/oracle/oci-cloud-controller-manager/issues/new |
| 150 | +[4]: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/ |
| 151 | +[5]: https://github.com/kubernetes/kubernetes/pull/49017 |
| 152 | +[6]: https://kubernetes.io/docs/concepts/containers/images/#creating-a-secret-with-a-docker-config |
| 153 | +[7]: https://github.com/oracle/oci-cloud-controller-manager/tree/master/manifests/cloud-provider-example.yaml |
| 154 | +[8]: https://github.com/oracle/oci-cloud-controller-manager/blob/master/docs/tutorial.md |
| 155 | +[9]: https://github.com/oracle/oci-cloud-controller-manager/blob/master/docs/tutorial-ssl.md |
0 commit comments