Skip to content

Commit b0a117b

Browse files
committed
Move around text in the readme, deprecate the v0.8.0 in favor for kubeadm
1 parent 7b4b009 commit b0a117b

File tree

2 files changed

+440
-337
lines changed

2 files changed

+440
-337
lines changed

OLDREADME.md

+360
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,360 @@
1+
# Readme for the deprecated v0.8.0 of kubernetes-on-arm
2+
3+
## Download and build a SD Card
4+
5+
The first thing you will do, is to create a SD Card for your Pi. Alternatively, you may use the [`.deb` deployment](#deb-deployment)
6+
7+
Supported OSes/boards:
8+
- Arch Linux ARM **(archlinux)**
9+
- Raspberry Pi 1 A, A+, B, B+, (ZERO,) armv6 **(rpi)**
10+
- Raspberry Pi 2 Model B, armv7 **(rpi-2)**
11+
- Raspberry Pi 3 Model B, armv8, _armv7 rootfs_ **(rpi-3)**
12+
- Parallella armv7, [read more](docs/parallella-status.md) **(parallella)**
13+
- Cubietruck, armv7 **(cubietruck)**
14+
- Banana Pro, armv7 **(bananapro)**
15+
- HypriotOS **(hypriotos)**
16+
- Raspberry Pi 1 A, A+, B, B+, armv6 **(rpi)**
17+
- Raspberry Pi 2 Model B, armv7 **(rpi-2)**
18+
- Raspberry Pi 3 Model B, armv8, _armv7 rootfs_ **(rpi-3)**
19+
20+
21+
```bash
22+
# Go to our home folder, if you want
23+
cd ~
24+
25+
# Install git if needed and download this project
26+
# sudo apt-get install git
27+
git clone https://github.com/luxas/kubernetes-on-arm
28+
29+
# Change to that directory
30+
cd kubernetes-on-arm
31+
32+
# See which letter your newly inserted SD Card has:
33+
sudo fdisk -l
34+
35+
# Another great command
36+
lsblk
37+
38+
# Get some help text about supported options
39+
sdcard/write.sh
40+
41+
# Template:
42+
sudo sdcard/write.sh /dev/sdX [board] [os] [rootfs]
43+
44+
# Example: Write the SD Card for Raspberry Pi 2, Arch Linux ARM and include this project's Kubernetes scripts
45+
sudo sdcard/write.sh /dev/sdX rpi-2 archlinux docker-multinode
46+
47+
# The installer will ask you if you want to erase all data on your card
48+
# Answer y/n on that question
49+
# Prepend the command with QUIET=1 if no security check should be made
50+
# Requires an internet connection
51+
# This script runs in 3-4 mins
52+
```
53+
54+
The README for the `docker-multinode` rootfs [is here](sdcard/rootfs/docker-multinode/etc/kubernetes/README.md)
55+
56+
## Setup your board from an SD Card
57+
58+
Boot your board and log into it.
59+
60+
Arch Linux users:
61+
- The user/password is: **root/root** or **alarm/alarm**
62+
- These scripts requires root. So if you login via **alarm**, then `su root` when you´re going to do some serious hacking :)
63+
64+
HypriotOS users:
65+
- The user/password is: **pirate/hypriot**
66+
- Remember to prepend all commands with `sudo` if you are the `pirate` user
67+
68+
69+
Yes, I know. Root enabled via ssh isn´t that good.
70+
But the task to enhance ssh security is left as an exercise to the user.
71+
72+
```bash
73+
# This script will install and setup docker etc.
74+
kube-config install
75+
76+
# First, it will install docker, if not present
77+
# Then it will download kube deploy
78+
79+
# It will ask for which hostname you want. Defaults to kubepi.
80+
81+
# The script will ask you for timezone. Defaults to Europe/Helsinki
82+
# Run "timedatectl list-timezones" before to check for values
83+
84+
# It will ask you if it should create a 1 GB swapfile.
85+
86+
# Last question is whether you want to reboot
87+
# You have to reboot in order to get the cgroups working
88+
89+
# If you want to run this script non-interactively, provide the user input beforehand:
90+
# "\n" is the delimiter, "\" the escape character, presssing enter can be simulated with "\n"
91+
# Example:
92+
# /bin/echo -e "rpi-3\nhypriotos\nnodename\nEurope\/Berlin\n\n\ny\nY" | sudo kube-config install
93+
# This script runs in 2-3 mins
94+
```
95+
96+
## Start Kubernetes!
97+
98+
Hmm, starting a complex system like Kubernetes should be a complex task, right?
99+
Well, not this time.
100+
101+
`enable-master` runs [master.sh](https://github.com/kubernetes/kube-deploy/blob/master/docker-multinode/master.sh)
102+
`enable-worker` runs [worker.sh](https://github.com/kubernetes/kube-deploy/blob/master/docker-multinode/worker.sh)
103+
104+
```bash
105+
# To set up your board as both a master and a node, run
106+
kube-config enable-master
107+
108+
# To set up your board as a node, run
109+
kube-config enable-worker [master-ip]
110+
```
111+
112+
## Package deployment
113+
If you have already made a SD Card and your device is up and running, what can you do instead?
114+
For that, I've made a `.deb` package, so you could install it easily
115+
116+
The README for the `docker-multinode` rootfs [is here](sdcard/rootfs/docker-multinode/etc/kubernetes/README.md)
117+
118+
If you already have set up your Pi with latest Raspbian OS for example, follow this guide.
119+
120+
#### Install the `.deb` package
121+
122+
Supported operating systems are HypriotOS, Raspbian, Arch Linux ARM and in some cases Debian/Ubuntu.
123+
124+
```bash
125+
# Download the latest package
126+
curl -sSL https://github.com/luxas/kubernetes-on-arm/releases/download/v0.8.0/docker-multinode.deb > docker-multinode.deb
127+
# or
128+
wget https://github.com/luxas/kubernetes-on-arm/releases/download/v0.8.0/docker-multinode.deb
129+
130+
# Requires dpkg, which is preinstalled in at least all Debian/Ubuntu OSes
131+
sudo dpkg -i docker-multinode.deb
132+
133+
# Setup the enviroinment
134+
# It will ask which board it's running on and which OS
135+
# A reboot is required for it to function properly, but not for HypriotOS
136+
kube-config install
137+
138+
## ----- REBOOT -----
139+
140+
# Start the master or worker
141+
kube-config enable-master
142+
kube-config enable-worker [master ip]
143+
144+
# Get some info about the node
145+
kube-config info
146+
```
147+
148+
149+
## Use Kubernetes (the fun part begins here)
150+
151+
Some notes on running Docker on ARM are [here](docs/docker-on-arm.md)
152+
153+
```bash
154+
# See which commands kubectl and kube-config has
155+
kubectl
156+
kube-config
157+
158+
# Get info about your machine and Kubernetes version
159+
kube-config info
160+
161+
# Make an replication controller with an image
162+
# Hopefully you will have some workers, so you is able to see how they spread across hosts
163+
# The nginx-test image will be downloaded from Docker Hub and is a nginx server which only is serving the message: "<p>WELCOME TO NGINX</p>"
164+
# Expose the replication controller "my-nginx" as a service
165+
kubectl run my-nginx --image=luxas/nginx-test --replicas=3 --expose --port=80
166+
167+
# The above command will make a deployment since v1.2. The deployment will run a replica set (the "new" kind of replication contoller)
168+
# You may also specify --hostport, e.g. --hostport=30001 for exposing the service on all nodes' port 30001
169+
170+
# The pull might take some minutes
171+
# See when the nginx container is running
172+
docker ps
173+
174+
# See which pods are running
175+
kubectl get pods
176+
177+
# See which nodes we have
178+
kubectl get nodes
179+
180+
# See which ip we may ping, by getting services
181+
kubectl get svc
182+
183+
# See if the nginx container is working
184+
# Replace $SERVICE_IP with the ip "kubectl get svc" returned
185+
curl $SERVICE_IP
186+
# --> <p>WELCOME TO NGINX</p>
187+
188+
# See which internal cluster services that are running
189+
kubectl --namespace=kube-system get pods,rc,svc
190+
191+
# Test dns
192+
curl my-nginx.default.svc.cluster.local
193+
# --> <p>WELCOME TO NGINX</p>
194+
195+
# By default, "search [domains]" is added to /etc/resolv.conf
196+
# In this case, these domains are searched: "default.svc.cluster.local svc.cluster.local cluster.local"
197+
# That means, that you may only write "my-nginx", and it will search in those domains
198+
curl my-nginx
199+
# --> <p>WELCOME TO NGINX</p>
200+
201+
# Start the registry
202+
kube-config enable-addon registry
203+
204+
# Wait a minute for it to start
205+
kubectl --namespace=kube-system get pods
206+
207+
# Tag an image
208+
docker tag my-name/my-image localhost:5000/my-name/my-image
209+
210+
# And push it to the registry
211+
docker push localhost:5000/my-name/my-image
212+
213+
# On another node, pull it
214+
docker pull localhost:5000/my-name/my-image
215+
216+
# The master also proxies the services so that they are accessible from outside
217+
# The -L flag is there because curl has to follow redirects
218+
# You may also type this URL in a web browser
219+
curl -L http://[master-ip]:8080/api/v1/proxy/namespaces/default/services/my-nginx
220+
221+
# Generic apiserver proxy URL
222+
# curl -L http://[master-ip]:8080/api/v1/proxy/namespaces/[namespace]/services/[service-name]:[port-name]
223+
224+
# See which ports are open
225+
netstat -nlp
226+
227+
# See cluster info
228+
kubectl cluster-info
229+
230+
# See master health in a web browser
231+
# cAdvisor in kubelet provides a web site that outputs all kind of stats in real time
232+
# http://$MASTER_IP:4194
233+
234+
# Turndown Kubernetes on this node. This always reverts the "kube-config enable-*" commands
235+
kube-config disable
236+
```
237+
238+
## Addons
239+
240+
To enable/disable addons is very easy: `kube-config enable-addon [addon-name]` and `kube-config disable-addon [addon-name]`
241+
[README for the addons](addons/README.md)
242+
243+
Three addons are available for the moment:
244+
- Kubernetes DNS:
245+
- Every service gets the hostname: `{{my-svc}}.{{my-namespace}}.svc.cluster.local`
246+
- The default namespace is `default` (surprise), so unless you manually edit something it will land there
247+
- Kubernetes internal addon services runs in the namespace `kube-system`
248+
- Example: `my-awesome-webserver.default.svc.cluster.local` or just `my-awesome-webserver` may resolve to ip `10.0.0.154`
249+
- Those DNS names is available both in containers and on the node itself (kube-config automatically adds the info to `/etc/resolv.conf`)
250+
- If you want to access the Kubernetes API easily, `curl -k https://kubernetes` or `curl -k https://10.0.0.1` if you remember numbers better (`-k` stands for insecure as apiserver has no signed certs by default)
251+
- The DNS server itself has allocated ip `10.0.0.10`
252+
- The DNS domain is `cluster.local`
253+
- This addon can't be disabled.
254+
- Kubernetes Dashboard:
255+
- The Kubernetes Dashboard project [is here](https://github.com/kubernetes/dashboard)
256+
- Access the dashboard on: `http://[master-ip]:8080/ui`
257+
- This addon can't be disabled.
258+
- Central image registry:
259+
- A registry for storing cluster images if e.g. the cluster has no internet connection for a while
260+
- Or for cluster-specific images that one not want to publish on Docker Hub
261+
- This service is available at `localhost:5000` on all nodes, which by default is a "trusted" location.
262+
- `localhost:5000` forwards the traffic to the internal IP of the registry service.
263+
- Just tag your image: `docker tag my-name/my-image localhost:5000/my-name/my-image`
264+
- And push it to the registry: `docker push localhost:5000/my-name/my-image`
265+
- Service loadbalancer:
266+
- Documentation [here](https://github.com/kubernetes/contrib/tree/master/service-loadbalancer)
267+
- You have to label at least one node `role=loadbalancer` like this: `kubectl label no [node_ip] role=loadbalancer`
268+
- The loadbalancer will expose http services in the default namespace on `http://[loadbalancer_ip]/[service_name]`. Only `http` services on port 80 are tested in this release. It should be pretty easy to add `https` support though.
269+
- You may see `haproxy` stats on `http://[loadbalancer_ip]:1936`
270+
- Not recommended for heavy use. Will be replaced with ingress in coming releases.
271+
- Cluster monitoring with heapster, influxdb and grafana
272+
- When this addon is enabled, the dashboard will show usage graphs in the CPU and RAM columns.
273+
- All heapster data is stored in an InfluxDB database. Data is written once a minute. Access the graphical InfluxDB UI: `http://[master-ip]:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb:http` and the raw api on: `http://[master-ip]:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb:api`
274+
- A nice `grafana` web dashboard that shows resource usage for the whole cluster as for individual pods is accessible at: `http://[master-ip]:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-grafana`. It may take some minutes for data to show up.
275+
276+
## Access your cluster
277+
278+
Here is some ways to make your outside devices reach the services running in the cluster.
279+
280+
- `apiserver` proxy:
281+
- This is enabled by default by apiserver
282+
- Type this URL in a browser or use `curl`
283+
- `curl -L http://[master-ip]:8080/api/v1/proxy/namespaces/[namespace]/services/[service-name][:[port-name]]`
284+
- You may build a proxy in front of this with `nginx` that forwards all requests to the apiserver proxy
285+
- Connect a computer to the `flannel` network
286+
- It's possible to start `flannel` and `kube-proxy` on another computer **in the same network** and access all services
287+
- Run these two commands from a `amd64` machine with docker:
288+
- `docker run --net=host -d --privileged -v /dev/net:/dev/net quay.io/coreos/flannel:0.6.1-amd64 /opt/bin/flanneld --etcd-endpoints=http://$MASTER_IP:2379`
289+
- `docker run --net=host -d --privileged gcr.io/google_containers/hyperkube-amd64:v1.3.6 /hyperkube proxy --master=http://$MASTER_IP:8080 --v=2`'
290+
- Replace $MASTER_IP with the actual ip of your master node
291+
- The consuming `amd64` computer can access all services
292+
- For example: `curl -k https://10.0.0.1`
293+
- Make a `service` with `externalIP`
294+
- Via `kubectl`: `kubectl expose rc {some rc} --port={the port this service will listen on} --container-port={the port the container exposes} --external-ip={the host you want to listen on}`
295+
- Example: `kubectl expose rc my-nginx --port=9060 --container-port=80 --external-ip=192.168.200.100`
296+
- This will make the service accessible at `192.168.200.100:9060`
297+
- Service `NodePort`
298+
- If one sets Service `.spec.type` to `NodePort`, Kubernetes automatically exposes the service on a random port on every node
299+
300+
#### See node health via `cAdvisor`
301+
302+
Go to a web browser and type: `{IP of your node}:4194` and a nice dashboard will be there and show you some nice real-time stats.
303+
304+
## Configuration
305+
306+
There is a configuration file: `/etc/kubernetes/k8s.conf`, where you can customize some things:
307+
- `K8S_MASTER_IP`: Points to the master in the cluster. If the node is master, it uses `127.0.0.1` (aka `localhost`). Default: `127.0.0.1`
308+
- The other options comes from [docker-multinode](https://github.com/kubernetes/kube-deploy/tree/master/docker-multinode#optionsconfiguration)
309+
310+
**Note:** You must change the values in `k8s.conf` before starting Kubernetes. Otherwise they won't have effect, just be able to harm your setup.
311+
312+
On Arch Linux, this file will override the default `eth0` settings. If you have a special `eth0` setup (or use some other network), edit this file to fit your use case: `/etc/systemd/network/dns.network`
313+
314+
## Docker versions
315+
316+
Only `docker-1.10` and higher is supported, `docker-1.11` is recommended.
317+
318+
## Troubleshooting
319+
320+
If your cluster won't start, try `kube-config disable` and choose to remove `/var/lib/kubelet`. That will remove all data you store in `/var/lib/kubelet` and kill most running docker images.
321+
322+
## Reboots
323+
324+
Will **not** work in this version. It's in the roadmap to enable reboots again.
325+
326+
## Contributing
327+
328+
I would be really glad to review your Pull Request! One thing would be good to remember though. I develop on the `dev` branch, so it would be great if you target that one instead of `master`
329+
330+
Thanks!
331+
332+
## Beta version
333+
334+
This project is under development.
335+
I develop things on the [`dev` branch](../../tree/dev)
336+
337+
[Changelog](CHANGELOG.md)
338+
339+
## Future work
340+
341+
See the [ROADMAP](ROADMAP.md)
342+
343+
## License
344+
345+
[MIT License](LICENSE)
346+
347+
## Goals for this project
348+
349+
This project ports [Kubernetes](https://github.com/kubernetes/kubernetes) to the ARM architecture.
350+
The primary boards used for testing is Raspberry Pi 2´s.
351+
352+
My goal for this project is that it should be as small as possible, while retaining its flexibility.
353+
It should also be as easy as possible for people, who don´t know anything about Kubernetes, to get started.
354+
355+
I also have opened a proposal for Kubernetes on ARM: [kubernetes/kubernetes#17981](https://github.com/kubernetes/kubernetes/issues/17981).
356+
The long-term goal most of this functionality should be present in core Kubernetes.
357+
358+
It should be easy in the future to add support for new boards and operating systems.
359+
360+
#### Feel free to create an issue if you find a bug or think that something should be added or removed!

0 commit comments

Comments
 (0)