Skip to content

Commit 4583227

Browse files
author
Jasper van Wanrooy
committed
Test client side session replication with with ansible and docker.
0 parents  commit 4583227

File tree

18 files changed

+552
-0
lines changed

18 files changed

+552
-0
lines changed

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/ansible/test.retry

‎README.md

+77
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# POC for PHP sessions backed by memcached
2+
3+
## Design considerations
4+
5+
Given we have to re-evaluate whether the sessions are going to work reliable we have to take a few things into consideration.
6+
7+
* The Memcached daemon does not have any features to act in a highly available manner. In case the daemon gets restarted it forgets everything what was stored in there. The `libmemcached` library supports a poor man's HA option though from the clients by writing to two nodes.
8+
* We have to Memcached nodes.
9+
* In case we lose one node, we don't want the use of sessions in PHP to be interrupted.
10+
11+
## How?
12+
13+
In order to be able to validate the session behaviour an isolated environment has been created with Docker. The `docker-compose.yml` contains a definition of the stack: a simple php application and two memcached nodes. The php web server gets the memcached extension installed. The configuration file for `pecl-memcached` is included from the `./php-app/php-memcached-opt.ini` file during container image build time. The memcached containers are started in very verbose mode in order to see all commands being issued. A simple PHP application starts a session, displays the variables in the session and adds all query parameters in the session.
14+
15+
In order to start playing around just start the stack with `docker-compose up --build`. This will start both the application and the memcached containers. Any change to the `./php-app/php-memcached-opt.ini` file needs restarting the stack. Any change to the php file is reflected right away. Simulating a memcached node being unavailable `docker stop pocphpmemcachedsessions-memcached-1` or `2` of course.
16+
17+
__Note:__ Want to test on PHP7? Look in the `docker-compose.yml` file which of the ports you have to use.
18+
19+
| PHP setting | Value | Reasoning |
20+
|:------------- |:--------------|:------|
21+
| `memcached.sess_locking` | `Off` | This feature results in only one php process can access the session for a single user. Although this is very useful, it was not supported in the former extension, so disabled. It adds a lot of chatter to the memcached daemon to handle the locking. We don't need it either. |
22+
| `memcached.sess_prefix` | `session.` | Logical separation of keys to prevent collisions. |
23+
| `memcached.sess_remove_failed` | `1` | Automatically remove a Memcached server from the pool as soon it becomes unavailable. As soon it becomes available again it will be available for sessions again. See Known quirks below. |
24+
| `memcached.sess_consistent_hash` | `On` | This is a very important setting for failure scenario's. The session configuration contains the two memcached servers. When PHP starts the session it assigns a Memcached server to act as a primary store based on the session Id. With this setting switched to off, it will assign the Memcached server statically. In case that server is unavailable, PHP will emit warnings and the session not able to be retrieved nor stored. Setting this value to `On` will result in a dynamic assignment of a primary Memcached server based on server availability. |
25+
| `memcached.sess_number_of_replicas` | `1` | This makes libmemcached store the session in both configured Memcached servers. |
26+
| `memcached.sess_binary` | `On` | Without this, the replication wouldn't work. No implications in user space though. |
27+
| `memcached.sess_randomize_replica_read` | `Off` | Setting is not properly documented. Setting this to off did not result in different behaviour combined with the other settings. Configured like this because it feels safer to only read from one server. Keep in mind though that each session is assigned a primary Memcached server, based on a consistent hashing algorithm. |
28+
| `memcached.sess_connect_timeout` | `100` | Time in milliseconds to use as a timeout to mark a Memcached server as unavailable. Given we only use a LAN any value higher than this will mean that we have bigger issues in the network. |
29+
| `session.save_handler` | `memcached` | Obvious |
30+
| `session.save_path` | `memcached-1:11211,memcached-2:11211` | Reference the two nodes. The documentation states that a retry_interval can be configured. It also states that a different syntax should be used. That's just nonsense. This works. |
31+
32+
__Note:__ the `session.gc_maxlifetime` has not been set, since it wasn't set earlier. Default value is 1440 seconds (24 min).
33+
34+
## Testing
35+
36+
In order to validate the behaviour of these settings with different versions of php and memcached extensions a suite of tests has been set up.
37+
38+
### Requirements
39+
* `docker` version `1.12`
40+
* `docker-compose` version `1.9.0-rc4`
41+
* `ansible` version `2.2.0.0`
42+
43+
### Environments
44+
The following environments are used for testing:
45+
* PHP `5.6.29` with pecl-memcached (latest) on port 83
46+
* PHP `7.0.14` with pecl-memcached (master branch) on port 84
47+
* PHP `5.6.29` with elasticache from git master branch HEAD on port 85
48+
* PHP `7.0.14` with elasticache from git php7 branch HEAD on port 87
49+
50+
### Tests
51+
The main ansible playbook `ansible/tests.yml` contains the definitions of the tests. With every test the stack defined in `docker-compose.yml` is destroyed and upped again. We do the following tests:
52+
* `ansible/roles/basic-set-values/tasks/main.yml` does basic interaction with the session in subsequent http requests.
53+
* `ansible/roles/single-node-failure/tasks/main.yml` validates the behaviour for primary and secondary memcached node failures.
54+
55+
### Running the tests
56+
* All tests for all environments:
57+
58+
ansible-playbook ansible/tests.yml -vv
59+
60+
* Running all tests based on php5:
61+
62+
ansible-playbook ansible/tests.yml -vv --tags php5
63+
64+
* Running only the elasticache tests for both php5 and php7:
65+
66+
ansible-playbook ansible/tests.yml -vv --tags elasticache
67+
68+
* Running the failing elasticache test for php7:
69+
70+
ansible-playbook ansible/tests.yml -vv --tags php7-elasticache-node-failure
71+
72+
* With a bit of creativity the other specific tests can be figured out. Or just look at the `tests.yml` file.
73+
74+
## Known quirks
75+
76+
* In case one Memcached daemon becomes unavailable for the web application servers we won't loose any functionalities in the site. As soon as this daemon becomes available again 50% of the sessions will be lost immediately.
77+
* Can't think of more really.

‎ansible/group_vars/all.yml

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
3+
php5_pecl_port: 83
4+
php7_pecl_port: 84
5+
php5_elasticache_port: 85
6+
php7_elasticache_port: 87
7+
8+
container_host: "localhost"

‎ansible/roles/basic-set-values/meta/main.yml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
dependencies:
3+
- { role: common }

‎ansible/roles/basic-set-values/tasks/main.yml

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
3+
- name: start a session and set a variable
4+
uri:
5+
url: "http://{{ container_host }}:{{ container_port }}/?initiator=php"
6+
return_content: true
7+
register: session_start
8+
9+
- name: set something else in the session
10+
uri:
11+
url: "http://{{ container_host }}:{{ container_port }}/?something=else"
12+
return_content: true
13+
HEADER_Cookie: "{{ session_start.set_cookie }}"
14+
15+
- name: assert the session contains the right values
16+
uri:
17+
url: "http://{{ container_host }}:{{ container_port }}/"
18+
return_content: true
19+
HEADER_Cookie: "{{ session_start.set_cookie }}"
20+
register: state
21+
failed_when: "('initiator = php' not in state.content) or ('something = else' not in state.content)"

‎ansible/roles/common/meta/main.yml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
3+
allow_duplicates: yes

‎ansible/roles/common/tasks/main.yml

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
---
2+
- name: cleanup docker-compose project
3+
docker_service:
4+
project_name: pocphpmemcachedsessions
5+
project_src: "../."
6+
state: absent
7+
8+
- name: initialize brand new docker-compose project
9+
docker_service:
10+
project_name: pocphpmemcachedsessions
11+
project_src: "../."
12+
build: yes
13+
state: present
14+
register: docker_compose

‎ansible/roles/single-node-failure/defaults/main.yml

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
container_port: 80 # only used as a default
3+
session_id: a
4+
primary_node: memcached-1 # this means that this node is used for reading this specific session, when the node is available. When this node joins empty again, it means the session is lost.
5+
replicate_node: memcached-2 # fallback node for reading

‎ansible/roles/single-node-failure/meta/main.yml

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
dependencies:
3+
- { role: common }

‎ansible/roles/single-node-failure/tasks/main.yml

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
3+
- name: start a session on port {{ container_port }} and set a replicate-test = yeap in the session
4+
uri:
5+
url: "http://{{ container_host }}:{{ container_port }}/?replicate-test=yeap"
6+
HEADER_Cookie: "PHPSESSID={{ session_id }}"
7+
8+
- name: stop the replicate node for this session
9+
docker_container:
10+
name: "pocphpmemcachedsessions-{{ replicate_node }}"
11+
state: stopped
12+
13+
- name: assert that the session contains the replicate-test = yeap content
14+
uri:
15+
url: "http://{{ container_host }}:{{ container_port }}/"
16+
return_content: true
17+
HEADER_Cookie: "PHPSESSID={{ session_id }}"
18+
register: state
19+
failed_when: "'replicate-test = yeap' not in state.content"
20+
21+
- name: start the replicate node for this session again
22+
docker_container:
23+
name: "pocphpmemcachedsessions-{{ replicate_node }}"
24+
state: started
25+
26+
- name: assert that the session contains the replicate-test = yeap content
27+
uri:
28+
url: "http://{{ container_host }}:{{ container_port }}/"
29+
return_content: true
30+
HEADER_Cookie: "PHPSESSID={{ session_id }}"
31+
register: state
32+
failed_when: "'replicate-test = yeap' not in state.content"
33+
34+
- name: stop the primary node for this session
35+
docker_container:
36+
name: "pocphpmemcachedsessions-{{ primary_node }}"
37+
state: stopped
38+
39+
- name: assert that the session contains the replicate-test = yeap content
40+
uri:
41+
url: "http://{{ container_host }}:{{ container_port }}/"
42+
return_content: true
43+
HEADER_Cookie: "PHPSESSID={{ session_id }}"
44+
register: state
45+
failed_when: "'replicate-test = yeap' not in state.content"
46+
47+
- name: start the primary node for this session again
48+
docker_container:
49+
name: "pocphpmemcachedsessions-{{ primary_node }}"
50+
state: started
51+
52+
- name: assert that the session is empty (unfortunate expected behaviour)
53+
uri:
54+
url: "http://{{ container_host }}:{{ container_port }}/"
55+
return_content: true
56+
HEADER_Cookie: "PHPSESSID={{ session_id }}"
57+
register: state
58+
failed_when: "'replicate-test = yeap' in state.content"

‎ansible/tests.yml

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
- name: test php5 pecl basic set values
3+
hosts: localhost
4+
gather_facts: false
5+
roles:
6+
- { role: basic-set-values, container_port: "{{ php5_pecl_port }}" }
7+
tags: [php5, pecl, php5-pecl-basic]
8+
9+
- name: test php7 pecl basic set values
10+
hosts: localhost
11+
gather_facts: false
12+
roles:
13+
- { role: basic-set-values, container_port: "{{ php7_pecl_port }}" }
14+
tags: [php7, pecl, php7-pecl-basic]
15+
16+
- name: test php5 elasticache basic set values
17+
hosts: localhost
18+
gather_facts: false
19+
roles:
20+
- { role: basic-set-values, container_port: "{{ php5_elasticache_port }}" }
21+
tags: [php5, elasticache, php5-elasticache-basic]
22+
23+
- name: test php7 elasticache basic set values
24+
hosts: localhost
25+
gather_facts: false
26+
roles:
27+
- { role: basic-set-values, container_port: "{{ php7_elasticache_port }}" }
28+
tags: [php7, elasticache, php7-elasticache-basic]
29+
30+
- name: test php5 pecl primary and replica node failure
31+
hosts: localhost
32+
gather_facts: false
33+
roles:
34+
- { role: single-node-failure, container_port: "{{ php5_pecl_port }}" }
35+
tags: [php5, pecl, php5-pecl-node-failure]
36+
37+
- name: test php7 pecl primary and replica node failure
38+
hosts: localhost
39+
gather_facts: false
40+
roles:
41+
- { role: single-node-failure, container_port: "{{ php7_pecl_port }}" }
42+
tags: [php7, pecl, php7-pecl-node-failure]
43+
44+
- name: test php5 elasticache primary and replica node failure
45+
hosts: localhost
46+
gather_facts: false
47+
roles:
48+
- { role: single-node-failure, container_port: "{{ php5_elasticache_port }}" }
49+
tags: [php5, elasticache, php5-elasticache-node-failure]
50+
51+
- name: test php7 elasticache primary and replica node failure
52+
hosts: localhost
53+
gather_facts: false
54+
roles:
55+
- { role: single-node-failure, container_port: "{{ php7_elasticache_port }}" }
56+
tags: [php7, elasticache, php7-elasticache-node-failure]

‎docker-compose.yml

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
version: '2'
2+
services:
3+
php-app-1:
4+
build:
5+
context: php-app/
6+
dockerfile: Dockerfile-php5
7+
ports:
8+
- "83:80"
9+
volumes:
10+
- "./php-app/src/:/var/www/html"
11+
depends_on:
12+
- memcached-1
13+
- memcached-2
14+
container_name: pocphpmemcachedsessions-php-app-1
15+
php-app-2:
16+
build:
17+
context: php-app/
18+
dockerfile: Dockerfile-php7
19+
ports:
20+
- "84:80"
21+
volumes:
22+
- "./php-app/src/:/var/www/html"
23+
depends_on:
24+
- memcached-1
25+
- memcached-2
26+
container_name: pocphpmemcachedsessions-php-app-2
27+
php-app-3:
28+
build:
29+
context: php-app/
30+
dockerfile: Dockerfile-php5-amazon-elasticache
31+
ports:
32+
- "85:80"
33+
volumes:
34+
- "./php-app/src/:/var/www/html"
35+
depends_on:
36+
- memcached-1
37+
- memcached-2
38+
container_name: pocphpmemcachedsessions-php-app-3
39+
php-app-4:
40+
build:
41+
context: php-app/
42+
dockerfile: Dockerfile-php7-amazon-elasticache
43+
ports:
44+
- "87:80"
45+
volumes:
46+
- "./php-app/src/:/var/www/html"
47+
depends_on:
48+
- memcached-1
49+
- memcached-2
50+
container_name: pocphpmemcachedsessions-php-app-4
51+
memcached-1:
52+
image: memcached:alpine
53+
command: [memcached, -vv]
54+
container_name: pocphpmemcachedsessions-memcached-1
55+
memcached-2:
56+
image: memcached:alpine
57+
command: [memcached, -vv]
58+
container_name: pocphpmemcachedsessions-memcached-2

‎php-app/Dockerfile-php5

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
FROM php:5.6.29-apache
2+
3+
RUN apt-get update && apt-get install -y zlib1g-dev libmemcached-dev \
4+
&& pecl install memcached \
5+
&& docker-php-ext-enable memcached \
6+
&& rm -rf /var/lib/apt/lists
7+
8+
VOLUME /var/www/html
9+
10+
COPY src/ /var/www/html/
11+
COPY php-memcached-opt.ini /usr/local/etc/php/conf.d/docker-php-ext-memcached.ini

‎php-app/Dockerfile-php5-amazon-elasticache

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
FROM php:5.6.29-apache
2+
3+
RUN mkdir -p /usr/local/elasticache-libmemcached/ \
4+
&& curl -sSL https://github.com/awslabs/aws-elasticache-cluster-client-libmemcached/archive/master.tar.gz | tar xz -C /usr/local/elasticache-libmemcached/ --strip-components=1
5+
6+
RUN mkdir -p /usr/local/elasticache/ \
7+
&& curl -sSL https://github.com/awslabs/aws-elasticache-cluster-client-memcached-for-php/archive/master.tar.gz | tar xz -C /usr/local/elasticache/ --strip-components=1
8+
9+
RUN apt-get update && apt-get install -y zlib1g-dev libevent-dev \
10+
&& rm -rf /var/lib/apt/lists
11+
12+
RUN cd /usr/local/elasticache-libmemcached \
13+
&& ./configure \
14+
&& make \
15+
&& make install
16+
17+
RUN cd /usr/local/elasticache \
18+
&& phpize \
19+
&& ./configure --disable-memcached-sasl \
20+
&& make \
21+
&& make install
22+
23+
VOLUME /var/www/html
24+
25+
COPY src/ /var/www/html/
26+
COPY php-memcached-opt.ini /usr/local/etc/php/conf.d/docker-php-ext-memcached.ini

‎php-app/Dockerfile-php7

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
FROM php:7.0.14-apache
2+
3+
RUN mkdir -p /usr/local/php-memcached/ \
4+
&& curl -sSL https://github.com/php-memcached-dev/php-memcached/archive/master.tar.gz | tar xz -C /usr/local/php-memcached/ --strip-components=1
5+
6+
RUN apt-get update && apt-get install -y zlib1g-dev libmemcached-dev \
7+
&& rm -rf /var/lib/apt/lists
8+
9+
RUN cd /usr/local/php-memcached \
10+
&& phpize \
11+
&& ./configure --disable-memcached-sasl \
12+
&& make \
13+
&& make install
14+
15+
VOLUME /var/www/html
16+
17+
COPY src/ /var/www/html/
18+
COPY php-memcached-opt.ini /usr/local/etc/php/conf.d/docker-php-ext-memcached.ini

0 commit comments

Comments
 (0)