In this tutorial we are going to expand our examples with deploying a more complex microservice. The idea is to make you more comfortable with the platform and to show you how you can leverage it for more advanced scenarios.
In this example we are going to see deployment of:
- A Redis master
- Multiple Redis slaves
- A sample guestbook application that uses Redis as a store
We assume that you have already set up a Platform9 cluster with at least one node, and the cluster is ready.
Let’s start with the Redis parts.
Deploying and exposing a Redis Cluster
Redis is a key-value in-memory store that is used mainly as a cache service. In order to set up Clustering for Data Replication we need a Redis instance that acts as Master, together with additional instances as slaves. Then the guestbook application can use this instance to store data. The Redis master will propagate the writes to the slave nodes.
We can initiate a Redis Master deployment in a few different ways: either using the kubectl tool, the Platform9 UI or the Kubernetes UI. For convenience, we use the kubectl tool as it’s the most commonly understood in tutorials.
First we need to create a Redis Cluster Deployment. Looking at their documentation here, to set up a cluster, we need some configuration properties. We can leverage kubernetes configmaps to store and reference them in the deployment spec.
We need to save a script and a redis.conf file that is going to be used to configure the master and slave nodes.
Create the following config redis-cluster.config.yml
With these values
$ cat redis-cluster.config.yml --- apiVersion: v1 kind: ConfigMap metadata: name: redis-cluster-config data: update-ip.sh: | #!/bin/sh sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${IP}/" /data/nodes.conf exec "$@" redis.conf: |+ cluster-enabled yes cluster-config-file /data/nodes.conf appendonly yes
We define a script that will insert an IP value to the node.conf file. This is to fix an issue with Redis as referenced here. We use this script every time we deploy a new redis image.
Then we have the redis.conf, which applies the minimal cluster configuration.
Apply this spec into the cluster:
$ kubectl apply -f redis-cluster.config.yml
Then verify that it exists in the list of configmaps:
$ kubectl get configmaps
Next we need to define a spec for the redis cluster instances. We can use a Deployment or a StatefulSet to define 3 instances:
Here is the spec: redis-cluster.statefulset.yml
$ cat redis-cluster.statefulset.yml --- apiVersion: apps/v1 kind: StatefulSet metadata: name: redis-cluster spec: serviceName: redis-cluster replicas: 6 selector: matchLabels: app: redis-cluster template: metadata: labels: app: redis-cluster spec: containers: - name: redis image: redis:5.0.7-alpine ports: - containerPort: 6379 name: client - containerPort: 16379 name: gossip command: ["/conf/update-ip.sh", "redis-server", "/conf/redis.conf"] env: - name: IP valueFrom: fieldRef: fieldPath: status.podIP volumeMounts: - name: conf mountPath: /conf readOnly: false - name: data mountPath: /data readOnly: false volumes: - name: conf configMap: name: redis-cluster-config defaultMode: 0755 volumeClaimTemplates: - metadata: name: data spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi
In the above step we defined a few things:
- An IP environment variable that we need in the update-ip.sh script that we defined in the configmap earlier. This is the pod-specific IP address using the Downward API.
- Some shared volumes including the configmap that we defined earlier.
- Two container ports – 6379 and 16379 – for the gossip protocol.
With this spec we can deploy the Redis cluster instances:
$ kubectl apply -f redis-cluster.statefulset.yml
Once we verify that we have the deployment ready, we need to perform the last step, which is bootstrapping the cluster. Consulting the documentation here for creating the cluster, we need to ssh into one of the instances and run the redis-cli cluster create command. For example taken from the docs:
$ redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 \ 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \ --cluster-replicas 1
To do that in our case, we need to get the local pod IPs of the instances and feed them to that command.
We can query the IP using this command:
$ kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 '
So if we save them in a variable or a file, we can pipe them at the end of the redis-cli command:
$ POD_IPS = $(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
Then we can run the following command:
$ kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $POD_IPS
If everything is OK, you will see the following prompt. Enter ‘yes’ to accept and continue:
Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ........ [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
Then we can verify the cluster state by running the cluster info command:
$ kubectl exec -it redis-cluster-0 -- redis-cli cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:28 cluster_stats_messages_pong_sent:34 cluster_stats_messages_sent:62 cluster_stats_messages_ping_received:29 cluster_stats_messages_pong_received:28 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:62
Before we continue deploying the guestbook app, we need to offer a unified service frontend for the Redis Cluster so that it’s easily discoverable in the cluster.
Here is the service spec: redis-cluster.service.yml
$ cat redis-cluster.service.yml --- apiVersion: v1 kind: Service metadata: name: redis-master spec: type: ClusterIP ports: - port: 6379 targetPort: 6379 name: client - port: 16379 targetPort: 16379 name: gossip selector: app: redis-cluster
We expose the cluster as redis-master here, as the guestbook app will be looking for a host service to connect to with that name.
Once we apply this service spec, we can move on to deploying and exposing the Guestbook Application:
$ kubectl apply -f redis-cluster.service.yml
Deploying and exposing a GuestBook Application
The guestbook application is a simple php script that shows a form to submit a message. Initially it will attempt to connect to either the redis-master
host or the redis-slave
hosts. It needs the GET_HOSTS_FROM environment variable set pointing to the file with the following variables: REDIS_MASTER_SERVICE_HOST: of the master REDIS_SLAVE_SERVICE_HOST: of the slave
First, let’s define the deployment spec bellow:
php-guestbook.deployment.yml
$ cat php-guestbook.deployment.yml --- apiVersion: apps/v1 kind: Deployment metadata: name: guestbook spec: replicas: 1 selector: matchLabels: app: guestbook template: metadata: labels: app: guestbook spec: containers: - name: php-redis image: gcr.io/google-samples/gb-frontend:v6 resources: requests: cpu: 150m memory: 150Mi env: - name: GET_HOSTS_FROM value: env - name: REDIS_MASTER_SERVICE_HOST value: "redis-master" - name: REDIS_SLAVE_SERVICE_HOST value: "redis-master" ports: - containerPort: 80
The code of the gb-frontend image is located here.
Next is the the associated service spec:
--- apiVersion: v1 kind: Service metadata: name: guestbook-lb spec: type: NodePort ports: - port: 80 selector: app: guestbook
Note: NodePort will assign a random port over the public IP of the Node. In either case, we get a public host:port pair where we can inspect the application. Here is a screenshot of the app after we deployed it:
Cleaning up
Once we have finished experimenting with the application, we can clean up the resources and all the servers by issuing kubectl delete
statements. A convenient way is to delete by labels. For example:
$ kubectl delete statefulset redis-cluster $ kubectl delete service redis-master $ kubectl delete deployment guestbook $ kubectl delete service guestbook-lb $ kubectl delete configmap redis-cluster-config $ kubectl delete pvc -l app=redis-cluster
Next Steps
Deploying multi-container systems in Kubernetes with Platform9 is no different than usual. The big difference is that you gain a quality of server and a maintenance-free Kubernetes experience with first-class support for troubleshooting issues. That’s not to mention that you can host the cluster on your own bare-metal server or AWS, eliminating any vendor lock-ins. For more information visit Platform9 Managed Kubernetes page.