Kubeless is a serverless solution for on-premises deployment. It aims to be an AWS Lambda clone that you deploy on a Kubernetes cluster.
To showcase the power of Kubeless, we are going to send Tweets from AWS Simple Queue Service (SQS). What this will demonstrate is that a simple Python script can be invoked when an SQS message is published in your AWS console or via any other SQS client. That message is received by the Kubeless function, which in turn calls the Twitter API and sends the Tweet.
That fun example should give you a bunch of interesting ideas for building some advanced application pipelines that can leverage AWS services and your own on-premises functions.
First, let’s deploy Kubeless:
Deploy Kubeless on your Kubernetes cluster
With a Kubernetes cluster at hand, you’ll deploy Kubeless in two commands. Create a namespace and then create the released manifests.
kubectl create ns kubeless curl -sL https://github.com/kubeless/kubeless/releases/download/v0.3.4/kubeless-rbac-v0.3.4.yaml | kubectl create -f -
Kubeless is continuously tested on Minikube and Google GKE clusters, but it should work in any recent k8s versions (and also k8s distributions like OpenShift).
To get the CLI, if you are on OSX, do:
brew install kubeless
If you are not on OSX, you can grab a release binary from the GitHub release page.
Now, let’s create our Twitter application:
Creating a Twitter application
First, create a Twitter application by going to https://apps.twitter.com/ and make note of your application’s secrets and keys. Creating an application is really just a way to get some API keys so that you can write code to talk to the Twitter API.
You can then store those secrets as Kubernetes secrets, so that your function can later call the Twitter API.
kubectl create secret generic twitter --from-literal=consumer_key=vV2...
--from-literal=consumer_secret=Am0...
--from-literal=token_key=369...
--from-literal=token_secret=5ltu...
Creating a function
The main Kubeless README shows you two approaches to deploy a basic toy function. In this blog post, we are going to deploy a function in a more advanced way. We are going to write the Function object directly and create it using the Kubernetes client “kubectl”.
Why do this? Because a Kubeless function is really a Kubernetes object. The Kubernetes API server has been customized automatically to discover a new object kind called a “function”. This means that we can write a YAML manifest of our function. It will contain the function code as well as other function spec values.
Let’s break down the function of the function object which creates a function that receives SQS messages and sends Tweets:
apiVersion: k8s.io/v1
kind: Function
metadata:
name: sqstweet
spec:
deps: |
python-twitter
kubernetes==2.0.0
Above we see the standard “apiVersion”, “kind”, and “metadata” of any k8s objects. Then the “deps” section specifies the dependencies needed by our function. We are going to use the Python Twitter module, so we need to define it there. We also need the k8s Python client to retrieve the Twitter API keys from the k8s secret.
Below, we write the Python script directly in the function object:
function: |+
import base64
import twitter
from kubernetes import client, config
config.load_incluster_config()
v1=client.CoreV1Api()
for secrets in v1.list_secret_for_all_namespaces().items:
if secrets.metadata.name == 'twitter':
consumer_key = base64.b64decode(secrets.data['consumer_key'])
consumer_secret = base64.b64decode(secrets.data['consumer_secret'])
token_key = base64.b64decode(secrets.data['token_key'])
token_secret = base64.b64decode(secrets.data['token_secret'])
api = twitter.Api(consumer_key=consumer_key,
consumer_secret=consumer_secret,
access_token_key=token_key,
access_token_secret=token_secret)
def tweet(context):
msg = context.body
status = api.PostUpdate(msg)
…And then, some Kubeless function specific data. The “handler” is the file name containing the function (if we were to create it from the CLI) and the name of the method defined. The “runtime” specifies that it is a Python 2.7 function. The “template” is the Pod Template used by the function. Here we pass a set of environment variables so that the function can talk to SQS.
function-content-type: text
handler: foo.tweet
runtime: python2.7
template:
metadata:
creationTimestamp: null
spec:
containers:
- env:
- name: QUEUE_NAME
value: kubeless.fifo
- name: AWS_ACCESS_KEY_ID
value:
- name: AWS_SECRET_ACCESS_KEY
value:
image: kubeless/sqs-python
name: ""
resources: {}
timeout: "180"
type: HTTP
Save the function object into a file called “sqs-tweet.yaml”
Note that the AWS credentials should be created. You can also use an IAM profile. In addition, note that the function object contains the SQS FIFO queue name as an environment variables. You can create this FIFO queue through the SQS console.
Create the function object with “kubectl” like so:
kubectl create -f sqs-tweet.yaml
Calling the function from AWS SQS
The nice trick in this tutorial is that the function is not exposed over HTTP like the standard toy functions. This function is actually using a custom runtime “kubeless/sqs-python”, a special Docker image that has been built-in with an SQS consumer.
Our Twitter function defined in the function object above is loaded inside our custom runtime. When an SQS message is received, the payload is passed to the function, which sends it to Twitter:
def tweet(context):
msg = context.body
status = api.PostUpdate(msg)
Putting it together
We can head over to the AWS SQS console and use the debugging UI to send an SQS message. See the snaphshot below:
As soon as the message is sent, you can check the timeline of your Twitter account and you will see your Tweet. 🙂
Conclusions
The sky is the limit here, and while I did not write about the most basic hello world example I hope that you see the potential of Kubeless.
Check out the code on GitHub. We would love to get help. This is 100% open source, with no commercialization plans.
There are more interesting function examples at https://github.com/kubeless/functions
