kubless logo

Sending a Tweet from AWS SQS with Kubeless

1012 VIEWS

· · ·

Kubeless is a serverless solution for on-premises deployment. It aims to be an AWS Lambda clone that you deploy on a Kubernetes cluster.

To showcase the power of Kubeless, we are going to send Tweets from AWS Simple Queue Service (SQS). What this will demonstrate is that a simple Python script can be invoked when an SQS message is published in your AWS console or via any other SQS client. That message is received by the Kubeless function, which in turn calls the Twitter API and sends the Tweet.

That fun example should give you a bunch of interesting ideas for building some advanced application pipelines that can leverage AWS services and your own on-premises functions.

First, let’s deploy Kubeless:

Deploy Kubeless on your Kubernetes cluster

With a Kubernetes cluster at hand, you’ll deploy Kubeless in two commands. Create a namespace and then create the released manifests.

Kubeless is continuously tested on Minikube and Google GKE clusters, but it should work in any recent k8s versions (and also k8s distributions like OpenShift).

To get the CLI, if you are on OSX, do:

If you are not on OSX, you can grab a release binary from the GitHub release page.

Now, let’s create our Twitter application:

Creating a Twitter application

First, create a Twitter application by going to https://apps.twitter.com/ and make note of your application’s secrets and keys. Creating an application is really just a way to get some API keys so that you can write code to talk to the Twitter API.

You can then store those secrets as Kubernetes secrets, so that your function can later call the Twitter API.

Creating a function

The main Kubeless README shows you two approaches to deploy a basic toy function. In this blog post, we are going to deploy a function in a more advanced way. We are going to write the Function object directly and create it using the Kubernetes client “kubectl”.

Why do this? Because a Kubeless function is really a Kubernetes object. The Kubernetes API server has been customized automatically to discover a new object kind called a “function”. This means that we can write a YAML manifest of our function. It will contain the function code as well as other function spec values.

Let’s break down the function of the function object which creates a function that receives SQS messages and sends Tweets:

Above we see the standard “apiVersion”, “kind”, and “metadata” of any k8s objects. Then the “deps” section specifies the dependencies needed by our function. We are going to use the Python Twitter module, so we need to define it there. We also need the k8s Python client to retrieve the Twitter API keys from the k8s secret.

Below, we write the Python script directly in the function object:

…And then, some Kubeless function specific data. The “handler” is the file name containing the function (if we were to create it from the CLI) and the name of the method defined. The “runtime” specifies that it is a Python 2.7 function. The “template” is the Pod Template used by the function. Here we pass a set of environment variables so that the function can talk to SQS.

Save the function object into a file called “sqs-tweet.yaml”

Note that the AWS credentials should be created. You can also use an IAM profile. In addition, note that the function object contains the SQS FIFO queue name as an environment variables. You can create this FIFO queue through the SQS console.

Create the function object with “kubectl” like so:

Calling the function from AWS SQS

The nice trick in this tutorial is that the function is not exposed over HTTP like the standard toy functions. This function is actually using a custom runtime “kubeless/sqs-python”, a special Docker image that has been built-in with an SQS consumer.

Our Twitter function defined in the function object above is loaded inside our custom runtime. When an SQS message is received, the payload is passed to the function, which sends it to Twitter:

Putting it together

We can head over to the AWS SQS console and use the debugging UI to send an SQS message. See the snaphshot below:

SQS

As soon as the message is sent, you can check the timeline of your Twitter account and you will see your Tweet. 🙂

Sebastien Goasguen

Conclusions

The sky is the limit here, and while I did not write about the most basic hello world example I hope that you see the potential of Kubeless.

Check out the code on GitHub. We would love to get help. This is 100% open source, with no commercialization plans.

There are more interesting function examples at https://github.com/kubeless/functions

Do you think you can beat this Sweet post?

If so, you may have what it takes to become a Sweetcode contributor... Learn More.

Sebastien Goasguen is a 20-year open source veteran. A member of the Apache Software Foundation, he spent many years in R&D on High Performance and Grid computing before diving into the container world. He is the founder of Skippbox, a Kubernetes startup acquired by Bitnami, where he currently serves as Senior Director of Cloud Technologies. He has been working on Kubernetes since the early days in 2014. Sebastien is the author of the O’Reilly Docker Cookbook and the Kubernetes Cookbook.


Discussion

Click on a tab to select how you'd like to leave your comment

Leave a Comment

Your email address will not be published. Required fields are marked *

Menu