The software development lifecycle is shrinking. It used to be that new versions of source code would be released every month or so. But now, more development teams use a CI/CD model where every change is immediately and automatically integrated, tested and deployed. This ensures the latest version of code is available to the development team and that the newest features are available to the users of the application.
AWS offers a suite of tools for establishing a CI/CD pipeline. Although, for teams looking for a minimal solution, a full pipeline can be built using just four services: Git, S3, CodeCommit, and Lambda.
Establish version control
Having a version control system in place for tracking your code changes is a prerequisite for building a CI/CD pipeline. If you don’t yet have this in place, you can follow the documentation to install Git and initialize your repository. Currently, AWS CodeCommit requires a Git client that supports Git version 1.7.9 or later.
Create a repository
Next, you need to create a repository in AWS CodeCommit, which will connect to your Git client. This is as simple as navigating to the CodeCommit console and clicking “Create Repository” – you need only specify a name and accept the default options. To give CodeCommit access to your code, you’ll need to create an IAM user for it. In the IAM console, add a user with “Programmatic Access” and attach the “AWSCodeCommitFullAccess” policy to it. Then, under the “Security Credentials” tab for this user, you can generate a Git username and password. AWS offers detailed documentation for this process.
Host your application on S3
The latest version of your code will be kept in an S3 bucket. If your application is a static web app, you can then host it directly from S3. Alternatively, you can link other AWS services to your bucket to facilitate many other types of applications. However, once your code gets used, the idea is to have your application linked directly to this bucket so that the most current code is always in production. When you create your S3 bucket, you need to specify a unique name for it, but it’s fine to accept the default parameters.
Automate deployment
AWS includes a tool for continuous integration and deployment called CodePipeline. However, the configuration of this service requires an AWS Serverless Application Model to specify code builds and relies on AWS CloudFormation to manage changesets. For users who just want a simple solution, a single Lambda function can do the job of pushing any newly committed code to an S3-bucket that represents the latest version. This function can then be expanded to include any automatic integration you desire.
Since the Lambda function will need access to an S3-bucket, as well as the CodeCommit repositories, you need to define a new IAM role for it. For this role, you should specify ‘Lambda’ as the service that will use the role, and attach the following policies: AWSLambdaExecute and AWSCodeCommitReadOnly.
Build the Lambda function with Python
Now you’re ready to build the Lambda function. Create a new function in the Lambda console, select Python 3.7 as the language and give it the role you just created. The plan is to have this function triggered whenever a commit is pushed to our repository. To accomplish this, choose ‘CodeCommit’ from the ‘Add triggers’ list when creating the function. This will open the ’Trigger configuration’ window where you can select the name of the CodeCommit repository you created earlier and select ‘Push to existing branch’ as the event to trigger the function.
You’re ready now to script a function that ties it all together. Whenever you do a git push, your CodeCommit repository will mirror this push and will trigger the Lambda function. The function then needs to copy all the files in the newly-updated CodeCommit repo and upload them to your S3-bucket. To accomplish this, you can use codecommitToS3, a Python script by Michael Niedermayr. This code can be used in your Lambda function but you’ll need to define the names of the S3-bucket, the CodeCommit repository and the branch and the CodeCommit region as environment variables.
This code gives you the option of specifying a branch that gets copied in its entirety every time the function is triggered (otherwise it only copies new files). If your repository is large, you may need to increase the Timeout value for your function. Currently, the maximum execution time is 300 seconds.
This is an example of a minimal pipeline that enables continuous deployment and integration. But, the Lambda function that copies your new code can be expanded at will to include whatever integration and unit testing makes sense for your application.
A basic CI/CD pipeline with AWS Lambda and Python
These building blocks provide the foundation for a fully robust CI/CD pipeline. If you’re sold on the benefits of CI/CD and want to try it out without a big commitment, you can start here and add to it later, either with your own code or by incorporating other tools such as AWS CodePipeline. Whatever tools you use, making the effort to establish a CI/CD pipeline is well worth it and I hope this example shows that it can be quickly implemented.
