Getting Started with HashiCorp Vault

10779 VIEWS

6 years ago

Vault is one of the many open source products available from HashiCorp that allows companies to automate their infrastructure management using the Infrastructure as Code philosophy and secrets management. HashiCorp also has commercial offerings to provide enhanced support in multi-cloud and mission-critical situations. This post will focus on getting up and running with the open source version of Vault in a multi-stage deployment pipeline.

Introduction

Every application has secrets that it uses to access resources. These vary by environment, and can be everything from API keys to certificate passphrases or passwords. As long as multiple environments have existed, there has been a need to manage these secrets in a secure way so that only the processes and people which need to access them can.

Installing

The first step in installing Vault is to download the correct package for your platform. Vault is written in Go. Once you download the package, it will contain a single binary file. That single file can be put anywhere you want (ex: Applications on a Mac or /usr/local/bin on Linux).

Running the Server

I am running the server in dev mode to show how easy it can be to use. To start the server in dev mode, simply run “/path/to/vault server -dev”.

The output will look like:

 
[root@rhel ~]# vault server -dev
==> Vault server configuration:

                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", tls: "disabled")
               Log Level: 
                   Mlock: supported: true, enabled: false
        Redirect Address: http://127.0.0.1:8200
                 Storage: inmem
                 Version: Vault v0.9.1
             Version Sha: 87b6919dea55da61d7cd444b2442cabb8ede8ab1

==> WARNING: Dev mode is enabled!

In this mode, Vault is completely in-memory and unsealed.
Vault is configured to only have a single unseal key. The root
token has already been authenticated with the CLI, so you can
immediately begin using the Vault CLI.

The only step you need to take is to set the following
environment variables:

    export VAULT_ADDR='http://127.0.0.1:8200'

The unseal key and root token are reproduced below in case you
want to seal/unseal the Vault or play with authentication.

Unseal Key: wsyNFZfkeBfLk7XxQ4oHJJRAHHDjXLrHMoyoOLUhB2Y=
Root Token: 4e431ad8-2f94-ba38-63ec-556829fe7e56

==> Vault server started! Log data will stream in below:

For production use, dev mode is not a good idea, because it unseals the data automatically on startup. In production, you would want to do a full deployment setup that includes a proper storage backend, TLS support, and the full set of steps to seal and unseal the data for maximum security.

How to Use the Command Line Interface (CLI)

Before you can execute any commands, you need to set the VAULT_ADDR environment variable. The exact command to run it is in the output of the server startup command, which is most often “export VAULT_ADDR=’http://127.0.0.1:8200’”.

After running that command, you can execute other commands, like status.

 [root@rhel ~]# export VAULT_ADDR='http://127.0.0.1:8200'
[root@rhel ~]# vault status
Seal Type: shamir
Sealed: false
Key Shares: 1
Key Threshold: 1
Unseal Progress: 0
Unseal Nonce: 
Version: 0.9.1
Cluster Name: vault-cluster-ab4f3d86
Cluster ID: c8468659-ff05-b365-3baa-2b7518114556

High-Availability Enabled: false

Creating a Policy via the CLI

We are going to create a new policy that will be used to allow access to a subset of secrets stored within the Vault. There is a default root policy that grants access to everything, but do you really trust people with full access?

First let’s create the HCL file (HCL is 100% compatible with JSON):

 [root@rhel ~]# cat sweetcode-policy.hcl 
path "sweetcode/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

Now let’s add the policy to the Vault server:

 [root@rhel ~]# vault write sys/policy/sweetcode-policy [email protected]
Success! Data written to: sys/policy/sweetcode-policy
[root@rhel ~]# vault read sys/policy/sweetcode-policy
Key  	Value
---  	-----
name 	sweetcode-policy
rules	path "secret/sweetcode/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

Creating an Access Token via the CLI

It isn’t exactly best practice to give any client that wants to access Vault the root access token, so as a first step before using clients to add and remove tokens, we will also create the token using the new policy so it can only access the secrets we want to allow.

Tokens can be created to expire automatically after a set period of time or be valid indefinitely. Depending on your use case, this may matter. For the purposes of this demo, I’ll create one using the default expiration of one month.

 [root@rhel ~]# vault token-create -policy=sweetcode-policy
Key            	Value
---            	-----
token          	d4f1a758-9e5c-b37d-3aa1-eb813112a6c1
token_accessor 	3b251bfb-ca86-5276-6eb6-a6a96f84ae9c
token_duration 	768h0m0s
token_renewable	true
token_policies 	[default sweetcode-policy]

Accessing the REST API

The secret to using the REST API from the client of your choice is having a custom header called “X-Vault-Token” with the token as its value, so in curl it would look like:

 [root@rhel ~]# curl \
    -H "X-Vault-Token: d4f1a758-9e5c-b37d-3aa1-eb813112a6c1" \
    -X GET \
    http://localhost:8200/v1/sys/init
{"initialized":true}

Adding a Secret via REST API

Just to show that the policy we created works, this is the result of trying to create a secret in an unapproved location:

 # curl \
    -H "X-Vault-Token: d4f1a758-9e5c-b37d-3aa1-eb813112a6c1" \
    -X POST \
    -H "Content-Type: application/json" \
    -d '{"value":"this is a secret"}' \
    http://127.0.0.1:8200/v1/secret/phrase
{"errors":["permission denied"]}

And when it is successful, there is no error returned, but it shows up in the list.

 # curl \
    -H "X-Vault-Token: d4f1a758-9e5c-b37d-3aa1-eb813112a6c1" \
    -X POST \
    -H "Content-Type: application/json" \
    -d '{"value":"this is a secret"}' \
    http://127.0.0.1:8200/v1/secret/sweetcode/phrase
# curl \
    -H "X-Vault-Token: d4f1a758-9e5c-b37d-3aa1-eb813112a6c1" \
    -X GET \
    http://localhost:8200/v1/secret/sweetcode/?list=true
{"request_id":"5237b1b7-af59-1611-8e62-da76a07dbfdf","lease_id":"","renewable":false,"lease_duration":0,"data":{"keys":["phrase"]},"wrap_info":null,"warnings":null,"auth":null}

Reading a Secret via REST API

Almost any language or automation tool can read a JSON response and use the value as part of its runtime (from Graddle to Java to Python), so using the REST API is the best way to retrieve the secrets you need.

 # curl \
    -H "X-Vault-Token: d4f1a758-9e5c-b37d-3aa1-eb813112a6c1" \
    -X GET \
http://localhost:8200/v1/secret/sweetcode/phrase
{"request_id":"73b2bdd1-cf27-2bbd-5dcf-6bab5a13e539","lease_id":"","renewable":false,"lease_duration":2764800,"data":{"value":"this is a secret"},"wrap_info":null,"warnings":null,"auth":null}

Integrating with Automation Tools

Along with the REST API’s programmatic use, the large community of users of HashiCorp Vault has led to the development of modules for the most popular automation frameworks.

Some quick examples are:

1. Chef’s vault-ruby provided by HashiCorp
2. Seth Vargo’s Puppet module
3. Johan Haals’ Ansible module

Conclusion

While there may be other products within the Continuous Integration and Infrastructure as Code spaces which allow secrets to be encoded and decoded as required, Vault by HashiCorp stands alone in its ability to protect and audit access to the secrets it protects while being able to be integrated with multiple toolsets via both a command line and modern REST interface.