Like a kid with a shiny new toy, I have been waltzing around with Hashicorp’s Vault for a few weeks now. Vault is without a doubt one of Hashicorp’s most complicated and sophisticated tools yet. This post starts with an introduction to Vault and then goes on to explain two of Hashicorp’s recommended authentication backends.
Pre-Vault Ways of Storing Credentials:
- Storing in source code repositories
- Storing in text files (Blimey!)
- Distributing different credentials to different people, making it difficult to keep track of a user and revoke when he or she leaves an organisation
Enter Vault……
Vault is a single place of storage of all your secrets across all your environments, for all of your users. It also makes auditing your secrets easier, and if exposed, you can revoke them from a single point. Using Vault, you can also create dynamic credentials that are time-based (which can hopefully be covered in a different post).
Installing Vault
To bootstrap Vault:
Step 1: Begin by downloading and then unzipping the binary from:
https://www.vaultproject.io/downloads.html
Step 2: Create the Vault configuration file:
vi /etc/vault.conf
storage "consul" {
address = "127.0.0.1:8500"
token = ""
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = "true"
}
Step 3: Add Vault to the PATH and start Vault.
NOTE: Vault starts as a foreground process.
$vault server -config /etc/vault.conf
Initialising and Unsealing Vault
To start using Vault, you need to first initialise it and then unseal it. During initialisation, you generate keys. These consist of the master key, split keys, and a root token. The master key is typically split into multiple keys and shared among many users so that no one person is in charge of the master key. Keys are split using Shamir’s Secret Sharing Algorithm.
Initialising Vault allows you to specify the total number of key shares and the number of keys (key threshold) required to unseal the vault. Unsealing supplies the keys to Vault so Vault can decrypt encrypted data and start serving clients.
Initialise Vault with:
$ export VAULT_ADDR=http://localhost:8200 $ vault init -key-shares=5 -key-threshold=2 $ vault status -address=http://localhost:8200 $ vault init Unseal Key 1: puqJYaAnei00qbGW+OD+vIn1d6QCGl0pR/QxBqRmwLh1 Unseal Key 2: <> . . Initial Root Token: 3bb39c4f-016f-30e2-6910-d566f5b8d82e
Vault initialized with 5 keys and a key threshold of 2. Please
securely distribute the above keys. When the vault is re-sealed,
restarted, or stopped, you must provide at least 3 of these keys
to unseal it again.
Vault does not store the master key. Without at least 3 keys,
your vault will remain permanently sealed.
$ vault unseal Key (will be hidden): Sealed: false Key Shares: 5 Key Threshold: 2 Unseal Progress: 0 Unseal Nonce: $ export VAULT_TOKEN=3bb39c4f-016f-30e2-6910-d566f5b8d82e
Now that Vault is unsealed, you just need to log in to start using it!
NOTE: Using a root token to authenticate the user is not best practice, but this is a post to get started, so….
$ vault auth $VAULT_TOKEN
Successfully authenticated! You are now logged in.
Console UI showing the unsealed status of Vault
Secrets Management Using Vault
Vault authentication happens only using tokens. Each token is assigned a policy that decides its action and path. Policies apply rules based on path matching. This is one of the major concepts behind Vault’s authentication process.
Now, based on the needs of your organisation and application, you can either manually assign these tokens or use one of Hashicorp’s recommended authentication backends. Below is a discussion on two of Hashicorp’s authentication backends: token-based and using the AWS backend.
Token-based Authentication
You can create new tokens using:
$ vault token-create
Key Value
--- -----
token f4cdcc32-7a05-2a6e-1067-50d455e991e3
token_accessor 9f86f198-8010-913b-2227-c86a7253b82a
token_duration 0s
token_renewable false
token_policies [root]
$ vi policy.hcl
path "secrets/project/dev" {
policy = "read"
}
$ vault policy-write demo policy.hcl
Policy 'demo' written.
$ vault token-create --policy=demo
Key Value
--- -----
token b8a406ee-b8b8-dedb-1fcd-772568e12dfb
token_accessor 88186cca-1302-4e1e-94a7-46c028fc2802
token_duration 768h0m0s
token_renewable true
token_policies [default demo]
$ vault auth b8a406ee-b8b8-dedb-1fcd-772568e12dfb
Successfully authenticated! You are now logged in.
token: b8a406ee-b8b8-dedb-1fcd-772568e12dfb
token_duration: 2764553
token_policies: [default demo]
$ vault write secrets/project/dev pass=P@ssword
Error writing data to secrets/project/dev: Error making API request.
URL: PUT http://127.0.0.1:8200/v1/secrets/project/dev
Code: 403. Errors:
Note: If a token is deleted, you lose all the child tokens along with it, unless you specifically mention the token is an orphan (cliche’) token during creation, using:
$ vault token-create -orphan
AWS Backend
Vault allows for the dynamic creation of AWS IAM credentials with specific lease periods so that the application can either revoke a credential after use, or Vault will automatically delete the IAM credential after the lease expires.
Step 1: Mount the AWS backend:
$ vault mount aws Successfully mounted 'aws' at 'aws'! $ vault path-help aws/roles/
Step: 2 Create an IAM policy:
$ vault list aws/roles
No value found at aws/roles/
$ vi policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1426528957000",
"Effect": "Allow",
"Action": [
"ec2:*"
],
"Resource": [
"*"
]
}
]
}
Step 3: Create an AWS role based on the IAM role created in Step 2. (The example below also demonstrates how to make sure these credentials remain time-based by attaching a lease to the role.)
$ vault write aws/roles/deploy [email protected] Success! Data written to: aws/roles/deploy $ vault list aws/roles Keys ---- deploy $ vault write aws/config/lease lease=3h lease_max=24h Success! Data written to: aws/config/lease $ vault read aws/creds/deploy Key Value --- ----- lease_id aws/creds/deploy/59e20390-992b-f719-7aa0-b5213d37572b lease_duration 3h0m0s lease_renewable true access_key xxxxxxxxxxxxxxxxxxxxx secret_key xxxxxxxxxxxxxxxxxxxxx security_token
Step 4: Storing your AWS credentials with Vault—This can either be done by IAM auth method, or you can register the keys as shown below:
$ vault read aws/creds/deploy
Error reading aws/creds/deploy: Error making API request.
URL: GET http://127.0.0.1:8200/v1/aws/creds/deployCode: 400.
Errors:* Error creating IAM user: NoCredentialProviders:
no valid providers in chain. Deprecated. For verbose messaging
see aws.Config.CredentialsChainVerboseErrors
$ vault write aws/config/root access_key=xxxxxxx secret_key=xxxxxxxx
curl $VAULT_ADDR/v1/aws/creds/deploy \
--request GET \
--header "X-Vault-Token: $VAULT_TOKEN"
{"request_id":"ec074299-ffd5-286f-9e9a-48ba86159780",
"lease_id":"aws/creds/deploy/7e0901fb-d946-d262-a98c-c88651e3ae51",
"renewable":true,
"lease_duration":10800,
"data":{"access_key":"xxxxxxxxxxxxxxx",
"secret_key":"xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"security_token":null},
"wrap_info":null,
"warnings":null,
"auth":null}
Here are a few helper scripts to get you started:
https://github.com/dimpledalby07/Vault-Helper-Scripts
