Sensitive variables in code for local environment
The problem
There is a common bad practice of storing sensitive variables in code. Quite often it leads to secret leakage. By secrets I mean — API tokens, SSH keys, private keys and others. Some careless developers hardcore them and push it into common repository. In result the secret can be found in commits, branches, merge request’s and other places. This case is especially painful if the secret related to live environment. Since the secret compromised, DevOps team has to spend time to recreate another one and distribute it in proper manner. To avoid this problem, there are many different ways. But the most difficult is to wean developers from the habit to store secrets into source control system.
Enterprise vault out of scope
The general solution is a dedicated Vault in your infrastructure. It could be Hashicorp Vault, CyberArk Conjur, AWS Secret Manager or other. There are pros and cons for each tool. Centralized secret management is the best solution but this time it’s out of scope.
How to store secrets in your local environment?
1. Secrets into gitignore file.
2. Secrets in environment variables
3. Encrypt secrets
4. Secrets into password manager
Let’s have a look at benefits and drawbacks of each method.
1. Secrets into gitignore file
It’s the most common method to store sensitive data apart from version control system. You can find a lot of records about secrets in gitignore files in GitHub. This method is a typical way cause it’s simple. The main drawback is that secrets are being kept in plaintext. If your host has been compromised, all these secrets can leak. Everybody knows about risks of passwords in txt, but it still surprisingly “ok” to keep API keys and tokens in gitignore. However, it’s better than secrets hardcoding. The first level of security concern.
2. Secrets in environment variables
Environment variables are being used in every OS. Initially, they were designed for storing configuration data. But nowadays, a sensitive data can be found there. Technically it’s the same level of security as gitignore. The main layer of protection is access control in your OS. You can grant a dedicated permissions for your application to query the specific environment variable. At the same time, you can remove this permission from your regular account. In this way, the secret will not be compromised if an attacker breaks your account. This permission separation will provide you an extra level of protection by using OS access control features.
Another difference from gitignore secrets is low risk of human mistake. If you define a sensitive variable in env var and provide restrictive permissions, you will never send it to repository by mistake.
3. Encrypt secrets
There are built-in solutions for secrets in development platforms or IaC tools. Like git-secrets or Ansible Vault. They encrypt the sensitive values before send it to source control system. In source codes it looks like cypher strings. Seems simple and secure. It’s convenient but in this case key protection solely relies on encryption. Any mistake in encryption algorithm can reveal all your keys at once.
4. Secrets into password manager
At first glance it’s the best solution. You create a protected vault in some cloud and use it as central storage for secrets. To get access to the vault, you need to use “master password”. After this, you or your app will be able to query each secret inside the unlocked vault. The master password can be stored in environment variable (less secure) or hardware token like Yubikey (more secure).
However, not every password manager can be used this way. These tools are mainly focused on UI usage. So, API calling isn’t a typical feature for them. As sample, below I provide the scheme for Keepass + Yubikey + Dropbox for Python application.
Key links:
> Password manager — Keepass — https://keepass.info/
> Hardware token — Yubikey — https://www.yubico.com/
> Integration libraries — https://pypi.org/project/pykeepass/ and https://developers.yubico.com/python-yubico/
Optionally, you can set up TOTP (Time-based one-time password) by using external plugins
This approach has many benefits:
1. Your secrets stored centrally in encrypted container
2. Access to secrets enforced. Even if your host were compromised, attacker will need to get access to Dropbox and master password together
3. Easy to use. It’s just enough to unlock your crypto container and all secrets can be easily queried or modified by your application.
The drawbacks of this approach:
1. Your secret management relies on third party libraries. Usually no one performs their audit. It brings risks of backdoor or flaw
2. Technical compatibility. I tested this schema for Python application and used the libraries from the list above. There are no guarantee that it will work for your stack.
3. Service availability under the risk. Need to duplicate the physical key to have a spare for this.
The obvious question that came to mind — why not just keep all secrets into hardware token (Yubikey)? The answer is availability. It’s hard to provide typical key management functions for hardware devices, especially redundancy.
Let’s do a rough threat modelling against this approach
Revealed threats and mitigation
>> Intercepted password — use TOTP and PKCS#11
>> Vulnerability or backdoor in library — do a code review and static analysis of source code. Pay attention to contributors and resolved issues.
>> Leaked variables in case of host compromising — enforce access control and use different permissions for your app and your default account. It should be root.
>> Missed hardware key — have a spare of your hardware keys
>> Access to secrets — use strong encryption algorithm
So, this brief threat analysis showed us that most of potential attack vectors can be covered.
Conclusion
There are many ways to store secrets and invoke them by your application. I listed the most commons here. As usual in security, there is no silver bullet for this issue. We always need to keep in mind corresponding risks and react to them proactively.
