One of the most important components of building secure software is setting up centralized secrets management. Not doing so will lead to a problem called secret sprawl which could in turn lead to data breaches, leaks of sensitive credentials, and low developer efficiency.
Depending on organizations' use cases, there might be a number of different options to choose from:
Cross-cloud secrets management solutions.
Cloud-specific secret managers.
Open source secret management projects.
In-house built secret management solutions.
In this article, we are going to discuss advantages and drawbacks of each of these approaches as well as the most popular secrets management tools within each of these categories.
Cross-cloud secret managers
Cross-platform secrets management software is by definition not tied to any specific cloud. Instead, these solutions, as is the case with Infisical and Vault, can be self-hosted on any infrastructure (or used as a managed service). Some of these tools, due to their nature of not being open source, as is the case for Doppler, only have the ability to provide a managed service.
1. Infisical
Infisical is an open source secret management platform. It provides an end-to-end set of tools that cover all aspects of secret management: from secure version-controled secret storage, to secret rotation, to integrations across infrastructure, to secret scanning and leak prevention.
Some of the main Infisical's features include:
Automatic integrations across infrastructure tools (e.g., Docker, Kubernetes, Terraform) and 3rd-party services (e.g., GitHub Actions, Vercel, CircleCI)
Secret Workflows: a set of tools to facilitate developer on/off--boarding as well as compliant changes to secrets in various environments.
Dashboard UI as a central tool: a single place in which developers can access/modify secrets (depending on their permissions) while their managers are able to set specific roles, monitor activity, analyze audit logs, and more.
Infisical Radar: continuous secret scanning/monitoring and leak prevention to git.
Infisical CLI allows developers to very easily inject secrets into application processes as environment variables (more approaches supported through API, SDKs, etc).
Summary
Pros:
Infisical provides an end-to-end secrets management platform with a range of secret management features.
Infisical supports highly granular Access Controls, Audit Logs, and Compliance Policies.
Automatic integrations and secret referencing functionalities help establish a single source of truth for application secrets and fight secret sprawl.
Infisical is an open source product available under the MIT license, which means organizations can self-host it on their own infrastructure. At the same time, Infisical offers a managed cloud product for teams that don't want the overhead of running the software themselves.
Cons:
- For on-prem installations, Infisical backend DB options are only limited to Postgres and MongoDB.
2. HashiCorp Vault
HashiCorp Vault was one of the first secret management solutions out there. It is a thorough tool for securely storing and accessing secrets and other sensitive data (e.g., access tokens, API keys, encryption keys, password, certificates).
Vault's main way to access secrets is through the API based on security policies set up in advance. In addition it allows companies to maintain tight and very granular access controls and keep very thorough audit logs.
It provides a web interface for secret management, but it is important to keep in mind that some of Vault's features are not available in its Web UI (e.g., some of the dynamic secrets functionalities).
(Re)starting the HCP Vault always involves one or more operators to unseal it. HashiCorp Vault mainly works with token system. Each token is provided to a corresponding policy that may limit the available actions and the paths. Some other important features of HCP Vault are:
Vault is able to generate secrets on-demand (dynamically) for some operations (e.g., PostgreSQL database).
HCP Vault is able to set up replication across multiple data centers.
Summary
Pros:
Vault has one of the most comprehensive and feature-complete secret management solutions on the market.
HCP Vault integrates well across HashiCorp's product ecosystem.
HashiCorp Vault is source-available under the BSL license.
Cons:
HashiCorp Vault's pricing is very convoluted and is heavily focused on large-scale enterprises.
HashiCorp Vault is entirely focused on the platform engineering teams. This makes it fairly hard to understand at first, while at the same time it keeps developers out of the secret management workflows. In turn, this may lead to having to build developer tooling on top of Vault in future.
HashiCorp Vault is not open source and some companies might have their legal teams not approve HashiCorp's BSL license.
Vault's free source-available version comes with a very high setup and maintenance costs (in terms of platform engineering team's time but also compute resources).
3. Doppler
Compared to Infisical or Vault, Doppler is a fully closed-source secrets management solution. Similar to Infisical, Doppler is focused on developers and has a friendly UI dashboard at its heart. Within the dashboard developers are able to modify secret values, set permissions, monitor access logs, and more.
Doppler is generally supporting static secrets, but recently it has also added some dynamic secrets functionality (which is currently still in the beta stage of testing).
In addition, Doppler also provides a secret sharing product which allows developer to send secrets (e.g., password or any sensitive information) securely in between each other with just a few clicks.
Summary
Pros:
Doppler provides a CLI which is very easy to set up and get going with injecting secrets as environment variables.
Doppler provides an extensive list of automatic integrations with 3rd-party services and infrastructure tools (e.g., Docker, Kubernetes).
Cons:
Doppler comes only as a managed service; it does not have an open source or self-hosted option which might not satisfy compliance requirements of many engineering teams.
Doppler is entirely focusing on injecting secrets as environment variables and does not offer SDKs for any programming language.
Doppler's organizational structure for secrets is very flat (1 layer only) which makes it unusable for any large-scale software.
Doppler recently significantly reduced the capacity of its free tier, which made the solution less attractive for small- and mid-sized teams.
Cloud-specific secret managers
Cloud-specific services are generally very barebone tools that shold mainly only be considered if your applications are entirely built on top of their infrastructure ecosystem.
1. AWS Secret Manager
If your applications are entirely under the AWS stack, then AWS Secrets manager is one of the solutions that you should look into. AWS Secrets Manager is ultimately a very barebone but resilient solution that lets you rotate, manage, retrieve database credentials, API keys, and passwords with ease.
Main features of AWS Secrets Manager include comprehensive secrets manager APIs and secrets encryption.
2. GCP Secret Manager
If Google Cloud Platform is your cloud provider, then GCP Secret Manager could be an option for you. Some of the features that GCP Secret Manager includes are secret storage with full-fledged access controls, audit logs, secret versioning and rotation capabilities.
In addition, GCP Secret manager supports of a number of ways to access secrets through APIs, client libraries for various programming languages, etc.
3. Azure Key Vault
If your software and applications run on Azure, Microsoft provides a managed service called Azure Key Vault to ensure that all your secrets are managed in a single repository. This includes keys, certificates, connection strings, and passwords. In addition, you can also create and import encryption keys into Azure Key Vault and automate SSL/TLS certificate renewal.
In addition, you can automate SSL/TLS certificate renewal and rotation so that your connection always remains secure.
Pros and cons of cloud-specific secret managers:
Pros:
It could be a good solution if your application are running entirely on the ecosystem of one of the Big Clouds (AWS, GCP, Azure).
If you are part of a large organization, it is much easier to get compliance approval and budget allocation for the Big Cloud products.
Cons:
Developer experience on these platforms is below average (Azure being the worst of the three, according to the reviews online).
There is a higher level of vendor lock-in that could become a big issue if you eventually want to go multi-cloud or switch your cloud provider.
These miss all the functionalities such as automatic integrations with 3rd party tools, secret referencing, CLI to inject secrets, etc.
Open source projects
Another option available is choosing one of the open source secret management alternatives. The main benefit of these secret management systems is that they are much easier to receive compliance approvals for as they can be self-hosted on your own infrastructure – although this benefit can also be achieved with HCP Vault and Infisical.
Another benefit that companies think of is the ability to use these solution for free. However, if this is the concern, it should be considered along with the constant time investment that maintaining a solution of such importance could take away from the engineering team.
On the other hand, the ultimate problem with non-profit open source software is that there is no party who bears responsibility over maintaining it, fixing relevant bugs, and releasing security patches. In addition, none of these products have a managed service available for them (in other words, the absolutely have to be self-hosted).
1. SOPS
SOPS, which is created by Mozilla, is one of the popular secret management tools available.
It takes a fairly different approach compared to the other tools; SOPS allows users to store secrets securely in git repositories. In addition, developers are able to use Mozilla’s SOPS CLI to encrypt Kubernetes secrets with AWS KMS, GCP KMS, Azure Key Vault, and OpenPGP. SOPS supports file formats like .yaml
, .json
, .env
, and .ini
.
SOPS is based on a client-server model for encryption and decryption of the data keys. In the default setting, it runs a local key service within the process. The client sends encryption and decryption requests to the key service using protocol buffers and gRPC.
It should be noted that the key service connection currently does not provide authentication or encryption capabilities. To ensure proper security levels, it is highly recommended to authenticate and encrypt the connection through other such as SSH tunnels.
Finally, SOPS is able to generate audit logs for files in the specified environments. It records the activity (e.g., timestamps, usernames, file data) in a PostgreSQL database.
Pros:
- SOPS is a fully open source tool that can relatively easily adopted by an organization of any size.
Cons:
Due to budget cuts in Mozilla, project is currently not being properly maintained which could lead non-fixed bugs and security issues. It should be noted that the project is currently under a transfer to a different organization which might make its future more clear.
Secrets are stored in your code which is not always desired and/or convenient.
SOPS does not provide a web interface which has now become a standard for a great developer experience.
2. Confidant
Confidant is a secret management tool developed at Lyft. At its core, it stores secrets in DynamoDB while generating a unique KMS data key for every change in secrets (using Fernet symmetric authenticated cryptography).
In addition, Confidant provides a web interface built on AngularJS which can be used to perform various secret actions. Some of the other Confidant's features include:
At-rest encryption of secrets.
Secret versioning.
KMS-based Authentication.
Note: Confidant could be complicated to tie into your infrastructure as it is heavily designed with Lyft's infrastructure in mind.
3. Other alternatives
Overall, there a number of other open source secret manager approaches such as Keywhiz by Square, Chamber by Segment, Knox by Pinterest, etc. A common thread between those is that they were tailored specifically for the infrastructure of the companies where they were developed which might make it hard to use for general use cases.
Building in-house
Depending on the size of your organization, you might want to consider building in-house. There are, however, a few considerations to keep in mind:
It is no surprise that building an in-house solution of any kind (especially in the secret management space) comes with a lot of developer overhead. Building a software that is always up-to-date with the latest 3rd-party APIs and infrastructure setups requires constant thinking from a corresponding team of developers that could be working on something product-related and more important to the core business.
Companies/tools mentioned in this article spend 90% of their time thinking about compliance and security of their services. This helps prevent the absolute majority of "all the things that could go wrong." Building a solution in-house unfortunately does not always have the same level of dedication, and oftentimes security of such solutions becomes a second-tier concern over time.
Summary of key considerations for selecting the right tool for you
Ultimately, choosing the right tool to follow the best practices of secret management depends on your needs and requirements.
Only choose cloud-specific secret managers if your stack is built entirely within one of the major clouds' ecosystems (e.g., AWS, GCP, Azure). If your infrastructure is (or will be) multi-cloud in future, then using cloud-specific secret managers might not be the best idea, as using those will also increase the platform lock-in.
If your budget is very tight, then you might want to look into fully open-source projects like Confidant and Chamber. Keep in mind that most of these solutions will not have proper support and product quality as some companies and developers would expect.
If you want to take the most feature-complete solution on the market that is focused around the platform engineering team and you care about self-hosting, then HCP Vault may be the best tool for you.
If you care deeply about developer experience and efficiency of your core/platform engineering teams and you want to have an option to both self-host and use a managed service, then Infisical is likely the right choice for you.