Guide to Store & Manage Secrets in Docker

Nonetheless, today’s typical developer is expected to know the fundamentals of containerizing their apps. Docker is the most widely used containerization tool.

There are specific Docker ideas that most developers overlook despite the growing need for containers. Among these ideas is the handling of private information in containers. This can be seen in the fact that most developers hardcode private information into their Dockerfiles or

environment variables, such as API keys, database credentials, and other secrets.

This is not recommended as it may result in security flaws. The best practices for handling sensitive data in Docker containers will be walked through in this post. It will teach you how to secure your applications using Docker’s built-in secrets management capability. By the end of this tutorial, you will understand how to safely transfer sensitive data to your containers.

Required Conditions

There will be numerous real-world examples and helpful demos in this text.

You will need the following to follow along:

• A fundamental knowledge of containerization, particularly Docker.
• A PC that has Docker loaded. Install Docker on your computer by following the official installation instructions.
• A fundamental understanding of the ideas behind container orchestration (optional for the Docker Swarm portion).
• The installation of Docker Compose on your system. To install Docker Compose, follow the official installation instructions.

How Do Docker Secrets Work?

A feature of Docker containers called “Docker secrets” enables the safe and secure use and storage of sensitive data. SSH keys, API keys, database credentials, or any other information that shouldn’t be disclosed to anyone with container access could be considered sensitive data.

Data sent between Docker containers is not encrypted or protected by default, making it possible for someone with network access to intercept and read the data. Docker secrets are added to the mix in order to give your data the security it needs.

Why Are Docker Secrets Needed?

Suppose one of your secrets is your TLS certificates, which are kept in plaintext in your Dockerfile or Docker image. This implies that this secret is easily accessible to anyone who can access your Dockerfile or Docker image. You require Docker secrets for this reason.

In addition to its primary security purpose, Docker secrets are necessary for additional purposes as well.

Among these explanations are the following:

• Greater Security Than Environment Variables: In general, it is thought that Docker secrets offer a more secure way to store sensitive data than environment variables. Only containers given permission to access them can access the virtual filesystem that Docker uses to store its secrets. Conversely, environment variables are accessible to all container users, including processes that are operating inside of it.
• One Storage Location: All containers operating on the Docker host can access the one location where Docker secrets are kept. Managing secrets across several containers is made more accessible by this, which is typically the case when using Docker Swarm or Docker Compose.
• Ease Of Management: Using Docker secrets isn’t tricky despite not being used as frequently. Using and managing Docker secret with the Docker CLI is simple.
• Distinguishing Between Containers and Secrets: Suppose you have test, local, and production environments, among others. Your containers don’t need to know the exact credentials used in each environment if you utilize Docker secrets. Instead, they ignore the underlying details in favor of a constant hidden name. Because of its adaptability, you can install the same container image without making any changes in different situations.

Example of Docker Secrets

Let’s now examine a real-world application of Docker secrets. First, we will create a secret and then demonstrate how to use it inside a service.

Let’s start by making a new secret with a password:

$ echo “mypassword” | docker secret create my_secret –

The use of the — at the end of the command instructs Docker to read the secret’s value from standard input rather than a file. This command will create a new secret called my_secret, containing the password “password.” Let’s now develop an essential service that utilizes this secret. We’ll use the official NGINX image, and the password for basic authentication will be our secret:

$ docker service create –name nginx –secret my_secret nginx:latest

Using the standard NGINX image, we create a new service called nginx with this command. Additionally, we affix our previously generated secret, my_secret, to the function. The secret can now be accessed by the NGINX container through a file at /run/secrets/my_secret. To use the secret password for simple authentication, we can make a reference to this file in our nginx settings. Remember that the secret is exposed as a file rather than as an environment variable. This method adds another degree of protection because logs or process listings may reveal environment variables.

How Do My Docker Secrets Get Visible?

To see a list of your Docker secrets, run the ls command (docker secret). This command will show a table including the names, creation dates, and secret IDs. This is an illustration of the command:

$ docker secret ls

The output will look something like this:

ID                          NAME                CREATED             UPDATED

vqnz6btp7gru0z6gk8zvtey6k   my_secret           2minutes ago       2minutes ago

But the list of secrets and their information are the only things displayed by the Docker secret ls command. The actual contents of the secrets are not displayed. Use the docker secret inspect command, followed by the secret’s name or ID, to examine a secret and see its metadata in more detail:

$ Docker secret inspect my_secret

The secret’s accurate content will remain hidden even after this command returns the metadata in a JSON format. Since Docker secrets are meant to be secure, it would be counterproductive to reveal their content via CLI instructions. Instead, a secret’s content can be accessed by attaching it to a service in your Docker Swarm; the secret will then be accessible as a file within that service’s containers.

Optimal Methods for Docker Secrets

When utilizing Docker secrets, bear in mind the following recommended practices:

• Restrict access to secrets: Make sure that a secret is only available to the services that need it. One way to achieve this is to create a service, like in the previous example, and only attach the secrets that are required.
• When you need to modify the values of secrets, you can make new versions of them by using version control. This method makes it easier to stay on top of changes and quickly return to a previous version when needed.
• Steer clear of hardcoding secrets: Although it may be alluring, hardcoding secrets into your application’s source code or configuration files is not a good idea. Alternatively, you can store sensitive data in Docker secrets and utilize them as references in your application.
• Rotate secrets frequently: To reduce the possibility of unwanted access, rotate your secrets frequently. Create a procedure for updating services that utilize secrets and rotating them. This could entail generating new secrets, updating services to use them, and expunging previous secrets.
• Never store secrets in version control systems: Make sure that your version control system (such as Git) is never used to store secrets. Put secrets-containing files in your.gitignore file or something similar, and handle and store them safely using a secrets management system like Docker secrets.
• Employ robust, distinct secrets: Create robust, distinct secrets for any service or component that needs them. Steer clear of sharing the same secret with more than one service, as this raises the possibility of a breach.
•  At rest, encrypt secrets: When secrets are kept in the Swarm, Docker secrets are automatically encrypted. Make careful to encrypt your secrets while they’re at rest if you utilize a different kind of secret management software.

How to Handle Secrets in Docker

You can manage secrets with a collection of subcommands provided by the docker secret command. Some of the most popular subcommands are listed below:

As can be observed, the command “docker secret ls” provides a list of all the secrets kept on the Docker host.

Docker secret inspects: This command offers comprehensive details on a particular secret. It allows you to access a secret’s metadata, including its ID, name, and creation date.

Docker private examine my_secret

docker secret inspect my_secret

[

{

“ID”: “kotjiwncv3kp8uyyj8d7kzesn”,

“Version”: {

“Index”: 11

},

“CreatedAt”: “2024-03-19T12:07:44.93980171Z”,

“UpdatedAt”: “2024-03-19T12:07:44.93980171Z”,

“Spec”: {

“Name”: “my_secret”,

“Labels”: {}

}

}

]

To remove a secret from the Docker host, execute the following command: docker secret rm. By providing the secret ID or name as an argument, you can utilize it to erase a secret.

docker secret rm my_secret

my_secret

• The secret known as my_secret will be deleted from the Docker host following the execution of this command.

Docker Swarm Integration with Docker Secrets

Services operating on the same Docker Swarm are the only ones with access to Docker secrets. This implies that only services included in a Docker Swarm are eligible to use Docker secrets.

For those unfamiliar, Docker Swarm is a platform for container orchestration that lets you oversee a group of Docker servers.

You must first establish a secret and provide service access to it in order to use Docker secrets with Docker Swarm. To accomplish this, modify the docker service creation command to include the –secret parameter.

The example that follows will build a redis service on Docker Swarm that has a secret.

• Set up a cluster for Docker Swarm. Should you haven’t done so previously, execute the subsequent command:

docker swarm init

• Use the docker secret create command to generate a secret:

echo “mysecretpassword” | docker secret create redis_password –

qg8cpym1q5v96jg67qkj4q5rc

• Establish the Redis service and use the –secret flag to give it access to the secret:

docker service create –name redis –secret redis_password redis:alpine

p3rmff7x5ta4gi4804cp9suj3

overall progress: 1 out of 1 tasks

1/1: running   [==================================================>]

verify: Service converged

You are establishing a redis service and giving it access to the redis_password secret with this command. The service is created using the redis: alpine image. This command creates the redis service and grants it access to the redis_password secret. The application will access the secret once installed within the service’s containers.

• Verify the redis service is operational:

docker service ps redis

• The following command can be used to check if the secret is mounted on the redis container:

docker container exec $(docker ps –filter name=redis -q) ls -l /run/secrets

total 4

-r–r–r–    1 root     root            16 Mar 19 12:51 redis_password

• Use the following command to prevent the redis service from accessing the secret:

docker container exec $(docker ps –filter name=redis -q) ls -l /run/secrets

ls: /run/secrets: No such file or directory

The redis service won’t be able to access the redis_password secret after executing this command.

• Use the command from step 5 to verify the service is no longer able to access the secret.

docker container exec $(docker ps –filter name=redis -q) ls -l /run/secrets

ls: /run/secrets: No such file or directory

There is no such file or directory ls: /run/secrets

• After completing the demo, use the following command to end the service:

docker service rm redis

redis

Combining Docker Compose with Docker Secrets

As was previously noted, Docker compose enables you to run several containers simultaneously. Still, Docker secrets can only be used on multiple containers. This implies that Docker Compose and Docker secrets can be used together.

When using Docker secrets with Docker Compose, there are two syntaxes available. The long and short syntaxes are these. Whereas the short syntax just allows you to specify the secret’s name, the extended syntax will enable you to define many attributes of the secret.

In the following demo, Docker composes will be used to apply a secret to a Redis service using the short syntax.

• Make the file or files that will hold your secret value. You can save the hidden value in this file, which you can call my_secret.txt.

echo “mysecretpassword” > my_secret.txt

• Produce a docker-compose.yml file containing the information below:

version: “3.9”

services:

redis:

image: redis:alpine

secrets:

– my_secret

secrets:

my_secret:

file: ./my_secret.txt

You are establishing a service called redis in this file and giving it access to my private secret. The service is created using the Redis: alpine image. The value is read from the my_secret.txt file and used to generate the my_secret secret using the file attribute.

• To launch the Redis service, type the following command:

Docker-compose up -d

This command will build the Redis service and allow access to the my_secret secret. The application will have access to the secret once installed within the service’s containers.

• Verify that redis is operational:

docker-compose ps

Name                   Command               State   Ports

—————————————————————

ubuntu_redis_1   docker-entrypoint.sh redis …   Up

Name Command State Ports

—————————————————

redis docker-entrypoint.sh ubuntu_redis_1… Raising

• Using the same command from the Docker Swarm section, you can check if the secret is mounted on the Redis container.

docker container exec $(docker ps –filter name=redis -q) ls -l /run/secrets

total 4

-rw-rw-r–    1 1000     redis           17 Mar 19 18:12 my_secret

Now that the redis secret is mounted into the redis container, the application can access it.

Note: SemaphoreCI also leverages secrets in your CI/CD pipelines to manage sensitive data much more quickly. You can specify your secret using a YAML file, the CLI, or the UI.

Here on SemaphoreCI, you can see how to manage your secrets.

In summary

The primary lesson to be learned from this paper is the importance of secrets management in container security. Using Docker secrets, you may safely handle and store sensitive data in your containers. The inner workings of Docker secrets, their management, and how to incorporate them into Docker Swarm and Docker Compose are all covered in detail in this article. You can safely store and utilize your secrets in your applications if you follow the examples.

Emblem