Docker – Setting Memory And CPU Limits

Nowadays, container technologies are very popular. We can see that variety of applications are running as containers. One of the prime reasons for its wide adoption is that it increases computing utilization by sharing and/or isolating its resources. Resource sharing is noteworthy, but an unthoughtful configuration can lead to a noisy neighbor situation.

In computing, a noisy neighbor situation occurs when a process or group of processes over-consumes a host’s resources. This situation negatively impacts other processes running on the same host. It ultimately degrades the overall system’s performance as the host becomes less responsive due to low resources.

This tutorial will discuss how to set memory and CPU limits in Docker to mitigate the noisy neighbor situation.

1. Creating Container Without Limits

By default, Docker doesn’t set any resource constraints on a container. It allows the container to access all system resources. Let’s understand this with an example.

First, display the CPU and memory configuration of the host machine:

$ docker info | grep -iE "CPUs|Memory"

CPUs: 4
Total Memory: 7.714GiB

The above output shows that the host machine has 4 CPUs and 7.714 GiB of memory.

Now, let’s create a new container and observe its resource limits using the docker stats command:

$ docker container run --rm -it -d --name web-server nginx:alpine

$ docker stats web-server --no-stream --format "{{ json . }}" | python3 -m json.tool

{
"BlockIO": "0B / 12.3kB",
"CPUPerc": "0.00%",
"Container": "web-server",
"ID": "2def8ff5e138",
"MemPerc": "0.05%",
"MemUsage": "3.734MiB / 7.714GiB",
"Name": "web-server",
"NetIO": "5.46kB / 0B",
"PIDs": "5"
}

The above output shows that the memory limit is 7.714 GiB which is the same as the host’s memory. The MemUsage field indicates it.

In this example, the output is redirected to the Python interpreter to print it in a pretty format.

2. Setting Resources Limits when Creating a Container

In Docker, we can create a container using the following ways:

• The container run child command
• The docker-compose.

Docker supports setting up resource limits using both of these options.

2.1. Using container run Child Command

The simplest way to set resource limits is using the container run child command. This command allows us to set memory and CPU limits.

2.1.1. Setting Memory Limits

To set a hard memory limit, use the --memory option with the container run child command. Docker doesn’t allow a container to use more than a given amount of user or system memory after setting this limit.

$ docker container run --rm -it -d --name mem-limit-demo --memory=256m nginx:alpine

Now, let’s check its memory limits:

$ docker stats mem-limit-demo --no-stream --format "{{ json . }}" | python3 -m json.tool

{
    "BlockIO": "0B / 12.3kB",
    "CPUPerc": "0.00%",
    "Container": "mem-limit-demo",
    "ID": "b46befb6d196",
    "MemPerc": "1.45%",
    "MemUsage": "3.711MiB / 256MiB",
    "Name": "mem-limit-demo",
    "NetIO": "3.83kB / 0B",
    "PIDs": "5"
}

In the above output, we can see that Docker has applied a memory limit of 256 MiB. The MemUsage field indicates this. The –memory option accepts a positive integer as an argument, followed by a suffix of b, k, m, g, to indicate bytes, kilobytes, megabytes, and gigabytes, respectively.

In addition to this, Docker allows us to set soft memory limits. This option allows a container to use as much memory as it requires unless the kernel detects memory contention. To set soft memory limits, use the –memory-reservation option with the container run child command:

$ docker container run --rm -it -d --name soft-mem-limit-demo --memory=1g --memory-reservation=512m nginx:alpine

In this example, we have set the hard memory limits to 1GiB and the soft memory limits to 512MiB.

We should note that the soft memory limits must be less than the hard ones. Because it is a soft limit and there isn’t a guarantee that the container will not exceed it.

Now, let’s verify the hard memory limits of the container:

$ docker stats soft-mem-limit-demo --no-stream --format "{{ json . }}" | python3 -m json.tool

{
    "BlockIO": "135kB / 12.3kB",
    "CPUPerc": "0.00%",
    "Container": "soft-mem-limit-demo",
    "ID": "9b748ec04b2a",
    "MemPerc": "0.38%",
    "MemUsage": "3.863MiB / 1GiB",
    "Name": "soft-mem-limit-demo",
    "NetIO": "3.98kB / 0B",
    "PIDs": "5"
}

Docker doesn’t provide any command to show the soft memory limits. However, we can find these details from the cgroups.

Let’s go inside the container and list the/sys/fs/cgroup/memory contents.low file:

$ docker exec -it soft-mem-limit-dem sh
# cat /sys/fs/cgroup/memory.low 
536870912
# exit

This file contains an integer value. It’s a Byte representation of 512 MiB, the same as the soft limits.

2.1.2. Setting CPU Limits

By default, containers can use the unlimited cycles of the host machine’s CPU. To restrict it, use the –cpus option with the container run child command.

$ docker container run --rm -it -d --name cpu-limit-demo --cpus=1 nginx:alpine

With this setting, Docker guarantees that the container can use at most 1 CPU at any given time.

Additionally, Docker allows us to set limits on a specific CPU. To achieve this, use the –cpuset-cpus option:

$ docker container run --rm -it -d --name cpu-sets-demo --cpus=1 --cpuset-cpus=2 nginx:alpine

In this example, we have set limits on the third CPU.

Now, to verify this, let’s go inside the container and list the contents of the /sys/fs/cgroup/cpuset.cpus file:

$ docker exec -it cpu-sets-demo sh
# cat /sys/fs/cgroup/cpuset.cpus
2
# exit

This file contains an integer value. It represents a CPU number. It is important to note that the CPU numbering starts from 0.

2.2. Using docker-compose

Docker allows us to set resource limits using the docker-compose as well. Let’s understand this with an example.

First, create a docker-compose.yml configuration file:

$ cat docker-compose.yml 
version: "3.9"
services:
  web-server:
    image: nginx:alpine
    container_name: res-limits-demo
    mem_limit: "1g"
    mem_reservation: "512m"
    cpus: "1"
    cpuset: "2"

In this configuration:

• mem_limit – represents the hard memory limits. We have set it to 1GiB
• mem_reservation: represents the soft memory limits. We have set it to 512MiB
• cpus – represents the CPU limit. We have set it to 1
• cpuset – represents the limit on a specific CPU. We have set it on the third CPU

Let’s deploy the configuration to create a container with the specified limits:

$ docker-compose up -d

Now, let’s verify the CPU and memory limits of the container:

$ docker stats res-limits-demo --no-stream --format "{{ json . }}" | python3 -m json.tool

{
    "BlockIO": "0B / 12.3kB",
    "CPUPerc": "0.00%",
    "Container": "res-limits-demo",
    "ID": "0d7055f8eb31",
    "MemPerc": "0.18%",
    "MemUsage": "1.836MiB / 1GiB",
    "Name": "res-limits-demo",
    "NetIO": "11.2kB / 0B",
    "PIDs": "2"
}
 
$ docker exec -it res-limits-demo sh

# cat /sys/fs/cgroup/memory.low
536870912
# cat /sys/fs/cgroup/cpuset.cpus
2
# exit

Here, we can see that the docker-compose has set CPU and memory limits correctly.

3. Setting Resources Limits on a Running Container

We used container run and docker-compose commands in the previous section to set resource limits. However, one of the major drawbacks of these commands is that they don’t allow updating limits dynamically. It means that we have to specify all the limits while creating a container. One more limitation of the container run command is that it only sets limits on a single container.

To overcome all these limitations, use the container update command. This command dynamically updates the configuration of the multiple containers. Hence we can use it with the running containers.

To understand this, first create a container without any resource limits:

$ docker container run --rm -it -d --name no-limits-demo nginx:alpine

Now, let’s set the CPU and memory limits using the container update command:

$ docker update --memory=1g --memory-reservation=512m --cpus=1 --cpuset-cpus=2 --memory-swap -1 no-limits-demo

In this example, we have used the –memory-swap -1 option. This option increases the swap memory limit up to the amount available on the host machine. In the absence of this option, the command generated the following error:

Error response from daemon: Cannot update container: Memory limit should be smaller than already set memoryswap limit, update the memoryswap at the same time

Let’s verify that the CPU and memory limits were appropriately updated for the container:

$ docker stats no-limits-demo --no-stream --format "{{ json . }}" | python3 -m json.tool

{
    "BlockIO": "0B / 12.3kB",
    "CPUPerc": "0.00%",
    "Container": "no-limits-demo",
    "ID": "ef6bd714038a",
    "MemPerc": "0.36%",
    "MemUsage": "3.691MiB / 1GiB",
    "Name": "no-limits-demo",
    "NetIO": "5.29kB / 0B",
    "PIDs": "5"
}
 
$ docker exec -it no-limits-demo sh
# cat /sys/fs/cgroup/memory.low
536870912
# cat /sys/fs/cgroup/cpuset.cpus
2
/ # exit

It is important to note that, the container update child command is not supported for the containers that are running on the Windows host.

4. Conclusion

This tutorial discussed setting CPU and memory limits to keep the host machine healthy and responsive. First, we discussed using container run and docker-compose commands to set limits while creating containers. Then we saw how to dynamically use the container update child command to update the configuration.

Happy Learning !!