京公网安备11010502063487号容器里限制了 CPU,Go 程序默认起多少个 P?Go 1.25 之前它看不见 cgroup 限制,容易被限流拖慢。这篇用不同的 --cpus 设置、不同的 go.mod 语言版本和运行中修改限制,实测默认值怎么变,以及选错的代价。
Docker Desktop 虚拟机:10 个 CPU,linux/aarch64,cgroup 版本 2 镜像 g50:new 的 go.mod 写 go 1.27;镜像 g50:old 的 go.mod 写 go 1.24(同一份源码,都由 Go 1.27.1 交叉编译为 linux/arm64 静态程序,用 docker import 做成只含这一个文件的镜像)
宿主是 Docker Desktop 的 Linux 虚拟机(10 个 CPU,cgroup 版本 2),所以读到的 cpu.max 是 cgroup v2 的文件。程序是同一份源码交叉编译的静态二进制,没有其它依赖;cgroup v1 的情形我没有测。
[不限制] NumCPU=10 GOMAXPROCS=10 | cpu.max="max 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpus=4] NumCPU=10 GOMAXPROCS=4 | cpu.max="400000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpus=2] NumCPU=10 GOMAXPROCS=2 | cpu.max="200000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpus=2.5] NumCPU=10 GOMAXPROCS=3 | cpu.max="250000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpus=1] NumCPU=10 GOMAXPROCS=2 | cpu.max="100000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpus=0.5] NumCPU=10 GOMAXPROCS=2 | cpu.max="50000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpuset-cpus=0-2] NumCPU=3 GOMAXPROCS=3 | cpu.max="max 100000" | 环境变量 GOMAXPROCS="" GODEBUG=""
go.mod 写 go 1.27 的程序,GOMAXPROCS 默认取 NumCPU 与 cgroup 限制中小的那个,非整数向上取整(2.5 → 3)。限制低于 2 个 CPU 时也是 2,文档写明“除非逻辑 CPU 数低于 2,否则不会低于 2”。
[--cpus=2] NumCPU=10 GOMAXPROCS=10 | cpu.max="200000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="" [--cpus=0.5] NumCPU=10 GOMAXPROCS=10 | cpu.max="50000 100000" | 环境变量 GOMAXPROCS="" GODEBUG=""
同一份源码、同一个编译器,仅 go.mod 的 go 行从 1.27 改成 1.24,容器限制 2 个 CPU 时 GOMAXPROCS 就从 2 变成 10。godebug 文档说明 containermaxprocs=0 是 go 1.24 及以下的默认。所以老项目升级编译器但没改 go.mod,不会自动获得这个行为。
[环境变量 GOMAXPROCS=8] NumCPU=10 GOMAXPROCS=8 | cpu.max="200000 100000" | 环境变量 GOMAXPROCS="8" GODEBUG="" [GODEBUG=containermaxprocs=0] NumCPU=10 GOMAXPROCS=10 | cpu.max="200000 100000" | 环境变量 GOMAXPROCS="" GODEBUG="containermaxprocs=0"
环境变量 GOMAXPROCS=8 无视 cgroup 直接取 8;GODEBUG=containermaxprocs=0 则回到 NumCPU。
t= 0.0s GOMAXPROCS=2 cpu.max="200000 100000" t= 0.5s GOMAXPROCS=2 cpu.max="200000 100000" t= 1.0s GOMAXPROCS=2 cpu.max="200000 100000" t= 1.5s GOMAXPROCS=2 cpu.max="200000 100000" t= 2.0s GOMAXPROCS=2 cpu.max="200000 100000" t= 2.5s GOMAXPROCS=2 cpu.max="400000 100000" t= 3.0s GOMAXPROCS=2 cpu.max="400000 100000" t= 3.5s GOMAXPROCS=4 cpu.max="400000 100000" t= 4.0s GOMAXPROCS=4 cpu.max="400000 100000" t= 4.5s GOMAXPROCS=4 cpu.max="100000 100000" t= 5.0s GOMAXPROCS=4 cpu.max="100000 100000" t= 5.5s GOMAXPROCS=2 cpu.max="100000 100000" t= 6.0s GOMAXPROCS=2 cpu.max="100000 100000" t= 6.5s GOMAXPROCS=2 cpu.max="100000 100000" t= 7.0s GOMAXPROCS=2 cpu.max="100000 100000" t= 7.5s GOMAXPROCS=2 cpu.max="100000 100000"
docker update --cpus 改了限制后,cpu.max 立刻变化,GOMAXPROCS 在大约 1 秒后跟上,与文档里“最多每秒更新一次”一致。
[默认] GOMAXPROCS=2:3 秒内 nr_periods=31 nr_throttled=30 throttled_usec=3004489;1ms 睡眠 77 次,最坏超时 86.3 ms [GOMAXPROCS=10] GOMAXPROCS=10:3 秒内 nr_periods=31 nr_throttled=31 throttled_usec=27151434;1ms 睡眠 21 次,最坏超时 214.5 ms [默认] GOMAXPROCS=2:3 秒内 nr_periods=30 nr_throttled=30 throttled_usec=2969165;1ms 睡眠 69 次,最坏超时 104.8 ms [GOMAXPROCS=10] GOMAXPROCS=10:3 秒内 nr_periods=32 nr_throttled=32 throttled_usec=28169325;1ms 睡眠 22 次,最坏超时 207.7 ms [默认] GOMAXPROCS=2:3 秒内 nr_periods=30 nr_throttled=30 throttled_usec=3007044;1ms 睡眠 74 次,最坏超时 95.4 ms [GOMAXPROCS=10] GOMAXPROCS=10:3 秒内 nr_periods=31 nr_throttled=31 throttled_usec=27330454;1ms 睡眠 20 次,最坏超时 206.3 ms
在 --cpus=1 下让 10 个线程并行空转,cgroup 记录的限流时间是默认值(2)的约 9 倍,1 ms 睡眠最坏被拖到约 210 ms(默认约 90 ms)。注意默认值 2 仍高于 1 个 CPU 的限制,所以仍被限流,只是轻得多。throttled_usec 是累计量,为什么恰好是 3 秒和 27 秒左右,我没有推导,只报告观察到的数。
simultaneously and returns the previous setting. If n < 1, it does not
change the current setting.
# Default
If the GOMAXPROCS environment variable is set to a positive whole number,
GOMAXPROCS defaults to that value.
Otherwise, the Go runtime selects an appropriate default value from a
combination of
- the number of logical CPUs on the machine,
- the process’s CPU affinity mask,
- and, on Linux, the process’s average CPU throughput limit based on
cgroup CPU quota, if any.
If GODEBUG=containermaxprocs=0 is set and GOMAXPROCS is not set by the
environment variable, then GOMAXPROCS instead defaults to the value of
runtime.NumCPU. Note that GODEBUG=containermaxprocs=0 is default for
language version 1.24 and below.
# Updates
The Go runtime periodically updates the default value based on changes to
the total logical CPU count, the CPU affinity mask, or cgroup quota. Setting
a custom value with the GOMAXPROCS environment variable or by calling
GOMAXPROCS disables automatic updates. The default value and automatic
updates can be restored by calling SetDefaultGOMAXPROCS.
If GODEBUG=updatemaxprocs=0 is set, the Go runtime does not perform
automatic GOMAXPROCS updating. Note that GODEBUG=updatemaxprocs=0 is default
for language version 1.24 and below.
# Compatibility
Note that the default GOMAXPROCS behavior may change as the scheduler
improves, especially the implementation detail below.
# Implementation details
When computing default GOMAXPROCS via cgroups, the Go runtime computes the
"average CPU throughput limit" as the cgroup CPU quota / period. In cgroup
v2, these values come from the cpu.max file. In cgroup v1, they come from
cpu.cfs_quota_us and cpu.cfs_period_us, respectively. In container runtimes
that allow configuring CPU limits, this value usually corresponds to the
"CPU limit" option, not "CPU request".
The Go runtime typically selects the default GOMAXPROCS as the minimum
of the logical CPU count, the CPU affinity mask count, or the cgroup CPU
throughput limit. However, it will never set GOMAXPROCS less than 2 unless
the logical CPU count or CPU affinity mask count are below 2.
If the cgroup CPU throughput limit is not a whole number, the Go runtime
rounds up to the next whole number.
GOMAXPROCS updates are performed up to once per second, or less if the
application is idle.
Go 1.25 added a new `containermaxprocs` setting that controls whether the Go
runtime will consider cgroup CPU limits when setting the default GOMAXPROCS.
The default value `containermaxprocs=1` will use cgroup limits in addition to
the total logical CPU count and CPU affinity. `containermaxprocs=0` will
disable consideration of cgroup limits. This setting only affects Linux.
Go 1.25 added a new `updatemaxprocs` setting that controls whether the Go
runtime will periodically update GOMAXPROCS for new CPU affinity or cgroup
limits. The default value `updatemaxprocs=1` will enable periodic updates.
`updatemaxprocs=0` will disable periodic updates.
Go 1.25 disabled SHA-1 signature algorithms in TLS 1.2 according to RFC 9155.
The default can be reverted using the `tlssha1=1` setting.
规则都来自 go doc runtime.GOMAXPROCS 与 godebug 文档:三者取最小、不低于 2、向上取整、每秒最多更新一次。实验的每一项都与这几条吻合。