京公网安备11010502063487号RuntimeClass 除了选运行时,还有什么用?Pod Overhead 是怎么被算进调度和配额的?它会不会影响容器的 cgroup 限制?handler 没装时 Pod 什么表现?这篇用 runc 的两个 RuntimeClass 把这些环节逐个量出来。
RuntimeClass runc-heavy:handler=runc;overhead.podFixed={cpu: 250m, memory: 120Mi}
Pod plain(没写 runtimeClassName):spec.overhead=None;Pod heavy(runtimeClassName: runc-heavy):spec.overhead={'cpu': '250m', 'memory': '120Mi'}(RuntimeClass 准入控制器写进去的,容器 requests 都是 100m / 64Mi)
节点 worker 的 Allocated resources(requests):加 plain 前 150m / 114Mi → 加 plain 后 250m / 178Mi → 再加 heavy 后 600m / 362Mi
plain 增加 cpu 100m、heavy 增加 cpu 350m(= 100m + 250m overhead)RuntimeClass runc-heavy:handler=runc,overhead.podFixed={cpu: 250m, memory: 120Mi}。Pod plain 不写 runtimeClassName,spec.overhead 是空;Pod heavy 写了 runtimeClassName: runc-heavy,spec.overhead={cpu: 250m, memory: 120Mi}——RuntimeClass 准入控制器写进去的,两个 Pod 的容器 requests 都是 100m / 64Mi。节点 worker 的 Allocated resources(requests):150m / 114Mi → 加 plain 后 250m / 178Mi → 再加 heavy 后 600m / 362Mi:plain 增加 100m,heavy 增加 350m(100m + 250m overhead)。
plain(qosClass=Guaranteed):Pod 级 memory.max=64 MiB cpu.max=10000 100000;容器级 memory.max=64 MiB cpu.max=10000 100000 heavy(qosClass=Guaranteed):Pod 级 memory.max=184 MiB cpu.max=35000 100000;容器级 memory.max=64 MiB cpu.max=10000 100000 Pod 级限制 = 容器 limits 之和 + overhead;容器自己的限制不变——overhead 是给沙箱/运行时组件留的空间
plain(Guaranteed)的 Pod 级 memory.max=64 MiB、cpu.max=10000 100000,容器级相同;heavy 的 Pod 级 memory.max=184 MiB(64 + 120)、cpu.max=35000 100000(100m + 250m),而容器级 memory.max 仍是 64 MiB、cpu.max 10000 100000。Pod 级限制 = 容器 limits 之和 + overhead;应用的 OOM 边界仍是它自己的 limits,overhead 是给沙箱和运行时组件留的空间。
命名空间已有 plain + heavy,ResourceQuota requests.cpu 上限 500m,已用 450m(plain 100m + heavy 100m + overhead 250m)
再创建一个 heavy 类的 Pod(容器 100m + overhead 250m = 350m,合计会到 800m):退出码 1;['Error from server (Forbidden): error when creating "STDIN": pods "third" is forbidden: exceeded quota: q, requested: requests.cpu=350m, used: requests.cpu=450m, limited: requests.cpu=500m']
再创建一个普通 Pod(100m,合计 550m 也超了):退出码 1;['Error from server (Forbidden): error when creating "STDIN": pods "small" is forbidden: exceeded quota: q, requested: requests.cpu=100m, used: requests.cpu=450m, limited: requests.cpu=500m']
overhead 是 100 个 CPU 的 RuntimeClass runc-huge:Pod huge 的 phase=Pending;[('PodScheduled', 'Unschedulable', '0/3 nodes are available: 1 node(s) had untolerated taint(s), 2 Insufficient cpu. preemption: 0/3 nodes are available: 3 Preemption is not helpful for ')]命名空间已有 plain + heavy,ResourceQuota requests.cpu 上限 500m,已用 450m(plain 100m + heavy 100m + overhead 250m)。再创建一个 heavy 类的 Pod(容器 100m + overhead 250m = 350m)被拒:exceeded quota: q, requested: requests.cpu=350m, used: requests.cpu=450m, limited: requests.cpu=500m;一个普通的 100m Pod 也被拒(450 + 100 > 500)。overhead 是 100 个 CPU 的 RuntimeClass runc-huge:Pod huge 一直 Pending,PodScheduled=False,0/3 nodes are available: … 2 Insufficient cpu。
runtimeClassName: nope(没有这个 RuntimeClass 对象):退出码 1;['Error from server (Forbidden): error when creating "STDIN": pods "missing-rc" is forbidden: pod rejected: RuntimeClass "nope" not found'] RuntimeClass 存在但 handler: gvisor-not-installed 节点上没配:phase=Pending;None;容器状态 ['waiting'] 事件:['8s Warning FailedCreatePodSandBox pod/nohandler Failed to create pod sandbox: rpc error: code = Unknown desc = unable to get OCI runtime for sandbox "56b72be1f8aa9a3932c60360ac3625379dda49ab164fda83d946d2007eddb085": no runtime for "gvisor-not-installed" is co']
runtimeClassName 写了不存在的 RuntimeClass:创建被拒,pod rejected: RuntimeClass "nope" not found。RuntimeClass 存在但 handler: gvisor-not-installed 在节点 containerd 里没配置:Pod 能创建,停在 Pending,容器 waiting,事件 FailedCreatePodSandBox: … no runtime for "gvisor-not-installed" is configured。