Skip to content

Commit

Permalink
cgroup: add cgroup v2 support in containers
Browse files Browse the repository at this point in the history
  • Loading branch information
criyle committed Dec 25, 2021
1 parent 359fda5 commit dd4c6ee
Show file tree
Hide file tree
Showing 5 changed files with 23 additions and 4 deletions.
6 changes: 6 additions & 0 deletions README.cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,12 @@ docker run -it --rm --privileged --shm-size=256m -p 5050:5050 criyle/executorser

### 注意

#### cgroup v2

`executorserver` 目前已经支持 cgroup v2 鉴于越来越多的 Linux 发行版默认启用 cgroup v2 而不是 v1 (比如 Ubuntu 21.10+,Fedora 31+)。然而,因为 cgroup v2 在内存控制器里面缺少 `memory.max_usage_in_bytes`,内存使用量计数会转而采用 `maxrss` 指标。这项指标会显示的比使用 cgroup v1 时候要稍多,在运行使用内存较少的程序时比较明显。

同时,如果本程序在容器中运行,容器中的进程会被移到 `/init` cgroup v2 控制器中来开启 cgroup v2 嵌套支持。

#### CentOS 7

需要开启 user 命名空间来使用 [stack overflow](https://superuser.com/questions/1294215/is-it-safe-to-enable-user-namespaces-in-centos-7-4-and-how-to-do-it/1294246#1294246)
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,12 @@ To customize mount points, please look at example `mount.yaml` file.

### Notice

#### cgroup v2 support

The cgroup v2 is supported by `executorserver` now when running as root since more Linux distribution are enabling cgroup v2 by default (e.g. Ubuntu 21.10+, Fedora 31+). However, due to missing `memory.max_usage_in_bytes` in `memory` controller, the memory usage is now accounted by `maxrss` returned by `wait4` syscall. Thus, the memory usage appears higher than those who uses cgroup v1.

When running in containers, the `executorserver` will migrate all processed into `/init` hierarchy to enable nesting support.

#### CentOS 7

By default, user namespace is disabled and it can be enabled following [stack overflow](https://superuser.com/questions/1294215/is-it-safe-to-enable-user-namespaces-in-centos-7-4-and-how-to-do-it/1294246#1294246)
Expand Down
9 changes: 8 additions & 1 deletion env/env_linux.go
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,14 @@ func NewBuilder(c Config) (pool.EnvBuilder, error) {
ContainerUID: cUID,
ContainerGID: cGID,
}
cgb := cgroup.NewBuilder(c.CgroupPrefix).DetectType().WithCPUAcct().WithMemory().WithPids().WithCPUSet()
t := cgroup.DetectType()
if t == cgroup.CgroupTypeV2 {
c.Info("Enable cgroup v2 nesting support")
if err := cgroup.EnableV2Nesting(); err != nil {
c.Warn("Enable cgroup v2 failed", err)
}
}
cgb := cgroup.NewBuilder(c.CgroupPrefix).WithType(t).WithCPUAcct().WithMemory().WithPids().WithCPUSet()
if c.EnableCPURate {
cgb = cgb.WithCPU()
}
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.17

require (
github.com/creack/pty v1.1.17
github.com/criyle/go-sandbox v0.9.0
github.com/criyle/go-sandbox v0.9.1
github.com/elastic/go-seccomp-bpf v1.2.0
github.com/elastic/go-ucfg v0.8.4
github.com/gin-contrib/pprof v1.3.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,8 @@ github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWH
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/creack/pty v1.1.17 h1:QeVUsEDNrLBW4tMgZHvxy18sKtr6VI492kBhUfhDJNI=
github.com/creack/pty v1.1.17/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/criyle/go-sandbox v0.9.0 h1:GsaQE+AKZqZD/piL5uGmT3mBVbrzyEF43fNrW7TVvLM=
github.com/criyle/go-sandbox v0.9.0/go.mod h1:+w66tiv66TUTslkseyfUHUdYZaawGbC6tSeRSGKR9uI=
github.com/criyle/go-sandbox v0.9.1 h1:T+82p3cROHC4kU6kOMmOZvf/871Eky7chHb41DeUk1w=
github.com/criyle/go-sandbox v0.9.1/go.mod h1:+w66tiv66TUTslkseyfUHUdYZaawGbC6tSeRSGKR9uI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
Expand Down

0 comments on commit dd4c6ee

Please sign in to comment.