5.2.1.3. Настройка ПО
В этом разделе описан процесс настройка и конфигурация компонентов Kubernetes, которые обеспечивают корректную работу кластера.
- containerd
- kubelet
- crictl
- kubeadm
Настройка containerd
● Обязателен к применению
Настройка containerd
● Обязателен к применению
Этапы настройки компонента
- Конфигурация компонента
- Настройка Systemd Unit компонента
- Старт Systemd Unit
Обратите внимание!
Данный раздел зависит от следующих документов:
Конфигурация компонента
- Bash
- Cloud-init
Создание рабочих директорий
mkdir -p /etc/containerd/
mkdir -p /etc/containerd/conf.d
mkdir -p /etc/containerd/certs.d
Базовый конфигурационный файл
cat <<"EOF" > /etc/containerd/config.toml
version = 2
imports = ["/etc/containerd/conf.d/*.toml"]
EOF
Шаблон кастомного конфигурационного файла
cat <<"EOF" > /etc/containerd/conf.d/in-cloud.toml
version = 2
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.k8s.io/pause:3.9"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d/"
EOF
Базовый конфигурационный файл
- path: /etc/containerd/config.toml
owner: root:root
permissions: '0644'
content: |
version = 2
imports = ["/etc/containerd/conf.d/*.toml"]
Шаблон кастомного конфигурационного файла
- path: /etc/containerd/conf.d/in-cloud.toml
owner: root:root
permissions: '0644'
content: |
version = 2
[plugins]
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.k8s.io/pause:3.9"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d/"
Настройка Systemd Unit компонента
- Bash
- Cloud-init
cat <<EOF > /usr/lib/systemd/system/containerd.service
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target containerd-install.service runc-install.service
Wants=containerd-install.service runc-install.service
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
EOF
systemctl enable containerd
systemctl start containerd
# write_files:
- path: /usr/lib/systemd/system/containerd.service
owner: root:root
permissions: '0644'
content: |
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target containerd-install.service runc-install.service
Wants=containerd-install.service runc-install.service
[Service]
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
TasksMax=infinity
OOMScoreAdjust=-999
[Install]
WantedBy=multi-user.target
#runcmd:
- systemctl enable containerd
- systemctl start containerd
Проверка настройки
Проверка настройки
tree /etc/containerd/
Вывод команды
/etc/containerd/
├── certs.d
├── conf.d
│ └── cloud.toml
└── config.toml
systemctl status containerd
Вывод команды
● containerd.service - containerd container runtime
Loaded: loaded (/usr/lib/systemd/system/containerd.service; enabled; preset: enabled)
Active: active (running) since Tue 2024-12-31 17:26:21 UTC; 2min 30s ago
Docs: https://containerd.io
Main PID: 839 (containerd)
Tasks: 7 (limit: 2274)
Memory: 62.0M (peak: 62.5M)
CPU: 375ms
CGroup: /system.slice/containerd.service
└─839 /usr/local/bin/containerd
***** level=info msg="Start subscribing containerd event"
***** level=info msg="Start recovering state"
***** level=info msg="Start event monitor"
***** level=info msg="Start snapshots syncer"
***** level=info msg="Start cni network conf syncer for default"
***** level=info msg="Start streaming server"
***** level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc
***** level=info msg=serving... address=/run/containerd/containerd.sock
***** level=info msg="containerd successfully booted in 0.065807s"
***** Started containerd.service - containerd container runtime.
Настройка kubelet
● Обязателен к применению
Настройка kubelet
● Обязателен к применению
Этапы настройки компонента
- Настройка Systemd Unit компонента
- Добавить в автозагрузку Systemd Unit
- Кастомная конфигурация компонента
Обратите внимание!
Данный раздел зависит от следующих документов:
Настройка Systemd Unit компонента
- Bash
- Cloud-init
mkdir -p /usr/lib/systemd/system/kubelet.service.d
mkdir -p /var/lib/kubelet/
Systemd Unit
cat <<EOF > /usr/lib/systemd/system/kubelet.service
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/
Wants=network-online.target containerd.service
After=network-online.target containerd.service
[Service]
ExecStart=/usr/local/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
Systemd Unit Config
cat <<EOF > /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet/extra-args.env
ExecStart=
ExecStart=/usr/local/bin/kubelet \$KUBELET_KUBECONFIG_ARGS \$KUBELET_CONFIG_ARGS \$KUBELET_KUBEADM_ARGS \$KUBELET_EXTRA_ARGS
EOF
- Default
- Cloud Controller Manager
Systemd Unit ENV
cat <<EOF > /etc/default/kubelet/extra-args.env
KUBELET_EXTRA_ARGS=""
EOF
Systemd Unit ENV
cat <<EOF > /etc/default/kubelet/extra-args.env
KUBELET_EXTRA_ARGS="--cloud-provider=external"
EOF
Добавить в автозагрузку Systemd Unit
systemctl enable kubelet
Systemd Unit ENV
# write_files:
- path: /usr/lib/systemd/system/kubelet.service
owner: root:root
permissions: '0644'
content: |
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/
Wants=network-online.target containerd.service
After=network-online.target containerd.service
[Service]
ExecStart=/usr/local/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10
[Install]
WantedBy=multi-user.target
Инструкция загрузки
# write_files:
- path: /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
owner: root:root
permissions: '0644'
content: |
# Note: This dropin only works with kubeadm and kubelet v1.11+
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet/extra-args.env
ExecStart=
ExecStart=/usr/local/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
Systemd Unit Download
# write_files:
- path: /etc/default/kubelet/extra-args.env
owner: root:root
permissions: '0644'
content: |
KUBELET_EXTRA_ARGS="--cloud-provider=external"
Systemd Unit Custom ENV
Внимание
Данный блок конфигурации применим только при установке Kubernetes вручную (методом «Kubernetes the Hard Way»). При использовании утил иты kubeadm конфигурационный файл будет создан автоматически на основе спецификации, указанной в файле kubeadm-config
.
# write_files:
- path: /var/lib/kubelet/kubeadm-flags.env
owner: root:root
permissions: '0644'
content: |
KUBELET_KUBEADM_ARGS="--container-runtime-endpoint=unix:///var/run/containerd/containerd.sock --pod-infra-container-image=registry.k8s.io/pause:3.9 --config=/var/lib/kubelet/config-custom.yaml"
Добавить в автозагрузку Systemd Unit
# runcmd:
- systemctl enable kubelet
Кастомная конфигурация компонента
Kubelet config
- Bash
- Cloud-init
Кастомный конфигурационный файл kubelet
cat <<EOF > /var/lib/kubelet/config-custom.yaml
---
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 0s
enabled: true
x509:
clientCAFile: "/etc/kubernetes/pki/ca.crt"
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 0s
cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
containerLogMaxSize: "50Mi"
containerRuntimeEndpoint: "/var/run/containerd/containerd.sock"
cpuManagerReconcilePeriod: 0s
evictionPressureTransitionPeriod: 5s
fileCheckFrequency: 0s
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 0s
imageGCHighThresholdPercent: 55
imageGCLowThresholdPercent: 50
imageMaximumGCAge: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
logging:
flushFrequency: 0
options:
json:
infoBufferSize: "0"
text:
infoBufferSize: "0"
verbosity: 0
kubeAPIQPS: 50
kubeAPIBurst: 100
maxPods: 250
memorySwap: {}
nodeStatusReportFrequency: 1s
nodeStatusUpdateFrequency: 1s
podPidsLimit: 4096
registerNode: true
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 0s
serializeImagePulls: false
serverTLSBootstrap: true
shutdownGracePeriod: 15s
shutdownGracePeriodCriticalPods: 5s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
tlsMinVersion: "VersionTLS12"
volumeStatsAggPeriod: 0s
featureGates:
RotateKubeletServerCertificate: true
APIPriorityAndFairness: true
tlsCipherSuites:
- "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
- "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
- "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
- "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
- "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256"
- "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256"
EOF
Кастомный конфигурационный файл kubelet
- path: /var/lib/kubelet/config-custom.yaml
owner: root:root
permissions: '0644'
content: |
---
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 0s
enabled: true
x509:
clientCAFile: "/etc/kubernetes/pki/ca.crt"
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 0s
cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
containerLogMaxSize: "50Mi"
containerRuntimeEndpoint: "/var/run/containerd/containerd.sock"
cpuManagerReconcilePeriod: 0s
evictionPressureTransitionPeriod: 5s
fileCheckFrequency: 0s
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 0s
imageGCHighThresholdPercent: 55
imageGCLowThresholdPercent: 50
imageMaximumGCAge: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
logging:
flushFrequency: 0
options:
json:
infoBufferSize: "0"
text:
infoBufferSize: "0"
verbosity: 0
kubeAPIQPS: 50
kubeAPIBurst: 100
maxPods: 250
memorySwap: {}
nodeStatusReportFrequency: 1s
nodeStatusUpdateFrequency: 1s
podPidsLimit: 4096
registerNode: true
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 0s
serializeImagePulls: false
serverTLSBootstrap: true
shutdownGracePeriod: 15s
shutdownGracePeriodCriticalPods: 5s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
tlsMinVersion: "VersionTLS12"
volumeStatsAggPeriod: 0s
featureGates:
RotateKubeletServerCertificate: true
APIPriorityAndFairness: true
tlsCipherSuites:
- "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
- "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
- "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"
- "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
- "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256"
- "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256"