kube-proxy and weave-net keep hanging in ContainerCreating state on master node
1
We are running a small kubernetes cluster with 4 nodes. After the master-node hang, we had to restart it.
Now kube-proxy and weave-net pods on the master node keep hanging in state ContainerCreating, while running fine on all other nodes:
kubectl get pod -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE
coredns-78fcdf6894-67wkh 1/1 Running 0 12h 10.42.0.40 i101v182
coredns-78fcdf6894-8xwkq 1/1 Running 3 122d 10.44.0.65 i101v181
etcd-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
heapster-74db55987-sthnp 1/1 Running 1 40d 10.44.0.60 i101v181
kube-apiserver-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-controller-manager-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-6dqzf 0/1 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-nvghr 1/1 Running 9 187d 10.1.101.182 i101v182
kube-proxy-spchz 1/1 Running 8 187d 10.1.101.181 i101v181
kube-proxy-xhg77 1/1 Running 7 187d 10.1.101.183 i101v183
kube-scheduler-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kubernetes-dashboard-767dc7d4d-ws79h 1/1 Running 0 12h 10.42.0.41 i101v182
metrics-server-696868464d-b8njh 1/1 Running 0 12h 10.42.0.42 i101v182
monitoring-influxdb-848b9b66f6-gwbf5 1/1 Running 1 46d 10.44.0.61 i101v181
weave-net-465np 2/2 Running 3 64d 10.1.101.182 i101v182
weave-net-5mgdp 0/2 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
weave-net-chsqv 2/2 Running 2 40d 10.1.101.183 i101v183
weave-net-mcn77 2/2 Running 18 187d 10.1.101.181 i101v181
There are no errors in the log, nor any events that would give a reason for this:
kubectl describe pod kube-proxy-6dqzf -n kube-system
Name: kube-proxy-6dqzf
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:32:51 +0100
Labels: controller-revision-hash=1151982146
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID:
Image: k8s.gcr.io/kube-proxy-amd64:v1.11.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/usr/local/bin/kube-proxy
--config=/var/lib/kube-proxy/config.conf
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/kube-proxy from kube-proxy (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-proxy-token-9t8cp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-proxy:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-proxy
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
kube-proxy-token-9t8cp:
Type: Secret (a volume populated by a Secret)
SecretName: kube-proxy-token-9t8cp
Optional: false
QoS Class: BestEffort
Node-Selectors: beta.kubernetes.io/arch=amd64
Tolerations: CriticalAddonsOnly
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
kubectl describe pod weave-net-5mgdp -n kube-system
Name: weave-net-5mgdp
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:21:57 +0100
Labels: controller-revision-hash=2963469815
name=weave-net
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/weave-net
Containers:
weave:
Container ID:
Image: weaveworks/weave-kube:2.3.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/home/weave/launch.sh
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Liveness: http-get http://127.0.0.1:6784/status delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/host/etc from cni-conf (rw)
/host/home from cni-bin2 (rw)
/host/opt from cni-bin (rw)
/host/var/lib/dbus from dbus (rw)
/lib/modules from lib-modules (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
/weavedb from weavedb (rw)
weave-npc:
Container ID:
Image: weaveworks/weave-npc:2.3.0
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
weavedb:
Type: HostPath (bare host directory volume)
Path: /var/lib/weave
HostPathType:
cni-bin:
Type: HostPath (bare host directory volume)
Path: /opt
HostPathType:
cni-bin2:
Type: HostPath (bare host directory volume)
Path: /home
HostPathType:
cni-conf:
Type: HostPath (bare host directory volume)
Path: /etc
HostPathType:
dbus:
Type: HostPath (bare host directory volume)
Path: /var/lib/dbus
HostPathType:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType:
weave-net-token-dcjzm:
Type: Secret (a volume populated by a Secret)
SecretName: weave-net-token-dcjzm
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
Any ideas what might be the cause of this?
linux virtualization
asked Jan 16 at 14:52
PeterPeter
61
add a comment |
1
We are running a small kubernetes cluster with 4 nodes. After the master-node hang, we had to restart it.
Now kube-proxy and weave-net pods on the master node keep hanging in state ContainerCreating, while running fine on all other nodes:
kubectl get pod -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE
coredns-78fcdf6894-67wkh 1/1 Running 0 12h 10.42.0.40 i101v182
coredns-78fcdf6894-8xwkq 1/1 Running 3 122d 10.44.0.65 i101v181
etcd-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
heapster-74db55987-sthnp 1/1 Running 1 40d 10.44.0.60 i101v181
kube-apiserver-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-controller-manager-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-6dqzf 0/1 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-nvghr 1/1 Running 9 187d 10.1.101.182 i101v182
kube-proxy-spchz 1/1 Running 8 187d 10.1.101.181 i101v181
kube-proxy-xhg77 1/1 Running 7 187d 10.1.101.183 i101v183
kube-scheduler-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kubernetes-dashboard-767dc7d4d-ws79h 1/1 Running 0 12h 10.42.0.41 i101v182
metrics-server-696868464d-b8njh 1/1 Running 0 12h 10.42.0.42 i101v182
monitoring-influxdb-848b9b66f6-gwbf5 1/1 Running 1 46d 10.44.0.61 i101v181
weave-net-465np 2/2 Running 3 64d 10.1.101.182 i101v182
weave-net-5mgdp 0/2 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
weave-net-chsqv 2/2 Running 2 40d 10.1.101.183 i101v183
weave-net-mcn77 2/2 Running 18 187d 10.1.101.181 i101v181
There are no errors in the log, nor any events that would give a reason for this:
kubectl describe pod kube-proxy-6dqzf -n kube-system
Name: kube-proxy-6dqzf
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:32:51 +0100
Labels: controller-revision-hash=1151982146
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID:
Image: k8s.gcr.io/kube-proxy-amd64:v1.11.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/usr/local/bin/kube-proxy
--config=/var/lib/kube-proxy/config.conf
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/kube-proxy from kube-proxy (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-proxy-token-9t8cp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-proxy:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-proxy
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
kube-proxy-token-9t8cp:
Type: Secret (a volume populated by a Secret)
SecretName: kube-proxy-token-9t8cp
Optional: false
QoS Class: BestEffort
Node-Selectors: beta.kubernetes.io/arch=amd64
Tolerations: CriticalAddonsOnly
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
kubectl describe pod weave-net-5mgdp -n kube-system
Name: weave-net-5mgdp
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:21:57 +0100
Labels: controller-revision-hash=2963469815
name=weave-net
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/weave-net
Containers:
weave:
Container ID:
Image: weaveworks/weave-kube:2.3.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/home/weave/launch.sh
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Liveness: http-get http://127.0.0.1:6784/status delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/host/etc from cni-conf (rw)
/host/home from cni-bin2 (rw)
/host/opt from cni-bin (rw)
/host/var/lib/dbus from dbus (rw)
/lib/modules from lib-modules (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
/weavedb from weavedb (rw)
weave-npc:
Container ID:
Image: weaveworks/weave-npc:2.3.0
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
weavedb:
Type: HostPath (bare host directory volume)
Path: /var/lib/weave
HostPathType:
cni-bin:
Type: HostPath (bare host directory volume)
Path: /opt
HostPathType:
cni-bin2:
Type: HostPath (bare host directory volume)
Path: /home
HostPathType:
cni-conf:
Type: HostPath (bare host directory volume)
Path: /etc
HostPathType:
dbus:
Type: HostPath (bare host directory volume)
Path: /var/lib/dbus
HostPathType:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType:
weave-net-token-dcjzm:
Type: Secret (a volume populated by a Secret)
SecretName: weave-net-token-dcjzm
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
Any ideas what might be the cause of this?
linux virtualization
asked Jan 16 at 14:52
PeterPeter
61
add a comment |
1
1
1
We are running a small kubernetes cluster with 4 nodes. After the master-node hang, we had to restart it.
Now kube-proxy and weave-net pods on the master node keep hanging in state ContainerCreating, while running fine on all other nodes:
kubectl get pod -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE
coredns-78fcdf6894-67wkh 1/1 Running 0 12h 10.42.0.40 i101v182
coredns-78fcdf6894-8xwkq 1/1 Running 3 122d 10.44.0.65 i101v181
etcd-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
heapster-74db55987-sthnp 1/1 Running 1 40d 10.44.0.60 i101v181
kube-apiserver-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-controller-manager-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-6dqzf 0/1 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-nvghr 1/1 Running 9 187d 10.1.101.182 i101v182
kube-proxy-spchz 1/1 Running 8 187d 10.1.101.181 i101v181
kube-proxy-xhg77 1/1 Running 7 187d 10.1.101.183 i101v183
kube-scheduler-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kubernetes-dashboard-767dc7d4d-ws79h 1/1 Running 0 12h 10.42.0.41 i101v182
metrics-server-696868464d-b8njh 1/1 Running 0 12h 10.42.0.42 i101v182
monitoring-influxdb-848b9b66f6-gwbf5 1/1 Running 1 46d 10.44.0.61 i101v181
weave-net-465np 2/2 Running 3 64d 10.1.101.182 i101v182
weave-net-5mgdp 0/2 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
weave-net-chsqv 2/2 Running 2 40d 10.1.101.183 i101v183
weave-net-mcn77 2/2 Running 18 187d 10.1.101.181 i101v181
There are no errors in the log, nor any events that would give a reason for this:
kubectl describe pod kube-proxy-6dqzf -n kube-system
Name: kube-proxy-6dqzf
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:32:51 +0100
Labels: controller-revision-hash=1151982146
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID:
Image: k8s.gcr.io/kube-proxy-amd64:v1.11.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/usr/local/bin/kube-proxy
--config=/var/lib/kube-proxy/config.conf
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/kube-proxy from kube-proxy (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-proxy-token-9t8cp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-proxy:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-proxy
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
kube-proxy-token-9t8cp:
Type: Secret (a volume populated by a Secret)
SecretName: kube-proxy-token-9t8cp
Optional: false
QoS Class: BestEffort
Node-Selectors: beta.kubernetes.io/arch=amd64
Tolerations: CriticalAddonsOnly
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
kubectl describe pod weave-net-5mgdp -n kube-system
Name: weave-net-5mgdp
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:21:57 +0100
Labels: controller-revision-hash=2963469815
name=weave-net
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/weave-net
Containers:
weave:
Container ID:
Image: weaveworks/weave-kube:2.3.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/home/weave/launch.sh
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Liveness: http-get http://127.0.0.1:6784/status delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/host/etc from cni-conf (rw)
/host/home from cni-bin2 (rw)
/host/opt from cni-bin (rw)
/host/var/lib/dbus from dbus (rw)
/lib/modules from lib-modules (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
/weavedb from weavedb (rw)
weave-npc:
Container ID:
Image: weaveworks/weave-npc:2.3.0
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
weavedb:
Type: HostPath (bare host directory volume)
Path: /var/lib/weave
HostPathType:
cni-bin:
Type: HostPath (bare host directory volume)
Path: /opt
HostPathType:
cni-bin2:
Type: HostPath (bare host directory volume)
Path: /home
HostPathType:
cni-conf:
Type: HostPath (bare host directory volume)
Path: /etc
HostPathType:
dbus:
Type: HostPath (bare host directory volume)
Path: /var/lib/dbus
HostPathType:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType:
weave-net-token-dcjzm:
Type: Secret (a volume populated by a Secret)
SecretName: weave-net-token-dcjzm
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
Any ideas what might be the cause of this?
linux virtualization
asked Jan 16 at 14:52
PeterPeter
61
We are running a small kubernetes cluster with 4 nodes. After the master-node hang, we had to restart it.
Now kube-proxy and weave-net pods on the master node keep hanging in state ContainerCreating, while running fine on all other nodes:
kubectl get pod -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE
coredns-78fcdf6894-67wkh 1/1 Running 0 12h 10.42.0.40 i101v182
coredns-78fcdf6894-8xwkq 1/1 Running 3 122d 10.44.0.65 i101v181
etcd-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
heapster-74db55987-sthnp 1/1 Running 1 40d 10.44.0.60 i101v181
kube-apiserver-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-controller-manager-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-6dqzf 0/1 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
kube-proxy-nvghr 1/1 Running 9 187d 10.1.101.182 i101v182
kube-proxy-spchz 1/1 Running 8 187d 10.1.101.181 i101v181
kube-proxy-xhg77 1/1 Running 7 187d 10.1.101.183 i101v183
kube-scheduler-i101v180.intra.graz.at 1/1 Running 2 11h 10.1.101.180 i101v180.intra.graz.at
kubernetes-dashboard-767dc7d4d-ws79h 1/1 Running 0 12h 10.42.0.41 i101v182
metrics-server-696868464d-b8njh 1/1 Running 0 12h 10.42.0.42 i101v182
monitoring-influxdb-848b9b66f6-gwbf5 1/1 Running 1 46d 10.44.0.61 i101v181
weave-net-465np 2/2 Running 3 64d 10.1.101.182 i101v182
weave-net-5mgdp 0/2 ContainerCreating 0 1h 10.1.101.180 i101v180.intra.graz.at
weave-net-chsqv 2/2 Running 2 40d 10.1.101.183 i101v183
weave-net-mcn77 2/2 Running 18 187d 10.1.101.181 i101v181
There are no errors in the log, nor any events that would give a reason for this:
kubectl describe pod kube-proxy-6dqzf -n kube-system
Name: kube-proxy-6dqzf
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:32:51 +0100
Labels: controller-revision-hash=1151982146
k8s-app=kube-proxy
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/kube-proxy
Containers:
kube-proxy:
Container ID:
Image: k8s.gcr.io/kube-proxy-amd64:v1.11.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/usr/local/bin/kube-proxy
--config=/var/lib/kube-proxy/config.conf
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/lib/modules from lib-modules (ro)
/run/xtables.lock from xtables-lock (rw)
/var/lib/kube-proxy from kube-proxy (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-proxy-token-9t8cp (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-proxy:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kube-proxy
Optional: false
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType: FileOrCreate
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
kube-proxy-token-9t8cp:
Type: Secret (a volume populated by a Secret)
SecretName: kube-proxy-token-9t8cp
Optional: false
QoS Class: BestEffort
Node-Selectors: beta.kubernetes.io/arch=amd64
Tolerations: CriticalAddonsOnly
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
kubectl describe pod weave-net-5mgdp -n kube-system
Name: weave-net-5mgdp
Namespace: kube-system
Node: i101v180.intra.graz.at/10.1.101.180
Start Time: Wed, 16 Jan 2019 14:21:57 +0100
Labels: controller-revision-hash=2963469815
name=weave-net
pod-template-generation=1
Annotations: <none>
Status: Pending
IP: 10.1.101.180
Controlled By: DaemonSet/weave-net
Containers:
weave:
Container ID:
Image: weaveworks/weave-kube:2.3.0
Image ID:
Port: <none>
Host Port: <none>
Command:
/home/weave/launch.sh
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Liveness: http-get http://127.0.0.1:6784/status delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/host/etc from cni-conf (rw)
/host/home from cni-bin2 (rw)
/host/opt from cni-bin (rw)
/host/var/lib/dbus from dbus (rw)
/lib/modules from lib-modules (rw)
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
/weavedb from weavedb (rw)
weave-npc:
Container ID:
Image: weaveworks/weave-npc:2.3.0
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 10m
Environment:
HOSTNAME: (v1:spec.nodeName)
Mounts:
/run/xtables.lock from xtables-lock (rw)
/var/run/secrets/kubernetes.io/serviceaccount from weave-net-token-dcjzm (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
weavedb:
Type: HostPath (bare host directory volume)
Path: /var/lib/weave
HostPathType:
cni-bin:
Type: HostPath (bare host directory volume)
Path: /opt
HostPathType:
cni-bin2:
Type: HostPath (bare host directory volume)
Path: /home
HostPathType:
cni-conf:
Type: HostPath (bare host directory volume)
Path: /etc
HostPathType:
dbus:
Type: HostPath (bare host directory volume)
Path: /var/lib/dbus
HostPathType:
lib-modules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
xtables-lock:
Type: HostPath (bare host directory volume)
Path: /run/xtables.lock
HostPathType:
weave-net-token-dcjzm:
Type: Secret (a volume populated by a Secret)
SecretName: weave-net-token-dcjzm
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: :NoSchedule
node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
Events: <none>
Any ideas what might be the cause of this?
linux virtualization
linux virtualization
asked Jan 16 at 14:52
PeterPeter
61
asked Jan 16 at 14:52
PeterPeter
61
asked Jan 16 at 14:52
PeterPeter
61
asked Jan 16 at 14:52
PeterPeter
61
asked Jan 16 at 14:52
PeterPeter
61
61
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1394977%2fkube-proxy-and-weave-net-keep-hanging-in-containercreating-state-on-master-node%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1394977%2fkube-proxy-and-weave-net-keep-hanging-in-containercreating-state-on-master-node%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
