+ export WORKSPACE=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release + WORKSPACE=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release + [[ k8s-1.11.0-release =~ openshift-.* ]] + [[ k8s-1.11.0-release =~ .*-1.9.3-.* ]] + export KUBEVIRT_PROVIDER=k8s-1.10.3 + KUBEVIRT_PROVIDER=k8s-1.10.3 + export KUBEVIRT_NUM_NODES=2 + KUBEVIRT_NUM_NODES=2 + export NFS_WINDOWS_DIR=/home/nfs/images/windows2016 + NFS_WINDOWS_DIR=/home/nfs/images/windows2016 + export NAMESPACE=kube-system + NAMESPACE=kube-system + trap '{ make cluster-down; }' EXIT SIGINT SIGTERM SIGSTOP + make cluster-down ./cluster/down.sh + make cluster-up ./cluster/up.sh Downloading ....... Downloading ....... 2018/07/26 10:39:50 Waiting for host: 192.168.66.101:22 2018/07/26 10:39:53 Problem with dial: dial tcp 192.168.66.101:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:40:01 Problem with dial: dial tcp 192.168.66.101:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:40:13 Connected to tcp://192.168.66.101:22 + kubeadm init --config /etc/kubernetes/kubeadm.conf [init] Using Kubernetes version: v1.10.3 [init] Using Authorization modes: [Node RBAC] [preflight] Running pre-flight checks. [WARNING FileExisting-crictl]: crictl not found in system path Suggestion: go get github.com/kubernetes-incubator/cri-tools/cmd/crictl Flag --admission-control has been deprecated, Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version. [certificates] Generated ca certificate and key. [certificates] Generated apiserver certificate and key. [certificates] apiserver serving cert is signed for DNS names [node01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.66.101] [certificates] Generated apiserver-kubelet-client certificate and key. [certificates] Generated etcd/ca certificate and key. [certificates] Generated etcd/server certificate and key. [certificates] etcd/server serving cert is signed for DNS names [localhost] and IPs [127.0.0.1] [certificates] Generated etcd/peer certificate and key. [certificates] etcd/peer serving cert is signed for DNS names [node01] and IPs [192.168.66.101] [certificates] Generated etcd/healthcheck-client certificate and key. [certificates] Generated apiserver-etcd-client certificate and key. [certificates] Generated sa key and public key. [certificates] Generated front-proxy-ca certificate and key. [certificates] Generated front-proxy-client certificate and key. [certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf" [controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml" [controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml" [controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml" [etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml" [init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests". [init] This might take a minute or longer if the control plane images have to be pulled. [apiclient] All control plane components are healthy after 26.509969 seconds [uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [markmaster] Will mark node node01 as master by adding a label and a taint [markmaster] Master node01 tainted and labelled with key/value: node-role.kubernetes.io/master="" [bootstraptoken] Using token: abcdef.1234567890123456 [bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [addons] Applied essential addon: kube-dns [addons] Applied essential addon: kube-proxy Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join 192.168.66.101:6443 --token abcdef.1234567890123456 --discovery-token-ca-cert-hash sha256:6984a27a3655c87ca21f63b9bbdcd6ab8f8068d8250aa433627753671a1494ae + kubectl --kubeconfig=/etc/kubernetes/admin.conf apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml clusterrole.rbac.authorization.k8s.io "flannel" created clusterrolebinding.rbac.authorization.k8s.io "flannel" created serviceaccount "flannel" created configmap "kube-flannel-cfg" created daemonset.extensions "kube-flannel-ds" created + kubectl --kubeconfig=/etc/kubernetes/admin.conf taint nodes node01 node-role.kubernetes.io/master:NoSchedule- node "node01" untainted 2018/07/26 10:41:04 Waiting for host: 192.168.66.102:22 2018/07/26 10:41:07 Problem with dial: dial tcp 192.168.66.102:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:41:15 Problem with dial: dial tcp 192.168.66.102:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:41:21 Problem with dial: dial tcp 192.168.66.102:22: getsockopt: connection refused. Sleeping 5s 2018/07/26 10:41:26 Connected to tcp://192.168.66.102:22 + kubeadm join --token abcdef.1234567890123456 192.168.66.101:6443 --ignore-preflight-errors=all --discovery-token-unsafe-skip-ca-verification=true [preflight] Running pre-flight checks. [discovery] Trying to connect to API Server "192.168.66.101:6443" [WARNING FileExisting-crictl]: crictl not found in system path Suggestion: go get github.com/kubernetes-incubator/cri-tools/cmd/crictl [discovery] Created cluster-info discovery client, requesting info from "https://192.168.66.101:6443" [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "192.168.66.101:6443" [discovery] Successfully established connection with API Server "192.168.66.101:6443" This node has joined the cluster: * Certificate signing request was sent to master and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the master to see this node join the cluster. Sending file modes: C0755 39588992 kubectl Sending file modes: C0600 5450 admin.conf Cluster "kubernetes" set. Cluster "kubernetes" set. + set +e + kubectl get nodes --no-headers + cluster/kubectl.sh get nodes --no-headers node01 Ready master 1m v1.10.3 node02 Ready 15s v1.10.3 + kubectl_rc=0 + '[' 0 -ne 0 ']' ++ kubectl get nodes --no-headers ++ cluster/kubectl.sh get nodes --no-headers ++ grep NotReady + '[' -n '' ']' + set -e + echo 'Nodes are ready:' Nodes are ready: + kubectl get nodes + cluster/kubectl.sh get nodes NAME STATUS ROLES AGE VERSION node01 Ready master 1m v1.10.3 node02 Ready 17s v1.10.3 + make cluster-sync ./cluster/build.sh Building ... sha256:dcf2b21fa2ed11dcf9dbba21b1cca0ee3fad521a0e9aee61c06d0b0b66a4b200 go version go1.10 linux/amd64 go version go1.10 linux/amd64 make[1]: Entering directory `/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt' hack/dockerized "./hack/check.sh && KUBEVIRT_VERSION= ./hack/build-go.sh install " && ./hack/build-copy-artifacts.sh sha256:dcf2b21fa2ed11dcf9dbba21b1cca0ee3fad521a0e9aee61c06d0b0b66a4b200 go version go1.10 linux/amd64 go version go1.10 linux/amd64 find: '/root/go/src/kubevirt.io/kubevirt/_out/cmd': No such file or directory Compiling tests... compiled tests.test hack/build-docker.sh build Sending build context to Docker daemon 40.37 MB Step 1/8 : FROM fedora:28 ---> cc510acfcd70 Step 2/8 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/8 : RUN useradd -u 1001 --create-home -s /bin/bash virt-controller ---> Using cache ---> b00c84523b53 Step 4/8 : WORKDIR /home/virt-controller ---> Using cache ---> b76b8bd8cd39 Step 5/8 : USER 1001 ---> Using cache ---> b6d9ad9ed232 Step 6/8 : COPY virt-controller /usr/bin/virt-controller ---> Using cache ---> 845fe74a71a8 Step 7/8 : ENTRYPOINT /usr/bin/virt-controller ---> Using cache ---> a2e1dca2ae16 Step 8/8 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "virt-controller" '' ---> Running in 172b2f187bca ---> 5eea9e7f1743 Removing intermediate container 172b2f187bca Successfully built 5eea9e7f1743 Sending build context to Docker daemon 43.3 MB Step 1/10 : FROM kubevirt/libvirt:4.2.0 ---> 5f0bfe81a3e0 Step 2/10 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 945996802736 Step 3/10 : RUN dnf -y install socat genisoimage util-linux libcgroup-tools ethtool net-tools sudo && dnf -y clean all && test $(id -u qemu) = 107 # make sure that the qemu user really is 107 ---> Using cache ---> 672f9ab56316 Step 4/10 : COPY virt-launcher /usr/bin/virt-launcher ---> Using cache ---> da718e0335b8 Step 5/10 : COPY kubevirt-sudo /etc/sudoers.d/kubevirt ---> Using cache ---> b08e20fcf072 Step 6/10 : RUN setcap CAP_NET_BIND_SERVICE=+eip /usr/bin/qemu-system-x86_64 ---> Using cache ---> b00411f9ceb3 Step 7/10 : RUN mkdir -p /usr/share/kubevirt/virt-launcher ---> Using cache ---> 2770d8d12615 Step 8/10 : COPY entrypoint.sh libvirtd.sh sock-connector /usr/share/kubevirt/virt-launcher/ ---> Using cache ---> ff27aff105c0 Step 9/10 : ENTRYPOINT /usr/share/kubevirt/virt-launcher/entrypoint.sh ---> Using cache ---> 455bdbace995 Step 10/10 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "virt-launcher" '' ---> Running in 0f807ce334be ---> 6be7f0b8a794 Removing intermediate container 0f807ce334be Successfully built 6be7f0b8a794 Sending build context to Docker daemon 41.67 MB Step 1/5 : FROM fedora:28 ---> cc510acfcd70 Step 2/5 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/5 : COPY virt-handler /usr/bin/virt-handler ---> Using cache ---> 0edb2a7803b2 Step 4/5 : ENTRYPOINT /usr/bin/virt-handler ---> Using cache ---> 8c695be5d077 Step 5/5 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "virt-handler" '' ---> Running in 6e2ade9ce657 ---> 9151724cbdd2 Removing intermediate container 6e2ade9ce657 Successfully built 9151724cbdd2 Sending build context to Docker daemon 38.81 MB Step 1/8 : FROM fedora:28 ---> cc510acfcd70 Step 2/8 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/8 : RUN useradd -u 1001 --create-home -s /bin/bash virt-api ---> Using cache ---> ed1ebf600ee1 Step 4/8 : WORKDIR /home/virt-api ---> Using cache ---> 0769dad023e5 Step 5/8 : USER 1001 ---> Using cache ---> 0cb65afb0c2b Step 6/8 : COPY virt-api /usr/bin/virt-api ---> Using cache ---> c2acee27a863 Step 7/8 : ENTRYPOINT /usr/bin/virt-api ---> Using cache ---> 6fe533d27c62 Step 8/8 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "virt-api" '' ---> Running in 56f431ed7764 ---> 7dffc44f78a3 Removing intermediate container 56f431ed7764 Successfully built 7dffc44f78a3 Sending build context to Docker daemon 4.096 kB Step 1/7 : FROM fedora:28 ---> cc510acfcd70 Step 2/7 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/7 : ENV container docker ---> Using cache ---> 62847a2a1fa8 Step 4/7 : RUN mkdir -p /images/custom /images/alpine && truncate -s 64M /images/custom/disk.img && curl http://dl-cdn.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso > /images/alpine/disk.img ---> Using cache ---> 02134835a6aa Step 5/7 : ADD entrypoint.sh / ---> Using cache ---> ec0843818da7 Step 6/7 : CMD /entrypoint.sh ---> Using cache ---> 754029bb4bd2 Step 7/7 : LABEL "disks-images-provider" '' "kubevirt-functional-tests-k8s-1.11.0-release0" '' ---> Running in 0affe5029745 ---> 2ce1eec77729 Removing intermediate container 0affe5029745 Successfully built 2ce1eec77729 Sending build context to Docker daemon 2.56 kB Step 1/5 : FROM fedora:28 ---> cc510acfcd70 Step 2/5 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/5 : ENV container docker ---> Using cache ---> 62847a2a1fa8 Step 4/5 : RUN dnf -y install procps-ng nmap-ncat && dnf -y clean all ---> Using cache ---> 207487abe7b2 Step 5/5 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "vm-killer" '' ---> Running in b1bfb379cabd ---> 51b62a0ecd10 Removing intermediate container b1bfb379cabd Successfully built 51b62a0ecd10 Sending build context to Docker daemon 5.12 kB Step 1/7 : FROM debian:sid ---> 68f33cf86aab Step 2/7 : MAINTAINER "David Vossel" \ ---> Using cache ---> 5734d749eb5c Step 3/7 : ENV container docker ---> Using cache ---> f8775a77966f Step 4/7 : RUN apt-get update && apt-get install -y bash curl bzip2 qemu-utils && mkdir -p /disk && rm -rf /var/lib/apt/lists/* ---> Using cache ---> 1a40cf222a61 Step 5/7 : ADD entry-point.sh / ---> Using cache ---> 77b545d92fe7 Step 6/7 : CMD /entry-point.sh ---> Using cache ---> dfe20d463305 Step 7/7 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "registry-disk-v1alpha" '' ---> Running in 343577507a71 ---> d47274c50623 Removing intermediate container 343577507a71 Successfully built d47274c50623 Sending build context to Docker daemon 2.56 kB Step 1/4 : FROM localhost:32860/kubevirt/registry-disk-v1alpha:devel ---> d47274c50623 Step 2/4 : MAINTAINER "David Vossel" \ ---> Running in 4068caee687c ---> 94c21c65872a Removing intermediate container 4068caee687c Step 3/4 : RUN curl https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img > /disk/cirros.img ---> Running in b9030631873b  % Total % Received % Xferd Average Speed Time Time Time Current  Dload Upload Total Spent Left Speed 0 0 0  0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 12.1M 0 49152 0 0 81783 0 0:02:35 --:--:--  0:02:35 81783 84 12.1M 84 10.2M 0 0 6631k 0 0:00:01 0:00:01 --:--:-- 6626k 100 12.1M 100 12.1M 0 0 7651k 0 0:00:01 0:00:01 --:--:-- 7646k  ---> 48605500da55 Removing intermediate container b9030631873b Step 4/4 : LABEL "cirros-registry-disk-demo" '' "kubevirt-functional-tests-k8s-1.11.0-release0" '' ---> Running in db2b7ffb4c7e ---> 4b8200fc9235 Removing intermediate container db2b7ffb4c7e Successfully built 4b8200fc9235 Sending build context to Docker daemon 2.56 kB Step 1/4 : FROM localhost:32860/kubevirt/registry-disk-v1alpha:devel ---> d47274c50623 Step 2/4 : MAINTAINER "The KubeVirt Project" ---> Running in d08c619b7b7d ---> 8e698fc18d9d Removing intermediate container d08c619b7b7d Step 3/4 : RUN curl -g -L https://download.fedoraproject.org/pub/fedora/linux/releases/27/CloudImages/x86_64/images/Fedora-Cloud-Base-27-1.6.x86_64.qcow2 > /disk/fedora.qcow2 ---> Running in e72acfc3d4e7  % Total % Received % Xferd Average Speed Time Time Time Current   Dload Upload  Total Spent Left Speed 0 0 0 0 0 0  0   0 --:--:-- --:--:-- --:--:--   0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--  0 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0 0 0 0 0 0  0 0 0 --:--:-- 0:00:01 --:--:--  0  0 221M 0 103k  0  0 41776 0 1:32:41 0:00:02 1:32:39 41776 0 221M 0 1231k  0   0 349k   0 0:10:48 0:00:03 0:10:45 1157k 1 221M 1 2473k 0 0 549k 0 0:06:52 0:00:04 0:06:48 1214k 1 221M 1 3811k 0 0 695k 0 0:05:26 0:00:05 0:05:21 1264k 2 221M 2 5386k 0 0 828k 0 0:04:33 0:00:06 0:04:27 1336k 3 221M 3 6902k 0 0 923k 0  0:04:05 0:00:07 0:03:58 1379k 3 221M 3 8422k 0 0 993k 0 0:03:48 0:00:08 0:03:40 1450k 4 221M 4 9.8M 0 0 1064k  0 0:03:33 0:00:09 0:03:24 1527k 5 221M 5 11.5M 0 0 1124k 0 0:03:21 0:00:10 0:03:11 1594k 5 221M 5 13.0M 0 0 1140k 0 0:03:18 0:00:11 0:03:07 1530k 6 221M 6 13.7M 0 0 1129k  0 0:03:20 0:00:12 0:03:08 1437k 6 221M 6 14.5M 0 0 1101k 0 0:03:25 0:00:13 0:03:12 1284k 6 221M 6 15.3M 0 0 1083k 0 0:03:29 0:00:14 0:03:15 1119k 7 221M 7 16.2M 0 0 1075k 0 0:03:30 0:00:15 0:03:15 972k 7 221M 7 17.3M 0 0 1074k 0  0:03:31 0:00:16 0:03:15 914k 8 221M 8 18.4M 0 0 1078k 0 0:03:30 0:00:17 0:03:13 951k 8 221M 8 19.6M 0 0 1089k 0 0:03:28 0:00:18 0:03:10 1055k 9 221M 9 21.0M 0 0 1105k 0 0:03:25 0:00:19 0:03:06 1168k 10 221M 10 22.8M  0 0 1140k   0 0:03:18 0:00:20 0:02:58 1341k 11 221M 11 25.2M 0 0 1204k 0 0:03:08 0:00:21 0:02:47 1636k 12 221M 12 27.7M  0 0 1265k  0 0:02:59 0:00:22 0:02:37 1918k 13 221M 13 30.4M 0 0 1325k 0 0:02:51  0:00:23  0:02:28 2200k 15 221M 15 33.2M 0 0 1391k 0 0:02:43 0:00:24 0:02:19 2506k 16 221M 16 36.3M 0 0 1460k 0 0:02:35 0:00:25  0:02:10 2772k 17 221M 17 39.4M 0 0 1524k  0 0:02:28 0:00:26 0:02:02 2897k 19 221M 19 42.4M 0 0 1583k  0 0:02:23 0:00:27 0:01:56 3011k 20 221M  20 45.5M  0   0 1638k   0 0:02:18  0:00:28  0:01:50 3115k 22 221M 22 48.7M 0 0 1693k  0 0:02:13 0:00:29 0:01:44 3165k 23 221M 23 51.9M 0 0 1745k 0 0:02:09 0:00:30 0:01:39 3198k 25 221M 25 55.4M 0 0 1802k 0 0:02:05 0:00:31 0:01:34 3282k 26 221M 26 58.0M 0 0 1827k  0 0:02:04 0:00:32 0:01:32 3165k 27 221M 27 60.0M 0 0 1836k 0 0:02:03 0:00:33 0:01:30 2961k 28 221M 28 62.2M 0 0 1849k 0 0:02:02  0:00:34 0:01:28 2772k 29 221M 29 64.4M 0 0 1860k  0 0:02:01 0:00:35 0:01:26 2558k 29 221M 29 66.2M 0 0 1859k 0 0:02:02 0:00:36 0:01:26 2217k 30 221M 30 68.2M 0 0 1865k 0 0:02:01 0:00:37 0:01:24 2108k  31 221M  31 70.5M  0 0 1877k 0 0:02:00 0:00:38 0:01:22 2147k 32 221M 32 72.7M 0 0 1887k  0 0:02:00 0:00:39  0:01:21 2150k 33 221M 33 75.0M 0 0 1896k 0 0:01:59  0:00:40 0:01:19 2154k 34 221M 34 77.2M 0 0 1907k    0 0:01:58 0:00:41 0:01:17 2254k 35 221M 35 79.3M 0 0 1907k 0 0:01:58 0:00:42 0:01:16 2221k 36 221M 36 80.7M  0 0 1900k 0 0:01:59 0:00:43 0:01:16 2080k 37 221M 37 82.1M 0 0 1891k  0 0:01:59 0:00:44 0:01:15 1917k 37 221M 37 83.6M 0 0 1883k 0 0:02:00 0:00:45 0:01:15 1773k 38 221M 38 85.2M 0 0 1876k 0 0:02:00 0:00:46 0:01:14 1622k 39 221M 39 86.8M 0 0 1872k 0 0:02:01 0:00:47 0:01:14 1561k 39 221M 39 88.2M 0 0 1863k 0 0:02:01 0:00:48 0:01:13 1544k 40 221M 40 89.8M 0 0 1857k  0 0:02:02 0:00:49  0:01:13 1565k 41 221M 41 91.2M 0 0 1851k 0 0:02:02  0:00:50 0:01:12 1562k 41 221M 41 92.6M 0 0 1842k 0 0:02:03 0:00:51 0:01:12 1524k 42 221M 42 93.8M 0 0 1830k 0 0:02:03 0:00:52 0:01:11 1434k 42 221M 42 94.9M 0 0 1818k 0 0:02:04 0:00:53 0:01:11 1373k 43 221M 43 96.1M 0 0 1807k 0 0:02:05 0:00:54 0:01:11 1305k 43 221M 43 97.4M 0 0 1799k  0 0:02:06 0:00:55 0:01:11 1271k 44 221M 44 98.8M 0 0 1792k 0 0:02:06 0:00:56 0:01:10 1282k 45 221M 45 100M 0 0 1788k 0 0:02:06 0:00:57 0:01:09 1347k 46 221M 46 101M 0 0 1785k 0 0:02:07 0:00:58 0:01:09 1441k 46 221M 46 103M 0 0 1785k 0 0:02:07 0:00:59 0:01:08 1550k 47 221M 47 105M 0 0 1789k 0 0:02:06  0:01:00 0:01:06 1681k 48 221M 48 108M 0 0 1801k 0 0:02:05 0:01:01 0:01:04 1897k 49 221M 49 110M 0 0 1808k  0 0:02:05 0:01:02 0:01:03 2031k 50 221M 50 112M 0 0 1819k 0 0:02:04 0:01:03 0:01:01 2213k 52 221M 52 115M 0 0 1833k 0  0:02:03 0:01:04 0:00:59 2405k  53 221M 53  118M  0  0 1848k 0 0:02:02 0:01:05 0:00:57 2565k 54 221M 54 120M 0 0 1863k 0 0:02:01 0:01:06 0:00:55 2628k 55 221M 55  123M 0 0 1877k 0 0:02:00 0:01:07 0:00:53 2748k 57 221M 57 126M 0 0 1892k 0 0:01:59  0:01:08 0:00:51 2811k 58 221M 58 129M 0 0 1906k 0 0:01:58  0:01:09 0:00:49 2849k 59 221M 59 132M  0 0 1924k 0 0:01:57  0:01:10 0:00:47 2907k 61 221M 61 135M 0 0 1944k 0 0:01:56 0:01:11 0:00:45 3018k 62 221M 62 139M 0  0 1971k 0 0:01:55 0:01:12 0:00:43 3242k 64 221M 64 143M 0 0 1996k  0 0:01:53  0:01:13 0:00:40 3390k 65 221M 65 145M 0 0 1994k 0 0:01:53 0:01:14 0:00:39 3206k 66 221M 66 146M 0 0 1986k 0 0:01:54 0:01:15 0:00:39 2870k 66 221M 66 147M 0 0 1979k 0 0:01:54 0:01:16 0:00:38 2483k 67 221M 67 149M 0 0 1973k  0 0:01:54 0:01:17 0:00:37 2000k 68 221M 68 150M 0 0 1969k 0 0:01:55 0:01:18 0:00:37 1564k 68 221M 68 152M 0 0 1966k  0 0:01:55 0:01:19 0:00:36 1551k 69 221M 69 154M 0 0 1964k 0 0:01:55 0:01:20 0:00:35 1637k 70 221M 70 156M 0 0 1965k  0 0:01:55 0:01:21  0:00:34 1746k 71 221M 71 158M 0 0 1966k  0 0:01:55 0:01:22 0:00:33 1857k 72 221M 72 160M 0 0 1973k 0 0:01:54  0:01:23 0:00:31 2035k 73 221M 73 163M 0 0 1980k 0 0:01:54 0:01:24 0:00:30 2199k 74 221M  74 165M  0  0 1986k  0 0:01:54 0:01:25 0:00:29 2344k 76 221M 76 168M  0 0 1997k  0 0:01:53 0:01:26 0:00:27 2512k  77 221M 77  171M 0  0 2008k 0 0:01:52 0:01:27  0:00:25 2693k 78 221M 78  174M 0 0 2019k 0  0:01:52 0:01:28 0:00:24 2788k 80 221M 80 177M 0 0 2031k   0 0:01:51 0:01:29 0:00:22 2889k 81 221M 81 180M 0  0 2042k 0 0:01:51 0:01:30  0:00:21 2982k 82 221M 82 183M 0 0 2053k 0 0:01:50 0:01:31 0:00:19 3028k 83 221M 83 185M 0 0 2058k 0 0:01:50 0:01:32 0:00:18 2935k 85 221M 85  188M   0 0 2065k  0 0:01:49 0:01:33 0:00:16 2880k  86 221M 86 191M  0 0  2075k 0 0:01:49  0:01:34 0:00:15 2867k 87 221M   87 193M 0  0 2079k  0 0:01:49 0:01:35 0:00:14 2763k 88 221M 88  196M 0 0 2083k    0 0:01:48 0:01:36 0:00:12 2641k 89 221M 89 198M 0 0 2089k  0 0:01:48 0:01:37 0:00:11 2663k 90 221M 90 201M  0 0 2094k 0 0:01:48 0:01:38 0:00:10 2628k 92 221M 92 203M 0  0 2097k 0 0:01:48 0:01:39 0:00:09 2523k 92 221M 92 205M 0 0 2098k 0 0:01:48 0:01:40 0:00:08 2446k 93 221M 93 207M  0  0 2097k   0 0:01:48 0:01:41 0:00:07 2366k 94 221M 94 209M 0 0 2095k 0 0:01:48 0:01:42 0:00:06 2219k 95 221M 95 211M 0 0 2090k 0 0:01:48 0:01:43  0:00:05 2012k 96  221M 96  212M 0 0 2085k 0 0:01:48 0:01:44 0:00:04 1836k 96 221M 96 214M 0 0 2081k 0 0:01:48 0:01:45 0:00:03 1747k 97 221M 97 216M 0 0 2078k 0 0:01:49 0:01:46 0:00:03 1679k 98 221M 98 217M 0 0 2076k 0 0:01:49  0:01:47 0:00:02 1673k 99 221M 99 219M 0 0 2075k 0 0:01:49 0:01:48 0:00:01 1760k 100 221M 100 221M 0 0 2074k 0 0:01:49 0:01:49 --:--:-- 1850k  ---> e9c68c4dc91c Removing intermediate container e72acfc3d4e7 Step 4/4 : LABEL "fedora-cloud-registry-disk-demo" '' "kubevirt-functional-tests-k8s-1.11.0-release0" '' ---> Running in 0fe819d9d16c ---> 98232324771d Removing intermediate container 0fe819d9d16c Successfully built 98232324771d Sending build context to Docker daemon 2.56 kB Step 1/4 : FROM localhost:32860/kubevirt/registry-disk-v1alpha:devel ---> d47274c50623 Step 2/4 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 8e698fc18d9d Step 3/4 : RUN curl http://dl-cdn.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso > /disk/alpine.iso ---> Running in a7c47b114bc6  % Total % Received % Xferd Average Speed Time Time Time Current  Dload  Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2 37.0M 2 800k 0 0 1878k 0 0:00:20 --:--:-- 0:00:20 1878k 28 37.0M 28 10.4M 0 0 7492k 0 0:00:05 0:00:01 0:00:04 7487k 100 37.0M 100 37.0M 0 0 16.0M 0 0:00:02 0:00:02 --:--:-- 16.0M  ---> 38176d1a3214 Removing intermediate container a7c47b114bc6 Step 4/4 : LABEL "alpine-registry-disk-demo" '' "kubevirt-functional-tests-k8s-1.11.0-release0" '' ---> Running in 4b77f5eb1080 ---> 8c0a79fc816d Removing intermediate container 4b77f5eb1080 Successfully built 8c0a79fc816d Sending build context to Docker daemon 35.59 MB Step 1/8 : FROM fedora:28 ---> cc510acfcd70 Step 2/8 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/8 : RUN useradd -u 1001 --create-home -s /bin/bash virtctl ---> Using cache ---> 985fe391c056 Step 4/8 : WORKDIR /home/virtctl ---> Using cache ---> 3b2cae8ac543 Step 5/8 : USER 1001 ---> Using cache ---> 0c06e5b4a900 Step 6/8 : COPY subresource-access-test /subresource-access-test ---> Using cache ---> 60c9649ce3ab Step 7/8 : ENTRYPOINT /subresource-access-test ---> Using cache ---> 764db85f918c Step 8/8 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "subresource-access-test" '' ---> Running in d82b4922bb22 ---> c51c4ae65da4 Removing intermediate container d82b4922bb22 Successfully built c51c4ae65da4 Sending build context to Docker daemon 3.072 kB Step 1/9 : FROM fedora:28 ---> cc510acfcd70 Step 2/9 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> bfe77d5699ed Step 3/9 : ENV container docker ---> Using cache ---> 62847a2a1fa8 Step 4/9 : RUN dnf -y install make git gcc && dnf -y clean all ---> Using cache ---> d3456b1644b1 Step 5/9 : ENV GIMME_GO_VERSION 1.9.2 ---> Using cache ---> 0ba81fddbba1 Step 6/9 : RUN mkdir -p /gimme && curl -sL https://raw.githubusercontent.com/travis-ci/gimme/master/gimme | HOME=/gimme bash >> /etc/profile.d/gimme.sh ---> Using cache ---> 5d33abe3f819 Step 7/9 : ENV GOPATH "/go" GOBIN "/usr/bin" ---> Using cache ---> 783826523be1 Step 8/9 : RUN mkdir -p /go && source /etc/profile.d/gimme.sh && go get github.com/masterzen/winrm-cli ---> Using cache ---> 711bc8d15952 Step 9/9 : LABEL "kubevirt-functional-tests-k8s-1.11.0-release0" '' "winrmcli" '' ---> Running in 709766babce9 ---> 671374ffcc63 Removing intermediate container 709766babce9 Successfully built 671374ffcc63 Sending build context to Docker daemon 36.78 MB Step 1/5 : FROM fedora:27 ---> 9110ae7f579f Step 2/5 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> e3238544ad97 Step 3/5 : COPY example-hook-sidecar /example-hook-sidecar ---> Using cache ---> 726968e92e6c Step 4/5 : ENTRYPOINT /example-hook-sidecar ---> Using cache ---> 238676912ae9 Step 5/5 : LABEL "example-hook-sidecar" '' "kubevirt-functional-tests-k8s-1.11.0-release0" '' ---> Running in dbfcb27b7731 ---> db61718d8f78 Removing intermediate container dbfcb27b7731 Successfully built db61718d8f78 hack/build-docker.sh push The push refers to a repository [localhost:32860/kubevirt/virt-controller] 5fffd2af17d7: Preparing aa89340cf7a8: Preparing 891e1e4ef82a: Preparing aa89340cf7a8: Pushed 5fffd2af17d7: Pushed 891e1e4ef82a: Pushed devel: digest: sha256:a958e18460185a07ecc8ee2efb006e6e8f82228986b2bf1d766fd704453d473b size: 949 The push refers to a repository [localhost:32860/kubevirt/virt-launcher] fc9921594a7c: Preparing cbbfc893304c: Preparing 3a0ba8999a7f: Preparing 93326e288fcf: Preparing b9717cc5681f: Preparing 633427c64a24: Preparing da38cf808aa5: Preparing b83399358a92: Preparing 186d8b3e4fd8: Preparing da38cf808aa5: Waiting fa6154170bf5: Preparing 186d8b3e4fd8: Waiting 5eefb9960a36: Preparing fa6154170bf5: Waiting 891e1e4ef82a: Preparing 5eefb9960a36: Waiting cbbfc893304c: Pushed 93326e288fcf: Pushed fc9921594a7c: Pushed da38cf808aa5: Pushed b83399358a92: Pushed fa6154170bf5: Pushed 186d8b3e4fd8: Pushed 891e1e4ef82a: Mounted from kubevirt/virt-controller 3a0ba8999a7f: Pushed 633427c64a24: Pushed b9717cc5681f: Pushed 5eefb9960a36: Pushed devel: digest: sha256:a837e352e82378a2452d2a064c52b201a08281c485c76d2c27e8738fc2cfc023 size: 2828 The push refers to a repository [localhost:32860/kubevirt/virt-handler] 7e4b7a575caa: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/virt-launcher 7e4b7a575caa: Pushed devel: digest: sha256:1f7a47ef0b7aa188f42b8f9d4bc8c6ece33f863f7027ba623ea9d6b1b887754a size: 741 The push refers to a repository [localhost:32860/kubevirt/virt-api] 0fdb68624272: Preparing 82fc744c99b4: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/virt-handler 82fc744c99b4: Pushed 0fdb68624272: Pushed devel: digest: sha256:7bf36422ff4e8d82b86805be55c759c781bf3082f65469c6d6957752c7e36dc9 size: 948 The push refers to a repository [localhost:32860/kubevirt/disks-images-provider] 71ad31feb2c5: Preparing 21d4b721776e: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/virt-api 71ad31feb2c5: Pushed 21d4b721776e: Pushed devel: digest: sha256:cd187e542554d6c701bfb0cb08f65a76d23fa75be34176dc4188c76ec468c9f4 size: 948 The push refers to a repository [localhost:32860/kubevirt/vm-killer] c4cfadeeaf5f: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/disks-images-provider c4cfadeeaf5f: Pushed devel: digest: sha256:72afae217b15b5f36f1509e4a78d76f16b21e607b318d2e701cfe8567738c9dc size: 740 The push refers to a repository [localhost:32860/kubevirt/registry-disk-v1alpha] 661cce8d8e52: Preparing 41e0baba3077: Preparing 25edbec0eaea: Preparing 661cce8d8e52: Pushed 41e0baba3077: Pushed 25edbec0eaea: Pushed devel: digest: sha256:4beab74b750c384fed6e9272292cd4f3885cf63160b4cc8be85dcb7eb12fd910 size: 948 The push refers to a repository [localhost:32860/kubevirt/cirros-registry-disk-demo] 796d82cd42db: Preparing 661cce8d8e52: Preparing 41e0baba3077: Preparing 25edbec0eaea: Preparing 41e0baba3077: Mounted from kubevirt/registry-disk-v1alpha 661cce8d8e52: Mounted from kubevirt/registry-disk-v1alpha 25edbec0eaea: Mounted from kubevirt/registry-disk-v1alpha 796d82cd42db: Pushed devel: digest: sha256:caa473d35dda5da1c955c07c18584bca173fc3ac1655c04509b42465d78e6565 size: 1160 The push refers to a repository [localhost:32860/kubevirt/fedora-cloud-registry-disk-demo] 3cf3799e71d4: Preparing 661cce8d8e52: Preparing 41e0baba3077: Preparing 25edbec0eaea: Preparing 41e0baba3077: Mounted from kubevirt/cirros-registry-disk-demo 661cce8d8e52: Mounted from kubevirt/cirros-registry-disk-demo 25edbec0eaea: Mounted from kubevirt/cirros-registry-disk-demo 3cf3799e71d4: Pushed devel: digest: sha256:949d4de4d95938a99ac539d20780234a3f06cc98e3d0113d383d45e98b360c10 size: 1161 The push refers to a repository [localhost:32860/kubevirt/alpine-registry-disk-demo] c59c11514491: Preparing 661cce8d8e52: Preparing 41e0baba3077: Preparing 25edbec0eaea: Preparing 25edbec0eaea: Mounted from kubevirt/fedora-cloud-registry-disk-demo 661cce8d8e52: Mounted from kubevirt/fedora-cloud-registry-disk-demo 41e0baba3077: Mounted from kubevirt/fedora-cloud-registry-disk-demo c59c11514491: Pushed devel: digest: sha256:f95f7a6825d39ae4570cd8b9e1fb8d77f390e328d9d8a0e516f222809071b61c size: 1160 The push refers to a repository [localhost:32860/kubevirt/subresource-access-test] 5f17182f8235: Preparing 25cb73590a9d: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/vm-killer 25cb73590a9d: Pushed 5f17182f8235: Pushed devel: digest: sha256:e61a760c9df0adde0b2995a1a039074d06c14aabd1a70cde3332f1c151ea264e size: 948 The push refers to a repository [localhost:32860/kubevirt/winrmcli] f8083e002d0b: Preparing 53c709abc882: Preparing 9ca98a0f492b: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/subresource-access-test f8083e002d0b: Pushed 9ca98a0f492b: Pushed 53c709abc882: Pushed devel: digest: sha256:a1fae1967d100edf5b23924a81a158c98d867422874226220754a604b2c256af size: 1165 The push refers to a repository [localhost:32860/kubevirt/example-hook-sidecar] 54e0f0f027d9: Preparing 39bae602f753: Preparing 54e0f0f027d9: Pushed 39bae602f753: Pushed devel: digest: sha256:29356b63b8c16bc444aa3c870837148a5b856bfbd9081c7ce06970f544290325 size: 740 make[1]: Leaving directory `/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt' Done ./cluster/clean.sh + source hack/common.sh ++++ dirname 'hack/common.sh[0]' +++ cd hack/../ +++ pwd ++ KUBEVIRT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt ++ OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out ++ VENDOR_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/vendor ++ CMD_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/cmd ++ TESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/tests ++ APIDOCS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/apidocs ++ MANIFESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests ++ MANIFEST_TEMPLATES_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/templates/manifests ++ PYTHON_CLIENT_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/client-python ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_NUM_NODES=2 ++ '[' -z kubevirt-functional-tests-k8s-1.11.0-release ']' ++ provider_prefix=kubevirt-functional-tests-k8s-1.11.0-release0 ++ job_prefix=kubevirt-functional-tests-k8s-1.11.0-release0 +++ kubevirt_version +++ '[' -n '' ']' +++ '[' -d /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/.git ']' ++++ git describe --always --tags +++ echo v0.7.0-126-g81b07a4 ++ KUBEVIRT_VERSION=v0.7.0-126-g81b07a4 + source cluster/k8s-1.10.3/provider.sh ++ set -e ++ image=k8s-1.10.3@sha256:d6290260e7e6b84419984f12719cf592ccbe327373b8df76aa0481f8ec01d357 ++ source cluster/ephemeral-provider-common.sh +++ set -e +++ _cli='docker run --privileged --net=host --rm -v /var/run/docker.sock:/var/run/docker.sock kubevirtci/gocli@sha256:aa7f295a7908fa333ab5e98ef3af0bfafbabfd3cee2b83f9af47f722e3000f6a' + source hack/config.sh ++ unset binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig manifest_docker_prefix namespace ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ source hack/config-default.sh source hack/config-k8s-1.10.3.sh +++ binaries='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virtctl cmd/fake-qemu-process cmd/virt-api cmd/subresource-access-test cmd/example-hook-sidecar' +++ docker_images='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virt-api images/disks-images-provider images/vm-killer cmd/registry-disk-v1alpha images/cirros-registry-disk-demo images/fedora-cloud-registry-disk-demo images/alpine-registry-disk-demo cmd/subresource-access-test images/winrmcli cmd/example-hook-sidecar' +++ docker_prefix=kubevirt +++ docker_tag=latest +++ master_ip=192.168.200.2 +++ network_provider=flannel +++ namespace=kube-system ++ test -f hack/config-provider-k8s-1.10.3.sh ++ source hack/config-provider-k8s-1.10.3.sh +++ master_ip=127.0.0.1 +++ docker_tag=devel +++ kubeconfig=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubeconfig +++ kubectl=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubectl +++ docker_prefix=localhost:32860/kubevirt +++ manifest_docker_prefix=registry:5000/kubevirt ++ test -f hack/config-local.sh ++ export binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig namespace + echo 'Cleaning up ...' Cleaning up ... + cluster/kubectl.sh get vmis --all-namespaces -o=custom-columns=NAME:.metadata.name,NAMESPACE:.metadata.namespace,FINALIZERS:.metadata.finalizers --no-headers + grep foregroundDeleteVirtualMachine + read p error: the server doesn't have a resource type "vmis" + _kubectl delete ds -l kubevirt.io -n kube-system --cascade=false --grace-period 0 No resources found + _kubectl delete pods -n kube-system -l=kubevirt.io=libvirt --force --grace-period 0 No resources found + _kubectl delete pods -n kube-system -l=kubevirt.io=virt-handler --force --grace-period 0 No resources found + namespaces=(default ${namespace}) + for i in '${namespaces[@]}' + _kubectl -n default delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete apiservices -l kubevirt.io No resources found + _kubectl -n default delete deployment -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete deployment -l kubevirt.io No resources found + _kubectl -n default delete rs -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete rs -l kubevirt.io No resources found + _kubectl -n default delete services -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete services -l kubevirt.io No resources found + _kubectl -n default delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete apiservices -l kubevirt.io No resources found + _kubectl -n default delete validatingwebhookconfiguration -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete validatingwebhookconfiguration -l kubevirt.io No resources found + _kubectl -n default delete secrets -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete secrets -l kubevirt.io No resources found + _kubectl -n default delete pv -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete pv -l kubevirt.io No resources found + _kubectl -n default delete pvc -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete pvc -l kubevirt.io No resources found + _kubectl -n default delete ds -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete ds -l kubevirt.io No resources found + _kubectl -n default delete customresourcedefinitions -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete customresourcedefinitions -l kubevirt.io No resources found + _kubectl -n default delete pods -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete pods -l kubevirt.io No resources found + _kubectl -n default delete clusterrolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete clusterrolebinding -l kubevirt.io No resources found + _kubectl -n default delete rolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete rolebinding -l kubevirt.io No resources found + _kubectl -n default delete roles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete roles -l kubevirt.io No resources found + _kubectl -n default delete clusterroles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete clusterroles -l kubevirt.io No resources found + _kubectl -n default delete serviceaccounts -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete serviceaccounts -l kubevirt.io No resources found ++ _kubectl -n default get crd offlinevirtualmachines.kubevirt.io ++ wc -l ++ export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ cluster/k8s-1.10.3/.kubectl -n default get crd offlinevirtualmachines.kubevirt.io Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "offlinevirtualmachines.kubevirt.io" not found + '[' 0 -gt 0 ']' + for i in '${namespaces[@]}' + _kubectl -n kube-system delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete apiservices -l kubevirt.io No resources found + _kubectl -n kube-system delete deployment -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete deployment -l kubevirt.io No resources found + _kubectl -n kube-system delete rs -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete rs -l kubevirt.io No resources found + _kubectl -n kube-system delete services -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete services -l kubevirt.io No resources found + _kubectl -n kube-system delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete apiservices -l kubevirt.io No resources found + _kubectl -n kube-system delete validatingwebhookconfiguration -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete validatingwebhookconfiguration -l kubevirt.io No resources found + _kubectl -n kube-system delete secrets -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete secrets -l kubevirt.io No resources found + _kubectl -n kube-system delete pv -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete pv -l kubevirt.io No resources found + _kubectl -n kube-system delete pvc -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete pvc -l kubevirt.io No resources found + _kubectl -n kube-system delete ds -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete ds -l kubevirt.io No resources found + _kubectl -n kube-system delete customresourcedefinitions -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete customresourcedefinitions -l kubevirt.io No resources found + _kubectl -n kube-system delete pods -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete pods -l kubevirt.io No resources found + _kubectl -n kube-system delete clusterrolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete clusterrolebinding -l kubevirt.io No resources found + _kubectl -n kube-system delete rolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete rolebinding -l kubevirt.io No resources found + _kubectl -n kube-system delete roles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete roles -l kubevirt.io No resources found + _kubectl -n kube-system delete clusterroles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete clusterroles -l kubevirt.io No resources found + _kubectl -n kube-system delete serviceaccounts -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete serviceaccounts -l kubevirt.io No resources found ++ _kubectl -n kube-system get crd offlinevirtualmachines.kubevirt.io ++ wc -l ++ export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ cluster/k8s-1.10.3/.kubectl -n kube-system get crd offlinevirtualmachines.kubevirt.io Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "offlinevirtualmachines.kubevirt.io" not found + '[' 0 -gt 0 ']' + sleep 2 + echo Done Done ./cluster/deploy.sh + source hack/common.sh ++++ dirname 'hack/common.sh[0]' +++ cd hack/../ +++ pwd ++ KUBEVIRT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt ++ OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out ++ VENDOR_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/vendor ++ CMD_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/cmd ++ TESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/tests ++ APIDOCS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/apidocs ++ MANIFESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests ++ MANIFEST_TEMPLATES_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/templates/manifests ++ PYTHON_CLIENT_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/client-python ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_NUM_NODES=2 ++ '[' -z kubevirt-functional-tests-k8s-1.11.0-release ']' ++ provider_prefix=kubevirt-functional-tests-k8s-1.11.0-release0 ++ job_prefix=kubevirt-functional-tests-k8s-1.11.0-release0 +++ kubevirt_version +++ '[' -n '' ']' +++ '[' -d /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/.git ']' ++++ git describe --always --tags +++ echo v0.7.0-126-g81b07a4 ++ KUBEVIRT_VERSION=v0.7.0-126-g81b07a4 + source cluster/k8s-1.10.3/provider.sh ++ set -e ++ image=k8s-1.10.3@sha256:d6290260e7e6b84419984f12719cf592ccbe327373b8df76aa0481f8ec01d357 ++ source cluster/ephemeral-provider-common.sh +++ set -e +++ _cli='docker run --privileged --net=host --rm -v /var/run/docker.sock:/var/run/docker.sock kubevirtci/gocli@sha256:aa7f295a7908fa333ab5e98ef3af0bfafbabfd3cee2b83f9af47f722e3000f6a' + source hack/config.sh ++ unset binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig manifest_docker_prefix namespace ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ source hack/config-default.sh source hack/config-k8s-1.10.3.sh +++ binaries='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virtctl cmd/fake-qemu-process cmd/virt-api cmd/subresource-access-test cmd/example-hook-sidecar' +++ docker_images='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virt-api images/disks-images-provider images/vm-killer cmd/registry-disk-v1alpha images/cirros-registry-disk-demo images/fedora-cloud-registry-disk-demo images/alpine-registry-disk-demo cmd/subresource-access-test images/winrmcli cmd/example-hook-sidecar' +++ docker_prefix=kubevirt +++ docker_tag=latest +++ master_ip=192.168.200.2 +++ network_provider=flannel +++ namespace=kube-system ++ test -f hack/config-provider-k8s-1.10.3.sh ++ source hack/config-provider-k8s-1.10.3.sh +++ master_ip=127.0.0.1 +++ docker_tag=devel +++ kubeconfig=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubeconfig +++ kubectl=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubectl +++ docker_prefix=localhost:32860/kubevirt +++ manifest_docker_prefix=registry:5000/kubevirt ++ test -f hack/config-local.sh ++ export binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig namespace + echo 'Deploying ...' Deploying ... + [[ -z k8s-1.11.0-release ]] + [[ k8s-1.11.0-release =~ .*-dev ]] + [[ k8s-1.11.0-release =~ .*-release ]] + for manifest in '${MANIFESTS_OUT_DIR}/release/*' + [[ /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/demo-content.yaml =~ .*demo.* ]] + continue + for manifest in '${MANIFESTS_OUT_DIR}/release/*' + [[ /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/kubevirt.yaml =~ .*demo.* ]] + _kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/kubevirt.yaml + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/kubevirt.yaml clusterrole.rbac.authorization.k8s.io "kubevirt.io:admin" created clusterrole.rbac.authorization.k8s.io "kubevirt.io:edit" created clusterrole.rbac.authorization.k8s.io "kubevirt.io:view" created serviceaccount "kubevirt-apiserver" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-apiserver" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-apiserver-auth-delegator" created rolebinding.rbac.authorization.k8s.io "kubevirt-apiserver" created role.rbac.authorization.k8s.io "kubevirt-apiserver" created clusterrole.rbac.authorization.k8s.io "kubevirt-apiserver" created clusterrole.rbac.authorization.k8s.io "kubevirt-controller" created serviceaccount "kubevirt-controller" created serviceaccount "kubevirt-privileged" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-controller" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-controller-cluster-admin" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-privileged-cluster-admin" created clusterrole.rbac.authorization.k8s.io "kubevirt.io:default" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt.io:default" created service "virt-api" created deployment.extensions "virt-api" created deployment.extensions "virt-controller" created daemonset.extensions "virt-handler" created customresourcedefinition.apiextensions.k8s.io "virtualmachineinstances.kubevirt.io" created customresourcedefinition.apiextensions.k8s.io "virtualmachineinstancereplicasets.kubevirt.io" created customresourcedefinition.apiextensions.k8s.io "virtualmachineinstancepresets.kubevirt.io" created customresourcedefinition.apiextensions.k8s.io "virtualmachines.kubevirt.io" created + _kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests/testing -R + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/go/src/kubevirt.io/kubevirt/_out/manifests/testing -R persistentvolumeclaim "disk-alpine" created persistentvolume "host-path-disk-alpine" created persistentvolumeclaim "disk-custom" created persistentvolume "host-path-disk-custom" created daemonset.extensions "disks-images-provider" created serviceaccount "kubevirt-testing" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-testing-cluster-admin" created + [[ k8s-1.10.3 =~ os-* ]] + echo Done Done + namespaces=(kube-system default) + [[ kube-system != \k\u\b\e\-\s\y\s\t\e\m ]] + timeout=300 + sample=30 + for i in '${namespaces[@]}' + current_time=0 ++ kubectl get pods -n kube-system --no-headers ++ cluster/kubectl.sh get pods -n kube-system --no-headers ++ grep -v Running + '[' -n 'virt-api-7d79764579-c2ffk 0/1 ContainerCreating 0 7s virt-api-7d79764579-lpjfs 0/1 ContainerCreating 0 7s virt-controller-7d57d96b65-bdcj9 0/1 ContainerCreating 0 7s virt-handler-5pv4z 0/1 ContainerCreating 0 7s virt-handler-7x9x5 0/1 ContainerCreating 0 7s' ']' + echo 'Waiting for kubevirt pods to enter the Running state ...' Waiting for kubevirt pods to enter the Running state ... + kubectl get pods -n kube-system --no-headers + cluster/kubectl.sh get pods -n kube-system --no-headers + grep -v Running disks-images-provider-kcn9c 0/1 Pending 0 2s disks-images-provider-mbnmx 0/1 Pending 0 1s virt-api-7d79764579-c2ffk 0/1 ContainerCreating 0 8s virt-api-7d79764579-lpjfs 0/1 ContainerCreating 0 8s virt-controller-7d57d96b65-bdcj9 0/1 ContainerCreating 0 8s virt-handler-5pv4z 0/1 ContainerCreating 0 8s virt-handler-7x9x5 0/1 ContainerCreating 0 8s + sleep 30 + current_time=30 + '[' 30 -gt 300 ']' ++ kubectl get pods -n kube-system --no-headers ++ cluster/kubectl.sh get pods -n kube-system --no-headers ++ grep -v Running + '[' -n '' ']' + current_time=0 ++ grep false ++ kubectl get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ cluster/kubectl.sh get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers + '[' -n '' ']' + kubectl get pods -n kube-system + cluster/kubectl.sh get pods -n kube-system NAME READY STATUS RESTARTS AGE disks-images-provider-kcn9c 1/1 Running 0 40s disks-images-provider-mbnmx 1/1 Running 0 39s etcd-node01 1/1 Running 0 16m kube-apiserver-node01 1/1 Running 0 16m kube-controller-manager-node01 1/1 Running 0 16m kube-dns-86f4d74b45-4p6j6 3/3 Running 0 17m kube-flannel-ds-5njff 1/1 Running 0 16m kube-flannel-ds-6kqx9 1/1 Running 0 17m kube-proxy-6ghnq 1/1 Running 0 16m kube-proxy-gxp7c 1/1 Running 0 17m kube-scheduler-node01 1/1 Running 0 16m virt-api-7d79764579-c2ffk 1/1 Running 0 46s virt-api-7d79764579-lpjfs 1/1 Running 0 46s virt-controller-7d57d96b65-8tb6l 1/1 Running 0 46s virt-controller-7d57d96b65-bdcj9 1/1 Running 0 46s virt-handler-5pv4z 1/1 Running 0 46s virt-handler-7x9x5 1/1 Running 0 46s + for i in '${namespaces[@]}' + current_time=0 ++ kubectl get pods -n default --no-headers ++ cluster/kubectl.sh get pods -n default --no-headers ++ grep -v Running No resources found. + '[' -n '' ']' + current_time=0 ++ kubectl get pods -n default '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ grep false ++ cluster/kubectl.sh get pods -n default '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers + '[' -n '' ']' + kubectl get pods -n default + cluster/kubectl.sh get pods -n default No resources found. + kubectl version + cluster/kubectl.sh version Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:17:39Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:05:37Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} + ginko_params='--ginkgo.noColor --junit-output=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/junit.xml' + [[ k8s-1.11.0-release =~ windows.* ]] + FUNC_TEST_ARGS='--ginkgo.noColor --junit-output=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.11.0-release/junit.xml' + make functest hack/dockerized "hack/build-func-tests.sh" sha256:dcf2b21fa2ed11dcf9dbba21b1cca0ee3fad521a0e9aee61c06d0b0b66a4b200 go version go1.10 linux/amd64 go version go1.10 linux/amd64 Compiling tests... compiled tests.test hack/functests.sh Running Suite: Tests Suite ========================== Random Seed: 1532602731 Will run 148 of 148 specs ••••••••••• ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.002 seconds] Templates /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:42 Launching VMI from VM Template [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:60 with given Fedora Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:193 should succeed to generate a VM JSON file using oc-process command /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:150 Skip test that requires oc binary /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1393 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.002 seconds] Templates /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:42 Launching VMI from VM Template [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:60 with given Fedora Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:193 with given VM JSON from the Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:152 should succeed to create a VM using oc-create command /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:156 Skip test that requires oc binary /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1393 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.002 seconds] Templates /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:42 Launching VMI from VM Template [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:60 with given Fedora Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:193 with given VM JSON from the Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:152 with given VM from the VM JSON /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:158 should succeed to launch a VMI using oc-patch command /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:161 Skip test that requires oc binary /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1393 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.002 seconds] Templates /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:42 Launching VMI from VM Template [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:60 with given Fedora Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:193 with given VM JSON from the Template /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:152 with given VM from the VM JSON /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:158 with given VMI from the VM /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:163 should succeed to terminate the VMI using oc-patch command /root/go/src/kubevirt.io/kubevirt/tests/template_test.go:166 Skip test that requires oc binary /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1393 ------------------------------ Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 10:57:45 http: TLS handshake error from 10.244.1.1:49152: EOF 2018/07/26 10:57:55 http: TLS handshake error from 10.244.1.1:49158: EOF 2018/07/26 10:58:05 http: TLS handshake error from 10.244.1.1:49164: EOF 2018/07/26 10:58:15 http: TLS handshake error from 10.244.1.1:49212: EOF 2018/07/26 10:58:25 http: TLS handshake error from 10.244.1.1:49218: EOF 2018/07/26 10:58:35 http: TLS handshake error from 10.244.1.1:49244: EOF 2018/07/26 10:58:45 http: TLS handshake error from 10.244.1.1:49250: EOF level=info timestamp=2018-07-26T10:58:52.652114Z pos=validating-webhook.go:84 component=virt-api msg="rejected vmi admission" 2018/07/26 10:58:55 http: TLS handshake error from 10.244.1.1:49256: EOF 2018/07/26 10:59:05 http: TLS handshake error from 10.244.1.1:49316: EOF 2018/07/26 10:59:15 http: TLS handshake error from 10.244.1.1:49322: EOF 2018/07/26 10:59:25 http: TLS handshake error from 10.244.1.1:49328: EOF 2018/07/26 10:59:35 http: TLS handshake error from 10.244.1.1:49334: EOF 2018/07/26 10:59:45 http: TLS handshake error from 10.244.1.1:49340: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 10:58:50 http: TLS handshake error from 10.244.0.1:53692: EOF level=info timestamp=2018-07-26T10:58:57.770136Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:58:57.774256Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:59:00 http: TLS handshake error from 10.244.0.1:53718: EOF 2018/07/26 10:59:10 http: TLS handshake error from 10.244.0.1:53764: EOF level=info timestamp=2018-07-26T10:59:15.072778Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:59:17.031639Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:59:18.106881Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:59:20 http: TLS handshake error from 10.244.0.1:53820: EOF 2018/07/26 10:59:30 http: TLS handshake error from 10.244.0.1:53844: EOF 2018/07/26 10:59:40 http: TLS handshake error from 10.244.0.1:53868: EOF level=info timestamp=2018-07-26T10:59:45.266506Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:59:47.102897Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:59:47.588280Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:59:50 http: TLS handshake error from 10.244.0.1:53892: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:54.528476Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgkph kind= uid=e34eabb1-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:54.528862Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgkph kind= uid=e34eabb1-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:55.114093Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmig4ql9 kind= uid=e3ac195e-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:55.114252Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmig4ql9 kind= uid=e3ac195e-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:55.823872Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmig4ql9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmig4ql9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e3ac195e-90c2-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmig4ql9" level=info timestamp=2018-07-26T10:58:56.279416Z pos=preset.go:161 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstance is excluded from VirtualMachinePresets" level=info timestamp=2018-07-26T10:58:56.279595Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:57.470473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470749Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-conflict-v9x5j matches VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470936Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-memory-79p4p matches VirtualMachineInstance" level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T10:57:35.961233Z pos=virt-handler.go:87 component=virt-handler hostname=node02 level=info timestamp=2018-07-26T10:57:35.966100Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:57:35.966960Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:57:36.068387Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:57:36.120368Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" level=info timestamp=2018-07-26T10:57:36.122849Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:50.462296Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:50.470985Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:50.471271Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:50.525134Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:50.571748Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:50.572042Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:50.659584Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:50.659851Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:50.659998Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:50.832580Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:50.832799Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:50.832928Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:51.156014Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:51.160153Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:51.162159Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" Pod name: virt-launcher-testvmiwg8w2-fw8zh Pod phase: Running goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42041c820, 0xc420110000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4203eb720, 0xc4200b6b50, 0xc42059fd00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200fcde0, 0xc42000e350, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bbd40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200fcde0, 0xc42000e350, 0x13, 0xc42005cf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42074bac0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42029c7c0, 0xc4200d3630, 0xc4216d2aa0, 0xc4216d2ab0, 0xc42191f780, 0xc4204ec2c0, 0x10f5e80, 0xc4200b6b50, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 10:59:35 http: TLS handshake error from 10.244.1.1:49334: EOF 2018/07/26 10:59:45 http: TLS handshake error from 10.244.1.1:49340: EOF 2018/07/26 10:59:55 http: TLS handshake error from 10.244.1.1:49358: EOF 2018/07/26 11:00:05 http: TLS handshake error from 10.244.1.1:49364: EOF 2018/07/26 11:00:15 http: TLS handshake error from 10.244.1.1:49370: EOF 2018/07/26 11:00:25 http: TLS handshake error from 10.244.1.1:49376: EOF 2018/07/26 11:00:35 http: TLS handshake error from 10.244.1.1:49382: EOF 2018/07/26 11:00:45 http: TLS handshake error from 10.244.1.1:49388: EOF 2018/07/26 11:00:55 http: TLS handshake error from 10.244.1.1:49394: EOF 2018/07/26 11:01:05 http: TLS handshake error from 10.244.1.1:49400: EOF 2018/07/26 11:01:15 http: TLS handshake error from 10.244.1.1:49406: EOF 2018/07/26 11:01:25 http: TLS handshake error from 10.244.1.1:49412: EOF 2018/07/26 11:01:35 http: TLS handshake error from 10.244.1.1:49418: EOF 2018/07/26 11:01:45 http: TLS handshake error from 10.244.1.1:49424: EOF 2018/07/26 11:01:55 http: TLS handshake error from 10.244.1.1:49430: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:00:55.651292Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:01:00 http: TLS handshake error from 10.244.0.1:54070: EOF 2018/07/26 11:01:10 http: TLS handshake error from 10.244.0.1:54094: EOF level=info timestamp=2018-07-26T11:01:15.812164Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:01:16.912729Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:01:18.041815Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:20 http: TLS handshake error from 10.244.0.1:54118: EOF 2018/07/26 11:01:30 http: TLS handshake error from 10.244.0.1:54142: EOF 2018/07/26 11:01:40 http: TLS handshake error from 10.244.0.1:54166: EOF level=info timestamp=2018-07-26T11:01:45.933733Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:01:46.935000Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:01:48.182677Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:50 http: TLS handshake error from 10.244.0.1:54190: EOF level=info timestamp=2018-07-26T11:01:55.655989Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:01:55.663366Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:54.528476Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgkph kind= uid=e34eabb1-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:54.528862Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgkph kind= uid=e34eabb1-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:55.114093Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmig4ql9 kind= uid=e3ac195e-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:55.114252Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmig4ql9 kind= uid=e3ac195e-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:55.823872Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmig4ql9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmig4ql9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e3ac195e-90c2-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmig4ql9" level=info timestamp=2018-07-26T10:58:56.279416Z pos=preset.go:161 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstance is excluded from VirtualMachinePresets" level=info timestamp=2018-07-26T10:58:56.279595Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:57.470473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470749Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-conflict-v9x5j matches VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470936Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-memory-79p4p matches VirtualMachineInstance" level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T10:57:35.961233Z pos=virt-handler.go:87 component=virt-handler hostname=node02 level=info timestamp=2018-07-26T10:57:35.966100Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:57:35.966960Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:57:36.068387Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:57:36.120368Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" level=info timestamp=2018-07-26T10:57:36.122849Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T10:59:51.827722Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:51.827899Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:53.110090Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:53.113067Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmiwg8w2-fw8zh Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42041c820, 0xc420110000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4203eb720, 0xc4200b6b50, 0xc42059fd00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200fcde0, 0xc42000e350, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bbd40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200fcde0, 0xc42000e350, 0x13, 0xc42005cf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42074bac0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42029c7c0, 0xc4200d3630, 0xc4216d2aa0, 0xc4216d2ab0, 0xc42191f780, 0xc4204ec2c0, 0x10f5e80, 0xc4200b6b50, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.937 seconds] HookSidecars /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:40 VMI definition /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:58 with SM BIOS hook sidecar /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:59 should successfully start with hook sidecar annotation [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:60 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting a VMI level=info timestamp=2018-07-26T10:58:59.463090Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmiwg8w2-fw8zh" level=info timestamp=2018-07-26T10:59:50.613506Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmiwg8w2-fw8zh" level=error timestamp=2018-07-26T10:59:51.260020Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 10:59:55 http: TLS handshake error from 10.244.1.1:49358: EOF 2018/07/26 11:00:05 http: TLS handshake error from 10.244.1.1:49364: EOF 2018/07/26 11:00:15 http: TLS handshake error from 10.244.1.1:49370: EOF 2018/07/26 11:00:25 http: TLS handshake error from 10.244.1.1:49376: EOF 2018/07/26 11:00:35 http: TLS handshake error from 10.244.1.1:49382: EOF 2018/07/26 11:00:45 http: TLS handshake error from 10.244.1.1:49388: EOF 2018/07/26 11:00:55 http: TLS handshake error from 10.244.1.1:49394: EOF 2018/07/26 11:01:05 http: TLS handshake error from 10.244.1.1:49400: EOF 2018/07/26 11:01:15 http: TLS handshake error from 10.244.1.1:49406: EOF 2018/07/26 11:01:25 http: TLS handshake error from 10.244.1.1:49412: EOF 2018/07/26 11:01:35 http: TLS handshake error from 10.244.1.1:49418: EOF 2018/07/26 11:01:45 http: TLS handshake error from 10.244.1.1:49424: EOF 2018/07/26 11:01:55 http: TLS handshake error from 10.244.1.1:49430: EOF 2018/07/26 11:02:05 http: TLS handshake error from 10.244.1.1:49436: EOF 2018/07/26 11:02:15 http: TLS handshake error from 10.244.1.1:49442: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:01:18.041815Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:20 http: TLS handshake error from 10.244.0.1:54118: EOF 2018/07/26 11:01:30 http: TLS handshake error from 10.244.0.1:54142: EOF 2018/07/26 11:01:40 http: TLS handshake error from 10.244.0.1:54166: EOF level=info timestamp=2018-07-26T11:01:45.933733Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:01:46.935000Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:01:48.182677Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:50 http: TLS handshake error from 10.244.0.1:54190: EOF level=info timestamp=2018-07-26T11:01:55.655989Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:01:55.663366Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:02:00 http: TLS handshake error from 10.244.0.1:54214: EOF 2018/07/26 11:02:10 http: TLS handshake error from 10.244.0.1:54238: EOF level=info timestamp=2018-07-26T11:02:16.064115Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:02:16.945750Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:02:18.304727Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:55.823872Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmig4ql9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmig4ql9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e3ac195e-90c2-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmig4ql9" level=info timestamp=2018-07-26T10:58:56.279416Z pos=preset.go:161 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstance is excluded from VirtualMachinePresets" level=info timestamp=2018-07-26T10:58:56.279595Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:57.470473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470749Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-conflict-v9x5j matches VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470936Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-memory-79p4p matches VirtualMachineInstance" level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:02:19.023462Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:19.023592Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:19.023802Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:19.028717Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:19.028973Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:19.029099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:19.049453Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:19.049605Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:19.049686Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:19.090401Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:19.090827Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:19.091176Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:19.171890Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:19.172477Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:19.172787Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmisk7xk-5vxdq Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a73a0, 0xc42010e000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c0c0, 0xc4207905b0, 0xc4205b9400, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420469f18, 0x3, 0x3, 0xc4200ba280, 0x226e776f6e6b6e55, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e078, 0x13, 0xc4204c1718, 0x3, 0x3, 0x2020200a202c2234, 0x7369222020202020, 0x72616c75706f705f) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bad00, 0xc4200b8e60, 0xc421951470, 0xc421951480, 0xc42043c300, 0xc4219522c0, 0x10f5e80, 0xc4207905b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:02:35 http: TLS handshake error from 10.244.1.1:49454: EOF 2018/07/26 11:02:45 http: TLS handshake error from 10.244.1.1:49460: EOF 2018/07/26 11:02:55 http: TLS handshake error from 10.244.1.1:49466: EOF 2018/07/26 11:03:05 http: TLS handshake error from 10.244.1.1:49472: EOF 2018/07/26 11:03:15 http: TLS handshake error from 10.244.1.1:49478: EOF 2018/07/26 11:03:25 http: TLS handshake error from 10.244.1.1:49484: EOF 2018/07/26 11:03:35 http: TLS handshake error from 10.244.1.1:49490: EOF 2018/07/26 11:03:45 http: TLS handshake error from 10.244.1.1:49496: EOF 2018/07/26 11:03:55 http: TLS handshake error from 10.244.1.1:49502: EOF 2018/07/26 11:04:05 http: TLS handshake error from 10.244.1.1:49508: EOF 2018/07/26 11:04:15 http: TLS handshake error from 10.244.1.1:49514: EOF 2018/07/26 11:04:25 http: TLS handshake error from 10.244.1.1:49520: EOF 2018/07/26 11:04:35 http: TLS handshake error from 10.244.1.1:49526: EOF 2018/07/26 11:04:45 http: TLS handshake error from 10.244.1.1:49532: EOF 2018/07/26 11:04:55 http: TLS handshake error from 10.244.1.1:49538: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:03:50 http: TLS handshake error from 10.244.0.1:54478: EOF level=info timestamp=2018-07-26T11:03:55.669901Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:03:55.673022Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:04:00 http: TLS handshake error from 10.244.0.1:54502: EOF 2018/07/26 11:04:10 http: TLS handshake error from 10.244.0.1:54526: EOF level=info timestamp=2018-07-26T11:04:16.582920Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:16.893840Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:04:18.843129Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:20 http: TLS handshake error from 10.244.0.1:54550: EOF 2018/07/26 11:04:30 http: TLS handshake error from 10.244.0.1:54574: EOF 2018/07/26 11:04:40 http: TLS handshake error from 10.244.0.1:54598: EOF level=info timestamp=2018-07-26T11:04:46.708917Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:46.989042Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:04:49.043946Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:50 http: TLS handshake error from 10.244.0.1:54622: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:55.823872Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmig4ql9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmig4ql9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e3ac195e-90c2-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmig4ql9" level=info timestamp=2018-07-26T10:58:56.279416Z pos=preset.go:161 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstance is excluded from VirtualMachinePresets" level=info timestamp=2018-07-26T10:58:56.279595Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirfwtn kind= uid=e45ae668-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:57.470473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470749Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-conflict-v9x5j matches VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470936Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-memory-79p4p matches VirtualMachineInstance" level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:02:20.298515Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:20.298813Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:21.579772Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:21.580258Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:21.580855Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:24.142451Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:24.142809Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:24.142943Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:29.263646Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:29.264092Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:29.264467Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:02:39.508824Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:02:39.585754Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:02:39.589854Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:02:39.590021Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmisk7xk-5vxdq Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a73a0, 0xc42010e000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c0c0, 0xc4207905b0, 0xc4205b9400, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420469f18, 0x3, 0x3, 0xc4200ba280, 0x226e776f6e6b6e55, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e078, 0x13, 0xc4204c1718, 0x3, 0x3, 0x2020200a202c2234, 0x7369222020202020, 0x72616c75706f705f) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bad00, 0xc4200b8e60, 0xc421951470, 0xc421951480, 0xc42043c300, 0xc4219522c0, 0x10f5e80, 0xc4207905b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.813 seconds] HookSidecars /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:40 VMI definition /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:58 with SM BIOS hook sidecar /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:59 should call Collect and OnDefineDomain on the hook sidecar [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:67 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Getting hook-sidecar logs level=info timestamp=2018-07-26T11:02:00.138985Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmisk7xk-5vxdq" level=info timestamp=2018-07-26T11:02:19.568227Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmisk7xk-5vxdq" level=error timestamp=2018-07-26T11:02:20.045445Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:02:55 http: TLS handshake error from 10.244.1.1:49466: EOF 2018/07/26 11:03:05 http: TLS handshake error from 10.244.1.1:49472: EOF 2018/07/26 11:03:15 http: TLS handshake error from 10.244.1.1:49478: EOF 2018/07/26 11:03:25 http: TLS handshake error from 10.244.1.1:49484: EOF 2018/07/26 11:03:35 http: TLS handshake error from 10.244.1.1:49490: EOF 2018/07/26 11:03:45 http: TLS handshake error from 10.244.1.1:49496: EOF 2018/07/26 11:03:55 http: TLS handshake error from 10.244.1.1:49502: EOF 2018/07/26 11:04:05 http: TLS handshake error from 10.244.1.1:49508: EOF 2018/07/26 11:04:15 http: TLS handshake error from 10.244.1.1:49514: EOF 2018/07/26 11:04:25 http: TLS handshake error from 10.244.1.1:49520: EOF 2018/07/26 11:04:35 http: TLS handshake error from 10.244.1.1:49526: EOF 2018/07/26 11:04:45 http: TLS handshake error from 10.244.1.1:49532: EOF 2018/07/26 11:04:55 http: TLS handshake error from 10.244.1.1:49538: EOF 2018/07/26 11:05:05 http: TLS handshake error from 10.244.1.1:49544: EOF 2018/07/26 11:05:15 http: TLS handshake error from 10.244.1.1:49550: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:03:55.673022Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:04:00 http: TLS handshake error from 10.244.0.1:54502: EOF 2018/07/26 11:04:10 http: TLS handshake error from 10.244.0.1:54526: EOF level=info timestamp=2018-07-26T11:04:16.582920Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:16.893840Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:04:18.843129Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:20 http: TLS handshake error from 10.244.0.1:54550: EOF 2018/07/26 11:04:30 http: TLS handshake error from 10.244.0.1:54574: EOF 2018/07/26 11:04:40 http: TLS handshake error from 10.244.0.1:54598: EOF level=info timestamp=2018-07-26T11:04:46.708917Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:46.989042Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:04:49.043946Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:50 http: TLS handshake error from 10.244.0.1:54622: EOF 2018/07/26 11:05:00 http: TLS handshake error from 10.244.0.1:54646: EOF 2018/07/26 11:05:10 http: TLS handshake error from 10.244.0.1:54670: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:57.470749Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-conflict-v9x5j matches VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470936Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-memory-79p4p matches VirtualMachineInstance" level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:05:15.829319Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:15.829632Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:15.829902Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:15.850643Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:15.851058Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:15.851460Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:15.892222Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:15.892739Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:15.893029Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:15.973811Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:15.974185Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:15.974517Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:16.135430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:16.136140Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:16.136468Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:05:35 http: TLS handshake error from 10.244.1.1:49562: EOF 2018/07/26 11:05:45 http: TLS handshake error from 10.244.1.1:49568: EOF 2018/07/26 11:05:55 http: TLS handshake error from 10.244.1.1:49574: EOF 2018/07/26 11:06:05 http: TLS handshake error from 10.244.1.1:49580: EOF 2018/07/26 11:06:15 http: TLS handshake error from 10.244.1.1:49586: EOF 2018/07/26 11:06:25 http: TLS handshake error from 10.244.1.1:49592: EOF 2018/07/26 11:06:35 http: TLS handshake error from 10.244.1.1:49598: EOF 2018/07/26 11:06:45 http: TLS handshake error from 10.244.1.1:49604: EOF 2018/07/26 11:06:55 http: TLS handshake error from 10.244.1.1:49610: EOF 2018/07/26 11:07:05 http: TLS handshake error from 10.244.1.1:49616: EOF 2018/07/26 11:07:15 http: TLS handshake error from 10.244.1.1:49622: EOF 2018/07/26 11:07:25 http: TLS handshake error from 10.244.1.1:49628: EOF 2018/07/26 11:07:35 http: TLS handshake error from 10.244.1.1:49634: EOF 2018/07/26 11:07:45 http: TLS handshake error from 10.244.1.1:49640: EOF 2018/07/26 11:07:55 http: TLS handshake error from 10.244.1.1:49646: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:06:50 http: TLS handshake error from 10.244.0.1:54910: EOF level=info timestamp=2018-07-26T11:06:54.745185Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:06:54.750947Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:07:00 http: TLS handshake error from 10.244.0.1:54934: EOF 2018/07/26 11:07:10 http: TLS handshake error from 10.244.0.1:54958: EOF level=info timestamp=2018-07-26T11:07:17.049904Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:07:17.491646Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:07:19.800678Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:07:20 http: TLS handshake error from 10.244.0.1:54982: EOF 2018/07/26 11:07:30 http: TLS handshake error from 10.244.0.1:55006: EOF 2018/07/26 11:07:40 http: TLS handshake error from 10.244.0.1:55030: EOF level=info timestamp=2018-07-26T11:07:46.910043Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:07:47.647868Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:07:49.921890Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:07:50 http: TLS handshake error from 10.244.0.1:55054: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:57.470749Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-conflict-v9x5j matches VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:57.470936Z pos=preset.go:202 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachineInstancePreset test-memory-79p4p matches VirtualMachineInstance" level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:05:17.099526Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:17.099853Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:18.380464Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:18.380891Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:18.381060Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:20.943669Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:20.946124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:20.947150Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:26.068539Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:26.069158Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:26.069565Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:05:36.310544Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:05:36.370611Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:05:36.371061Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:05:36.371239Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.719 seconds] HookSidecars /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:40 VMI definition /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:58 with SM BIOS hook sidecar /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:59 should update domain XML with SM BIOS properties [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_hook_sidecar_test.go:83 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Reading domain XML using virsh level=info timestamp=2018-07-26T11:05:00.926548Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi22vpz-5cbbn" level=info timestamp=2018-07-26T11:05:16.515039Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi22vpz-5cbbn" level=error timestamp=2018-07-26T11:05:16.860204Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:06:35 http: TLS handshake error from 10.244.1.1:49598: EOF 2018/07/26 11:06:45 http: TLS handshake error from 10.244.1.1:49604: EOF 2018/07/26 11:06:55 http: TLS handshake error from 10.244.1.1:49610: EOF 2018/07/26 11:07:05 http: TLS handshake error from 10.244.1.1:49616: EOF 2018/07/26 11:07:15 http: TLS handshake error from 10.244.1.1:49622: EOF 2018/07/26 11:07:25 http: TLS handshake error from 10.244.1.1:49628: EOF 2018/07/26 11:07:35 http: TLS handshake error from 10.244.1.1:49634: EOF 2018/07/26 11:07:45 http: TLS handshake error from 10.244.1.1:49640: EOF 2018/07/26 11:07:55 http: TLS handshake error from 10.244.1.1:49646: EOF 2018/07/26 11:08:05 http: TLS handshake error from 10.244.1.1:49652: EOF 2018/07/26 11:08:15 http: TLS handshake error from 10.244.1.1:49658: EOF 2018/07/26 11:08:25 http: TLS handshake error from 10.244.1.1:49664: EOF 2018/07/26 11:08:35 http: TLS handshake error from 10.244.1.1:49670: EOF 2018/07/26 11:08:45 http: TLS handshake error from 10.244.1.1:49676: EOF 2018/07/26 11:08:55 http: TLS handshake error from 10.244.1.1:49682: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:08:00 http: TLS handshake error from 10.244.0.1:55078: EOF 2018/07/26 11:08:10 http: TLS handshake error from 10.244.0.1:55102: EOF level=info timestamp=2018-07-26T11:08:17.106232Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:08:17.789156Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:08:20.059913Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:08:20 http: TLS handshake error from 10.244.0.1:55126: EOF 2018/07/26 11:08:30 http: TLS handshake error from 10.244.0.1:55150: EOF 2018/07/26 11:08:40 http: TLS handshake error from 10.244.0.1:55174: EOF level=info timestamp=2018-07-26T11:08:47.095865Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:08:47.936154Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:08:50.200920Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:08:50 http: TLS handshake error from 10.244.0.1:55198: EOF level=info timestamp=2018-07-26T11:08:54.757733Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:08:54.761613Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:09:00 http: TLS handshake error from 10.244.0.1:55222: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=error timestamp=2018-07-26T10:58:57.471891Z pos=preset.go:362 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="VirtualMachinePresets cannot be applied due to conflicts: presets 'test-memory-79p4p' and 'test-conflict-v9x5j' conflict: spec.resources.requests[memory]: {{128 6} {} 128M DecimalSI} != {{256 6} {} 256M DecimalSI}" level=warning timestamp=2018-07-26T10:58:57.472176Z pos=preset.go:154 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as failed" level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:08:00.198943Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:00.204685Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:08:18.550836Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:08:18.551136Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:08:19.831801Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:08:19.832217Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:08:19.832620Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:08:22.393513Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:08:22.394856Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:08:22.395200Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:08:27.516002Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:08:27.516641Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:08:27.516959Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:08:37.758528Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:08:37.807384Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:08:37.811450Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:08:37.811679Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [60.536 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose service on a VM /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:61 Expose ClusterIP service [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:68 Should expose a Cluster IP service on a VMI and connect to it /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:71 Timed out after 30.012s. Timed out waiting for VMI to enter Running phase Expected : false to equal : true /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1049 ------------------------------ level=info timestamp=2018-07-26T11:08:01.434872Z pos=utils.go:254 component=tests msg="Created virtual machine pod virt-launcher-testvmigdx98-vrjjv" level=info timestamp=2018-07-26T11:08:17.812176Z pos=utils.go:254 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmigdx98-vrjjv" level=error timestamp=2018-07-26T11:08:18.283923Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="unexpected EOF" level=error timestamp=2018-07-26T11:08:18.317907Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:18.347239Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:18.368417Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:18.391420Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:18.470574Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:18.644131Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:18.956194Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:19.595234Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:20.907154Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:23.447922Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:08:28.583237Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:07:35 http: TLS handshake error from 10.244.1.1:49634: EOF 2018/07/26 11:07:45 http: TLS handshake error from 10.244.1.1:49640: EOF 2018/07/26 11:07:55 http: TLS handshake error from 10.244.1.1:49646: EOF 2018/07/26 11:08:05 http: TLS handshake error from 10.244.1.1:49652: EOF 2018/07/26 11:08:15 http: TLS handshake error from 10.244.1.1:49658: EOF 2018/07/26 11:08:25 http: TLS handshake error from 10.244.1.1:49664: EOF 2018/07/26 11:08:35 http: TLS handshake error from 10.244.1.1:49670: EOF 2018/07/26 11:08:45 http: TLS handshake error from 10.244.1.1:49676: EOF 2018/07/26 11:08:55 http: TLS handshake error from 10.244.1.1:49682: EOF 2018/07/26 11:09:05 http: TLS handshake error from 10.244.1.1:49688: EOF 2018/07/26 11:09:15 http: TLS handshake error from 10.244.1.1:49694: EOF 2018/07/26 11:09:25 http: TLS handshake error from 10.244.1.1:49700: EOF 2018/07/26 11:09:35 http: TLS handshake error from 10.244.1.1:49706: EOF 2018/07/26 11:09:45 http: TLS handshake error from 10.244.1.1:49712: EOF 2018/07/26 11:09:55 http: TLS handshake error from 10.244.1.1:49718: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:08:54.757733Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:08:54.761613Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:09:00 http: TLS handshake error from 10.244.0.1:55222: EOF 2018/07/26 11:09:10 http: TLS handshake error from 10.244.0.1:55246: EOF level=info timestamp=2018-07-26T11:09:17.004728Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:09:18.066518Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:09:20.342869Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:09:20 http: TLS handshake error from 10.244.0.1:55270: EOF 2018/07/26 11:09:30 http: TLS handshake error from 10.244.0.1:55294: EOF 2018/07/26 11:09:40 http: TLS handshake error from 10.244.0.1:55318: EOF level=info timestamp=2018-07-26T11:09:46.964297Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:09:48.225095Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:09:50 http: TLS handshake error from 10.244.0.1:55342: EOF level=info timestamp=2018-07-26T11:09:50.465226Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:10:00 http: TLS handshake error from 10.244.0.1:55366: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:57.472248Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitqx9j kind= uid=e50ca79d-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:58.170170Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:08:00.198943Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:00.204685Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:09:00.708485Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:09:00.711682Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:09:16.944058Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:16.944458Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmic6v68" level=info timestamp=2018-07-26T11:09:18.225853Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:18.226226Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:18.226598Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmic6v68" level=info timestamp=2018-07-26T11:09:20.787169Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:20.787708Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:20.787965Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmic6v68" level=info timestamp=2018-07-26T11:09:25.908841Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:25.909462Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:25.909794Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmic6v68" level=info timestamp=2018-07-26T11:09:36.151521Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:09:36.221144Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:09:36.221554Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:09:36.221732Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic6v68-f7s2c Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201bcc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc420514108, 0xc420340940, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420106120, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420089900, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420106120, 0xc42000e070, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ba0c0, 0xc4200b8410, 0xc4216380b0, 0xc4216380c0, 0xc42194c300, 0xc42191ed40, 0x10f5e80, 0xc420514108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [60.719 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose service on a VM /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:61 Expose ClusterIP service with string target-port [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:98 Should expose a ClusterIP service and connect to the vm on port 80 /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:101 Timed out after 30.011s. Timed out waiting for VMI to enter Running phase Expected : false to equal : true /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1049 ------------------------------ level=info timestamp=2018-07-26T11:09:01.945592Z pos=utils.go:254 component=tests msg="Created virtual machine pod virt-launcher-testvmic6v68-f7s2c" level=info timestamp=2018-07-26T11:09:16.343598Z pos=utils.go:254 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmic6v68-f7s2c" level=error timestamp=2018-07-26T11:09:16.684227Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="unexpected EOF" level=error timestamp=2018-07-26T11:09:16.697296Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:16.708239Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:16.726225Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:16.772121Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:16.857632Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:17.022170Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:17.350058Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:17.987421Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:19.266211Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:21.858677Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:09:26.963767Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:08:35 http: TLS handshake error from 10.244.1.1:49670: EOF 2018/07/26 11:08:45 http: TLS handshake error from 10.244.1.1:49676: EOF 2018/07/26 11:08:55 http: TLS handshake error from 10.244.1.1:49682: EOF 2018/07/26 11:09:05 http: TLS handshake error from 10.244.1.1:49688: EOF 2018/07/26 11:09:15 http: TLS handshake error from 10.244.1.1:49694: EOF 2018/07/26 11:09:25 http: TLS handshake error from 10.244.1.1:49700: EOF 2018/07/26 11:09:35 http: TLS handshake error from 10.244.1.1:49706: EOF 2018/07/26 11:09:45 http: TLS handshake error from 10.244.1.1:49712: EOF 2018/07/26 11:09:55 http: TLS handshake error from 10.244.1.1:49718: EOF 2018/07/26 11:10:05 http: TLS handshake error from 10.244.1.1:49724: EOF 2018/07/26 11:10:15 http: TLS handshake error from 10.244.1.1:49730: EOF 2018/07/26 11:10:25 http: TLS handshake error from 10.244.1.1:49736: EOF 2018/07/26 11:10:35 http: TLS handshake error from 10.244.1.1:49742: EOF 2018/07/26 11:10:45 http: TLS handshake error from 10.244.1.1:49748: EOF 2018/07/26 11:10:55 http: TLS handshake error from 10.244.1.1:49754: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:10:17.134261Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:10:18.361184Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:10:20 http: TLS handshake error from 10.244.0.1:55414: EOF level=info timestamp=2018-07-26T11:10:20.573240Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:10:30 http: TLS handshake error from 10.244.0.1:55438: EOF 2018/07/26 11:10:40 http: TLS handshake error from 10.244.0.1:55462: EOF level=info timestamp=2018-07-26T11:10:46.811222Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:10:47.160602Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:10:47.169507Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:10:48.603603Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:10:50 http: TLS handshake error from 10.244.0.1:55486: EOF level=info timestamp=2018-07-26T11:10:50.731085Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:10:56.713717Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:10:56.717225Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:11:00 http: TLS handshake error from 10.244.0.1:55510: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T10:58:58.170498Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.006089Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:08:00.198943Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:00.204685Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:09:00.708485Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:09:00.711682Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:10:01.448843Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:10:01.450833Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:10:17.897808Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:10:17.898064Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bc9x" level=info timestamp=2018-07-26T11:10:19.178928Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:10:19.179544Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:10:19.179992Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bc9x" level=info timestamp=2018-07-26T11:10:21.741075Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:10:21.741703Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:10:21.741986Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bc9x" level=info timestamp=2018-07-26T11:10:26.863178Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:10:26.863961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:10:26.864354Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bc9x" level=info timestamp=2018-07-26T11:10:37.106026Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:10:37.155079Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:10:37.159096Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:10:37.159395Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi9bc9x-tm7wf Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fa0b0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928080, 0xc4204b4b28, 0xc4203911a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200965a0, 0xc4200b6728, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200bbb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200965a0, 0xc4200b6728, 0x13, 0xc421951718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ab0c0, 0xc4200b8780, 0xc421934790, 0xc4219347a0, 0xc421946300, 0xc42196e4a0, 0x10f5e80, 0xc4204b4b28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic6v68-f7s2c Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201bcc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc420514108, 0xc420340940, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420106120, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420089900, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420106120, 0xc42000e070, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ba0c0, 0xc4200b8410, 0xc4216380b0, 0xc4216380c0, 0xc42194c300, 0xc42191ed40, 0x10f5e80, 0xc420514108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [60.977 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose service on a VM /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:61 Expose NodePort service [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:124 Should expose a NodePort service on a VMI and connect to it /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:129 Timed out after 30.011s. Timed out waiting for VMI to enter Running phase Expected : false to equal : true /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1049 ------------------------------ level=info timestamp=2018-07-26T11:10:02.635970Z pos=utils.go:254 component=tests msg="Created virtual machine pod virt-launcher-testvmi9bc9x-tm7wf" level=info timestamp=2018-07-26T11:10:17.279327Z pos=utils.go:254 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi9bc9x-tm7wf" level=error timestamp=2018-07-26T11:10:17.649330Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="unexpected EOF" level=error timestamp=2018-07-26T11:10:17.677215Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:17.703309Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:17.723055Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:17.755381Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:17.948137Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:18.016835Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:18.298887Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:18.938590Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:20.227145Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:22.786976Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:10:27.934032Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:09:35 http: TLS handshake error from 10.244.1.1:49706: EOF 2018/07/26 11:09:45 http: TLS handshake error from 10.244.1.1:49712: EOF 2018/07/26 11:09:55 http: TLS handshake error from 10.244.1.1:49718: EOF 2018/07/26 11:10:05 http: TLS handshake error from 10.244.1.1:49724: EOF 2018/07/26 11:10:15 http: TLS handshake error from 10.244.1.1:49730: EOF 2018/07/26 11:10:25 http: TLS handshake error from 10.244.1.1:49736: EOF 2018/07/26 11:10:35 http: TLS handshake error from 10.244.1.1:49742: EOF 2018/07/26 11:10:45 http: TLS handshake error from 10.244.1.1:49748: EOF 2018/07/26 11:10:55 http: TLS handshake error from 10.244.1.1:49754: EOF 2018/07/26 11:11:05 http: TLS handshake error from 10.244.1.1:49760: EOF 2018/07/26 11:11:15 http: TLS handshake error from 10.244.1.1:49766: EOF 2018/07/26 11:11:25 http: TLS handshake error from 10.244.1.1:49772: EOF 2018/07/26 11:11:35 http: TLS handshake error from 10.244.1.1:49778: EOF 2018/07/26 11:11:45 http: TLS handshake error from 10.244.1.1:49784: EOF 2018/07/26 11:11:55 http: TLS handshake error from 10.244.1.1:49790: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:11:00 http: TLS handshake error from 10.244.0.1:55510: EOF 2018/07/26 11:11:10 http: TLS handshake error from 10.244.0.1:55534: EOF level=info timestamp=2018-07-26T11:11:16.909862Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:11:18.730592Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:20 http: TLS handshake error from 10.244.0.1:55558: EOF level=info timestamp=2018-07-26T11:11:20.854882Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:30 http: TLS handshake error from 10.244.0.1:55582: EOF 2018/07/26 11:11:40 http: TLS handshake error from 10.244.0.1:55606: EOF level=info timestamp=2018-07-26T11:11:46.997225Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:11:48.886976Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:50 http: TLS handshake error from 10.244.0.1:55630: EOF level=info timestamp=2018-07-26T11:11:51.018192Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:11:56.725155Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:11:56.729475Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:12:00 http: TLS handshake error from 10.244.0.1:55654: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:01:59.009703Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisk7xk kind= uid=51464b1a-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:59.114249Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:01:59.162459Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisk7xk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisk7xk" level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:08:00.198943Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:00.204685Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:09:00.708485Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:09:00.711682Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:10:01.448843Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:10:01.450833Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:02.405335Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:02.408750Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:11:19.300004Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:11:19.300541Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:11:20.581915Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:11:20.582979Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:11:20.583470Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:11:23.144703Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:11:23.145240Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:11:23.145504Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:11:28.266844Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:11:28.267742Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:11:28.268118Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:11:38.509201Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:11:38.559441Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:11:38.559761Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:11:38.559902Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi9bc9x-tm7wf Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fa0b0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928080, 0xc4204b4b28, 0xc4203911a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200965a0, 0xc4200b6728, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200bbb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200965a0, 0xc4200b6728, 0x13, 0xc421951718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ab0c0, 0xc4200b8780, 0xc421934790, 0xc4219347a0, 0xc421946300, 0xc42196e4a0, 0x10f5e80, 0xc4204b4b28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic6v68-f7s2c Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201bcc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc420514108, 0xc420340940, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420106120, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420089900, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420106120, 0xc42000e070, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ba0c0, 0xc4200b8410, 0xc4216380b0, 0xc4216380c0, 0xc42194c300, 0xc42191ed40, 0x10f5e80, 0xc420514108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmihrgsm-4fmm5 Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42031c480, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42059f260, 0xc4206f4638, 0xc42054dbc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2480, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200ba040, 0xc420171fa0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2480, 0xc42000e100, 0x13, 0xc421901718, 0x3, 0x3, 0xc4201720e0, 0xc420172100, 0xc420172120) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346480, 0xc4200e47d0, 0xc4219712c0, 0xc4219712d0, 0xc42196fe00, 0xc42000c0e0, 0x10f5e80, 0xc4206f4638, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [61.120 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose UDP service on a VMI /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:166 Expose ClusterIP UDP service [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:173 Should expose a ClusterIP service on a VMI and connect to it /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:177 Timed out after 30.010s. Timed out waiting for VMI to enter Running phase Expected : false to equal : true /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1049 ------------------------------ level=info timestamp=2018-07-26T11:11:03.588860Z pos=utils.go:254 component=tests msg="Created virtual machine pod virt-launcher-testvmihrgsm-4fmm5" level=info timestamp=2018-07-26T11:11:18.704420Z pos=utils.go:254 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmihrgsm-4fmm5" level=error timestamp=2018-07-26T11:11:19.031079Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="unexpected EOF" level=error timestamp=2018-07-26T11:11:19.057574Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:19.081039Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:19.101618Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:19.131849Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:19.212867Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:19.371734Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:19.706169Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:20.341191Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:21.630797Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:24.199017Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:11:29.382668Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:10:35 http: TLS handshake error from 10.244.1.1:49742: EOF 2018/07/26 11:10:45 http: TLS handshake error from 10.244.1.1:49748: EOF 2018/07/26 11:10:55 http: TLS handshake error from 10.244.1.1:49754: EOF 2018/07/26 11:11:05 http: TLS handshake error from 10.244.1.1:49760: EOF 2018/07/26 11:11:15 http: TLS handshake error from 10.244.1.1:49766: EOF 2018/07/26 11:11:25 http: TLS handshake error from 10.244.1.1:49772: EOF 2018/07/26 11:11:35 http: TLS handshake error from 10.244.1.1:49778: EOF 2018/07/26 11:11:45 http: TLS handshake error from 10.244.1.1:49784: EOF 2018/07/26 11:11:55 http: TLS handshake error from 10.244.1.1:49790: EOF 2018/07/26 11:12:05 http: TLS handshake error from 10.244.1.1:49796: EOF 2018/07/26 11:12:15 http: TLS handshake error from 10.244.1.1:49802: EOF 2018/07/26 11:12:25 http: TLS handshake error from 10.244.1.1:49808: EOF 2018/07/26 11:12:35 http: TLS handshake error from 10.244.1.1:49814: EOF 2018/07/26 11:12:45 http: TLS handshake error from 10.244.1.1:49820: EOF 2018/07/26 11:12:55 http: TLS handshake error from 10.244.1.1:49826: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:11:56.725155Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:11:56.729475Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:12:00 http: TLS handshake error from 10.244.0.1:55654: EOF 2018/07/26 11:12:10 http: TLS handshake error from 10.244.0.1:55678: EOF level=info timestamp=2018-07-26T11:12:17.055201Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:12:19.064742Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:12:20 http: TLS handshake error from 10.244.0.1:55702: EOF level=info timestamp=2018-07-26T11:12:21.149197Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:12:30 http: TLS handshake error from 10.244.0.1:55726: EOF 2018/07/26 11:12:40 http: TLS handshake error from 10.244.0.1:55750: EOF level=info timestamp=2018-07-26T11:12:46.980980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:12:49.184007Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:12:50 http: TLS handshake error from 10.244.0.1:55774: EOF level=info timestamp=2018-07-26T11:12:51.264874Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:13:00 http: TLS handshake error from 10.244.0.1:55798: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:04:59.691791Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:59.696400Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22vpz kind= uid=bcf672db-90c3-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:59.898476Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:04:59.933667Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi22vpz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi22vpz" level=info timestamp=2018-07-26T11:08:00.198943Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:00.204685Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigdx98 kind= uid=2888770c-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:09:00.708485Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:09:00.711682Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic6v68 kind= uid=4c9def3a-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:10:01.448843Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:10:01.450833Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bc9x kind= uid=70cf4189-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:02.405335Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:02.408750Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihrgsm kind= uid=95280578-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:12:03.535612Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:12:03.539040Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:12:03.798723Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmix88mj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:12:21.000791Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:21.001099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:12:22.282820Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:22.283552Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:22.283891Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:12:24.845023Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:24.845664Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:24.845919Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:12:29.966671Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:29.967139Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:29.967592Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:12:40.208485Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:12:40.251482Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:12:40.254733Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:12:40.254929Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmix88mj kind= uid=b9968abd-90c4-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi9bc9x-tm7wf Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fa0b0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928080, 0xc4204b4b28, 0xc4203911a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200965a0, 0xc4200b6728, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200bbb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200965a0, 0xc4200b6728, 0x13, 0xc421951718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ab0c0, 0xc4200b8780, 0xc421934790, 0xc4219347a0, 0xc421946300, 0xc42196e4a0, 0x10f5e80, 0xc4204b4b28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic6v68-f7s2c Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201bcc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc420514108, 0xc420340940, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420106120, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420089900, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420106120, 0xc42000e070, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ba0c0, 0xc4200b8410, 0xc4216380b0, 0xc4216380c0, 0xc42194c300, 0xc42191ed40, 0x10f5e80, 0xc420514108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmihrgsm-4fmm5 Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42031c480, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42059f260, 0xc4206f4638, 0xc42054dbc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2480, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200ba040, 0xc420171fa0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2480, 0xc42000e100, 0x13, 0xc421901718, 0x3, 0x3, 0xc4201720e0, 0xc420172100, 0xc420172120) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346480, 0xc4200e47d0, 0xc4219712c0, 0xc4219712d0, 0xc42196fe00, 0xc42000c0e0, 0x10f5e80, 0xc4206f4638, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmix88mj-6ndbf Pod phase: Failed goroutine 24 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202b40d0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420560000, 0xc42000e648, 0xc42194d5e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b6060, 0xc4201de4a8, 0x13, 0x131ed4d, 0x4, 0xc421954f18, 0x3, 0x3, 0xc420088080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b6060, 0xc4201de4a8, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420326a40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420474080, 0xc4201dc410, 0xc420620640, 0xc420620650, 0xc421933c00, 0xc42000c0e0, 0x10f5e80, 0xc42000e648, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [61.332 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose UDP service on a VMI /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:166 Expose NodePort UDP service [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:205 Should expose a NodePort service on a VMI and connect to it /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:210 Timed out after 30.014s. Timed out waiting for VMI to enter Running phase Expected : false to equal : true /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1049 ------------------------------ level=info timestamp=2018-07-26T11:12:04.818331Z pos=utils.go:254 component=tests msg="Created virtual machine pod virt-launcher-testvmix88mj-6ndbf" level=info timestamp=2018-07-26T11:12:20.404259Z pos=utils.go:254 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmix88mj-6ndbf" level=error timestamp=2018-07-26T11:12:20.745160Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="unexpected EOF" level=error timestamp=2018-07-26T11:12:20.768966Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:20.797018Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:20.817980Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:20.836894Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:20.919417Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:21.074823Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:21.437644Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:22.037084Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:23.324451Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:25.899366Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T11:12:31.030837Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:12:35 http: TLS handshake error from 10.244.1.1:49814: EOF 2018/07/26 11:12:45 http: TLS handshake error from 10.244.1.1:49820: EOF 2018/07/26 11:12:55 http: TLS handshake error from 10.244.1.1:49826: EOF 2018/07/26 11:13:05 http: TLS handshake error from 10.244.1.1:49832: EOF 2018/07/26 11:13:15 http: TLS handshake error from 10.244.1.1:49838: EOF 2018/07/26 11:13:25 http: TLS handshake error from 10.244.1.1:49844: EOF 2018/07/26 11:13:35 http: TLS handshake error from 10.244.1.1:49850: EOF 2018/07/26 11:13:45 http: TLS handshake error from 10.244.1.1:49856: EOF 2018/07/26 11:13:55 http: TLS handshake error from 10.244.1.1:49862: EOF 2018/07/26 11:14:05 http: TLS handshake error from 10.244.1.1:49868: EOF 2018/07/26 11:14:15 http: TLS handshake error from 10.244.1.1:49874: EOF 2018/07/26 11:14:25 http: TLS handshake error from 10.244.1.1:49880: EOF 2018/07/26 11:14:35 http: TLS handshake error from 10.244.1.1:49886: EOF 2018/07/26 11:14:45 http: TLS handshake error from 10.244.1.1:49892: EOF 2018/07/26 11:14:55 http: TLS handshake error from 10.244.1.1:49898: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:13:56.738864Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:13:56.745074Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:14:00 http: TLS handshake error from 10.244.0.1:55942: EOF 2018/07/26 11:14:10 http: TLS handshake error from 10.244.0.1:55966: EOF level=info timestamp=2018-07-26T11:14:16.940714Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:14:19.573992Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:20 http: TLS handshake error from 10.244.0.1:55990: EOF level=info timestamp=2018-07-26T11:14:21.644963Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:30 http: TLS handshake error from 10.244.0.1:56014: EOF 2018/07/26 11:14:40 http: TLS handshake error from 10.244.0.1:56038: EOF level=info timestamp=2018-07-26T11:14:46.975958Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:14:49.695675Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:50 http: TLS handshake error from 10.244.0.1:56062: EOF level=info timestamp=2018-07-26T11:14:51.759202Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:15:00 http: TLS handshake error from 10.244.0.1:56086: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:14:17.462208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczw6x2lq kind= uid=096e23fd-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:14:18.407482Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiggczw6x2lq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczw6x2lq" level=error timestamp=2018-07-26T11:14:18.606508Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetjnxg7 kind= uid=de25864d-90c4-11e8-992e-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetjnxg7\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T11:14:18.606602Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetjnxg7\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetjnxg7" level=info timestamp=2018-07-26T11:14:53.276888Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczwv9pls kind= uid=1ec3fd9a-90c5-11e8-992e-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T11:14:53.279555Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczwv9pls kind= uid=1ec3fd9a-90c5-11e8-992e-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmiggczwv9pls" level=info timestamp=2018-07-26T11:14:53.279800Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczwv9pls kind= uid=1ec3fd9a-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:14:53.280385Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczwv9pls kind= uid=1ec3fd9a-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:14:53.449015Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczw7jnd7 kind= uid=1edb937c-90c5-11e8-992e-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T11:14:53.449224Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczw7jnd7 kind= uid=1edb937c-90c5-11e8-992e-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmiggczw7jnd7" level=info timestamp=2018-07-26T11:14:53.449454Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczw7jnd7 kind= uid=1edb937c-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:14:53.449594Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiggczw7jnd7 kind= uid=1edb937c-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T11:14:53.600428Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetjnxg7 kind= uid=de25864d-90c4-11e8-992e-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetjnxg7\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T11:14:53.600597Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetjnxg7\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetjnxg7" level=info timestamp=2018-07-26T11:14:54.440586Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiggczw7jnd7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczw7jnd7" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:14:53.208324Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind= uid=096e23fd-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:53.212008Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind= uid=096e23fd-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:14:53.212983Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind= uid=096e23fd-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:53.322484Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind= uid=096e23fd-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:14:53.324547Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind= uid=096e23fd-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:53.326770Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind= uid=0929cf11-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:14:53.373761Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind= uid=0929cf11-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:53.374062Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind= uid=0929cf11-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:14:53.374177Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind= uid=0929cf11-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:53.555357Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind= uid=0929cf11-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:14:53.555869Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind= uid=0929cf11-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:53.579250Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:14:53.579580Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczw6x2lq kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:14:54.247100Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:14:54.248235Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwd5qkw kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T10:59:53.113830Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T10:59:55.675675Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:55.675956Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:55.676168Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:00.797081Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:00:00.799222Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:00:00.804940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwg8w2" level=info timestamp=2018-07-26T11:00:11.057838Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.130729Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:00:11.138277Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:00:11.138862Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:58.918278Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:58.927594Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind= uid=e577f972-90c2-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:01:59.003516Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:01:59.004047Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwg8w2 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi9bc9x-tm7wf Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fa0b0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928080, 0xc4204b4b28, 0xc4203911a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200965a0, 0xc4200b6728, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200bbb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200965a0, 0xc4200b6728, 0x13, 0xc421951718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ab0c0, 0xc4200b8780, 0xc421934790, 0xc4219347a0, 0xc421946300, 0xc42196e4a0, 0x10f5e80, 0xc4204b4b28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic6v68-f7s2c Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201bcc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc420514108, 0xc420340940, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420106120, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420089900, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420106120, 0xc42000e070, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ba0c0, 0xc4200b8410, 0xc4216380b0, 0xc4216380c0, 0xc42194c300, 0xc42191ed40, 0x10f5e80, 0xc420514108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiggczw7jnd7-rxxx5 Pod phase: Running level=info timestamp=2018-07-26T11:14:58.411864Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:14:58.412039Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:14:58.413676Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" Pod name: virt-launcher-testvmiggczwv9pls-jsj4m Pod phase: Running level=info timestamp=2018-07-26T11:14:57.587363Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:14:57.588043Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:14:57.589649Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" Pod name: virt-launcher-testvmihrgsm-4fmm5 Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42031c480, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42059f260, 0xc4206f4638, 0xc42054dbc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2480, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200ba040, 0xc420171fa0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2480, 0xc42000e100, 0x13, 0xc421901718, 0x3, 0x3, 0xc4201720e0, 0xc420172100, 0xc420172120) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346480, 0xc4200e47d0, 0xc4219712c0, 0xc4219712d0, 0xc42196fe00, 0xc42000c0e0, 0x10f5e80, 0xc4206f4638, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmix88mj-6ndbf Pod phase: Failed goroutine 24 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202b40d0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420560000, 0xc42000e648, 0xc42194d5e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b6060, 0xc4201de4a8, 0x13, 0x131ed4d, 0x4, 0xc421954f18, 0x3, 0x3, 0xc420088080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b6060, 0xc4201de4a8, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420326a40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420474080, 0xc4201dc410, 0xc420620640, 0xc420620650, 0xc421933c00, 0xc42000c0e0, 0x10f5e80, 0xc42000e648, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [121.721 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose service on a VMI replica set /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:253 Expose ClusterIP service [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:286 Should create a ClusterIP service on VMRS and connect to it /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:290 Timed out after 120.000s. Expected : 0 to equal : 2 /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:271 ------------------------------ STEP: Creating a VMRS object with 2 replicas STEP: Start the replica set STEP: Checking the number of ready replicas Service cluster-ip-vm successfully exposed for virtualmachine testvmin9mdj VM testvmin9mdj was scheduled to start Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:14:45 http: TLS handshake error from 10.244.1.1:49892: EOF 2018/07/26 11:14:55 http: TLS handshake error from 10.244.1.1:49898: EOF 2018/07/26 11:15:05 http: TLS handshake error from 10.244.1.1:49904: EOF 2018/07/26 11:15:15 http: TLS handshake error from 10.244.1.1:49910: EOF 2018/07/26 11:15:25 http: TLS handshake error from 10.244.1.1:49916: EOF 2018/07/26 11:15:35 http: TLS handshake error from 10.244.1.1:49922: EOF 2018/07/26 11:15:45 http: TLS handshake error from 10.244.1.1:49928: EOF 2018/07/26 11:15:55 http: TLS handshake error from 10.244.1.1:49934: EOF 2018/07/26 11:16:05 http: TLS handshake error from 10.244.1.1:49940: EOF 2018/07/26 11:16:15 http: TLS handshake error from 10.244.1.1:49946: EOF 2018/07/26 11:16:25 http: TLS handshake error from 10.244.1.1:49952: EOF 2018/07/26 11:16:35 http: TLS handshake error from 10.244.1.1:49958: EOF 2018/07/26 11:16:45 http: TLS handshake error from 10.244.1.1:49964: EOF 2018/07/26 11:16:55 http: TLS handshake error from 10.244.1.1:49970: EOF 2018/07/26 11:17:05 http: TLS handshake error from 10.244.1.1:49976: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:16:00 http: TLS handshake error from 10.244.0.1:56230: EOF 2018/07/26 11:16:10 http: TLS handshake error from 10.244.0.1:56254: EOF level=info timestamp=2018-07-26T11:16:17.028944Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:16:20.159662Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:16:20 http: TLS handshake error from 10.244.0.1:56278: EOF level=info timestamp=2018-07-26T11:16:22.027050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:16:30 http: TLS handshake error from 10.244.0.1:56302: EOF 2018/07/26 11:16:40 http: TLS handshake error from 10.244.0.1:56326: EOF level=info timestamp=2018-07-26T11:16:47.008575Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:16:50.282953Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:16:50 http: TLS handshake error from 10.244.0.1:56350: EOF level=info timestamp=2018-07-26T11:16:52.155250Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:16:54.488922Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:16:54.511199Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:00 http: TLS handshake error from 10.244.0.1:56374: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:16:56.892993Z pos=vm.go:470 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=6874c3b6-90c5-11e8-992e-525500d15501 msg="VirtualMachineInstance created bacause testvmin9mdj was added." level=info timestamp=2018-07-26T11:16:56.893166Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=6874c3b6-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:16:56.893614Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=6874c3b6-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:16:56.930808Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:56.930970Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:56.932384Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:56.932581Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:56.971622Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:56.971803Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.018235Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:57.018490Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.046453Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:57.069444Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:57.069960Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.083055Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:17:00.615866Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwvq6kq kind= uid=5f913328-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:00.616450Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwvq6kq kind= uid=5f913328-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:00.616672Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczwvq6kq" level=info timestamp=2018-07-26T11:17:01.145636Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwgswnl kind= uid=603bc098-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:01.145855Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwgswnl kind= uid=603bc098-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:01.145947Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczwgswnl" level=info timestamp=2018-07-26T11:17:02.427126Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwgswnl kind= uid=603bc098-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:02.427854Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwgswnl kind= uid=603bc098-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:02.427970Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczwgswnl" level=info timestamp=2018-07-26T11:17:03.177195Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwvq6kq kind= uid=5f913328-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:03.177918Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwvq6kq kind= uid=5f913328-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:03.178025Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczwvq6kq" level=info timestamp=2018-07-26T11:17:04.988896Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwgswnl kind= uid=603bc098-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:04.989408Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiggczwgswnl kind= uid=603bc098-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:04.989580Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiggczwgswnl" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22vpz-5cbbn Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420248430, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191d0e0, 0xc42000f858, 0xc420212120, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4202034a0, 0xc4202080e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204800c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4202034a0, 0xc4202080e0, 0x13, 0xc42196df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42027cb80, 0xc420206870, 0xc421a62010, 0xc421a62020, 0xc421994000, 0xc42194a260, 0x10f5e80, 0xc42000f858, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi9bc9x-tm7wf Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fa0b0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928080, 0xc4204b4b28, 0xc4203911a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200965a0, 0xc4200b6728, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200bbb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200965a0, 0xc4200b6728, 0x13, 0xc421951718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ab0c0, 0xc4200b8780, 0xc421934790, 0xc4219347a0, 0xc421946300, 0xc42196e4a0, 0x10f5e80, 0xc4204b4b28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic6v68-f7s2c Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201bcc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc420514108, 0xc420340940, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420106120, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420089900, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420106120, 0xc42000e070, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ba0c0, 0xc4200b8410, 0xc4216380b0, 0xc4216380c0, 0xc42194c300, 0xc42191ed40, 0x10f5e80, 0xc420514108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigdx98-vrjjv Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278540, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928c80, 0xc421ac4998, 0xc4203888e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420224f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421901b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba640, 0xc4200b8960, 0xc421930dc0, 0xc421930dd0, 0xc42192bd80, 0xc4219320a0, 0x10f5e80, 0xc421ac4998, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiggczwgswnl-f72rb Pod phase: Failed goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fe0b0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4200c60a0, 0xc4200b8bf8, 0xc42063e3a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e180, 0xc42000e0a8, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bc900, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e180, 0xc42000e0a8, 0x13, 0xc4201d7f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207863c0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac040, 0xc4201cc5a0, 0xc421725090, 0xc4217250a0, 0xc420272300, 0xc42000d2c0, 0x10f5e80, 0xc4200b8bf8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiggczwvq6kq-slmzb Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0c0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192c040, 0xc42000f818, 0xc420188560, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e4180, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42008a380, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e4180, 0xc42000e078, 0x13, 0xc42005ff18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a080, 0xc4200e65f0, 0xc42052a030, 0xc42052a050, 0xc42194a300, 0xc42191cd20, 0x10f5e80, 0xc42000f818, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmihrgsm-4fmm5 Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42031c480, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42059f260, 0xc4206f4638, 0xc42054dbc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2480, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200ba040, 0xc420171fa0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2480, 0xc42000e100, 0x13, 0xc421901718, 0x3, 0x3, 0xc4201720e0, 0xc420172100, 0xc420172120) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346480, 0xc4200e47d0, 0xc4219712c0, 0xc4219712d0, 0xc42196fe00, 0xc42000c0e0, 0x10f5e80, 0xc4206f4638, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmin9mdj-6chpv Pod phase: Running level=info timestamp=2018-07-26T11:17:00.296867Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:17:00.296998Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:17:00.298584Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" Pod name: virt-launcher-testvmix88mj-6ndbf Pod phase: Failed goroutine 24 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202b40d0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420560000, 0xc42000e648, 0xc42194d5e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b6060, 0xc4201de4a8, 0x13, 0x131ed4d, 0x4, 0xc421954f18, 0x3, 0x3, 0xc420088080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b6060, 0xc4201de4a8, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420326a40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420474080, 0xc4201dc410, 0xc420620640, 0xc420620650, 0xc421933c00, 0xc42000c0e0, 0x10f5e80, 0xc42000e648, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [122.201 seconds] Expose /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:53 Expose service on an VM /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:318 Expose ClusterIP service [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:362 Connect to ClusterIP services that was set when VM was offline /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:363 Timed out after 120.000s. Expected : false to be true /root/go/src/kubevirt.io/kubevirt/tests/expose_test.go:349 ------------------------------ STEP: Creating an VM object STEP: Creating the VM STEP: Exposing a service on the VM using virtctl STEP: Calling the start command STEP: Getting the status of the VMI •• Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:15:05 http: TLS handshake error from 10.244.1.1:49904: EOF 2018/07/26 11:15:15 http: TLS handshake error from 10.244.1.1:49910: EOF 2018/07/26 11:15:25 http: TLS handshake error from 10.244.1.1:49916: EOF 2018/07/26 11:15:35 http: TLS handshake error from 10.244.1.1:49922: EOF 2018/07/26 11:15:45 http: TLS handshake error from 10.244.1.1:49928: EOF 2018/07/26 11:15:55 http: TLS handshake error from 10.244.1.1:49934: EOF 2018/07/26 11:16:05 http: TLS handshake error from 10.244.1.1:49940: EOF 2018/07/26 11:16:15 http: TLS handshake error from 10.244.1.1:49946: EOF 2018/07/26 11:16:25 http: TLS handshake error from 10.244.1.1:49952: EOF 2018/07/26 11:16:35 http: TLS handshake error from 10.244.1.1:49958: EOF 2018/07/26 11:16:45 http: TLS handshake error from 10.244.1.1:49964: EOF 2018/07/26 11:16:55 http: TLS handshake error from 10.244.1.1:49970: EOF 2018/07/26 11:17:05 http: TLS handshake error from 10.244.1.1:49976: EOF 2018/07/26 11:17:15 http: TLS handshake error from 10.244.1.1:49982: EOF 2018/07/26 11:17:25 http: TLS handshake error from 10.244.1.1:49988: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:16:30 http: TLS handshake error from 10.244.0.1:56302: EOF 2018/07/26 11:16:40 http: TLS handshake error from 10.244.0.1:56326: EOF level=info timestamp=2018-07-26T11:16:47.008575Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:16:50.282953Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:16:50 http: TLS handshake error from 10.244.0.1:56350: EOF level=info timestamp=2018-07-26T11:16:52.155250Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:16:54.488922Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:16:54.511199Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:00 http: TLS handshake error from 10.244.0.1:56374: EOF level=info timestamp=2018-07-26T11:17:09.734096Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2/version proto=HTTP/2.0 statusCode=200 contentLength=247 2018/07/26 11:17:10 http: TLS handshake error from 10.244.0.1:56398: EOF level=info timestamp=2018-07-26T11:17:16.923163Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:20 http: TLS handshake error from 10.244.0.1:56422: EOF level=info timestamp=2018-07-26T11:17:20.392401Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:17:22.277209Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:16:56.971803Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.018235Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:57.018490Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.046453Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:57.069444Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:57.069960Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.083055Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:17:09.433019Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigdx98\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmigdx98, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2888770c-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:17:09.635153Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihrgsm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmihrgsm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 95280578-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:17:09.733252Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:09.733868Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:09.835718Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmix88mj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmix88mj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b9968abd-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:17:10.022155Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:10.022436Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:10.245031Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2827r\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi2827r, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 701ca9ac-90c5-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2827r" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:17:26.751916Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:26.752407Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:26.752601Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:26.773170Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:26.773628Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:26.773871Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:26.830770Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:26.831161Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:26.831501Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:26.912722Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:26.913140Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:26.913486Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:27.074736Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:27.075458Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:27.075796Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi2827r-67kjn Pod phase: Running level=info timestamp=2018-07-26T11:17:14.817171Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:17:14.817367Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:17:14.819107Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:17:24.829351Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:17:24.908945Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi2827r" level=info timestamp=2018-07-26T11:17:24.921135Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:17:24.921848Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmi5p86f-vn9xd Pod phase: Running goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203200c0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420246040, 0xc4200b8d20, 0xc4206c88c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e240, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4202ac7c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e240, 0xc42000e078, 0x13, 0xc42021bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568040, 0xc4201a2370, 0xc420694690, 0xc4206946a0, 0xc42030c300, 0xc420186660, 0x10f5e80, 0xc4200b8d20, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:17:45 http: TLS handshake error from 10.244.1.1:50000: EOF 2018/07/26 11:17:55 http: TLS handshake error from 10.244.1.1:50006: EOF 2018/07/26 11:18:05 http: TLS handshake error from 10.244.1.1:50012: EOF 2018/07/26 11:18:15 http: TLS handshake error from 10.244.1.1:50018: EOF 2018/07/26 11:18:25 http: TLS handshake error from 10.244.1.1:50024: EOF 2018/07/26 11:18:35 http: TLS handshake error from 10.244.1.1:50030: EOF 2018/07/26 11:18:45 http: TLS handshake error from 10.244.1.1:50036: EOF 2018/07/26 11:18:55 http: TLS handshake error from 10.244.1.1:50042: EOF 2018/07/26 11:19:05 http: TLS handshake error from 10.244.1.1:50048: EOF 2018/07/26 11:19:15 http: TLS handshake error from 10.244.1.1:50054: EOF 2018/07/26 11:19:25 http: TLS handshake error from 10.244.1.1:50060: EOF 2018/07/26 11:19:35 http: TLS handshake error from 10.244.1.1:50066: EOF 2018/07/26 11:19:45 http: TLS handshake error from 10.244.1.1:50072: EOF 2018/07/26 11:19:55 http: TLS handshake error from 10.244.1.1:50078: EOF 2018/07/26 11:20:05 http: TLS handshake error from 10.244.1.1:50084: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:18:54.524215Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:19:00 http: TLS handshake error from 10.244.0.1:56662: EOF 2018/07/26 11:19:10 http: TLS handshake error from 10.244.0.1:56686: EOF level=info timestamp=2018-07-26T11:19:17.100757Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:19:20 http: TLS handshake error from 10.244.0.1:56710: EOF level=info timestamp=2018-07-26T11:19:20.948626Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:19:22.731614Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:19:30 http: TLS handshake error from 10.244.0.1:56734: EOF 2018/07/26 11:19:40 http: TLS handshake error from 10.244.0.1:56758: EOF level=info timestamp=2018-07-26T11:19:47.062102Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:19:50 http: TLS handshake error from 10.244.0.1:56782: EOF level=info timestamp=2018-07-26T11:19:51.080706Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:19:52.848401Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:20:00 http: TLS handshake error from 10.244.0.1:56806: EOF 2018/07/26 11:20:10 http: TLS handshake error from 10.244.0.1:56830: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:16:56.971803Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.018235Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:57.018490Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.046453Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:57.069444Z pos=vm.go:135 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Started processing VM" level=info timestamp=2018-07-26T11:16:57.069960Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.083055Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:17:09.433019Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigdx98\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmigdx98, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2888770c-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:17:09.635153Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihrgsm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmihrgsm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 95280578-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:17:09.733252Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:09.733868Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:09.835718Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmix88mj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmix88mj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b9968abd-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:17:10.022155Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:10.022436Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:10.245031Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2827r\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi2827r, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 701ca9ac-90c5-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2827r" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:17:28.038067Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:28.038396Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:29.319554Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:29.320017Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:29.320600Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:31.881473Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:31.882747Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:31.883047Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:37.003799Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:17:37.004213Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:17:37.004597Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5p86f" level=info timestamp=2018-07-26T11:17:47.245532Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:17:47.286526Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:17:47.286878Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:17:47.287029Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi2827r-67kjn Pod phase: Running level=info timestamp=2018-07-26T11:17:14.817171Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:17:14.817367Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:17:14.819107Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:17:24.829351Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:17:24.908945Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi2827r" level=info timestamp=2018-07-26T11:17:24.921135Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:17:24.921848Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmi5p86f-vn9xd Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203200c0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420246040, 0xc4200b8d20, 0xc4206c88c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e240, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4202ac7c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e240, 0xc42000e078, 0x13, 0xc42021bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568040, 0xc4201a2370, 0xc420694690, 0xc4206946a0, 0xc42030c300, 0xc420186660, 0x10f5e80, 0xc4200b8d20, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.821 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 should start it [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:76 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:17:12.834284Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi5p86f-vn9xd" level=info timestamp=2018-07-26T11:17:27.493406Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi5p86f-vn9xd" level=error timestamp=2018-07-26T11:17:27.735413Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:18:05 http: TLS handshake error from 10.244.1.1:50012: EOF 2018/07/26 11:18:15 http: TLS handshake error from 10.244.1.1:50018: EOF 2018/07/26 11:18:25 http: TLS handshake error from 10.244.1.1:50024: EOF 2018/07/26 11:18:35 http: TLS handshake error from 10.244.1.1:50030: EOF 2018/07/26 11:18:45 http: TLS handshake error from 10.244.1.1:50036: EOF 2018/07/26 11:18:55 http: TLS handshake error from 10.244.1.1:50042: EOF 2018/07/26 11:19:05 http: TLS handshake error from 10.244.1.1:50048: EOF 2018/07/26 11:19:15 http: TLS handshake error from 10.244.1.1:50054: EOF 2018/07/26 11:19:25 http: TLS handshake error from 10.244.1.1:50060: EOF 2018/07/26 11:19:35 http: TLS handshake error from 10.244.1.1:50066: EOF 2018/07/26 11:19:45 http: TLS handshake error from 10.244.1.1:50072: EOF 2018/07/26 11:19:55 http: TLS handshake error from 10.244.1.1:50078: EOF 2018/07/26 11:20:05 http: TLS handshake error from 10.244.1.1:50084: EOF 2018/07/26 11:20:15 http: TLS handshake error from 10.244.1.1:50090: EOF 2018/07/26 11:20:25 http: TLS handshake error from 10.244.1.1:50096: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:19:20 http: TLS handshake error from 10.244.0.1:56710: EOF level=info timestamp=2018-07-26T11:19:20.948626Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:19:22.731614Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:19:30 http: TLS handshake error from 10.244.0.1:56734: EOF 2018/07/26 11:19:40 http: TLS handshake error from 10.244.0.1:56758: EOF level=info timestamp=2018-07-26T11:19:47.062102Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:19:50 http: TLS handshake error from 10.244.0.1:56782: EOF level=info timestamp=2018-07-26T11:19:51.080706Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:19:52.848401Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:20:00 http: TLS handshake error from 10.244.0.1:56806: EOF 2018/07/26 11:20:10 http: TLS handshake error from 10.244.0.1:56830: EOF level=info timestamp=2018-07-26T11:20:17.044818Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:20:20 http: TLS handshake error from 10.244.0.1:56854: EOF level=info timestamp=2018-07-26T11:20:21.237158Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:20:22.951953Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:16:57.069960Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.083055Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:17:09.433019Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigdx98\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmigdx98, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2888770c-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:17:09.635153Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihrgsm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmihrgsm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 95280578-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:17:09.733252Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:09.733868Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:09.835718Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmix88mj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmix88mj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b9968abd-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:17:10.022155Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:10.022436Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:10.245031Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2827r\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi2827r, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 701ca9ac-90c5-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2827r" level=info timestamp=2018-07-26T11:20:10.943972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:20:10.949461Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:20:11.135420Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.204193Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.216370Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:20:26.590654Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:26.590970Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:26.591252Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:26.611967Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:26.612468Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:26.612732Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:26.653438Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:26.653879Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:26.654167Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:26.734935Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:26.735467Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:26.735722Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:26.896426Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:26.896813Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:26.897106Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi2827r-67kjn Pod phase: Failed level=info timestamp=2018-07-26T11:17:14.817171Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:17:14.817367Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:17:14.819107Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:17:24.829351Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:17:24.908945Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi2827r" level=info timestamp=2018-07-26T11:17:24.921135Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:17:24.921848Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" caught signal virt-launcher exited with code 127 Pod name: virt-launcher-testvmi9qfpq-89vxk Pod phase: Running goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fe3a0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191ac20, 0xc4200b8c08, 0xc420246180, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8cc0, 0xc42000e0c8, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034b280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8cc0, 0xc42000e0c8, 0x13, 0xc421925f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a26240) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202acb80, 0xc42014c4b0, 0xc420242050, 0xc420242060, 0xc42191dd80, 0xc421940140, 0x10f5e80, 0xc4200b8c08, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:20:45 http: TLS handshake error from 10.244.1.1:50108: EOF 2018/07/26 11:20:55 http: TLS handshake error from 10.244.1.1:50114: EOF 2018/07/26 11:21:05 http: TLS handshake error from 10.244.1.1:50120: EOF 2018/07/26 11:21:15 http: TLS handshake error from 10.244.1.1:50126: EOF 2018/07/26 11:21:25 http: TLS handshake error from 10.244.1.1:50132: EOF 2018/07/26 11:21:35 http: TLS handshake error from 10.244.1.1:50138: EOF 2018/07/26 11:21:45 http: TLS handshake error from 10.244.1.1:50144: EOF 2018/07/26 11:21:55 http: TLS handshake error from 10.244.1.1:50150: EOF 2018/07/26 11:22:05 http: TLS handshake error from 10.244.1.1:50156: EOF 2018/07/26 11:22:15 http: TLS handshake error from 10.244.1.1:50162: EOF 2018/07/26 11:22:25 http: TLS handshake error from 10.244.1.1:50168: EOF 2018/07/26 11:22:35 http: TLS handshake error from 10.244.1.1:50174: EOF 2018/07/26 11:22:45 http: TLS handshake error from 10.244.1.1:50180: EOF 2018/07/26 11:22:55 http: TLS handshake error from 10.244.1.1:50186: EOF 2018/07/26 11:23:05 http: TLS handshake error from 10.244.1.1:50192: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:21:55.101108Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:22:00 http: TLS handshake error from 10.244.0.1:57094: EOF 2018/07/26 11:22:10 http: TLS handshake error from 10.244.0.1:57118: EOF level=info timestamp=2018-07-26T11:22:17.045141Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:22:20 http: TLS handshake error from 10.244.0.1:57142: EOF level=info timestamp=2018-07-26T11:22:21.763228Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:22:23.722790Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:22:30 http: TLS handshake error from 10.244.0.1:57166: EOF 2018/07/26 11:22:40 http: TLS handshake error from 10.244.0.1:57190: EOF level=info timestamp=2018-07-26T11:22:47.028805Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:22:50 http: TLS handshake error from 10.244.0.1:57214: EOF level=info timestamp=2018-07-26T11:22:51.897694Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:22:53.851881Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:23:00 http: TLS handshake error from 10.244.0.1:57238: EOF 2018/07/26 11:23:10 http: TLS handshake error from 10.244.0.1:57262: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:16:57.069960Z pos=vm.go:186 component=virt-controller service=http namespace=kubevirt-test-default name=testvmin9mdj kind= uid=26b353b8-90c5-11e8-992e-525500d15501 msg="Creating or the VirtualMachineInstance: true" level=info timestamp=2018-07-26T11:16:57.083055Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmin9mdj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:17:09.433019Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigdx98\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmigdx98, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2888770c-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigdx98" level=info timestamp=2018-07-26T11:17:09.635153Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihrgsm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmihrgsm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 95280578-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihrgsm" level=info timestamp=2018-07-26T11:17:09.733252Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:09.733868Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2827r kind= uid=701ca9ac-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:09.835718Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmix88mj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmix88mj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b9968abd-90c4-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmix88mj" level=info timestamp=2018-07-26T11:17:10.022155Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:17:10.022436Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5p86f kind= uid=7048f68e-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:17:10.245031Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2827r\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi2827r, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 701ca9ac-90c5-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2827r" level=info timestamp=2018-07-26T11:20:10.943972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:20:10.949461Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:20:11.135420Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.204193Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.216370Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:20:27.862228Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:27.862574Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:29.143418Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:29.143876Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:29.144187Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:31.704979Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:31.705526Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:31.705804Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:36.826910Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:20:36.827715Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:20:36.828060Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:47.069917Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:20:47.151071Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:20:47.151703Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:20:47.151863Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi9qfpq-89vxk Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fe3a0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191ac20, 0xc4200b8c08, 0xc420246180, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8cc0, 0xc42000e0c8, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034b280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8cc0, 0xc42000e0c8, 0x13, 0xc421925f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a26240) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202acb80, 0xc42014c4b0, 0xc420242050, 0xc420242060, 0xc42191dd80, 0xc421940140, 0x10f5e80, 0xc4200b8c08, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.775 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 should attach virt-launcher to it [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:82 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:20:12.219849Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi9qfpq-89vxk" level=info timestamp=2018-07-26T11:20:27.293580Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi9qfpq-89vxk" level=error timestamp=2018-07-26T11:20:27.596161Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" •••• Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:21:15 http: TLS handshake error from 10.244.1.1:50126: EOF 2018/07/26 11:21:25 http: TLS handshake error from 10.244.1.1:50132: EOF 2018/07/26 11:21:35 http: TLS handshake error from 10.244.1.1:50138: EOF 2018/07/26 11:21:45 http: TLS handshake error from 10.244.1.1:50144: EOF 2018/07/26 11:21:55 http: TLS handshake error from 10.244.1.1:50150: EOF 2018/07/26 11:22:05 http: TLS handshake error from 10.244.1.1:50156: EOF 2018/07/26 11:22:15 http: TLS handshake error from 10.244.1.1:50162: EOF 2018/07/26 11:22:25 http: TLS handshake error from 10.244.1.1:50168: EOF 2018/07/26 11:22:35 http: TLS handshake error from 10.244.1.1:50174: EOF 2018/07/26 11:22:45 http: TLS handshake error from 10.244.1.1:50180: EOF 2018/07/26 11:22:55 http: TLS handshake error from 10.244.1.1:50186: EOF 2018/07/26 11:23:05 http: TLS handshake error from 10.244.1.1:50192: EOF level=info timestamp=2018-07-26T11:23:11.897679Z pos=validating-webhook.go:84 component=virt-api msg="rejected vmi admission" 2018/07/26 11:23:15 http: TLS handshake error from 10.244.1.1:50198: EOF 2018/07/26 11:23:25 http: TLS handshake error from 10.244.1.1:50204: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:22:20 http: TLS handshake error from 10.244.0.1:57142: EOF level=info timestamp=2018-07-26T11:22:21.763228Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:22:23.722790Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:22:30 http: TLS handshake error from 10.244.0.1:57166: EOF 2018/07/26 11:22:40 http: TLS handshake error from 10.244.0.1:57190: EOF level=info timestamp=2018-07-26T11:22:47.028805Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:22:50 http: TLS handshake error from 10.244.0.1:57214: EOF level=info timestamp=2018-07-26T11:22:51.897694Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:22:53.851881Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:23:00 http: TLS handshake error from 10.244.0.1:57238: EOF 2018/07/26 11:23:10 http: TLS handshake error from 10.244.0.1:57262: EOF level=info timestamp=2018-07-26T11:23:17.130891Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:23:20 http: TLS handshake error from 10.244.0.1:57286: EOF level=info timestamp=2018-07-26T11:23:22.031122Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:23:24.003078Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:20:10.949461Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:20:11.135420Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.204193Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.216370Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:23:12.142317Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.148821Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.312441Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.326254Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.367039Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi9prg2, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 481ddb5d-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.517014Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.517860Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:23:27.728234Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:27.728736Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:27.729015Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:27.749687Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:27.750033Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:27.750401Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:27.791055Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:27.791605Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:27.791903Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:27.872602Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:27.873138Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:27.873519Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:28.034227Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:28.034757Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:28.035084Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmixvk7j-fjn52 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027ae60, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190cca0, 0xc4200b91d8, 0xc42038ef20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098f60, 0xc4200b85c0, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008b9c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098f60, 0xc4200b85c0, 0x13, 0xc4206dff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202cfd80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406640, 0xc4200baaa0, 0xc42194b610, 0xc42194b620, 0xc42190fe00, 0xc4219180e0, 0x10f5e80, 0xc4200b91d8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:23:45 http: TLS handshake error from 10.244.1.1:50216: EOF 2018/07/26 11:23:55 http: TLS handshake error from 10.244.1.1:50222: EOF 2018/07/26 11:24:05 http: TLS handshake error from 10.244.1.1:50228: EOF 2018/07/26 11:24:15 http: TLS handshake error from 10.244.1.1:50234: EOF 2018/07/26 11:24:25 http: TLS handshake error from 10.244.1.1:50240: EOF 2018/07/26 11:24:35 http: TLS handshake error from 10.244.1.1:50246: EOF 2018/07/26 11:24:45 http: TLS handshake error from 10.244.1.1:50252: EOF 2018/07/26 11:24:55 http: TLS handshake error from 10.244.1.1:50258: EOF 2018/07/26 11:25:05 http: TLS handshake error from 10.244.1.1:50264: EOF 2018/07/26 11:25:15 http: TLS handshake error from 10.244.1.1:50270: EOF 2018/07/26 11:25:25 http: TLS handshake error from 10.244.1.1:50276: EOF 2018/07/26 11:25:35 http: TLS handshake error from 10.244.1.1:50282: EOF 2018/07/26 11:25:45 http: TLS handshake error from 10.244.1.1:50288: EOF 2018/07/26 11:25:55 http: TLS handshake error from 10.244.1.1:50294: EOF 2018/07/26 11:26:05 http: TLS handshake error from 10.244.1.1:50300: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:25:10 http: TLS handshake error from 10.244.0.1:57550: EOF level=info timestamp=2018-07-26T11:25:17.074290Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:25:20 http: TLS handshake error from 10.244.0.1:57574: EOF level=info timestamp=2018-07-26T11:25:22.553114Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:25:24.518789Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:25:30 http: TLS handshake error from 10.244.0.1:57598: EOF 2018/07/26 11:25:40 http: TLS handshake error from 10.244.0.1:57622: EOF level=info timestamp=2018-07-26T11:25:47.034105Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:25:50 http: TLS handshake error from 10.244.0.1:57646: EOF level=info timestamp=2018-07-26T11:25:52.756009Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:25:54.634626Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:25:55.383074Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:25:55.392665Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:26:00 http: TLS handshake error from 10.244.0.1:57670: EOF 2018/07/26 11:26:10 http: TLS handshake error from 10.244.0.1:57694: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:20:10.949461Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9qfpq kind= uid=dc1c8804-90c5-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:20:11.135420Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.204193Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.216370Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:23:12.142317Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.148821Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.312441Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.326254Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.367039Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi9prg2, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 481ddb5d-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.517014Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.517860Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:23:28.998023Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:28.998491Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:30.279172Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:30.279799Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:30.280075Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:32.841234Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:32.842792Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:32.843110Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:37.964462Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:23:37.964965Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:23:37.965358Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixvk7j" level=info timestamp=2018-07-26T11:23:48.207026Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:23:48.254218Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:23:48.258255Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:23:48.258582Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmixvk7j-fjn52 Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027ae60, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190cca0, 0xc4200b91d8, 0xc42038ef20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098f60, 0xc4200b85c0, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008b9c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098f60, 0xc4200b85c0, 0x13, 0xc4206dff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202cfd80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406640, 0xc4200baaa0, 0xc42194b610, 0xc42194b620, 0xc42190fe00, 0xc4219180e0, 0x10f5e80, 0xc4200b91d8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.731 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with boot order /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:170 should be able to boot from selected disk /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 Alpine as first boot [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: defining a VirtualMachineInstance with an Alpine disk STEP: adding a Cirros Disk STEP: setting boot order STEP: starting VirtualMachineInstance STEP: Waiting the VirtualMachineInstance start level=info timestamp=2018-07-26T11:23:14.073665Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmixvk7j-fjn52" level=info timestamp=2018-07-26T11:23:28.304679Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmixvk7j-fjn52" level=error timestamp=2018-07-26T11:23:28.701310Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:24:05 http: TLS handshake error from 10.244.1.1:50228: EOF 2018/07/26 11:24:15 http: TLS handshake error from 10.244.1.1:50234: EOF 2018/07/26 11:24:25 http: TLS handshake error from 10.244.1.1:50240: EOF 2018/07/26 11:24:35 http: TLS handshake error from 10.244.1.1:50246: EOF 2018/07/26 11:24:45 http: TLS handshake error from 10.244.1.1:50252: EOF 2018/07/26 11:24:55 http: TLS handshake error from 10.244.1.1:50258: EOF 2018/07/26 11:25:05 http: TLS handshake error from 10.244.1.1:50264: EOF 2018/07/26 11:25:15 http: TLS handshake error from 10.244.1.1:50270: EOF 2018/07/26 11:25:25 http: TLS handshake error from 10.244.1.1:50276: EOF 2018/07/26 11:25:35 http: TLS handshake error from 10.244.1.1:50282: EOF 2018/07/26 11:25:45 http: TLS handshake error from 10.244.1.1:50288: EOF 2018/07/26 11:25:55 http: TLS handshake error from 10.244.1.1:50294: EOF 2018/07/26 11:26:05 http: TLS handshake error from 10.244.1.1:50300: EOF 2018/07/26 11:26:15 http: TLS handshake error from 10.244.1.1:50306: EOF 2018/07/26 11:26:25 http: TLS handshake error from 10.244.1.1:50312: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:25:24.518789Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:25:30 http: TLS handshake error from 10.244.0.1:57598: EOF 2018/07/26 11:25:40 http: TLS handshake error from 10.244.0.1:57622: EOF level=info timestamp=2018-07-26T11:25:47.034105Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:25:50 http: TLS handshake error from 10.244.0.1:57646: EOF level=info timestamp=2018-07-26T11:25:52.756009Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:25:54.634626Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:25:55.383074Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:25:55.392665Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:26:00 http: TLS handshake error from 10.244.0.1:57670: EOF 2018/07/26 11:26:10 http: TLS handshake error from 10.244.0.1:57694: EOF level=info timestamp=2018-07-26T11:26:16.971663Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:26:20 http: TLS handshake error from 10.244.0.1:57718: EOF level=info timestamp=2018-07-26T11:26:22.886619Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:26:24.748606Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:20:11.204193Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.216370Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:23:12.142317Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.148821Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.312441Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.326254Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.367039Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi9prg2, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 481ddb5d-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.517014Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.517860Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:26:29.207073Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:29.207552Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:29.207863Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:29.209409Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:29.209693Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:29.209873Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:29.230626Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:29.230987Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:29.231372Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:29.271976Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:29.272430Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:29.272682Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:29.353593Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:29.353958Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:29.354176Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmihr4hp-nnn5r Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200483d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190ece0, 0xc42000ea10, 0xc4203919a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098f60, 0xc4200b8688, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4202ac100, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098f60, 0xc4200b8688, 0x13, 0xc4200cff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42069f180) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406700, 0xc4200baaf0, 0xc421925e50, 0xc421925e60, 0xc421911d80, 0xc4219260a0, 0x10f5e80, 0xc42000ea10, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:26:45 http: TLS handshake error from 10.244.1.1:50324: EOF 2018/07/26 11:26:55 http: TLS handshake error from 10.244.1.1:50330: EOF 2018/07/26 11:27:05 http: TLS handshake error from 10.244.1.1:50336: EOF 2018/07/26 11:27:15 http: TLS handshake error from 10.244.1.1:50342: EOF 2018/07/26 11:27:25 http: TLS handshake error from 10.244.1.1:50348: EOF 2018/07/26 11:27:35 http: TLS handshake error from 10.244.1.1:50354: EOF 2018/07/26 11:27:45 http: TLS handshake error from 10.244.1.1:50360: EOF 2018/07/26 11:27:55 http: TLS handshake error from 10.244.1.1:50366: EOF 2018/07/26 11:28:05 http: TLS handshake error from 10.244.1.1:50372: EOF 2018/07/26 11:28:15 http: TLS handshake error from 10.244.1.1:50378: EOF 2018/07/26 11:28:25 http: TLS handshake error from 10.244.1.1:50384: EOF 2018/07/26 11:28:35 http: TLS handshake error from 10.244.1.1:50390: EOF 2018/07/26 11:28:45 http: TLS handshake error from 10.244.1.1:50396: EOF 2018/07/26 11:28:55 http: TLS handshake error from 10.244.1.1:50402: EOF 2018/07/26 11:29:05 http: TLS handshake error from 10.244.1.1:50408: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:28:10 http: TLS handshake error from 10.244.0.1:57982: EOF level=info timestamp=2018-07-26T11:28:17.032766Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:28:20 http: TLS handshake error from 10.244.0.1:58006: EOF level=info timestamp=2018-07-26T11:28:23.347968Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:28:25.273857Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:28:30 http: TLS handshake error from 10.244.0.1:58030: EOF 2018/07/26 11:28:40 http: TLS handshake error from 10.244.0.1:58054: EOF level=info timestamp=2018-07-26T11:28:46.994615Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:28:50 http: TLS handshake error from 10.244.0.1:58078: EOF level=info timestamp=2018-07-26T11:28:53.466861Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:28:55.393889Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:28:55.419249Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:28:55.430266Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:29:00 http: TLS handshake error from 10.244.0.1:58102: EOF 2018/07/26 11:29:10 http: TLS handshake error from 10.244.0.1:58126: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:20:11.204193Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:20:11.216370Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9qfpq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9qfpq" level=info timestamp=2018-07-26T11:23:12.142317Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.148821Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9prg2 kind= uid=481ddb5d-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.312441Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.326254Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.367039Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi9prg2\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi9prg2, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 481ddb5d-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi9prg2" level=info timestamp=2018-07-26T11:23:12.517014Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.517860Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:26:30.483234Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:30.483571Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:31.764423Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:31.764908Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:31.765247Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:34.325976Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:34.326510Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:34.326745Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:39.447878Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:26:39.448480Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:26:39.448860Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr4hp" level=info timestamp=2018-07-26T11:26:49.690782Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:26:49.718563Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:26:49.722098Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:26:49.722368Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmihr4hp-nnn5r Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200483d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190ece0, 0xc42000ea10, 0xc4203919a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098f60, 0xc4200b8688, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4202ac100, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098f60, 0xc4200b8688, 0x13, 0xc4200cff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42069f180) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406700, 0xc4200baaf0, 0xc421925e50, 0xc421925e60, 0xc421911d80, 0xc4219260a0, 0x10f5e80, 0xc42000ea10, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.844 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with boot order /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:170 should be able to boot from selected disk /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 Cirros as first boot [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: defining a VirtualMachineInstance with an Alpine disk STEP: adding a Cirros Disk STEP: setting boot order STEP: starting VirtualMachineInstance STEP: Waiting the VirtualMachineInstance start level=info timestamp=2018-07-26T11:26:15.019112Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmihr4hp-nnn5r" level=info timestamp=2018-07-26T11:26:29.835242Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmihr4hp-nnn5r" level=error timestamp=2018-07-26T11:26:30.206826Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" • [SLOW TEST:15.865 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with user-data /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:201 without k8s secret /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:202 should retry starting the VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:203 ------------------------------ • [SLOW TEST:45.403 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with user-data /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:201 without k8s secret /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:202 should log warning and proceed once the secret is there /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:233 ------------------------------ Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:28:05 http: TLS handshake error from 10.244.1.1:50372: EOF 2018/07/26 11:28:15 http: TLS handshake error from 10.244.1.1:50378: EOF 2018/07/26 11:28:25 http: TLS handshake error from 10.244.1.1:50384: EOF 2018/07/26 11:28:35 http: TLS handshake error from 10.244.1.1:50390: EOF 2018/07/26 11:28:45 http: TLS handshake error from 10.244.1.1:50396: EOF 2018/07/26 11:28:55 http: TLS handshake error from 10.244.1.1:50402: EOF 2018/07/26 11:29:05 http: TLS handshake error from 10.244.1.1:50408: EOF 2018/07/26 11:29:15 http: TLS handshake error from 10.244.1.1:50414: EOF 2018/07/26 11:29:25 http: TLS handshake error from 10.244.1.1:50420: EOF 2018/07/26 11:29:35 http: TLS handshake error from 10.244.1.1:50426: EOF 2018/07/26 11:29:45 http: TLS handshake error from 10.244.1.1:50432: EOF 2018/07/26 11:29:55 http: TLS handshake error from 10.244.1.1:50438: EOF 2018/07/26 11:30:05 http: TLS handshake error from 10.244.1.1:50444: EOF 2018/07/26 11:30:15 http: TLS handshake error from 10.244.1.1:50450: EOF 2018/07/26 11:30:25 http: TLS handshake error from 10.244.1.1:50456: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:29:23.586859Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:29:25.536497Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:29:30 http: TLS handshake error from 10.244.0.1:58174: EOF 2018/07/26 11:29:40 http: TLS handshake error from 10.244.0.1:58198: EOF level=info timestamp=2018-07-26T11:29:46.985794Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:29:50 http: TLS handshake error from 10.244.0.1:58222: EOF level=info timestamp=2018-07-26T11:29:53.703460Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:29:55.655817Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:30:00 http: TLS handshake error from 10.244.0.1:58246: EOF 2018/07/26 11:30:10 http: TLS handshake error from 10.244.0.1:58270: EOF level=info timestamp=2018-07-26T11:30:16.970621Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:30:20 http: TLS handshake error from 10.244.0.1:58294: EOF level=info timestamp=2018-07-26T11:30:23.859147Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:30:25.745393Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:30:30 http: TLS handshake error from 10.244.0.1:58318: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:23:12.517014Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.517860Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:30:33.218423Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:33.218772Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:33.218966Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:33.239649Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:33.240040Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:33.240441Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:33.281140Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:33.281681Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:33.281932Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:33.362743Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:33.363120Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:33.363431Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:33.524546Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:33.525023Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:33.525448Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmis267s-2fbf5 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420236380, 0xc420118280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c020, 0xc420756d00, 0xc420782c60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e120, 0xc4200b8190, 0x13, 0x131ed4d, 0x4, 0xc421954f18, 0x3, 0x3, 0xc42008b880, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e120, 0xc4200b8190, 0x13, 0xc42005b718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420723b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202b4000, 0xc4200bab40, 0xc42075e4c0, 0xc42075e4d0, 0xc421945e00, 0xc420440080, 0x10f5e80, 0xc420756d00, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:30:55 http: TLS handshake error from 10.244.1.1:50474: EOF 2018/07/26 11:31:05 http: TLS handshake error from 10.244.1.1:50480: EOF 2018/07/26 11:31:15 http: TLS handshake error from 10.244.1.1:50486: EOF 2018/07/26 11:31:25 http: TLS handshake error from 10.244.1.1:50492: EOF 2018/07/26 11:31:35 http: TLS handshake error from 10.244.1.1:50498: EOF 2018/07/26 11:31:45 http: TLS handshake error from 10.244.1.1:50504: EOF 2018/07/26 11:31:55 http: TLS handshake error from 10.244.1.1:50510: EOF 2018/07/26 11:32:05 http: TLS handshake error from 10.244.1.1:50516: EOF 2018/07/26 11:32:15 http: TLS handshake error from 10.244.1.1:50522: EOF 2018/07/26 11:32:25 http: TLS handshake error from 10.244.1.1:50528: EOF 2018/07/26 11:32:35 http: TLS handshake error from 10.244.1.1:50534: EOF 2018/07/26 11:32:45 http: TLS handshake error from 10.244.1.1:50540: EOF 2018/07/26 11:32:55 http: TLS handshake error from 10.244.1.1:50546: EOF 2018/07/26 11:33:05 http: TLS handshake error from 10.244.1.1:50552: EOF 2018/07/26 11:33:15 http: TLS handshake error from 10.244.1.1:50558: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:31:56.092740Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:32:00 http: TLS handshake error from 10.244.0.1:58534: EOF 2018/07/26 11:32:10 http: TLS handshake error from 10.244.0.1:58558: EOF level=info timestamp=2018-07-26T11:32:17.085565Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:20 http: TLS handshake error from 10.244.0.1:58582: EOF level=info timestamp=2018-07-26T11:32:24.299108Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:32:26.192705Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:32:30 http: TLS handshake error from 10.244.0.1:58606: EOF 2018/07/26 11:32:40 http: TLS handshake error from 10.244.0.1:58630: EOF level=info timestamp=2018-07-26T11:32:47.018979Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:50 http: TLS handshake error from 10.244.0.1:58654: EOF level=info timestamp=2018-07-26T11:32:54.415604Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:32:56.316221Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:33:00 http: TLS handshake error from 10.244.0.1:58678: EOF 2018/07/26 11:33:10 http: TLS handshake error from 10.244.0.1:58702: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:23:12.517014Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.517860Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirtlkj kind= uid=4858b6ed-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:30:34.493815Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:34.494218Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:35.775572Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:35.776103Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:35.776423Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:38.337398Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:38.337795Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:38.337993Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:43.458918Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:43.459726Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:43.460044Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:30:53.701589Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:30:53.763742Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:53.764166Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:30:53.764445Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmis267s-2fbf5 Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420236380, 0xc420118280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c020, 0xc420756d00, 0xc420782c60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e120, 0xc4200b8190, 0x13, 0x131ed4d, 0x4, 0xc421954f18, 0x3, 0x3, 0xc42008b880, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e120, 0xc4200b8190, 0x13, 0xc42005b718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420723b80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202b4000, 0xc4200bab40, 0xc42075e4c0, 0xc42075e4d0, 0xc421945e00, 0xc420440080, 0x10f5e80, 0xc420756d00, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.784 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-launcher crashes /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:281 should be stopped and have Failed phase [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:282 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:30:17.028369Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmis267s-2fbf5" level=info timestamp=2018-07-26T11:30:33.910765Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmis267s-2fbf5" level=error timestamp=2018-07-26T11:30:34.200285Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:31:05 http: TLS handshake error from 10.244.1.1:50480: EOF 2018/07/26 11:31:15 http: TLS handshake error from 10.244.1.1:50486: EOF 2018/07/26 11:31:25 http: TLS handshake error from 10.244.1.1:50492: EOF 2018/07/26 11:31:35 http: TLS handshake error from 10.244.1.1:50498: EOF 2018/07/26 11:31:45 http: TLS handshake error from 10.244.1.1:50504: EOF 2018/07/26 11:31:55 http: TLS handshake error from 10.244.1.1:50510: EOF 2018/07/26 11:32:05 http: TLS handshake error from 10.244.1.1:50516: EOF 2018/07/26 11:32:15 http: TLS handshake error from 10.244.1.1:50522: EOF 2018/07/26 11:32:25 http: TLS handshake error from 10.244.1.1:50528: EOF 2018/07/26 11:32:35 http: TLS handshake error from 10.244.1.1:50534: EOF 2018/07/26 11:32:45 http: TLS handshake error from 10.244.1.1:50540: EOF 2018/07/26 11:32:55 http: TLS handshake error from 10.244.1.1:50546: EOF 2018/07/26 11:33:05 http: TLS handshake error from 10.244.1.1:50552: EOF 2018/07/26 11:33:15 http: TLS handshake error from 10.244.1.1:50558: EOF 2018/07/26 11:33:25 http: TLS handshake error from 10.244.1.1:50564: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:32:24.299108Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:32:26.192705Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:32:30 http: TLS handshake error from 10.244.0.1:58606: EOF 2018/07/26 11:32:40 http: TLS handshake error from 10.244.0.1:58630: EOF level=info timestamp=2018-07-26T11:32:47.018979Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:50 http: TLS handshake error from 10.244.0.1:58654: EOF level=info timestamp=2018-07-26T11:32:54.415604Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:32:56.316221Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:33:00 http: TLS handshake error from 10.244.0.1:58678: EOF 2018/07/26 11:33:10 http: TLS handshake error from 10.244.0.1:58702: EOF level=info timestamp=2018-07-26T11:33:17.036142Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:33:20 http: TLS handshake error from 10.244.0.1:58726: EOF level=info timestamp=2018-07-26T11:33:24.532187Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:33:26.442009Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:33:30 http: TLS handshake error from 10.244.0.1:58750: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:33:16.617201Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:16.624115Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:33:32.459757Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:32.460115Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:32.460417Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:32.481692Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:32.482118Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:32.482443Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:32.523064Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:32.523483Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:32.523684Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:32.604448Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:32.604894Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:32.605158Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:32.766853Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:32.767377Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:32.767598Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmimbffm-5cv2w Pod phase: Running goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200480c0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c020, 0xc4205965b0, 0xc421921fe0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e41e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc4200bd0c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e41e0, 0xc42000e078, 0x13, 0xc42005bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa040, 0xc4200e6370, 0xc4209850f0, 0xc420985100, 0xc421923e00, 0xc4200c60e0, 0x10f5e80, 0xc4205965b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:33:55 http: TLS handshake error from 10.244.1.1:50582: EOF 2018/07/26 11:34:05 http: TLS handshake error from 10.244.1.1:50588: EOF 2018/07/26 11:34:15 http: TLS handshake error from 10.244.1.1:50594: EOF 2018/07/26 11:34:25 http: TLS handshake error from 10.244.1.1:50600: EOF 2018/07/26 11:34:35 http: TLS handshake error from 10.244.1.1:50606: EOF 2018/07/26 11:34:45 http: TLS handshake error from 10.244.1.1:50612: EOF 2018/07/26 11:34:55 http: TLS handshake error from 10.244.1.1:50618: EOF 2018/07/26 11:35:05 http: TLS handshake error from 10.244.1.1:50624: EOF 2018/07/26 11:35:15 http: TLS handshake error from 10.244.1.1:50630: EOF 2018/07/26 11:35:25 http: TLS handshake error from 10.244.1.1:50636: EOF 2018/07/26 11:35:35 http: TLS handshake error from 10.244.1.1:50642: EOF 2018/07/26 11:35:45 http: TLS handshake error from 10.244.1.1:50648: EOF 2018/07/26 11:35:55 http: TLS handshake error from 10.244.1.1:50654: EOF 2018/07/26 11:36:05 http: TLS handshake error from 10.244.1.1:50660: EOF 2018/07/26 11:36:15 http: TLS handshake error from 10.244.1.1:50666: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:35:10 http: TLS handshake error from 10.244.0.1:58990: EOF level=info timestamp=2018-07-26T11:35:17.111271Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:35:20 http: TLS handshake error from 10.244.0.1:59014: EOF level=info timestamp=2018-07-26T11:35:25.105931Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:35:26.899841Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:35:30 http: TLS handshake error from 10.244.0.1:59038: EOF 2018/07/26 11:35:40 http: TLS handshake error from 10.244.0.1:59062: EOF level=info timestamp=2018-07-26T11:35:47.049615Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:35:50 http: TLS handshake error from 10.244.0.1:59086: EOF level=info timestamp=2018-07-26T11:35:54.324515Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:35:54.327719Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:35:55.234590Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:35:57.012684Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:36:00 http: TLS handshake error from 10.244.0.1:59110: EOF 2018/07/26 11:36:10 http: TLS handshake error from 10.244.0.1:59134: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:23:12.668793Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.824426Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmirtlkj\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmirtlkj, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4858b6ed-90c6-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmirtlkj" level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:33:16.617201Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:16.624115Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:33:33.731502Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:33.731751Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:35.012662Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:35.013115Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:35.013499Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:37.574151Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:37.574726Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:37.574953Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:42.695655Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:42.696044Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:42.696340Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimbffm" level=info timestamp=2018-07-26T11:33:52.937442Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:33:52.983115Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:33:52.986594Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:33:52.986819Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmimbffm-5cv2w Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200480c0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c020, 0xc4205965b0, 0xc421921fe0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e41e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc4200bd0c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e41e0, 0xc42000e078, 0x13, 0xc42005bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa040, 0xc4200e6370, 0xc4209850f0, 0xc420985100, 0xc421923e00, 0xc4200c60e0, 0x10f5e80, 0xc4205965b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.830 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-handler crashes /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:304 should recover and continue management [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:305 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:33:17.834383Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmimbffm-5cv2w" level=info timestamp=2018-07-26T11:33:33.155680Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmimbffm-5cv2w" level=error timestamp=2018-07-26T11:33:33.443414Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" • [SLOW TEST:54.694 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-handler is responsive /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:335 should indicate that a node is ready for vmis /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:336 ------------------------------ Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:35:05 http: TLS handshake error from 10.244.1.1:50624: EOF 2018/07/26 11:35:15 http: TLS handshake error from 10.244.1.1:50630: EOF 2018/07/26 11:35:25 http: TLS handshake error from 10.244.1.1:50636: EOF 2018/07/26 11:35:35 http: TLS handshake error from 10.244.1.1:50642: EOF 2018/07/26 11:35:45 http: TLS handshake error from 10.244.1.1:50648: EOF 2018/07/26 11:35:55 http: TLS handshake error from 10.244.1.1:50654: EOF 2018/07/26 11:36:05 http: TLS handshake error from 10.244.1.1:50660: EOF 2018/07/26 11:36:15 http: TLS handshake error from 10.244.1.1:50666: EOF 2018/07/26 11:36:25 http: TLS handshake error from 10.244.1.1:50672: EOF 2018/07/26 11:36:35 http: TLS handshake error from 10.244.1.1:50678: EOF 2018/07/26 11:36:45 http: TLS handshake error from 10.244.1.1:50684: EOF 2018/07/26 11:36:55 http: TLS handshake error from 10.244.1.1:50690: EOF 2018/07/26 11:37:05 http: TLS handshake error from 10.244.1.1:50696: EOF 2018/07/26 11:37:15 http: TLS handshake error from 10.244.1.1:50702: EOF 2018/07/26 11:37:25 http: TLS handshake error from 10.244.1.1:50708: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:36:27.165559Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:36:30 http: TLS handshake error from 10.244.0.1:59182: EOF 2018/07/26 11:36:40 http: TLS handshake error from 10.244.0.1:59206: EOF level=info timestamp=2018-07-26T11:36:47.218199Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:36:50 http: TLS handshake error from 10.244.0.1:59230: EOF level=info timestamp=2018-07-26T11:36:54.335197Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:36:54.344033Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:36:55.511959Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:36:57.259701Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:37:00 http: TLS handshake error from 10.244.0.1:59254: EOF 2018/07/26 11:37:10 http: TLS handshake error from 10.244.0.1:59278: EOF level=info timestamp=2018-07-26T11:37:17.142157Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:37:20 http: TLS handshake error from 10.244.0.1:59302: EOF level=info timestamp=2018-07-26T11:37:25.649037Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:37:27.372755Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:33:16.617201Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:16.624115Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:37:12.097792Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:37:12.102873Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:37:28.039703Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:28.040031Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:28.040350Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:28.061047Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:28.061508Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:28.061753Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:28.102352Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:28.102780Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:28.102986Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:28.183639Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:28.184034Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:28.184360Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:28.345085Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:28.345507Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:28.345795Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi4jvkj-jjrmn Pod phase: Running goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40b0, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4200c6080, 0xc4207765c8, 0xc420589b20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e40c0, 0xc42000e090, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bc080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e40c0, 0xc42000e090, 0x13, 0xc42049bf18, 0x3, 0x3, 0x105b2d1, 0x11a3360, 0xc420362660) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac100, 0xc4200e62d0, 0xc4216e30a0, 0xc4216e30b0, 0xc4215e0300, 0xc42000d440, 0x10f5e80, 0xc4207765c8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:37:45 http: TLS handshake error from 10.244.1.1:50720: EOF 2018/07/26 11:37:55 http: TLS handshake error from 10.244.1.1:50726: EOF 2018/07/26 11:38:05 http: TLS handshake error from 10.244.1.1:50732: EOF 2018/07/26 11:38:15 http: TLS handshake error from 10.244.1.1:50738: EOF 2018/07/26 11:38:25 http: TLS handshake error from 10.244.1.1:50744: EOF 2018/07/26 11:38:35 http: TLS handshake error from 10.244.1.1:50750: EOF 2018/07/26 11:38:45 http: TLS handshake error from 10.244.1.1:50756: EOF 2018/07/26 11:38:55 http: TLS handshake error from 10.244.1.1:50762: EOF 2018/07/26 11:39:05 http: TLS handshake error from 10.244.1.1:50768: EOF 2018/07/26 11:39:15 http: TLS handshake error from 10.244.1.1:50774: EOF 2018/07/26 11:39:25 http: TLS handshake error from 10.244.1.1:50780: EOF 2018/07/26 11:39:35 http: TLS handshake error from 10.244.1.1:50786: EOF 2018/07/26 11:39:45 http: TLS handshake error from 10.244.1.1:50792: EOF 2018/07/26 11:39:55 http: TLS handshake error from 10.244.1.1:50798: EOF 2018/07/26 11:40:05 http: TLS handshake error from 10.244.1.1:50804: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:38:57.715088Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:39:00 http: TLS handshake error from 10.244.0.1:59542: EOF 2018/07/26 11:39:10 http: TLS handshake error from 10.244.0.1:59566: EOF level=info timestamp=2018-07-26T11:39:17.146044Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:39:20 http: TLS handshake error from 10.244.0.1:59590: EOF level=info timestamp=2018-07-26T11:39:26.192755Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:27.808069Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:39:30 http: TLS handshake error from 10.244.0.1:59614: EOF 2018/07/26 11:39:40 http: TLS handshake error from 10.244.0.1:59638: EOF level=info timestamp=2018-07-26T11:39:47.134995Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:39:50 http: TLS handshake error from 10.244.0.1:59662: EOF level=info timestamp=2018-07-26T11:39:56.313690Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:57.945415Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:40:00 http: TLS handshake error from 10.244.0.1:59686: EOF 2018/07/26 11:40:10 http: TLS handshake error from 10.244.0.1:59710: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:23:12.932657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:23:12.932904Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixvk7j kind= uid=4898f807-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:26:13.840122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:33:16.617201Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:16.624115Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:37:12.097792Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:37:12.102873Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:37:29.309905Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:29.310208Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:30.591129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:30.591815Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:30.592145Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:33.153201Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:33.153903Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:33.154156Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:38.276805Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:38.277592Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:38.277882Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:48.518690Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:48.557054Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:48.559980Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:48.560225Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:16:38.880786Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:41.442042Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:41.443961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:41.444488Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:46.565823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:16:46.567073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:16:46.568558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmin9mdj" level=info timestamp=2018-07-26T11:16:56.809718Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.856668Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.869767Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:16:56.870026Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:56.891880Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:56.898016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind= uid=52efb26a-90c5-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:16:57.020103Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:16:57.021217Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmin9mdj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi4jvkj-jjrmn Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40b0, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4200c6080, 0xc4207765c8, 0xc420589b20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e40c0, 0xc42000e090, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bc080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e40c0, 0xc42000e090, 0x13, 0xc42049bf18, 0x3, 0x3, 0x105b2d1, 0x11a3360, 0xc420362660) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac100, 0xc4200e62d0, 0xc4216e30a0, 0xc4216e30b0, 0xc4215e0300, 0xc42000d440, 0x10f5e80, 0xc4207765c8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [180.707 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-handler is not responsive /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:366 the node controller should react [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:405 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:37:13.227319Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi4jvkj-jjrmn" level=info timestamp=2018-07-26T11:37:28.662794Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi4jvkj-jjrmn" level=error timestamp=2018-07-26T11:37:28.984871Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:38:05 http: TLS handshake error from 10.244.1.1:50732: EOF 2018/07/26 11:38:15 http: TLS handshake error from 10.244.1.1:50738: EOF 2018/07/26 11:38:25 http: TLS handshake error from 10.244.1.1:50744: EOF 2018/07/26 11:38:35 http: TLS handshake error from 10.244.1.1:50750: EOF 2018/07/26 11:38:45 http: TLS handshake error from 10.244.1.1:50756: EOF 2018/07/26 11:38:55 http: TLS handshake error from 10.244.1.1:50762: EOF 2018/07/26 11:39:05 http: TLS handshake error from 10.244.1.1:50768: EOF 2018/07/26 11:39:15 http: TLS handshake error from 10.244.1.1:50774: EOF 2018/07/26 11:39:25 http: TLS handshake error from 10.244.1.1:50780: EOF 2018/07/26 11:39:35 http: TLS handshake error from 10.244.1.1:50786: EOF 2018/07/26 11:39:45 http: TLS handshake error from 10.244.1.1:50792: EOF 2018/07/26 11:39:55 http: TLS handshake error from 10.244.1.1:50798: EOF 2018/07/26 11:40:05 http: TLS handshake error from 10.244.1.1:50804: EOF 2018/07/26 11:40:15 http: TLS handshake error from 10.244.1.1:50810: EOF 2018/07/26 11:40:25 http: TLS handshake error from 10.244.1.1:50816: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:39:20 http: TLS handshake error from 10.244.0.1:59590: EOF level=info timestamp=2018-07-26T11:39:26.192755Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:27.808069Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:39:30 http: TLS handshake error from 10.244.0.1:59614: EOF 2018/07/26 11:39:40 http: TLS handshake error from 10.244.0.1:59638: EOF level=info timestamp=2018-07-26T11:39:47.134995Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:39:50 http: TLS handshake error from 10.244.0.1:59662: EOF level=info timestamp=2018-07-26T11:39:56.313690Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:57.945415Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:40:00 http: TLS handshake error from 10.244.0.1:59686: EOF 2018/07/26 11:40:10 http: TLS handshake error from 10.244.0.1:59710: EOF level=info timestamp=2018-07-26T11:40:17.072673Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:40:20 http: TLS handshake error from 10.244.0.1:59734: EOF level=info timestamp=2018-07-26T11:40:26.415725Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:40:28.087361Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:33:16.617201Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:16.624115Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:37:12.097792Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:37:12.102873Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:40:13.006042Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:40:13.008555Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:40:13.100831Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmidgf88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:37:30.592145Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:33.153201Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:33.153903Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:33.154156Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:38.276805Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:38.277592Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:38.277882Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:48.518690Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:48.557054Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:48.559980Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:48.560225Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:12.705511Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:40:12.707553Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:12.766559Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:40:12.766986Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:40:28.439683Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:28.439803Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:28.439909Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:28.444798Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:28.444977Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:28.445739Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:28.469279Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:28.473239Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:28.473720Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:28.514097Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:28.514495Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:28.514698Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:28.595199Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:28.596903Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:28.597133Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" Pod name: virt-launcher-testvmidgf88-fcg8d Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420298100, 0xc4201b4c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420414080, 0xc4200f8630, 0xc4207994e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b66c0, 0xc42000e130, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200fc6c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b66c0, 0xc42000e130, 0x13, 0xc4204cdf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a380, 0xc4200ae370, 0xc42076a690, 0xc42076a6a0, 0xc4207d4300, 0xc4203a9020, 0x10f5e80, 0xc4200f8630, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:40:45 http: TLS handshake error from 10.244.1.1:50828: EOF 2018/07/26 11:40:55 http: TLS handshake error from 10.244.1.1:50834: EOF 2018/07/26 11:41:05 http: TLS handshake error from 10.244.1.1:50840: EOF 2018/07/26 11:41:15 http: TLS handshake error from 10.244.1.1:50846: EOF 2018/07/26 11:41:25 http: TLS handshake error from 10.244.1.1:50852: EOF 2018/07/26 11:41:35 http: TLS handshake error from 10.244.1.1:50858: EOF 2018/07/26 11:41:45 http: TLS handshake error from 10.244.1.1:50864: EOF 2018/07/26 11:41:55 http: TLS handshake error from 10.244.1.1:50870: EOF 2018/07/26 11:42:05 http: TLS handshake error from 10.244.1.1:50876: EOF 2018/07/26 11:42:15 http: TLS handshake error from 10.244.1.1:50882: EOF 2018/07/26 11:42:25 http: TLS handshake error from 10.244.1.1:50888: EOF 2018/07/26 11:42:35 http: TLS handshake error from 10.244.1.1:50894: EOF 2018/07/26 11:42:45 http: TLS handshake error from 10.244.1.1:50900: EOF 2018/07/26 11:42:55 http: TLS handshake error from 10.244.1.1:50906: EOF 2018/07/26 11:43:05 http: TLS handshake error from 10.244.1.1:50912: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:41:58.436836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:42:00 http: TLS handshake error from 10.244.0.1:59974: EOF 2018/07/26 11:42:10 http: TLS handshake error from 10.244.0.1:59998: EOF level=info timestamp=2018-07-26T11:42:17.082216Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:42:20 http: TLS handshake error from 10.244.0.1:60022: EOF level=info timestamp=2018-07-26T11:42:26.921883Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:28.553824Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:42:30 http: TLS handshake error from 10.244.0.1:60046: EOF 2018/07/26 11:42:40 http: TLS handshake error from 10.244.0.1:60070: EOF level=info timestamp=2018-07-26T11:42:47.205205Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:42:50 http: TLS handshake error from 10.244.0.1:60094: EOF level=info timestamp=2018-07-26T11:42:57.072847Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:58.665785Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:43:00 http: TLS handshake error from 10.244.0.1:60118: EOF 2018/07/26 11:43:10 http: TLS handshake error from 10.244.0.1:60142: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:26:13.843556Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr4hp kind= uid=b46a221f-90c6-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:14.684479Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:14.690553Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimnx2g kind= uid=20362784-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:29:30.439586Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:29:30.440877Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimk9qf kind= uid=299a558b-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:15.867530Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:15.870720Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmis267s kind= uid=44ae7225-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:30:16.026369Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmis267s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmis267s" level=info timestamp=2018-07-26T11:33:16.617201Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:16.624115Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimbffm kind= uid=b069895a-90c7-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:37:12.097792Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:37:12.102873Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:40:13.006042Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:40:13.008555Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:40:13.100831Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmidgf88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:37:30.592145Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:33.153201Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:33.153903Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:33.154156Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:38.276805Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:37:38.277592Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:37:38.277882Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4jvkj" level=info timestamp=2018-07-26T11:37:48.518690Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:48.557054Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:48.559980Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:48.560225Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:12.705511Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:40:12.707553Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind= uid=3cc15345-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:12.766559Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:40:12.766986Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4jvkj kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T11:40:29.722787Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:29.723557Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:31.004430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:31.006124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:31.006734Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:33.569135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:33.569995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:33.572611Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:38.694257Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:38.697947Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:38.699185Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:48.945708Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000684Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:49.000841Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000894Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmidgf88-fcg8d Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420298100, 0xc4201b4c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420414080, 0xc4200f8630, 0xc4207994e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b66c0, 0xc42000e130, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200fc6c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b66c0, 0xc42000e130, 0x13, 0xc4204cdf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a380, 0xc4200ae370, 0xc42076a690, 0xc42076a6a0, 0xc4207d4300, 0xc4203a9020, 0x10f5e80, 0xc4200f8630, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [181.086 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with node tainted /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:458 the vmi with tolerations should be scheduled [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:480 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:40:14.187808Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmidgf88-fcg8d" level=info timestamp=2018-07-26T11:40:29.054699Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmidgf88-fcg8d" level=error timestamp=2018-07-26T11:40:29.264516Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" • ------------------------------ S [SKIPPING] [0.482 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with non default namespace /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:530 should log libvirt start and stop lifecycle events of the domain /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 kubevirt-test-default [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Skip log query tests for JENKINS ci test environment /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:535 ------------------------------ S [SKIPPING] [0.151 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with non default namespace /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:530 should log libvirt start and stop lifecycle events of the domain /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 kubevirt-test-alternative [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Skip log query tests for JENKINS ci test environment /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:535 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.292 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 VirtualMachineInstance Emulation Mode /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:591 should enable emulation in virt-launcher [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:603 Software emulation is not enabled on this cluster /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:599 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.100 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 VirtualMachineInstance Emulation Mode /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:591 should be reflected in domain XML [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:640 Software emulation is not enabled on this cluster /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:599 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.096 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 VirtualMachineInstance Emulation Mode /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:591 should request a TUN device but not KVM [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:684 Software emulation is not enabled on this cluster /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:599 ------------------------------ •••• Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:41:05 http: TLS handshake error from 10.244.1.1:50840: EOF 2018/07/26 11:41:15 http: TLS handshake error from 10.244.1.1:50846: EOF 2018/07/26 11:41:25 http: TLS handshake error from 10.244.1.1:50852: EOF 2018/07/26 11:41:35 http: TLS handshake error from 10.244.1.1:50858: EOF 2018/07/26 11:41:45 http: TLS handshake error from 10.244.1.1:50864: EOF 2018/07/26 11:41:55 http: TLS handshake error from 10.244.1.1:50870: EOF 2018/07/26 11:42:05 http: TLS handshake error from 10.244.1.1:50876: EOF 2018/07/26 11:42:15 http: TLS handshake error from 10.244.1.1:50882: EOF 2018/07/26 11:42:25 http: TLS handshake error from 10.244.1.1:50888: EOF 2018/07/26 11:42:35 http: TLS handshake error from 10.244.1.1:50894: EOF 2018/07/26 11:42:45 http: TLS handshake error from 10.244.1.1:50900: EOF 2018/07/26 11:42:55 http: TLS handshake error from 10.244.1.1:50906: EOF 2018/07/26 11:43:05 http: TLS handshake error from 10.244.1.1:50912: EOF 2018/07/26 11:43:15 http: TLS handshake error from 10.244.1.1:50918: EOF 2018/07/26 11:43:25 http: TLS handshake error from 10.244.1.1:50924: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:42:26.921883Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:28.553824Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:42:30 http: TLS handshake error from 10.244.0.1:60046: EOF 2018/07/26 11:42:40 http: TLS handshake error from 10.244.0.1:60070: EOF level=info timestamp=2018-07-26T11:42:47.205205Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:42:50 http: TLS handshake error from 10.244.0.1:60094: EOF level=info timestamp=2018-07-26T11:42:57.072847Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:58.665785Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:43:00 http: TLS handshake error from 10.244.0.1:60118: EOF 2018/07/26 11:43:10 http: TLS handshake error from 10.244.0.1:60142: EOF level=info timestamp=2018-07-26T11:43:17.125228Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:43:20 http: TLS handshake error from 10.244.0.1:60166: EOF level=info timestamp=2018-07-26T11:43:27.232984Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:43:28.744922Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:43:30 http: TLS handshake error from 10.244.0.1:60190: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:14.107402Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:14.141970Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:14.177157Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:15.556831Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmik5zcw, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 14762f85-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:16.492929Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.493226Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:16.558462Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.745861Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmifx8l7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 15f90dd9-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.941048Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:43:33.010240Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:33.010628Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:33.010943Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:33.015563Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:33.016112Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:33.016529Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:33.037423Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:33.037865Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:33.038135Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:33.080347Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:33.080763Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:33.081037Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:33.161826Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:33.162243Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:33.162588Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:40:31.004430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:31.006124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:31.006734Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:33.569135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:33.569995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:33.572611Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:38.694257Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:38.697947Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:38.699185Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:48.945708Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000684Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:49.000841Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000894Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:13.928061Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:43:13.930165Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmih6zps-kjvh2 Pod phase: Running goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202e40f0, 0xc42019cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421938040, 0xc42000e9f8, 0xc42033f7c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098540, 0xc4200b85e0, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4202fab80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098540, 0xc4200b85e0, 0x13, 0xc42195c718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a16040) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bdd00, 0xc4200ba5f0, 0xc42196c6f0, 0xc42196c700, 0xc421956300, 0xc42192cd20, 0x10f5e80, 0xc42000e9f8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:43:55 http: TLS handshake error from 10.244.1.1:50942: EOF 2018/07/26 11:44:05 http: TLS handshake error from 10.244.1.1:50948: EOF 2018/07/26 11:44:15 http: TLS handshake error from 10.244.1.1:50954: EOF 2018/07/26 11:44:25 http: TLS handshake error from 10.244.1.1:50960: EOF 2018/07/26 11:44:35 http: TLS handshake error from 10.244.1.1:50966: EOF 2018/07/26 11:44:45 http: TLS handshake error from 10.244.1.1:50972: EOF 2018/07/26 11:44:55 http: TLS handshake error from 10.244.1.1:50978: EOF 2018/07/26 11:45:05 http: TLS handshake error from 10.244.1.1:50984: EOF 2018/07/26 11:45:15 http: TLS handshake error from 10.244.1.1:50990: EOF 2018/07/26 11:45:25 http: TLS handshake error from 10.244.1.1:50996: EOF 2018/07/26 11:45:35 http: TLS handshake error from 10.244.1.1:51002: EOF 2018/07/26 11:45:45 http: TLS handshake error from 10.244.1.1:51008: EOF 2018/07/26 11:45:55 http: TLS handshake error from 10.244.1.1:51014: EOF 2018/07/26 11:46:05 http: TLS handshake error from 10.244.1.1:51020: EOF 2018/07/26 11:46:15 http: TLS handshake error from 10.244.1.1:51026: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:45:17.052871Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:20 http: TLS handshake error from 10.244.0.1:60454: EOF level=info timestamp=2018-07-26T11:45:27.776875Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:45:29.240135Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:45:30 http: TLS handshake error from 10.244.0.1:60478: EOF 2018/07/26 11:45:40 http: TLS handshake error from 10.244.0.1:60502: EOF level=info timestamp=2018-07-26T11:45:47.285291Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:50 http: TLS handshake error from 10.244.0.1:60526: EOF level=info timestamp=2018-07-26T11:45:55.020117Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:45:55.023624Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:45:57.895905Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:45:59.346113Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:46:00 http: TLS handshake error from 10.244.0.1:60550: EOF 2018/07/26 11:46:10 http: TLS handshake error from 10.244.0.1:60574: EOF level=info timestamp=2018-07-26T11:46:17.082496Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:14.107402Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:14.141970Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:14.177157Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:15.556831Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmik5zcw, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 14762f85-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:16.492929Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.493226Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:16.558462Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.745861Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmifx8l7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 15f90dd9-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.941048Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:43:34.287041Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:34.287355Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:35.568017Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:35.568555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:35.568812Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:38.129539Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:38.129944Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:38.130231Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:43.250913Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:43:43.251540Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:43:43.251811Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:43:53.492618Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:53.540718Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:53.541062Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:53.541230Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:40:31.004430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:31.006124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:31.006734Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:33.569135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:33.569995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:33.572611Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:38.694257Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:38.697947Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:38.699185Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:48.945708Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000684Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:49.000841Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000894Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:13.928061Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:43:13.930165Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmih6zps-kjvh2 Pod phase: Failed goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202e40f0, 0xc42019cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421938040, 0xc42000e9f8, 0xc42033f7c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098540, 0xc4200b85e0, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4202fab80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098540, 0xc4200b85e0, 0x13, 0xc42195c718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a16040) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bdd00, 0xc4200ba5f0, 0xc42196c6f0, 0xc42196c700, 0xc421956300, 0xc42192cd20, 0x10f5e80, 0xc42000e9f8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.688 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Delete a VirtualMachineInstance's Pod /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:836 should result in the VirtualMachineInstance moving to a finalized state [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:837 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Creating the VirtualMachineInstance level=info timestamp=2018-07-26T11:43:18.932568Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmih6zps-kjvh2" level=info timestamp=2018-07-26T11:43:33.721883Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmih6zps-kjvh2" level=error timestamp=2018-07-26T11:43:33.977246Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:44:15 http: TLS handshake error from 10.244.1.1:50954: EOF 2018/07/26 11:44:25 http: TLS handshake error from 10.244.1.1:50960: EOF 2018/07/26 11:44:35 http: TLS handshake error from 10.244.1.1:50966: EOF 2018/07/26 11:44:45 http: TLS handshake error from 10.244.1.1:50972: EOF 2018/07/26 11:44:55 http: TLS handshake error from 10.244.1.1:50978: EOF 2018/07/26 11:45:05 http: TLS handshake error from 10.244.1.1:50984: EOF 2018/07/26 11:45:15 http: TLS handshake error from 10.244.1.1:50990: EOF 2018/07/26 11:45:25 http: TLS handshake error from 10.244.1.1:50996: EOF 2018/07/26 11:45:35 http: TLS handshake error from 10.244.1.1:51002: EOF 2018/07/26 11:45:45 http: TLS handshake error from 10.244.1.1:51008: EOF 2018/07/26 11:45:55 http: TLS handshake error from 10.244.1.1:51014: EOF 2018/07/26 11:46:05 http: TLS handshake error from 10.244.1.1:51020: EOF 2018/07/26 11:46:15 http: TLS handshake error from 10.244.1.1:51026: EOF 2018/07/26 11:46:25 http: TLS handshake error from 10.244.1.1:51032: EOF 2018/07/26 11:46:35 http: TLS handshake error from 10.244.1.1:51038: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:45:30 http: TLS handshake error from 10.244.0.1:60478: EOF 2018/07/26 11:45:40 http: TLS handshake error from 10.244.0.1:60502: EOF level=info timestamp=2018-07-26T11:45:47.285291Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:50 http: TLS handshake error from 10.244.0.1:60526: EOF level=info timestamp=2018-07-26T11:45:55.020117Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:45:55.023624Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:45:57.895905Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:45:59.346113Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:46:00 http: TLS handshake error from 10.244.0.1:60550: EOF 2018/07/26 11:46:10 http: TLS handshake error from 10.244.0.1:60574: EOF level=info timestamp=2018-07-26T11:46:17.082496Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:46:20 http: TLS handshake error from 10.244.0.1:60598: EOF level=info timestamp=2018-07-26T11:46:28.034875Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:46:29.513528Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:46:30 http: TLS handshake error from 10.244.0.1:60622: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:14.177157Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:15.556831Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmik5zcw, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 14762f85-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:16.492929Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.493226Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:16.558462Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.745861Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmifx8l7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 15f90dd9-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.941048Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:46:36.172098Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:36.172569Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:36.172827Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:36.177488Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:36.178059Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:36.178377Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:36.200615Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:36.201042Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:36.201603Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:36.242443Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:36.242818Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:36.243117Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:36.323872Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:36.324408Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:36.324719Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:40:31.004430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:31.006124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:31.006734Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:33.569135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:33.569995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:33.572611Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:38.694257Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:38.697947Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:38.699185Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:48.945708Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000684Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:49.000841Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000894Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:13.928061Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:43:13.930165Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmitr4vv-8dv6n Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 26 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6d70, 0xc420108280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420504060, 0xc42000e308, 0xc42192df80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8c60, 0xc4200b80a8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bd080, 0x2220202020202020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8c60, 0xc4200b80a8, 0x13, 0xc420268718, 0x3, 0x3, 0x22656d616e5f726f, 0x696a69654222203a, 0x6e7573654820676e) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c040, 0xc4200ba5f0, 0xc4205d8ef0, 0xc4205d8f00, 0xc42190be00, 0xc42000d2e0, 0x10f5e80, 0xc42000e308, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:46:55 http: TLS handshake error from 10.244.1.1:51050: EOF 2018/07/26 11:47:05 http: TLS handshake error from 10.244.1.1:51056: EOF 2018/07/26 11:47:15 http: TLS handshake error from 10.244.1.1:51062: EOF 2018/07/26 11:47:25 http: TLS handshake error from 10.244.1.1:51068: EOF 2018/07/26 11:47:35 http: TLS handshake error from 10.244.1.1:51074: EOF 2018/07/26 11:47:45 http: TLS handshake error from 10.244.1.1:51080: EOF 2018/07/26 11:47:55 http: TLS handshake error from 10.244.1.1:51086: EOF 2018/07/26 11:48:05 http: TLS handshake error from 10.244.1.1:51092: EOF 2018/07/26 11:48:15 http: TLS handshake error from 10.244.1.1:51098: EOF 2018/07/26 11:48:25 http: TLS handshake error from 10.244.1.1:51104: EOF 2018/07/26 11:48:35 http: TLS handshake error from 10.244.1.1:51110: EOF 2018/07/26 11:48:45 http: TLS handshake error from 10.244.1.1:51116: EOF 2018/07/26 11:48:55 http: TLS handshake error from 10.244.1.1:51122: EOF 2018/07/26 11:49:05 http: TLS handshake error from 10.244.1.1:51128: EOF 2018/07/26 11:49:15 http: TLS handshake error from 10.244.1.1:51134: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:48:17.102201Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:20 http: TLS handshake error from 10.244.0.1:60886: EOF level=info timestamp=2018-07-26T11:48:28.555017Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:48:29.942818Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:48:30 http: TLS handshake error from 10.244.0.1:60910: EOF 2018/07/26 11:48:40 http: TLS handshake error from 10.244.0.1:60934: EOF level=info timestamp=2018-07-26T11:48:47.071346Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:50 http: TLS handshake error from 10.244.0.1:60958: EOF level=info timestamp=2018-07-26T11:48:55.044917Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:55.054081Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:58.698257Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:49:00.075773Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:49:00 http: TLS handshake error from 10.244.0.1:60982: EOF 2018/07/26 11:49:10 http: TLS handshake error from 10.244.0.1:32774: EOF level=info timestamp=2018-07-26T11:49:17.194607Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:14.177157Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:15.556831Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik5zcw\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmik5zcw, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 14762f85-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik5zcw" level=info timestamp=2018-07-26T11:43:16.492929Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.493226Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifx8l7 kind= uid=15f90dd9-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:16.558462Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.745861Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmifx8l7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 15f90dd9-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.941048Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:46:37.449169Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:37.449533Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:38.730809Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:38.731465Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:38.731745Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:41.292947Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:41.293954Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:41.294513Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:46.415229Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:46.415664Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:46.415814Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:56.656756Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:56.695934Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:56.696904Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:56.697161Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:40:31.004430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:31.006124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:31.006734Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:33.569135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:33.569995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:33.572611Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:38.694257Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:40:38.697947Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:40:38.699185Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidgf88" level=info timestamp=2018-07-26T11:40:48.945708Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000684Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:49.000841Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:49.000894Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind= uid=a893f079-90c8-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:13.928061Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:43:13.930165Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidgf88 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmitr4vv-8dv6n Pod phase: Failed goroutine 26 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6d70, 0xc420108280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420504060, 0xc42000e308, 0xc42192df80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8c60, 0xc4200b80a8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bd080, 0x2220202020202020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8c60, 0xc4200b80a8, 0x13, 0xc420268718, 0x3, 0x3, 0x22656d616e5f726f, 0x696a69654222203a, 0x6e7573654820676e) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c040, 0xc4200ba5f0, 0xc4205d8ef0, 0xc4205d8f00, 0xc42190be00, 0xc42000d2e0, 0x10f5e80, 0xc42000e308, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.837 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Delete a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:868 with an active pod. /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:869 should result in pod being terminated [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:870 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Creating the VirtualMachineInstance level=info timestamp=2018-07-26T11:46:19.801685Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmitr4vv-8dv6n" level=info timestamp=2018-07-26T11:46:36.830959Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmitr4vv-8dv6n" level=error timestamp=2018-07-26T11:46:37.113887Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:47:05 http: TLS handshake error from 10.244.1.1:51056: EOF 2018/07/26 11:47:15 http: TLS handshake error from 10.244.1.1:51062: EOF 2018/07/26 11:47:25 http: TLS handshake error from 10.244.1.1:51068: EOF 2018/07/26 11:47:35 http: TLS handshake error from 10.244.1.1:51074: EOF 2018/07/26 11:47:45 http: TLS handshake error from 10.244.1.1:51080: EOF 2018/07/26 11:47:55 http: TLS handshake error from 10.244.1.1:51086: EOF 2018/07/26 11:48:05 http: TLS handshake error from 10.244.1.1:51092: EOF 2018/07/26 11:48:15 http: TLS handshake error from 10.244.1.1:51098: EOF 2018/07/26 11:48:25 http: TLS handshake error from 10.244.1.1:51104: EOF 2018/07/26 11:48:35 http: TLS handshake error from 10.244.1.1:51110: EOF 2018/07/26 11:48:45 http: TLS handshake error from 10.244.1.1:51116: EOF 2018/07/26 11:48:55 http: TLS handshake error from 10.244.1.1:51122: EOF 2018/07/26 11:49:05 http: TLS handshake error from 10.244.1.1:51128: EOF 2018/07/26 11:49:15 http: TLS handshake error from 10.244.1.1:51134: EOF 2018/07/26 11:49:25 http: TLS handshake error from 10.244.1.1:51140: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:48:30 http: TLS handshake error from 10.244.0.1:60910: EOF 2018/07/26 11:48:40 http: TLS handshake error from 10.244.0.1:60934: EOF level=info timestamp=2018-07-26T11:48:47.071346Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:50 http: TLS handshake error from 10.244.0.1:60958: EOF level=info timestamp=2018-07-26T11:48:55.044917Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:55.054081Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:58.698257Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:49:00.075773Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:49:00 http: TLS handshake error from 10.244.0.1:60982: EOF 2018/07/26 11:49:10 http: TLS handshake error from 10.244.0.1:32774: EOF level=info timestamp=2018-07-26T11:49:17.194607Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:49:20 http: TLS handshake error from 10.244.0.1:32798: EOF level=info timestamp=2018-07-26T11:49:28.826149Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:49:30.286606Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:49:30 http: TLS handshake error from 10.244.0.1:32822: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:16.558462Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.745861Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmifx8l7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 15f90dd9-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.941048Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.333766Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:49:19.339544Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.482875Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:19.544469Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:46:38.731745Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:41.292947Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:41.293954Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:41.294513Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:46.415229Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:46.415664Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:46.415814Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:56.656756Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:56.695934Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:56.696904Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:56.697161Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:19.105404Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:49:19.108685Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:19.129105Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:49:19.129542Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:49:35.061863Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:35.062196Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:35.071420Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:35.072725Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:35.073526Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:35.073910Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:35.094220Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:35.095676Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:35.095866Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:35.176634Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:35.177050Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:35.177914Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:35.341363Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:35.343418Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:35.343761Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" Pod name: virt-launcher-testvmilw2h5-fzqp8 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fe0b0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926040, 0xc42000f6b8, 0xc420391ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8600, 0xc4200b8618, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bdac0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8600, 0xc4200b8618, 0x13, 0xc421955718, 0x3, 0x3, 0x5cc549, 0xc4202e10b0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae340, 0xc4200ba5a0, 0xc421924850, 0xc421924860, 0xc42194c300, 0xc421918d40, 0x10f5e80, 0xc42000f6b8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:49:55 http: TLS handshake error from 10.244.1.1:51158: EOF 2018/07/26 11:50:05 http: TLS handshake error from 10.244.1.1:51164: EOF 2018/07/26 11:50:15 http: TLS handshake error from 10.244.1.1:51170: EOF 2018/07/26 11:50:25 http: TLS handshake error from 10.244.1.1:51176: EOF 2018/07/26 11:50:35 http: TLS handshake error from 10.244.1.1:51182: EOF 2018/07/26 11:50:45 http: TLS handshake error from 10.244.1.1:51188: EOF 2018/07/26 11:50:55 http: TLS handshake error from 10.244.1.1:51194: EOF 2018/07/26 11:51:05 http: TLS handshake error from 10.244.1.1:51200: EOF 2018/07/26 11:51:15 http: TLS handshake error from 10.244.1.1:51206: EOF 2018/07/26 11:51:25 http: TLS handshake error from 10.244.1.1:51212: EOF 2018/07/26 11:51:35 http: TLS handshake error from 10.244.1.1:51218: EOF 2018/07/26 11:51:45 http: TLS handshake error from 10.244.1.1:51224: EOF 2018/07/26 11:51:55 http: TLS handshake error from 10.244.1.1:51230: EOF 2018/07/26 11:52:05 http: TLS handshake error from 10.244.1.1:51236: EOF 2018/07/26 11:52:15 http: TLS handshake error from 10.244.1.1:51242: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:51:17.214024Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:20 http: TLS handshake error from 10.244.0.1:33086: EOF level=info timestamp=2018-07-26T11:51:29.406231Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:30 http: TLS handshake error from 10.244.0.1:33110: EOF level=info timestamp=2018-07-26T11:51:30.779601Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:40 http: TLS handshake error from 10.244.0.1:33134: EOF level=info timestamp=2018-07-26T11:51:47.042949Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:50 http: TLS handshake error from 10.244.0.1:33158: EOF level=info timestamp=2018-07-26T11:51:55.389722Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:51:55.400010Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:51:59.549386Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:52:00 http: TLS handshake error from 10.244.0.1:33182: EOF level=info timestamp=2018-07-26T11:52:00.897987Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:52:10 http: TLS handshake error from 10.244.0.1:33206: EOF level=info timestamp=2018-07-26T11:52:17.131879Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:16.558462Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.745861Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifx8l7\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmifx8l7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 15f90dd9-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifx8l7" level=info timestamp=2018-07-26T11:43:16.941048Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.333766Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:49:19.339544Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.482875Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:19.544469Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:46:38.731745Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:41.292947Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:41.293954Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:41.294513Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:46.415229Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:46.415664Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:46.415814Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitr4vv" level=info timestamp=2018-07-26T11:46:56.656756Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:56.695934Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:56.696904Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:56.697161Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:19.105404Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:49:19.108685Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:19.129105Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:49:19.129542Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitr4vv kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T11:49:36.309179Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:36.309493Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:37.592414Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:37.594241Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:37.595134Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:40.160012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:40.162441Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:40.164677Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:45.285768Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:45.287149Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:45.288070Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:55.529477Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.604788Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:55.607223Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.607625Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmilw2h5-fzqp8 Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fe0b0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926040, 0xc42000f6b8, 0xc420391ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8600, 0xc4200b8618, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bdac0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8600, 0xc4200b8618, 0x13, 0xc421955718, 0x3, 0x3, 0x5cc549, 0xc4202e10b0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae340, 0xc4200ba5a0, 0xc421924850, 0xc421924860, 0xc42194c300, 0xc421918d40, 0x10f5e80, 0xc42000f6b8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.749 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Delete a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:868 with grace period greater than 0 /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:893 should run graceful shutdown [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:894 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Setting a VirtualMachineInstance termination grace period to 5 STEP: Creating the VirtualMachineInstance level=info timestamp=2018-07-26T11:49:20.631394Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmilw2h5-fzqp8" level=info timestamp=2018-07-26T11:49:35.597728Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmilw2h5-fzqp8" level=error timestamp=2018-07-26T11:49:35.862159Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:50:15 http: TLS handshake error from 10.244.1.1:51170: EOF 2018/07/26 11:50:25 http: TLS handshake error from 10.244.1.1:51176: EOF 2018/07/26 11:50:35 http: TLS handshake error from 10.244.1.1:51182: EOF 2018/07/26 11:50:45 http: TLS handshake error from 10.244.1.1:51188: EOF 2018/07/26 11:50:55 http: TLS handshake error from 10.244.1.1:51194: EOF 2018/07/26 11:51:05 http: TLS handshake error from 10.244.1.1:51200: EOF 2018/07/26 11:51:15 http: TLS handshake error from 10.244.1.1:51206: EOF 2018/07/26 11:51:25 http: TLS handshake error from 10.244.1.1:51212: EOF 2018/07/26 11:51:35 http: TLS handshake error from 10.244.1.1:51218: EOF 2018/07/26 11:51:45 http: TLS handshake error from 10.244.1.1:51224: EOF 2018/07/26 11:51:55 http: TLS handshake error from 10.244.1.1:51230: EOF 2018/07/26 11:52:05 http: TLS handshake error from 10.244.1.1:51236: EOF 2018/07/26 11:52:15 http: TLS handshake error from 10.244.1.1:51242: EOF 2018/07/26 11:52:25 http: TLS handshake error from 10.244.1.1:51248: EOF 2018/07/26 11:52:35 http: TLS handshake error from 10.244.1.1:51254: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:51:30.779601Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:40 http: TLS handshake error from 10.244.0.1:33134: EOF level=info timestamp=2018-07-26T11:51:47.042949Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:50 http: TLS handshake error from 10.244.0.1:33158: EOF level=info timestamp=2018-07-26T11:51:55.389722Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:51:55.400010Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:51:59.549386Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:52:00 http: TLS handshake error from 10.244.0.1:33182: EOF level=info timestamp=2018-07-26T11:52:00.897987Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:52:10 http: TLS handshake error from 10.244.0.1:33206: EOF level=info timestamp=2018-07-26T11:52:17.131879Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:52:20 http: TLS handshake error from 10.244.0.1:33230: EOF level=info timestamp=2018-07-26T11:52:29.679410Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:52:30 http: TLS handshake error from 10.244.0.1:33254: EOF level=info timestamp=2018-07-26T11:52:31.014276Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.333766Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:49:19.339544Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.482875Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:19.544469Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:19.978884Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmilw2h5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ee3ca44a-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:20.117811Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:52:20.118456Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:52:36.729194Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:36.729828Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:36.730191Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:36.733130Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:36.733741Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:36.734006Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:36.754615Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:36.754963Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:36.755401Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:36.796215Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:36.796761Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:36.797011Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:36.877654Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:36.878146Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:36.878474Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:49:37.595134Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:40.160012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:40.162441Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:40.164677Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:45.285768Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:45.287149Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:45.288070Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:55.529477Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.604788Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:55.607223Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.607625Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.083631Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.089626Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.133985Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.134279Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmid9pwf-r8fd6 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a3d0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420401160, 0xc42000ef78, 0xc420603000, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098de0, 0xc4200b8250, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bdac0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098de0, 0xc4200b8250, 0x13, 0xc42005b718, 0x3, 0x3, 0x5cc549, 0xc420256420, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac4c0, 0xc4200baaa0, 0xc4215f9880, 0xc4215f9890, 0xc420231d80, 0xc4205020e0, 0x10f5e80, 0xc42000ef78, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:52:55 http: TLS handshake error from 10.244.1.1:51266: EOF 2018/07/26 11:53:05 http: TLS handshake error from 10.244.1.1:51272: EOF 2018/07/26 11:53:15 http: TLS handshake error from 10.244.1.1:51278: EOF 2018/07/26 11:53:25 http: TLS handshake error from 10.244.1.1:51284: EOF 2018/07/26 11:53:35 http: TLS handshake error from 10.244.1.1:51290: EOF 2018/07/26 11:53:45 http: TLS handshake error from 10.244.1.1:51296: EOF 2018/07/26 11:53:55 http: TLS handshake error from 10.244.1.1:51302: EOF 2018/07/26 11:54:05 http: TLS handshake error from 10.244.1.1:51308: EOF 2018/07/26 11:54:15 http: TLS handshake error from 10.244.1.1:51314: EOF 2018/07/26 11:54:25 http: TLS handshake error from 10.244.1.1:51320: EOF 2018/07/26 11:54:35 http: TLS handshake error from 10.244.1.1:51326: EOF 2018/07/26 11:54:45 http: TLS handshake error from 10.244.1.1:51332: EOF 2018/07/26 11:54:55 http: TLS handshake error from 10.244.1.1:51338: EOF 2018/07/26 11:55:05 http: TLS handshake error from 10.244.1.1:51344: EOF 2018/07/26 11:55:15 http: TLS handshake error from 10.244.1.1:51350: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:54:10 http: TLS handshake error from 10.244.0.1:33494: EOF level=info timestamp=2018-07-26T11:54:17.108849Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:20 http: TLS handshake error from 10.244.0.1:33518: EOF level=info timestamp=2018-07-26T11:54:30.237822Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:30 http: TLS handshake error from 10.244.0.1:33542: EOF level=info timestamp=2018-07-26T11:54:31.548181Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:40 http: TLS handshake error from 10.244.0.1:33566: EOF level=info timestamp=2018-07-26T11:54:47.162549Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:50 http: TLS handshake error from 10.244.0.1:33590: EOF 2018/07/26 11:55:00 http: TLS handshake error from 10.244.0.1:33614: EOF level=info timestamp=2018-07-26T11:55:00.388781Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:55:01.644945Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:55:10 http: TLS handshake error from 10.244.0.1:33638: EOF level=info timestamp=2018-07-26T11:55:17.043175Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:55:20 http: TLS handshake error from 10.244.0.1:33662: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:16.941253Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8b5ck kind= uid=163baa7e-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.072232Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.333766Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:49:19.339544Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.482875Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:19.544469Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:19.978884Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmilw2h5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ee3ca44a-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:20.117811Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:52:20.118456Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:52:38.003159Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:38.003494Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:39.284101Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:39.284562Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:39.284870Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:41.846018Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:41.846543Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:41.846904Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:46.967805Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:46.969542Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:46.969962Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid9pwf" level=info timestamp=2018-07-26T11:52:57.212128Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:57.270585Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:57.271052Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:57.271405Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:49:37.595134Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:40.160012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:40.162441Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:40.164677Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:45.285768Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:45.287149Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:45.288070Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:55.529477Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.604788Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:55.607223Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.607625Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.083631Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.089626Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.133985Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.134279Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmid9pwf-r8fd6 Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a3d0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420401160, 0xc42000ef78, 0xc420603000, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098de0, 0xc4200b8250, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bdac0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098de0, 0xc4200b8250, 0x13, 0xc42005b718, 0x3, 0x3, 0x5cc549, 0xc420256420, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac4c0, 0xc4200baaa0, 0xc4215f9880, 0xc4215f9890, 0xc420231d80, 0xc4205020e0, 0x10f5e80, 0xc42000ef78, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.809 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Killed VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:945 should be in Failed phase [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:946 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting a VirtualMachineInstance level=info timestamp=2018-07-26T11:52:21.524439Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmid9pwf-r8fd6" level=info timestamp=2018-07-26T11:52:37.406526Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmid9pwf-r8fd6" level=error timestamp=2018-07-26T11:52:37.666030Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:53:15 http: TLS handshake error from 10.244.1.1:51278: EOF 2018/07/26 11:53:25 http: TLS handshake error from 10.244.1.1:51284: EOF 2018/07/26 11:53:35 http: TLS handshake error from 10.244.1.1:51290: EOF 2018/07/26 11:53:45 http: TLS handshake error from 10.244.1.1:51296: EOF 2018/07/26 11:53:55 http: TLS handshake error from 10.244.1.1:51302: EOF 2018/07/26 11:54:05 http: TLS handshake error from 10.244.1.1:51308: EOF 2018/07/26 11:54:15 http: TLS handshake error from 10.244.1.1:51314: EOF 2018/07/26 11:54:25 http: TLS handshake error from 10.244.1.1:51320: EOF 2018/07/26 11:54:35 http: TLS handshake error from 10.244.1.1:51326: EOF 2018/07/26 11:54:45 http: TLS handshake error from 10.244.1.1:51332: EOF 2018/07/26 11:54:55 http: TLS handshake error from 10.244.1.1:51338: EOF 2018/07/26 11:55:05 http: TLS handshake error from 10.244.1.1:51344: EOF 2018/07/26 11:55:15 http: TLS handshake error from 10.244.1.1:51350: EOF 2018/07/26 11:55:25 http: TLS handshake error from 10.244.1.1:51356: EOF 2018/07/26 11:55:35 http: TLS handshake error from 10.244.1.1:51362: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T11:54:30.237822Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:30 http: TLS handshake error from 10.244.0.1:33542: EOF level=info timestamp=2018-07-26T11:54:31.548181Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:40 http: TLS handshake error from 10.244.0.1:33566: EOF level=info timestamp=2018-07-26T11:54:47.162549Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:50 http: TLS handshake error from 10.244.0.1:33590: EOF 2018/07/26 11:55:00 http: TLS handshake error from 10.244.0.1:33614: EOF level=info timestamp=2018-07-26T11:55:00.388781Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:55:01.644945Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:55:10 http: TLS handshake error from 10.244.0.1:33638: EOF level=info timestamp=2018-07-26T11:55:17.043175Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:55:20 http: TLS handshake error from 10.244.0.1:33662: EOF 2018/07/26 11:55:30 http: TLS handshake error from 10.244.0.1:33686: EOF level=info timestamp=2018-07-26T11:55:30.634643Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:55:31.775476Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.333766Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:49:19.339544Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.482875Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:19.544469Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:19.978884Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmilw2h5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ee3ca44a-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:20.117811Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:52:20.118456Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:55:20.970450Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:55:20.976694Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:55:37.243403Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:37.243579Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:37.243714Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:37.244550Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:37.244704Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:37.244940Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:37.265414Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:37.265608Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:37.265703Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:37.306110Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:37.306445Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:37.306571Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:37.388118Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:37.388495Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:37.388622Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:49:37.595134Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:40.160012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:40.162441Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:40.164677Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:45.285768Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:45.287149Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:45.288070Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:55.529477Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.604788Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:55.607223Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.607625Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.083631Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.089626Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.133985Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.134279Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmiwzk48-p6gjj Pod phase: Running goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a85d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928040, 0xc421a8cc40, 0xc4203409e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc4200b8818, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4202afbc0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc4200b8818, 0x13, 0xc42005ff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a8a940) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406080, 0xc4200ba780, 0xc421a960f0, 0xc421a96100, 0xc42193e300, 0xc421912d20, 0x10f5e80, 0xc421a8cc40, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:55:55 http: TLS handshake error from 10.244.1.1:51374: EOF 2018/07/26 11:56:05 http: TLS handshake error from 10.244.1.1:51380: EOF 2018/07/26 11:56:15 http: TLS handshake error from 10.244.1.1:51386: EOF 2018/07/26 11:56:25 http: TLS handshake error from 10.244.1.1:51392: EOF 2018/07/26 11:56:35 http: TLS handshake error from 10.244.1.1:51398: EOF 2018/07/26 11:56:45 http: TLS handshake error from 10.244.1.1:51404: EOF 2018/07/26 11:56:55 http: TLS handshake error from 10.244.1.1:51410: EOF 2018/07/26 11:57:05 http: TLS handshake error from 10.244.1.1:51416: EOF 2018/07/26 11:57:15 http: TLS handshake error from 10.244.1.1:51422: EOF 2018/07/26 11:57:25 http: TLS handshake error from 10.244.1.1:51428: EOF 2018/07/26 11:57:35 http: TLS handshake error from 10.244.1.1:51434: EOF 2018/07/26 11:57:45 http: TLS handshake error from 10.244.1.1:51440: EOF 2018/07/26 11:57:55 http: TLS handshake error from 10.244.1.1:51446: EOF 2018/07/26 11:58:05 http: TLS handshake error from 10.244.1.1:51452: EOF 2018/07/26 11:58:15 http: TLS handshake error from 10.244.1.1:51458: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:57:10 http: TLS handshake error from 10.244.0.1:33926: EOF level=info timestamp=2018-07-26T11:57:17.061176Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:20 http: TLS handshake error from 10.244.0.1:33950: EOF 2018/07/26 11:57:30 http: TLS handshake error from 10.244.0.1:33974: EOF level=info timestamp=2018-07-26T11:57:31.241272Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:57:32.283927Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:57:40 http: TLS handshake error from 10.244.0.1:33998: EOF level=info timestamp=2018-07-26T11:57:47.180642Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:50 http: TLS handshake error from 10.244.0.1:34022: EOF 2018/07/26 11:58:00 http: TLS handshake error from 10.244.0.1:34046: EOF level=info timestamp=2018-07-26T11:58:01.391439Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:58:02.418214Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:58:10 http: TLS handshake error from 10.244.0.1:34070: EOF level=info timestamp=2018-07-26T11:58:17.047497Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:58:20 http: TLS handshake error from 10.244.0.1:34094: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:43:17.217927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8b5ck\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8b5ck, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 163baa7e-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8b5ck" level=info timestamp=2018-07-26T11:43:17.687883Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:43:17.688122Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih6zps kind= uid=16aed46f-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:43:17.930894Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmih6zps\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmih6zps" level=info timestamp=2018-07-26T11:46:18.509195Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:46:18.513369Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitr4vv kind= uid=8273adc6-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.333766Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:49:19.339544Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:49:19.482875Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:19.544469Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:19.978884Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilw2h5\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmilw2h5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ee3ca44a-90c9-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:52:20.117811Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:52:20.118456Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:55:20.970450Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:55:20.976694Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T11:55:38.512006Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:38.512368Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:39.793230Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:39.793791Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:39.794150Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:42.355505Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:42.355977Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:42.356524Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:47.478107Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:47.479817Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:47.480534Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwzk48" level=info timestamp=2018-07-26T11:55:57.721641Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:57.783644Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:57.787743Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:57.788124Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:49:37.595134Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:40.160012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:40.162441Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:40.164677Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:45.285768Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:45.287149Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:45.288070Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilw2h5" level=info timestamp=2018-07-26T11:49:55.529477Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.604788Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:55.607223Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:55.607625Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.083631Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.089626Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind= uid=ee3ca44a-90c9-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:20.133985Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:52:20.134279Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilw2h5 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmiwzk48-p6gjj Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a85d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421928040, 0xc421a8cc40, 0xc4203409e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc4200b8818, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4202afbc0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc4200b8818, 0x13, 0xc42005ff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a8a940) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406080, 0xc4200ba780, 0xc421a960f0, 0xc421a96100, 0xc42193e300, 0xc421912d20, 0x10f5e80, 0xc421a8cc40, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.870 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Killed VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:945 should be left alone by virt-handler [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:973 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting a VirtualMachineInstance level=info timestamp=2018-07-26T11:55:22.230646Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmiwzk48-p6gjj" level=info timestamp=2018-07-26T11:55:37.978741Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmiwzk48-p6gjj" level=error timestamp=2018-07-26T11:55:38.188189Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:56:15 http: TLS handshake error from 10.244.1.1:51386: EOF 2018/07/26 11:56:25 http: TLS handshake error from 10.244.1.1:51392: EOF 2018/07/26 11:56:35 http: TLS handshake error from 10.244.1.1:51398: EOF 2018/07/26 11:56:45 http: TLS handshake error from 10.244.1.1:51404: EOF 2018/07/26 11:56:55 http: TLS handshake error from 10.244.1.1:51410: EOF 2018/07/26 11:57:05 http: TLS handshake error from 10.244.1.1:51416: EOF 2018/07/26 11:57:15 http: TLS handshake error from 10.244.1.1:51422: EOF 2018/07/26 11:57:25 http: TLS handshake error from 10.244.1.1:51428: EOF 2018/07/26 11:57:35 http: TLS handshake error from 10.244.1.1:51434: EOF 2018/07/26 11:57:45 http: TLS handshake error from 10.244.1.1:51440: EOF 2018/07/26 11:57:55 http: TLS handshake error from 10.244.1.1:51446: EOF 2018/07/26 11:58:05 http: TLS handshake error from 10.244.1.1:51452: EOF 2018/07/26 11:58:15 http: TLS handshake error from 10.244.1.1:51458: EOF 2018/07/26 11:58:25 http: TLS handshake error from 10.244.1.1:51464: EOF 2018/07/26 11:58:35 http: TLS handshake error from 10.244.1.1:51470: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 11:57:30 http: TLS handshake error from 10.244.0.1:33974: EOF level=info timestamp=2018-07-26T11:57:31.241272Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:57:32.283927Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:57:40 http: TLS handshake error from 10.244.0.1:33998: EOF level=info timestamp=2018-07-26T11:57:47.180642Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:50 http: TLS handshake error from 10.244.0.1:34022: EOF 2018/07/26 11:58:00 http: TLS handshake error from 10.244.0.1:34046: EOF level=info timestamp=2018-07-26T11:58:01.391439Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:58:02.418214Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:58:10 http: TLS handshake error from 10.244.0.1:34070: EOF level=info timestamp=2018-07-26T11:58:17.047497Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:58:20 http: TLS handshake error from 10.244.0.1:34094: EOF 2018/07/26 11:58:30 http: TLS handshake error from 10.244.0.1:34118: EOF level=info timestamp=2018-07-26T11:58:31.479772Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:58:32.538996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:52:20.117811Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:52:20.118456Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:55:20.970450Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:55:20.976694Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.865199Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.871138Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.897002Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.897352Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.931149Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.931460Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.999507Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.999809Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:22.116111Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.691036Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.904986Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmintpt5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmintpt5" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:58:38.493828Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:38.494058Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.494344Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:38.535002Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:38.535447Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.535611Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:38.616177Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:38.616676Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.616884Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:38.686601Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:38.686885Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.687151Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib7pjp" level=info timestamp=2018-07-26T11:58:38.779703Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:38.780163Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.780512Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:58:37.195160Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:37.195596Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:37.195744Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:37.276235Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:37.276521Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:37.276754Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:37.437711Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:37.438297Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:37.438564Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:37.759032Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:37.759403Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:37.760088Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:38.402089Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:38.403510Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.404630Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" Pod name: virt-launcher-testvmib7pjp-nqscq Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42026a400, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420560000, 0xc42000eae8, 0xc421923260, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e6c0, 0xc42000e2f8, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42029c980, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e6c0, 0xc42000e2f8, 0x13, 0xc421931718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207da540) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42029c700, 0xc4200d6eb0, 0xc4205191d0, 0xc4205191e0, 0xc421967d80, 0xc420560120, 0x10f5e80, 0xc42000eae8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmintpt5-pc479 Pod phase: Running goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420202450, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420ba75c0, 0xc42164ae90, 0xc421678520, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d20, 0xc4200b80d0, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008a040, 0x6166203a2272616c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d20, 0xc4200b80d0, 0x13, 0xc420bd4f18, 0x3, 0x3, 0x202020202020200a, 0x200a7b2020202020, 0x2020202020202020) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc700, 0xc4200ba870, 0xc420bbf270, 0xc420bbf280, 0xc420babd80, 0xc420bb20e0, 0x10f5e80, 0xc42164ae90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmip949s-7p2jz Pod phase: Running goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a9340, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421914ca0, 0xc42000e1f0, 0xc42029ec40, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e600, 0xc42000e0f8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bdb00, 0x100000114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e600, 0xc42000e0f8, 0x13, 0xc42197d718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4200bcb80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bce00, 0xc42023e8c0, 0xc42190bdb0, 0xc42190bdc0, 0xc421917d80, 0xc4219220a0, 0x10f5e80, 0xc42000e1f0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivkxfk-gtq6k Pod phase: Running goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc4d0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c2c0, 0xc42067c460, 0xc420694aa0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098840, 0xc4200b8240, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc42008a400, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098840, 0xc4200b8240, 0x13, 0xc42005bf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4206820c0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc640, 0xc4200ba7d0, 0xc4216818e8, 0xc421681900, 0xc420321d80, 0xc4200c60e0, 0x10f5e80, 0xc42067c460, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:58:55 http: TLS handshake error from 10.244.1.1:51482: EOF 2018/07/26 11:59:05 http: TLS handshake error from 10.244.1.1:51488: EOF 2018/07/26 11:59:15 http: TLS handshake error from 10.244.1.1:51494: EOF 2018/07/26 11:59:25 http: TLS handshake error from 10.244.1.1:51500: EOF 2018/07/26 11:59:35 http: TLS handshake error from 10.244.1.1:51506: EOF 2018/07/26 11:59:45 http: TLS handshake error from 10.244.1.1:51512: EOF 2018/07/26 11:59:55 http: TLS handshake error from 10.244.1.1:51518: EOF 2018/07/26 12:00:05 http: TLS handshake error from 10.244.1.1:51524: EOF 2018/07/26 12:00:15 http: TLS handshake error from 10.244.1.1:51530: EOF 2018/07/26 12:00:25 http: TLS handshake error from 10.244.1.1:51536: EOF 2018/07/26 12:00:35 http: TLS handshake error from 10.244.1.1:51542: EOF 2018/07/26 12:00:45 http: TLS handshake error from 10.244.1.1:51548: EOF 2018/07/26 12:00:55 http: TLS handshake error from 10.244.1.1:51554: EOF 2018/07/26 12:01:05 http: TLS handshake error from 10.244.1.1:51560: EOF 2018/07/26 12:01:15 http: TLS handshake error from 10.244.1.1:51566: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:00:30 http: TLS handshake error from 10.244.0.1:34406: EOF level=info timestamp=2018-07-26T12:00:32.036471Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:00:33.031919Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:00:40 http: TLS handshake error from 10.244.0.1:34430: EOF level=info timestamp=2018-07-26T12:00:46.828103Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:00:47.286099Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:50 http: TLS handshake error from 10.244.0.1:34454: EOF level=info timestamp=2018-07-26T12:00:55.384020Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:00:55.401559Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:01:00 http: TLS handshake error from 10.244.0.1:34478: EOF level=info timestamp=2018-07-26T12:01:02.159154Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:01:03.138572Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:01:10 http: TLS handshake error from 10.244.0.1:34502: EOF level=info timestamp=2018-07-26T12:01:17.060217Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:01:20 http: TLS handshake error from 10.244.0.1:34526: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:52:20.117811Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:52:20.118456Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid9pwf kind= uid=59fd1e8f-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:55:20.970450Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:55:20.976694Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwzk48 kind= uid=c5c866ee-90ca-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.865199Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.871138Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.897002Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.897352Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.931149Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.931460Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:21.999507Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:58:21.999809Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:22.116111Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.691036Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.904986Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmintpt5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmintpt5" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T11:58:48.705676Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:48.706181Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:48.706585Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:57.636973Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:57.666619Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:57.666977Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:57.667170Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:58.535605Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:58.569841Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:58.570152Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:58.570429Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmib7pjp kind= uid=31a1a3a1-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:58.947485Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:58.973909Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:58.974185Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:58.974448Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmip949s kind= uid=31999e8a-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T11:58:38.403510Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:38.404630Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:39.685682Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:39.686236Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:39.686766Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:42.249015Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:42.250257Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:42.250965Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:47.372135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:47.372911Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:47.373256Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:57.614204Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:57.661723Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:57.663676Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:57.663961Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmib7pjp-nqscq Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42026a400, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420560000, 0xc42000eae8, 0xc421923260, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e6c0, 0xc42000e2f8, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42029c980, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e6c0, 0xc42000e2f8, 0x13, 0xc421931718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207da540) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42029c700, 0xc4200d6eb0, 0xc4205191d0, 0xc4205191e0, 0xc421967d80, 0xc420560120, 0x10f5e80, 0xc42000eae8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmintpt5-pc479 Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420202450, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420ba75c0, 0xc42164ae90, 0xc421678520, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d20, 0xc4200b80d0, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008a040, 0x6166203a2272616c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d20, 0xc4200b80d0, 0x13, 0xc420bd4f18, 0x3, 0x3, 0x202020202020200a, 0x200a7b2020202020, 0x2020202020202020) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc700, 0xc4200ba870, 0xc420bbf270, 0xc420bbf280, 0xc420babd80, 0xc420bb20e0, 0x10f5e80, 0xc42164ae90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmip949s-7p2jz Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a9340, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421914ca0, 0xc42000e1f0, 0xc42029ec40, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e600, 0xc42000e0f8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bdb00, 0x100000114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e600, 0xc42000e0f8, 0x13, 0xc42197d718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4200bcb80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bce00, 0xc42023e8c0, 0xc42190bdb0, 0xc42190bdc0, 0xc421917d80, 0xc4219220a0, 0x10f5e80, 0xc42000e1f0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivkxfk-gtq6k Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc4d0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c2c0, 0xc42067c460, 0xc420694aa0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420098840, 0xc4200b8240, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc42008a400, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420098840, 0xc4200b8240, 0x13, 0xc42005bf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4206820c0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc640, 0xc4200ba7d0, 0xc4216818e8, 0xc421681900, 0xc420321d80, 0xc4200c60e0, 0x10f5e80, 0xc42067c460, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.379 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the Inbound VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:58:23.180658Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmip949s-7p2jz" level=info timestamp=2018-07-26T11:58:39.181407Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmip949s-7p2jz" level=error timestamp=2018-07-26T11:58:39.476123Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 11:59:15 http: TLS handshake error from 10.244.1.1:51494: EOF 2018/07/26 11:59:25 http: TLS handshake error from 10.244.1.1:51500: EOF 2018/07/26 11:59:35 http: TLS handshake error from 10.244.1.1:51506: EOF 2018/07/26 11:59:45 http: TLS handshake error from 10.244.1.1:51512: EOF 2018/07/26 11:59:55 http: TLS handshake error from 10.244.1.1:51518: EOF 2018/07/26 12:00:05 http: TLS handshake error from 10.244.1.1:51524: EOF 2018/07/26 12:00:15 http: TLS handshake error from 10.244.1.1:51530: EOF 2018/07/26 12:00:25 http: TLS handshake error from 10.244.1.1:51536: EOF 2018/07/26 12:00:35 http: TLS handshake error from 10.244.1.1:51542: EOF 2018/07/26 12:00:45 http: TLS handshake error from 10.244.1.1:51548: EOF 2018/07/26 12:00:55 http: TLS handshake error from 10.244.1.1:51554: EOF 2018/07/26 12:01:05 http: TLS handshake error from 10.244.1.1:51560: EOF 2018/07/26 12:01:15 http: TLS handshake error from 10.244.1.1:51566: EOF 2018/07/26 12:01:25 http: TLS handshake error from 10.244.1.1:51572: EOF 2018/07/26 12:01:35 http: TLS handshake error from 10.244.1.1:51578: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:00:40 http: TLS handshake error from 10.244.0.1:34430: EOF level=info timestamp=2018-07-26T12:00:46.828103Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:00:47.286099Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:50 http: TLS handshake error from 10.244.0.1:34454: EOF level=info timestamp=2018-07-26T12:00:55.384020Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:00:55.401559Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:01:00 http: TLS handshake error from 10.244.0.1:34478: EOF level=info timestamp=2018-07-26T12:01:02.159154Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:01:03.138572Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:01:10 http: TLS handshake error from 10.244.0.1:34502: EOF level=info timestamp=2018-07-26T12:01:17.060217Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:01:20 http: TLS handshake error from 10.244.0.1:34526: EOF 2018/07/26 12:01:30 http: TLS handshake error from 10.244.0.1:34550: EOF level=info timestamp=2018-07-26T12:01:32.253237Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:01:33.247723Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:58:21.999809Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:22.116111Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.691036Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.904986Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmintpt5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmintpt5" level=info timestamp=2018-07-26T12:01:23.750716Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.753792Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:23.816603Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.816932Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:23.877078Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.877458Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:23.957189Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:23.961584Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.961788Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:24.054956Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitz6wq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitz6wq" level=info timestamp=2018-07-26T12:01:24.063425Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:01:38.651794Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:38.652155Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:38.652489Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:38.674641Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:38.674999Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:38.675317Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:38.724373Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:38.724743Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:38.724963Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:38.806508Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:38.806709Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:38.806810Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:38.967097Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:38.967375Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:38.967503Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T11:58:42.249015Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:42.250257Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:42.250965Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:47.372135Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:47.372911Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:47.373256Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivkxfk" level=info timestamp=2018-07-26T11:58:57.614204Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:57.661723Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:57.663676Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:57.663961Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:23.197502Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T12:01:23.200515Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind= uid=31a56b18-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:23.429294Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T12:01:23.429816Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivkxfk kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:39.244907Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Processing vmi update" Pod name: virt-launcher-testvmi4q86q-76ftn Pod phase: Running level=info timestamp=2018-07-26T12:01:27.851385Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:01:27.851603Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:01:27.853242Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:01:37.862832Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:01:37.932137Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi4q86q" level=info timestamp=2018-07-26T12:01:37.934581Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:01:37.935189Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" level=info timestamp=2018-07-26T12:01:39.267901Z pos=cloud-init.go:254 component=virt-launcher msg="generated nocloud iso file /var/run/libvirt/kubevirt-ephemeral-disk/cloud-init-data/kubevirt-test-default/testvmi4q86q/noCloud.iso" level=error timestamp=2018-07-26T12:01:39.276397Z pos=common.go:126 component=virt-launcher msg="updated MAC for interface: eth0 - 0a:58:0a:35:aa:dd" level=info timestamp=2018-07-26T12:01:39.286788Z pos=converter.go:751 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \n`\u0000\n" level=info timestamp=2018-07-26T12:01:39.286876Z pos=converter.go:752 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local" level=info timestamp=2018-07-26T12:01:39.288432Z pos=dhcp.go:62 component=virt-launcher msg="Starting SingleClientDHCPServer" Pod name: virt-launcher-testvmi6m5wp-629vk Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0c0, 0xc42020c280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421656080, 0xc420beed18, 0xc420c294e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8540, 0xc42000e098, 0x13, 0x131ed4d, 0x4, 0xc42165bf18, 0x3, 0x3, 0xc4200bc080, 0x1e, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8540, 0xc42000e098, 0x13, 0xc421677f18, 0x3, 0x3, 0x1, 0x4, 0xc420437658) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420408040, 0xc4201a2190, 0xc421643f00, 0xc421643f10, 0xc42166c300, 0xc42164d100, 0x10f5e80, 0xc420beed18, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmic9wnt-l46r6 Pod phase: Running level=info timestamp=2018-07-26T12:01:29.321539Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:01:29.321795Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:01:29.323733Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" Pod name: virt-launcher-testvmitz6wq-vdbmz Pod phase: Running level=info timestamp=2018-07-26T12:01:28.151168Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:01:28.151310Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:01:28.152765Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:01:38.179613Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:01:38.225197Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmitz6wq" level=info timestamp=2018-07-26T12:01:38.239782Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:01:38.240006Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:01:55 http: TLS handshake error from 10.244.1.1:51590: EOF 2018/07/26 12:02:05 http: TLS handshake error from 10.244.1.1:51596: EOF 2018/07/26 12:02:15 http: TLS handshake error from 10.244.1.1:51602: EOF 2018/07/26 12:02:25 http: TLS handshake error from 10.244.1.1:51608: EOF 2018/07/26 12:02:35 http: TLS handshake error from 10.244.1.1:51614: EOF 2018/07/26 12:02:45 http: TLS handshake error from 10.244.1.1:51620: EOF 2018/07/26 12:02:55 http: TLS handshake error from 10.244.1.1:51626: EOF 2018/07/26 12:03:05 http: TLS handshake error from 10.244.1.1:51632: EOF 2018/07/26 12:03:15 http: TLS handshake error from 10.244.1.1:51638: EOF 2018/07/26 12:03:25 http: TLS handshake error from 10.244.1.1:51644: EOF 2018/07/26 12:03:35 http: TLS handshake error from 10.244.1.1:51650: EOF 2018/07/26 12:03:45 http: TLS handshake error from 10.244.1.1:51656: EOF 2018/07/26 12:03:55 http: TLS handshake error from 10.244.1.1:51662: EOF 2018/07/26 12:04:05 http: TLS handshake error from 10.244.1.1:51668: EOF 2018/07/26 12:04:15 http: TLS handshake error from 10.244.1.1:51674: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:03:20 http: TLS handshake error from 10.244.0.1:34814: EOF 2018/07/26 12:03:30 http: TLS handshake error from 10.244.0.1:34838: EOF level=info timestamp=2018-07-26T12:03:32.771466Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:03:33.722023Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:03:40 http: TLS handshake error from 10.244.0.1:34862: EOF level=info timestamp=2018-07-26T12:03:47.214509Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:50 http: TLS handshake error from 10.244.0.1:34886: EOF level=info timestamp=2018-07-26T12:03:55.460529Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:03:55.463850Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:04:00 http: TLS handshake error from 10.244.0.1:34910: EOF level=info timestamp=2018-07-26T12:04:02.901652Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:04:03.863614Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:04:10 http: TLS handshake error from 10.244.0.1:34934: EOF level=info timestamp=2018-07-26T12:04:17.100894Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:04:20 http: TLS handshake error from 10.244.0.1:34958: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T11:58:21.999809Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmintpt5 kind= uid=31ad39e4-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:58:22.116111Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.691036Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip949s\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip949s" level=info timestamp=2018-07-26T11:58:22.904986Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmintpt5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmintpt5" level=info timestamp=2018-07-26T12:01:23.750716Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.753792Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:23.816603Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.816932Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:23.877078Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.877458Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:23.957189Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:23.961584Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.961788Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:24.054956Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitz6wq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitz6wq" level=info timestamp=2018-07-26T12:01:24.063425Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:01:52.270915Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:52.271511Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:52.271838Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmic9wnt" level=info timestamp=2018-07-26T12:01:59.137679Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:59.198429Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:59.198844Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:59.198996Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6m5wp kind= uid=9e049608-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:02:00.563165Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:02:00.590480Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:02:00.590769Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:02:00.590938Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitz6wq kind= uid=9e0a9e24-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:02:02.512939Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:02:02.540742Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:02:02.543551Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:02:02.543728Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:01:40.675698Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:40.675944Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4q86q" level=info timestamp=2018-07-26T12:01:41.956780Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:41.957221Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:41.957639Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4q86q" level=info timestamp=2018-07-26T12:01:44.518443Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:44.518930Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:44.519189Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4q86q" level=info timestamp=2018-07-26T12:01:49.641145Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:49.641742Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:49.641951Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4q86q" level=info timestamp=2018-07-26T12:01:59.884929Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:59.915568Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:59.916212Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:59.918234Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4q86q kind= uid=9e1406ee-90cb-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi4q86q-76ftn Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d60b0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420644040, 0xc42000ec48, 0xc420d72740, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e120, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034c380, 0x1f, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e120, 0xc42000e078, 0x13, 0xc42066cf18, 0x3, 0x3, 0x1, 0x4, 0xc4205492f0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420564100, 0xc4200e65a0, 0xc420ce3730, 0xc420ce3740, 0xc420660300, 0xc4216b72e0, 0x10f5e80, 0xc42000ec48, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi6m5wp-629vk Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0c0, 0xc42020c280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421656080, 0xc420beed18, 0xc420c294e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8540, 0xc42000e098, 0x13, 0x131ed4d, 0x4, 0xc42165bf18, 0x3, 0x3, 0xc4200bc080, 0x1e, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8540, 0xc42000e098, 0x13, 0xc421677f18, 0x3, 0x3, 0x1, 0x4, 0xc420437658) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420408040, 0xc4201a2190, 0xc421643f00, 0xc421643f10, 0xc42166c300, 0xc42164d100, 0x10f5e80, 0xc420beed18, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmic9wnt-l46r6 Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42195e040, 0xc421be2388, 0xc42000db20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e180, 0xc42000e098, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bdb00, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e180, 0xc42000e098, 0x13, 0xc421986f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ae2c0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c080, 0xc42010e190, 0xc421949dc0, 0xc421949dd0, 0xc42197c300, 0xc421952d20, 0x10f5e80, 0xc421be2388, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmitz6wq-vdbmz Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027c0f0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421944080, 0xc42000e290, 0xc42000dae0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bdb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc42000e078, 0x13, 0xc421973718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc4201885f0, 0xc42002c050, 0xc42002c070, 0xc421966300, 0xc42193ad00, 0x10f5e80, 0xc42000e290, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.984 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the Inbound VirtualMachineInstance with pod network connectivity explicitly set /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:01:25.014203Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi6m5wp-629vk" level=info timestamp=2018-07-26T12:01:39.400453Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi6m5wp-629vk" level=error timestamp=2018-07-26T12:01:39.597384Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:02:15 http: TLS handshake error from 10.244.1.1:51602: EOF 2018/07/26 12:02:25 http: TLS handshake error from 10.244.1.1:51608: EOF 2018/07/26 12:02:35 http: TLS handshake error from 10.244.1.1:51614: EOF 2018/07/26 12:02:45 http: TLS handshake error from 10.244.1.1:51620: EOF 2018/07/26 12:02:55 http: TLS handshake error from 10.244.1.1:51626: EOF 2018/07/26 12:03:05 http: TLS handshake error from 10.244.1.1:51632: EOF 2018/07/26 12:03:15 http: TLS handshake error from 10.244.1.1:51638: EOF 2018/07/26 12:03:25 http: TLS handshake error from 10.244.1.1:51644: EOF 2018/07/26 12:03:35 http: TLS handshake error from 10.244.1.1:51650: EOF 2018/07/26 12:03:45 http: TLS handshake error from 10.244.1.1:51656: EOF 2018/07/26 12:03:55 http: TLS handshake error from 10.244.1.1:51662: EOF 2018/07/26 12:04:05 http: TLS handshake error from 10.244.1.1:51668: EOF 2018/07/26 12:04:15 http: TLS handshake error from 10.244.1.1:51674: EOF 2018/07/26 12:04:25 http: TLS handshake error from 10.244.1.1:51680: EOF 2018/07/26 12:04:35 http: TLS handshake error from 10.244.1.1:51686: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:03:40 http: TLS handshake error from 10.244.0.1:34862: EOF level=info timestamp=2018-07-26T12:03:47.214509Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:50 http: TLS handshake error from 10.244.0.1:34886: EOF level=info timestamp=2018-07-26T12:03:55.460529Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:03:55.463850Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:04:00 http: TLS handshake error from 10.244.0.1:34910: EOF level=info timestamp=2018-07-26T12:04:02.901652Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:04:03.863614Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:04:10 http: TLS handshake error from 10.244.0.1:34934: EOF level=info timestamp=2018-07-26T12:04:17.100894Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:04:20 http: TLS handshake error from 10.244.0.1:34958: EOF 2018/07/26 12:04:30 http: TLS handshake error from 10.244.0.1:34982: EOF level=info timestamp=2018-07-26T12:04:33.087868Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:04:33.951144Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:04:40 http: TLS handshake error from 10.244.0.1:35006: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:01:23.957189Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:23.961584Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.961788Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:24.054956Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitz6wq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitz6wq" level=info timestamp=2018-07-26T12:01:24.063425Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:04:25.517724Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.525624Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.532122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.532471Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.555451Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.555668Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.589189Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.589517Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.713783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikc5f9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikc5f9" level=info timestamp=2018-07-26T12:04:25.714974Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminwv78\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T12:04:41.454603Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.454870Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:04:41.475688Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:41.476074Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.476503Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:04:41.517073Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:41.517523Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.517763Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:04:41.599730Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:41.600101Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.600489Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:04:41.764604Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:41.764995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.765225Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:04:41.899803Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Processing vmi update" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:04:40.617133Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:40.618278Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:40.619060Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:40.699772Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:40.700263Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:40.700770Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:40.861139Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:40.861705Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:40.861878Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:41.185122Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:41.205934Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.207371Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:41.850365Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:41.850585Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.850681Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" Pod name: virt-launcher-testvmidvg6m-tkzm2 Pod phase: Running goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202000b0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930040, 0xc4202a65b0, 0xc420340860, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bc280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc42000e078, 0x13, 0xc42195f718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4219b2b00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a600, 0xc4200ba4b0, 0xc4208477a0, 0xc4208477b0, 0xc421956300, 0xc421922d00, 0x10f5e80, 0xc4202a65b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmikc5f9-mbzwz Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42026c0b0, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421946040, 0xc42000f2b8, 0xc420288080, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e120, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4205682c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e120, 0xc42000e078, 0x13, 0xc42196af18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4219ed340) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568100, 0xc4200d6280, 0xc421945440, 0xc421945450, 0xc421964300, 0xc42193ad20, 0x10f5e80, 0xc42000f2b8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvminwv78-nn9gq Pod phase: Running goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200480d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930040, 0xc421bb68e0, 0xc42029f6e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8480, 0xc4200b8080, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42034c680, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8480, 0xc4200b8080, 0x13, 0xc421952f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a100, 0xc4200e65f0, 0xc420242030, 0xc420242040, 0xc42194e300, 0xc421922ce0, 0x10f5e80, 0xc421bb68e0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmitv6wk-lp9wd Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0c0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202ec080, 0xc420670c90, 0xc4206a2960, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8660, 0xc42000e670, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc42008a0c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8660, 0xc42000e670, 0x13, 0xc4203d1718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42063fbc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc2c0, 0xc4200ba6e0, 0xc420678680, 0xc420678690, 0xc42022c300, 0xc42029cc80, 0x10f5e80, 0xc420670c90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:05:05 http: TLS handshake error from 10.244.1.1:51704: EOF 2018/07/26 12:05:15 http: TLS handshake error from 10.244.1.1:51710: EOF 2018/07/26 12:05:25 http: TLS handshake error from 10.244.1.1:51716: EOF 2018/07/26 12:05:35 http: TLS handshake error from 10.244.1.1:51722: EOF 2018/07/26 12:05:45 http: TLS handshake error from 10.244.1.1:51728: EOF 2018/07/26 12:05:55 http: TLS handshake error from 10.244.1.1:51734: EOF 2018/07/26 12:06:05 http: TLS handshake error from 10.244.1.1:51740: EOF 2018/07/26 12:06:15 http: TLS handshake error from 10.244.1.1:51746: EOF 2018/07/26 12:06:25 http: TLS handshake error from 10.244.1.1:51752: EOF 2018/07/26 12:06:35 http: TLS handshake error from 10.244.1.1:51758: EOF 2018/07/26 12:06:45 http: TLS handshake error from 10.244.1.1:51764: EOF 2018/07/26 12:06:55 http: TLS handshake error from 10.244.1.1:51770: EOF 2018/07/26 12:07:05 http: TLS handshake error from 10.244.1.1:51776: EOF 2018/07/26 12:07:15 http: TLS handshake error from 10.244.1.1:51782: EOF 2018/07/26 12:07:25 http: TLS handshake error from 10.244.1.1:51788: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:06:20 http: TLS handshake error from 10.244.0.1:35246: EOF 2018/07/26 12:06:30 http: TLS handshake error from 10.244.0.1:35270: EOF level=info timestamp=2018-07-26T12:06:33.567767Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:06:34.441687Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:06:40 http: TLS handshake error from 10.244.0.1:35294: EOF level=info timestamp=2018-07-26T12:06:47.121440Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:06:50 http: TLS handshake error from 10.244.0.1:35318: EOF level=info timestamp=2018-07-26T12:06:55.032996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:06:55.035648Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:07:00 http: TLS handshake error from 10.244.0.1:35342: EOF level=info timestamp=2018-07-26T12:07:03.703606Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:07:04.559602Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:07:10 http: TLS handshake error from 10.244.0.1:35366: EOF level=info timestamp=2018-07-26T12:07:17.087695Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:07:20 http: TLS handshake error from 10.244.0.1:35390: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:01:23.957189Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:01:23.961584Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:01:23.961788Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmic9wnt kind= uid=9e1e39ca-90cb-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:01:24.054956Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitz6wq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitz6wq" level=info timestamp=2018-07-26T12:01:24.063425Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6m5wp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6m5wp" level=info timestamp=2018-07-26T12:04:25.517724Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.525624Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.532122Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.532471Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.555451Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.555668Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.589189Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.589517Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.713783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikc5f9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikc5f9" level=info timestamp=2018-07-26T12:04:25.714974Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminwv78\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:04:52.543743Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:52.544211Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:52.544532Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitv6wk" level=info timestamp=2018-07-26T12:05:01.942008Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:01.966417Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:05:01.967675Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:01.968061Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminwv78 kind= uid=0a5dde22-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:05:02.557887Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:02.578868Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:05:02.579157Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:02.579370Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmikc5f9 kind= uid=0a61a262-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:05:02.785571Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:02.809699Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:05:02.810020Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:02.810164Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:04:41.850585Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:41.850681Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:43.132176Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:43.134025Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:43.135471Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:45.699223Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:45.699787Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:45.700087Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:04:50.820957Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:50.821614Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:50.821956Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidvg6m" level=info timestamp=2018-07-26T12:05:01.067998Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:01.119068Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:05:01.119436Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:05:01.119619Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidvg6m kind= uid=0a63e896-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmidvg6m-tkzm2 Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202000b0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930040, 0xc4202a65b0, 0xc420340860, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bc280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc42000e078, 0x13, 0xc42195f718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4219b2b00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a600, 0xc4200ba4b0, 0xc4208477a0, 0xc4208477b0, 0xc421956300, 0xc421922d00, 0x10f5e80, 0xc4202a65b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmikc5f9-mbzwz Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42026c0b0, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421946040, 0xc42000f2b8, 0xc420288080, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e120, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4205682c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e120, 0xc42000e078, 0x13, 0xc42196af18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4219ed340) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568100, 0xc4200d6280, 0xc421945440, 0xc421945450, 0xc421964300, 0xc42193ad20, 0x10f5e80, 0xc42000f2b8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminwv78-nn9gq Pod phase: Failed goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200480d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930040, 0xc421bb68e0, 0xc42029f6e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8480, 0xc4200b8080, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42034c680, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8480, 0xc4200b8080, 0x13, 0xc421952f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a100, 0xc4200e65f0, 0xc420242030, 0xc420242040, 0xc42194e300, 0xc421922ce0, 0x10f5e80, 0xc421bb68e0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmitv6wk-lp9wd Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0c0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202ec080, 0xc420670c90, 0xc4206a2960, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8660, 0xc42000e670, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc42008a0c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8660, 0xc42000e670, 0x13, 0xc4203d1718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42063fbc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc2c0, 0xc4200ba6e0, 0xc420678680, 0xc420678690, 0xc42022c300, 0xc42029cc80, 0x10f5e80, 0xc420670c90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.635 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the Inbound VirtualMachineInstance with custom MAC address /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:04:26.753266Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvminwv78-nn9gq" level=info timestamp=2018-07-26T12:04:42.019996Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvminwv78-nn9gq" level=error timestamp=2018-07-26T12:04:42.387146Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:05:25 http: TLS handshake error from 10.244.1.1:51716: EOF 2018/07/26 12:05:35 http: TLS handshake error from 10.244.1.1:51722: EOF 2018/07/26 12:05:45 http: TLS handshake error from 10.244.1.1:51728: EOF 2018/07/26 12:05:55 http: TLS handshake error from 10.244.1.1:51734: EOF 2018/07/26 12:06:05 http: TLS handshake error from 10.244.1.1:51740: EOF 2018/07/26 12:06:15 http: TLS handshake error from 10.244.1.1:51746: EOF 2018/07/26 12:06:25 http: TLS handshake error from 10.244.1.1:51752: EOF 2018/07/26 12:06:35 http: TLS handshake error from 10.244.1.1:51758: EOF 2018/07/26 12:06:45 http: TLS handshake error from 10.244.1.1:51764: EOF 2018/07/26 12:06:55 http: TLS handshake error from 10.244.1.1:51770: EOF 2018/07/26 12:07:05 http: TLS handshake error from 10.244.1.1:51776: EOF 2018/07/26 12:07:15 http: TLS handshake error from 10.244.1.1:51782: EOF 2018/07/26 12:07:25 http: TLS handshake error from 10.244.1.1:51788: EOF 2018/07/26 12:07:35 http: TLS handshake error from 10.244.1.1:51794: EOF 2018/07/26 12:07:45 http: TLS handshake error from 10.244.1.1:51800: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:06:40 http: TLS handshake error from 10.244.0.1:35294: EOF level=info timestamp=2018-07-26T12:06:47.121440Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:06:50 http: TLS handshake error from 10.244.0.1:35318: EOF level=info timestamp=2018-07-26T12:06:55.032996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:06:55.035648Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:07:00 http: TLS handshake error from 10.244.0.1:35342: EOF level=info timestamp=2018-07-26T12:07:03.703606Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:07:04.559602Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:07:10 http: TLS handshake error from 10.244.0.1:35366: EOF level=info timestamp=2018-07-26T12:07:17.087695Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:07:20 http: TLS handshake error from 10.244.0.1:35390: EOF 2018/07/26 12:07:30 http: TLS handshake error from 10.244.0.1:35414: EOF level=info timestamp=2018-07-26T12:07:33.777354Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:07:34.626542Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:07:40 http: TLS handshake error from 10.244.0.1:35438: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:04:25.589189Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.589517Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.713783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikc5f9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikc5f9" level=info timestamp=2018-07-26T12:04:25.714974Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminwv78\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:07:27.385339Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.388251Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.411192Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.411513Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.456637Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.457064Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.490563Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.490808Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.843217Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:07:27.899927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir6wg7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir6wg7" level=info timestamp=2018-07-26T12:07:28.524305Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:07:45.956739Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:45.957113Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:45.957563Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5jw9h" level=info timestamp=2018-07-26T12:07:45.998232Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:45.998735Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:45.998972Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5jw9h" level=info timestamp=2018-07-26T12:07:46.079912Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:46.080226Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:46.080530Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5jw9h" level=info timestamp=2018-07-26T12:07:46.209751Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:46.210220Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:46.210563Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:07:46.241140Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:46.241751Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:46.242025Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5jw9h" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:07:43.490166Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:43.491476Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:43.491652Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:43.652049Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:43.652282Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:43.653231Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:43.974881Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:43.975298Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:43.978692Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:44.620167Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:44.620871Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:44.621135Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:45.913990Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:45.921932Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:45.924560Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" Pod name: virt-launcher-testvmi5jw9h-lzg6j Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6510, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42029f300, 0xc42000e448, 0xc420693420, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8290, 0x13, 0x131ed4d, 0x4, 0xc420071f18, 0x3, 0x3, 0xc42008a080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8290, 0x13, 0xc42005b718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4216a0a00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae4c0, 0xc4200baaa0, 0xc4200d5260, 0xc4200d5270, 0xc4203cfd80, 0xc4202ec0c0, 0x10f5e80, 0xc42000e448, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmi7pdb8-nd5f9 Pod phase: Running goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420236380, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190acc0, 0xc42000fee8, 0xc420388d60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e660, 0xc42000e240, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc420348c80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e660, 0xc42000e240, 0x13, 0xc421a6cf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42198e940) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420348640, 0xc4200e6870, 0xc421994450, 0xc421994460, 0xc42190de00, 0xc4219160a0, 0x10f5e80, 0xc42000fee8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiq9w88-252rh Pod phase: Running goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202a20f0, 0xc420118000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420360440, 0xc4207c8740, 0xc4205f3f80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200f20c0, 0xc42000e090, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200fb8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200f20c0, 0xc42000e090, 0x13, 0xc420296f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420763d80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a300, 0xc4200a8370, 0xc4207c0ee0, 0xc4207c0ef0, 0xc4207d4300, 0xc4203111a0, 0x10f5e80, 0xc4207c8740, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmir6wg7-dj2qs Pod phase: Running goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203e8390, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202f1800, 0xc42000e450, 0xc4206ce280, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8de0, 0xc4200b8100, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008b400, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8de0, 0xc4200b8100, 0x13, 0xc42005bf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4216d4100) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202fe500, 0xc4200ba960, 0xc4200d49e0, 0xc4200d49f0, 0xc420483d80, 0xc42033e0e0, 0x10f5e80, 0xc42000e450, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:08:05 http: TLS handshake error from 10.244.1.1:51812: EOF 2018/07/26 12:08:15 http: TLS handshake error from 10.244.1.1:51818: EOF 2018/07/26 12:08:25 http: TLS handshake error from 10.244.1.1:51824: EOF 2018/07/26 12:08:35 http: TLS handshake error from 10.244.1.1:51830: EOF 2018/07/26 12:08:45 http: TLS handshake error from 10.244.1.1:51836: EOF 2018/07/26 12:08:55 http: TLS handshake error from 10.244.1.1:51842: EOF 2018/07/26 12:09:05 http: TLS handshake error from 10.244.1.1:51848: EOF 2018/07/26 12:09:15 http: TLS handshake error from 10.244.1.1:51854: EOF 2018/07/26 12:09:25 http: TLS handshake error from 10.244.1.1:51860: EOF 2018/07/26 12:09:35 http: TLS handshake error from 10.244.1.1:51866: EOF 2018/07/26 12:09:45 http: TLS handshake error from 10.244.1.1:51872: EOF 2018/07/26 12:09:55 http: TLS handshake error from 10.244.1.1:51878: EOF 2018/07/26 12:10:05 http: TLS handshake error from 10.244.1.1:51884: EOF 2018/07/26 12:10:15 http: TLS handshake error from 10.244.1.1:51890: EOF 2018/07/26 12:10:25 http: TLS handshake error from 10.244.1.1:51896: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:09:10 http: TLS handshake error from 10.244.0.1:35654: EOF level=info timestamp=2018-07-26T12:09:17.130216Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:09:20 http: TLS handshake error from 10.244.0.1:35678: EOF 2018/07/26 12:09:30 http: TLS handshake error from 10.244.0.1:35702: EOF level=info timestamp=2018-07-26T12:09:34.412682Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:09:35.151147Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:09:40 http: TLS handshake error from 10.244.0.1:35726: EOF level=info timestamp=2018-07-26T12:09:47.160381Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:09:50 http: TLS handshake error from 10.244.0.1:35750: EOF 2018/07/26 12:10:00 http: TLS handshake error from 10.244.0.1:35774: EOF level=info timestamp=2018-07-26T12:10:04.598802Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:10:05.236143Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:10:10 http: TLS handshake error from 10.244.0.1:35798: EOF level=info timestamp=2018-07-26T12:10:17.182292Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:10:20 http: TLS handshake error from 10.244.0.1:35822: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:04:25.589189Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:04:25.589517Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitv6wk kind= uid=0a6778cf-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:04:25.713783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikc5f9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikc5f9" level=info timestamp=2018-07-26T12:04:25.714974Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminwv78\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminwv78" level=info timestamp=2018-07-26T12:07:27.385339Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.388251Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.411192Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.411513Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.456637Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.457064Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.490563Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:07:27.490808Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.843217Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:07:27.899927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir6wg7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir6wg7" level=info timestamp=2018-07-26T12:07:28.524305Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:07:56.168430Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:56.168856Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:56.169082Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5jw9h" level=info timestamp=2018-07-26T12:08:05.097802Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:05.118186Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:08:05.126777Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:05.126965Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:08:06.057401Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:06.080120Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:08:06.080443Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:06.080559Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiq9w88 kind= uid=76c91363-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:08:06.410768Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:06.432999Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:08:06.436866Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:06.437122Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5jw9h kind= uid=76c1fcaa-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:07:44.620871Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:44.621135Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:45.913990Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:45.921932Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:45.924560Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:48.485778Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:48.486500Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:48.486876Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:07:53.607856Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:53.608909Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:53.609216Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:08:03.850984Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:03.955951Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:08:03.963995Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:08:03.964235Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi7pdb8 kind= uid=76cd1ccb-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi5jw9h-lzg6j Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6510, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42029f300, 0xc42000e448, 0xc420693420, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8290, 0x13, 0x131ed4d, 0x4, 0xc420071f18, 0x3, 0x3, 0xc42008a080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8290, 0x13, 0xc42005b718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4216a0a00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae4c0, 0xc4200baaa0, 0xc4200d5260, 0xc4200d5270, 0xc4203cfd80, 0xc4202ec0c0, 0x10f5e80, 0xc42000e448, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi7pdb8-nd5f9 Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420236380, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190acc0, 0xc42000fee8, 0xc420388d60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e660, 0xc42000e240, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc420348c80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e660, 0xc42000e240, 0x13, 0xc421a6cf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42198e940) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420348640, 0xc4200e6870, 0xc421994450, 0xc421994460, 0xc42190de00, 0xc4219160a0, 0x10f5e80, 0xc42000fee8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiq9w88-252rh Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202a20f0, 0xc420118000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420360440, 0xc4207c8740, 0xc4205f3f80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200f20c0, 0xc42000e090, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200fb8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200f20c0, 0xc42000e090, 0x13, 0xc420296f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420763d80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a300, 0xc4200a8370, 0xc4207c0ee0, 0xc4207c0ef0, 0xc4207d4300, 0xc4203111a0, 0x10f5e80, 0xc4207c8740, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmir6wg7-dj2qs Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203e8390, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202f1800, 0xc42000e450, 0xc4206ce280, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8de0, 0xc4200b8100, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008b400, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8de0, 0xc4200b8100, 0x13, 0xc42005bf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4216d4100) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202fe500, 0xc4200ba960, 0xc4200d49e0, 0xc4200d49f0, 0xc420483d80, 0xc42033e0e0, 0x10f5e80, 0xc42000e450, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.917 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the internet /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:07:28.600619Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi5jw9h-lzg6j" level=info timestamp=2018-07-26T12:07:46.724333Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi5jw9h-lzg6j" level=error timestamp=2018-07-26T12:07:46.941576Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:08:25 http: TLS handshake error from 10.244.1.1:51824: EOF 2018/07/26 12:08:35 http: TLS handshake error from 10.244.1.1:51830: EOF 2018/07/26 12:08:45 http: TLS handshake error from 10.244.1.1:51836: EOF 2018/07/26 12:08:55 http: TLS handshake error from 10.244.1.1:51842: EOF 2018/07/26 12:09:05 http: TLS handshake error from 10.244.1.1:51848: EOF 2018/07/26 12:09:15 http: TLS handshake error from 10.244.1.1:51854: EOF 2018/07/26 12:09:25 http: TLS handshake error from 10.244.1.1:51860: EOF 2018/07/26 12:09:35 http: TLS handshake error from 10.244.1.1:51866: EOF 2018/07/26 12:09:45 http: TLS handshake error from 10.244.1.1:51872: EOF 2018/07/26 12:09:55 http: TLS handshake error from 10.244.1.1:51878: EOF 2018/07/26 12:10:05 http: TLS handshake error from 10.244.1.1:51884: EOF 2018/07/26 12:10:15 http: TLS handshake error from 10.244.1.1:51890: EOF 2018/07/26 12:10:25 http: TLS handshake error from 10.244.1.1:51896: EOF 2018/07/26 12:10:35 http: TLS handshake error from 10.244.1.1:51902: EOF 2018/07/26 12:10:45 http: TLS handshake error from 10.244.1.1:51908: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T12:09:34.412682Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:09:35.151147Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:09:40 http: TLS handshake error from 10.244.0.1:35726: EOF level=info timestamp=2018-07-26T12:09:47.160381Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:09:50 http: TLS handshake error from 10.244.0.1:35750: EOF 2018/07/26 12:10:00 http: TLS handshake error from 10.244.0.1:35774: EOF level=info timestamp=2018-07-26T12:10:04.598802Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:10:05.236143Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:10:10 http: TLS handshake error from 10.244.0.1:35798: EOF level=info timestamp=2018-07-26T12:10:17.182292Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:10:20 http: TLS handshake error from 10.244.0.1:35822: EOF 2018/07/26 12:10:30 http: TLS handshake error from 10.244.0.1:35846: EOF level=info timestamp=2018-07-26T12:10:34.715917Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:10:35.355111Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:10:40 http: TLS handshake error from 10.244.0.1:35870: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:07:27.490808Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.843217Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:07:27.899927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir6wg7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir6wg7" level=info timestamp=2018-07-26T12:07:28.524305Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:10:28.915620Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi7pdb8\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi7pdb8, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 76cd1ccb-90cc-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:10:29.293700Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.294473Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.317084Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.317466Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.342401Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.342626Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.386669Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.387026Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.585430Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2vmft\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2vmft" level=info timestamp=2018-07-26T12:10:30.470809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilsnp6\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=error timestamp=2018-07-26T12:10:45.086214Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:45.086586Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:10:45.127703Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:45.128087Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:45.128380Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:10:45.209067Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:45.209530Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:45.209778Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:10:45.370291Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:45.370484Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:45.370622Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:10:45.691373Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:45.691857Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:45.692159Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:10:46.175945Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Processing vmi update" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:10:46.102255Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:46.103002Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:46.103283Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:46.112181Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:46.118614Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:46.118776Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:46.125709Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:46.126071Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:46.128794Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:46.173641Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:46.175820Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:46.176079Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:46.262424Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:46.262907Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:46.270684Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" Pod name: virt-launcher-testvmi2vmft-l754j Pod phase: Running level=info timestamp=2018-07-26T12:10:35.015368Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:10:35.015492Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:10:35.021053Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:10:45.221750Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:10:45.284794Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi2vmft" level=info timestamp=2018-07-26T12:10:45.286017Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:10:45.286321Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmi9h62g-rkf82 Pod phase: Running goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202680b0, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420230040, 0xc4207eac70, 0xc42081fb00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e060, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008a280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e060, 0xc42000e078, 0x13, 0xc42005e718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207ecbc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420334180, 0xc4200d6280, 0xc4207f4460, 0xc4207f4470, 0xc4208a4300, 0xc420190a40, 0x10f5e80, 0xc4207eac70, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigxvcn-w6psr Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027c3d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420564000, 0xc42000e2c8, 0xc42196cda0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8600, 0xc4200b8258, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bc480, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8600, 0xc4200b8258, 0x13, 0xc42005a718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207d5b40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c640, 0xc4200baa00, 0xc420308940, 0xc420308950, 0xc42196f600, 0xc4200c60e0, 0x10f5e80, 0xc42000e2c8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmilsnp6-qvkgv Pod phase: Running goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420550080, 0xc42000e4b0, 0xc420ba4b60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8120, 0xc4200b87a0, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034d8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8120, 0xc4200b87a0, 0x13, 0xc42005ff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42034d800) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac040, 0xc4200ba780, 0xc420b7e680, 0xc420b7e690, 0xc4216f4300, 0xc4200d9100, 0x10f5e80, 0xc42000e4b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:11:05 http: TLS handshake error from 10.244.1.1:51920: EOF 2018/07/26 12:11:15 http: TLS handshake error from 10.244.1.1:51926: EOF 2018/07/26 12:11:25 http: TLS handshake error from 10.244.1.1:51932: EOF 2018/07/26 12:11:35 http: TLS handshake error from 10.244.1.1:51938: EOF 2018/07/26 12:11:45 http: TLS handshake error from 10.244.1.1:51944: EOF 2018/07/26 12:11:55 http: TLS handshake error from 10.244.1.1:51950: EOF 2018/07/26 12:12:05 http: TLS handshake error from 10.244.1.1:51956: EOF 2018/07/26 12:12:15 http: TLS handshake error from 10.244.1.1:51962: EOF 2018/07/26 12:12:25 http: TLS handshake error from 10.244.1.1:51968: EOF 2018/07/26 12:12:35 http: TLS handshake error from 10.244.1.1:51974: EOF 2018/07/26 12:12:45 http: TLS handshake error from 10.244.1.1:51980: EOF 2018/07/26 12:12:55 http: TLS handshake error from 10.244.1.1:51986: EOF 2018/07/26 12:13:05 http: TLS handshake error from 10.244.1.1:51992: EOF 2018/07/26 12:13:15 http: TLS handshake error from 10.244.1.1:51998: EOF 2018/07/26 12:13:25 http: TLS handshake error from 10.244.1.1:52004: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:12:10 http: TLS handshake error from 10.244.0.1:36086: EOF level=info timestamp=2018-07-26T12:12:17.105825Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:12:20 http: TLS handshake error from 10.244.0.1:36110: EOF 2018/07/26 12:12:30 http: TLS handshake error from 10.244.0.1:36134: EOF level=info timestamp=2018-07-26T12:12:35.243618Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:12:35.757665Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:12:40 http: TLS handshake error from 10.244.0.1:36158: EOF level=info timestamp=2018-07-26T12:12:47.101980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:12:50 http: TLS handshake error from 10.244.0.1:36182: EOF 2018/07/26 12:13:00 http: TLS handshake error from 10.244.0.1:36206: EOF level=info timestamp=2018-07-26T12:13:05.382297Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:13:05.872981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:13:10 http: TLS handshake error from 10.244.0.1:36230: EOF level=info timestamp=2018-07-26T12:13:17.118892Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:13:20 http: TLS handshake error from 10.244.0.1:36254: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:07:27.490808Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir6wg7 kind= uid=76d32cbc-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:07:27.843217Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:07:27.899927Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir6wg7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir6wg7" level=info timestamp=2018-07-26T12:07:28.524305Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiq9w88\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiq9w88" level=info timestamp=2018-07-26T12:10:28.915620Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi7pdb8\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi7pdb8, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 76cd1ccb-90cc-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi7pdb8" level=info timestamp=2018-07-26T12:10:29.293700Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.294473Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.317084Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.317466Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.342401Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.342626Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.386669Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:10:29.387026Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.585430Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2vmft\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2vmft" level=info timestamp=2018-07-26T12:10:30.470809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilsnp6\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:10:57.425198Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:57.425695Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:57.425967Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi2vmft" level=info timestamp=2018-07-26T12:11:05.540993Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:05.584722Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:11:05.587166Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:05.587467Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:11:07.020940Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:07.041855Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:11:07.042190Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:07.042397Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigxvcn kind= uid=e3398377-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:11:07.666922Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:07.687553Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:11:07.687872Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:07.688027Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi2vmft kind= uid=e335ccc1-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:10:47.414209Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:47.418819Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:48.712535Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:48.713482Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:48.713845Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:51.274474Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:51.275163Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:51.275493Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:10:56.396437Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:10:56.396909Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:10:56.397226Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9h62g" level=info timestamp=2018-07-26T12:11:06.638273Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:06.661127Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:11:06.662278Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:11:06.663171Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9h62g kind= uid=e331304c-90cc-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi2vmft-l754j Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027c3e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b9e700, 0xc4200b8fa8, 0xc420395820, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b82c0, 0x13, 0x131ed4d, 0x4, 0xc420070f18, 0x3, 0x3, 0xc4200bd080, 0x2020202020202020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b82c0, 0x13, 0xc420c39f18, 0x3, 0x3, 0x5522203a22656d61, 0xa226e776f6e6b6e, 0x2020202020202020) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae480, 0xc4200ba870, 0xc42033d820, 0xc42033d830, 0xc420b9dd80, 0xc420ba80e0, 0x10f5e80, 0xc4200b8fa8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi9h62g-rkf82 Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202680b0, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420230040, 0xc4207eac70, 0xc42081fb00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e060, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc42008a280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e060, 0xc42000e078, 0x13, 0xc42005e718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207ecbc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420334180, 0xc4200d6280, 0xc4207f4460, 0xc4207f4470, 0xc4208a4300, 0xc420190a40, 0x10f5e80, 0xc4207eac70, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigxvcn-w6psr Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027c3d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420564000, 0xc42000e2c8, 0xc42196cda0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8600, 0xc4200b8258, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bc480, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8600, 0xc4200b8258, 0x13, 0xc42005a718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207d5b40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c640, 0xc4200baa00, 0xc420308940, 0xc420308950, 0xc42196f600, 0xc4200c60e0, 0x10f5e80, 0xc42000e2c8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmilsnp6-qvkgv Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420550080, 0xc42000e4b0, 0xc420ba4b60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8120, 0xc4200b87a0, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034d8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8120, 0xc4200b87a0, 0x13, 0xc42005ff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42034d800) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac040, 0xc4200ba780, 0xc420b7e680, 0xc420b7e690, 0xc4216f4300, 0xc4200d9100, 0x10f5e80, 0xc42000e4b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.832 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on the same node from Pod /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:10:30.861806Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi9h62g-rkf82" level=info timestamp=2018-07-26T12:10:46.579663Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi9h62g-rkf82" level=error timestamp=2018-07-26T12:10:46.873651Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:11:25 http: TLS handshake error from 10.244.1.1:51932: EOF 2018/07/26 12:11:35 http: TLS handshake error from 10.244.1.1:51938: EOF 2018/07/26 12:11:45 http: TLS handshake error from 10.244.1.1:51944: EOF 2018/07/26 12:11:55 http: TLS handshake error from 10.244.1.1:51950: EOF 2018/07/26 12:12:05 http: TLS handshake error from 10.244.1.1:51956: EOF 2018/07/26 12:12:15 http: TLS handshake error from 10.244.1.1:51962: EOF 2018/07/26 12:12:25 http: TLS handshake error from 10.244.1.1:51968: EOF 2018/07/26 12:12:35 http: TLS handshake error from 10.244.1.1:51974: EOF 2018/07/26 12:12:45 http: TLS handshake error from 10.244.1.1:51980: EOF 2018/07/26 12:12:55 http: TLS handshake error from 10.244.1.1:51986: EOF 2018/07/26 12:13:05 http: TLS handshake error from 10.244.1.1:51992: EOF 2018/07/26 12:13:15 http: TLS handshake error from 10.244.1.1:51998: EOF 2018/07/26 12:13:25 http: TLS handshake error from 10.244.1.1:52004: EOF 2018/07/26 12:13:35 http: TLS handshake error from 10.244.1.1:52010: EOF 2018/07/26 12:13:45 http: TLS handshake error from 10.244.1.1:52016: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T12:12:35.757665Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:12:40 http: TLS handshake error from 10.244.0.1:36158: EOF level=info timestamp=2018-07-26T12:12:47.101980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:12:50 http: TLS handshake error from 10.244.0.1:36182: EOF 2018/07/26 12:13:00 http: TLS handshake error from 10.244.0.1:36206: EOF level=info timestamp=2018-07-26T12:13:05.382297Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:13:05.872981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:13:10 http: TLS handshake error from 10.244.0.1:36230: EOF level=info timestamp=2018-07-26T12:13:17.118892Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:13:20 http: TLS handshake error from 10.244.0.1:36254: EOF 2018/07/26 12:13:30 http: TLS handshake error from 10.244.0.1:36278: EOF level=info timestamp=2018-07-26T12:13:35.540152Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:13:35.914746Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:13:40 http: TLS handshake error from 10.244.0.1:36302: EOF level=info timestamp=2018-07-26T12:13:47.256742Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:10:29.387026Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.585430Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2vmft\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2vmft" level=info timestamp=2018-07-26T12:10:30.470809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilsnp6\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:13:30.523099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilsnp6\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmilsnp6, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e33e8128-90cc-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:13:30.913899Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.914412Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:30.946779Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.947016Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:30.959868Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.960004Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:30.994926Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.995117Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:31.021877Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:31.069057Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi7cjg5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi7cjg5" level=info timestamp=2018-07-26T12:13:31.207980Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:13:47.643925Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:47.644499Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:47.644856Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:47.665525Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:47.665899Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:47.666154Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:47.706926Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:47.707441Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:47.707762Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:47.788392Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:47.788785Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:47.789010Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:47.950739Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:47.951089Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:47.951349Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:13:45.674468Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:45.674791Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:45.674950Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:45.755884Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:45.756218Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:45.757129Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:45.918026Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:45.919050Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:45.920721Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:46.253608Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:46.254259Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:46.254795Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:46.895699Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:46.896096Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:46.898653Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" Pod name: virt-launcher-testvmi7cjg5-tm2fw Pod phase: Running goroutine 24 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6380, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202464a0, 0xc4200b8b48, 0xc42066a6c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8cc0, 0xc42000e0d8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420408140, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8cc0, 0xc42000e0d8, 0x13, 0xc42005af18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420408080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c500, 0xc4201ae5f0, 0xc42062eda0, 0xc42062edb0, 0xc4203c3d80, 0xc42029e0e0, 0x10f5e80, 0xc4200b8b48, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmibnbk5-7psnx Pod phase: Running goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42026c0c0, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b62080, 0xc420b86108, 0xc42028c7c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b4060, 0xc42000e088, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420c4ec80, 0x2020202020202020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b4060, 0xc42000e088, 0x13, 0xc420b7ff18, 0x3, 0x3, 0x75706f705f736922, 0x6166203a2272616c, 0x20200a202c65736c) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ce2c0, 0xc4200cc370, 0xc420944520, 0xc420944530, 0xc420b7a300, 0xc420b58760, 0x10f5e80, 0xc420b86108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmisb2sg-2lqgf Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203200d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4216b60a0, 0xc42000f000, 0xc420d21ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e4e0, 0xc4200b8090, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc420408140, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e4e0, 0xc4200b8090, 0x13, 0xc420655f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420cf5c80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568140, 0xc4200ba5f0, 0xc4206d9ce0, 0xc4206d9cf0, 0xc420662300, 0xc4216a3c00, 0x10f5e80, 0xc42000f000, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmit4xvd-s8dp6 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202000c0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420676040, 0xc420c9a720, 0xc420ccd3a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e2a0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bc040, 0x14, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e2a0, 0xc42000e078, 0x13, 0xc4206a3718, 0x3, 0x3, 0x1, 0x4, 0xc42040ead8) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc4201ec5f0, 0xc420ae47f0, 0xc420ae4800, 0xc420698300, 0xc420663fc0, 0x10f5e80, 0xc420c9a720, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:14:05 http: TLS handshake error from 10.244.1.1:52028: EOF 2018/07/26 12:14:15 http: TLS handshake error from 10.244.1.1:52034: EOF 2018/07/26 12:14:25 http: TLS handshake error from 10.244.1.1:52040: EOF 2018/07/26 12:14:35 http: TLS handshake error from 10.244.1.1:52046: EOF 2018/07/26 12:14:45 http: TLS handshake error from 10.244.1.1:52052: EOF 2018/07/26 12:14:55 http: TLS handshake error from 10.244.1.1:52058: EOF 2018/07/26 12:15:05 http: TLS handshake error from 10.244.1.1:52064: EOF 2018/07/26 12:15:15 http: TLS handshake error from 10.244.1.1:52070: EOF 2018/07/26 12:15:25 http: TLS handshake error from 10.244.1.1:52076: EOF 2018/07/26 12:15:35 http: TLS handshake error from 10.244.1.1:52082: EOF 2018/07/26 12:15:45 http: TLS handshake error from 10.244.1.1:52088: EOF 2018/07/26 12:15:55 http: TLS handshake error from 10.244.1.1:52094: EOF 2018/07/26 12:16:05 http: TLS handshake error from 10.244.1.1:52100: EOF 2018/07/26 12:16:15 http: TLS handshake error from 10.244.1.1:52106: EOF 2018/07/26 12:16:25 http: TLS handshake error from 10.244.1.1:52112: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:15:30 http: TLS handshake error from 10.244.0.1:36566: EOF level=info timestamp=2018-07-26T12:15:36.135272Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:15:36.328601Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:15:40 http: TLS handshake error from 10.244.0.1:36590: EOF level=info timestamp=2018-07-26T12:15:47.325771Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:15:50 http: TLS handshake error from 10.244.0.1:36614: EOF level=info timestamp=2018-07-26T12:15:54.965762Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:15:54.970561Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:16:00 http: TLS handshake error from 10.244.0.1:36638: EOF level=info timestamp=2018-07-26T12:16:06.282466Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:16:06.420196Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:16:10 http: TLS handshake error from 10.244.0.1:36662: EOF level=info timestamp=2018-07-26T12:16:17.232002Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:16:20 http: TLS handshake error from 10.244.0.1:36686: EOF 2018/07/26 12:16:30 http: TLS handshake error from 10.244.0.1:36710: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:10:29.387026Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilsnp6 kind= uid=e33e8128-90cc-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:10:29.585430Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi2vmft\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi2vmft" level=info timestamp=2018-07-26T12:10:30.470809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilsnp6\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:13:30.523099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmilsnp6\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmilsnp6, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e33e8128-90cc-11e8-992e-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmilsnp6" level=info timestamp=2018-07-26T12:13:30.913899Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.914412Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:30.946779Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.947016Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:30.959868Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.960004Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:30.994926Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:13:30.995117Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:13:31.021877Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:31.069057Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi7cjg5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi7cjg5" level=info timestamp=2018-07-26T12:13:31.207980Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:13:58.620139Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:58.620717Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:58.621072Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4xvd" level=info timestamp=2018-07-26T12:14:06.604691Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:06.639701Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:14:06.640359Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:06.640551Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi7cjg5 kind= uid=4f775f5d-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:14:08.121091Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:08.138480Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:14:08.138840Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:08.138961Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisb2sg kind= uid=4f72de0f-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:14:08.864989Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:08.884174Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:14:08.884602Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:08.884731Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmit4xvd kind= uid=4f7d72f0-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:13:46.896096Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:46.898653Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:48.182784Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:48.183211Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:48.183972Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:50.745221Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:50.745959Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:50.746171Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:13:55.868006Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:13:55.869622Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:13:55.870125Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibnbk5" level=info timestamp=2018-07-26T12:14:06.111439Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:06.168972Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:14:06.179117Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:14:06.181054Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibnbk5 kind= uid=4f79fc4b-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi7cjg5-tm2fw Pod phase: Failed goroutine 24 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6380, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202464a0, 0xc4200b8b48, 0xc42066a6c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8cc0, 0xc42000e0d8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420408140, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8cc0, 0xc42000e0d8, 0x13, 0xc42005af18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420408080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c500, 0xc4201ae5f0, 0xc42062eda0, 0xc42062edb0, 0xc4203c3d80, 0xc42029e0e0, 0x10f5e80, 0xc4200b8b48, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmibnbk5-7psnx Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42026c0c0, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b62080, 0xc420b86108, 0xc42028c7c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b4060, 0xc42000e088, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420c4ec80, 0x2020202020202020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b4060, 0xc42000e088, 0x13, 0xc420b7ff18, 0x3, 0x3, 0x75706f705f736922, 0x6166203a2272616c, 0x20200a202c65736c) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ce2c0, 0xc4200cc370, 0xc420944520, 0xc420944530, 0xc420b7a300, 0xc420b58760, 0x10f5e80, 0xc420b86108, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmisb2sg-2lqgf Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203200d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4216b60a0, 0xc42000f000, 0xc420d21ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e4e0, 0xc4200b8090, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc420408140, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e4e0, 0xc4200b8090, 0x13, 0xc420655f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420cf5c80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568140, 0xc4200ba5f0, 0xc4206d9ce0, 0xc4206d9cf0, 0xc420662300, 0xc4216a3c00, 0x10f5e80, 0xc42000f000, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmit4xvd-s8dp6 Pod phase: Failed goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202000c0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420676040, 0xc420c9a720, 0xc420ccd3a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e2a0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bc040, 0x14, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e2a0, 0xc42000e078, 0x13, 0xc4206a3718, 0x3, 0x3, 0x1, 0x4, 0xc42040ead8) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc4201ec5f0, 0xc420ae47f0, 0xc420ae4800, 0xc420698300, 0xc420663fc0, 0x10f5e80, 0xc420c9a720, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.614 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on a different node from Pod /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:13:32.019045Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmisb2sg-2lqgf" level=info timestamp=2018-07-26T12:13:48.227247Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmisb2sg-2lqgf" level=error timestamp=2018-07-26T12:13:48.544338Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:14:25 http: TLS handshake error from 10.244.1.1:52040: EOF 2018/07/26 12:14:35 http: TLS handshake error from 10.244.1.1:52046: EOF 2018/07/26 12:14:45 http: TLS handshake error from 10.244.1.1:52052: EOF 2018/07/26 12:14:55 http: TLS handshake error from 10.244.1.1:52058: EOF 2018/07/26 12:15:05 http: TLS handshake error from 10.244.1.1:52064: EOF 2018/07/26 12:15:15 http: TLS handshake error from 10.244.1.1:52070: EOF 2018/07/26 12:15:25 http: TLS handshake error from 10.244.1.1:52076: EOF 2018/07/26 12:15:35 http: TLS handshake error from 10.244.1.1:52082: EOF 2018/07/26 12:15:45 http: TLS handshake error from 10.244.1.1:52088: EOF 2018/07/26 12:15:55 http: TLS handshake error from 10.244.1.1:52094: EOF 2018/07/26 12:16:05 http: TLS handshake error from 10.244.1.1:52100: EOF 2018/07/26 12:16:15 http: TLS handshake error from 10.244.1.1:52106: EOF 2018/07/26 12:16:25 http: TLS handshake error from 10.244.1.1:52112: EOF 2018/07/26 12:16:35 http: TLS handshake error from 10.244.1.1:52118: EOF 2018/07/26 12:16:45 http: TLS handshake error from 10.244.1.1:52124: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:15:50 http: TLS handshake error from 10.244.0.1:36614: EOF level=info timestamp=2018-07-26T12:15:54.965762Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:15:54.970561Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:16:00 http: TLS handshake error from 10.244.0.1:36638: EOF level=info timestamp=2018-07-26T12:16:06.282466Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:16:06.420196Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:16:10 http: TLS handshake error from 10.244.0.1:36662: EOF level=info timestamp=2018-07-26T12:16:17.232002Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:16:20 http: TLS handshake error from 10.244.0.1:36686: EOF 2018/07/26 12:16:30 http: TLS handshake error from 10.244.0.1:36710: EOF level=info timestamp=2018-07-26T12:16:36.462090Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:16:36.709220Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:16:40 http: TLS handshake error from 10.244.0.1:36734: EOF level=info timestamp=2018-07-26T12:16:47.091751Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:16:50 http: TLS handshake error from 10.244.0.1:36758: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:13:31.021877Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:31.069057Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi7cjg5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi7cjg5" level=info timestamp=2018-07-26T12:13:31.207980Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:16:32.748190Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.753438Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:32.772620Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.772940Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:32.811735Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.811957Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:32.858013Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.858430Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:33.135176Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibcc8z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:33.568079Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" level=info timestamp=2018-07-26T12:16:33.732004Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibcc8z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:34.137105Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:16:50.447384Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:50.447519Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:50.447603Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:50.468126Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:50.468455Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:50.468597Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:50.508921Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:50.509070Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:50.509200Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:50.590687Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:50.591095Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:50.591383Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:50.612001Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:50.612483Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:50.612783Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:16:48.715330Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:48.715523Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:48.716447Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:48.800643Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:48.800855Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:48.800981Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:48.961381Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:48.961771Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:48.961917Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:49.282720Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:49.283075Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:49.283279Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:49.924413Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:49.924819Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:49.925117Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" Pod name: virt-launcher-testvmib8n9r-b4cmv Pod phase: Running goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201d20b0, 0xc420190c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191e040, 0xc42000e498, 0xc42021a8e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200961e0, 0xc42000e090, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4204745c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200961e0, 0xc42000e090, 0x13, 0xc421954718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420474380) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42031e0c0, 0xc4200ae370, 0xc4204a40e0, 0xc4204a40f0, 0xc42194a300, 0xc421912d00, 0x10f5e80, 0xc42000e498, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmibcc8z-kd688 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d64b0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42064bbe0, 0xc420c7ce88, 0xc420cacba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420408180, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8100, 0x13, 0xc420afb718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c43c80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac3c0, 0xc4200ba960, 0xc420c8a4a0, 0xc420c8a4b0, 0xc420653d80, 0xc42065a0e0, 0x10f5e80, 0xc420c7ce88, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvminbsfk-fhnjb Pod phase: Running goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42023e0b0, 0xc420120000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205b6080, 0xc42000e620, 0xc420c40f20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8180, 0xc4200b8118, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc42008a9c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8180, 0xc4200b8118, 0x13, 0xc421655f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420410040, 0xc4200ba5f0, 0xc42065d0c0, 0xc42065d0d0, 0xc42164a300, 0xc4205ad100, 0x10f5e80, 0xc42000e620, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivnwdf-swwnp Pod phase: Running goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0d0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42166c040, 0xc420bf9138, 0xc420c254a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc4200b8090, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc42034a3c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc4200b8090, 0x13, 0xc42169ef18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c05ac0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc4200ba550, 0xc420bfb830, 0xc420bfb840, 0xc421692300, 0xc42165d940, 0x10f5e80, 0xc420bf9138, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:17:05 http: TLS handshake error from 10.244.1.1:52136: EOF 2018/07/26 12:17:15 http: TLS handshake error from 10.244.1.1:52142: EOF 2018/07/26 12:17:25 http: TLS handshake error from 10.244.1.1:52148: EOF 2018/07/26 12:17:35 http: TLS handshake error from 10.244.1.1:52154: EOF 2018/07/26 12:17:45 http: TLS handshake error from 10.244.1.1:52160: EOF 2018/07/26 12:17:55 http: TLS handshake error from 10.244.1.1:52166: EOF 2018/07/26 12:18:05 http: TLS handshake error from 10.244.1.1:52172: EOF 2018/07/26 12:18:15 http: TLS handshake error from 10.244.1.1:52178: EOF 2018/07/26 12:18:25 http: TLS handshake error from 10.244.1.1:52184: EOF 2018/07/26 12:18:35 http: TLS handshake error from 10.244.1.1:52190: EOF 2018/07/26 12:18:45 http: TLS handshake error from 10.244.1.1:52196: EOF 2018/07/26 12:18:55 http: TLS handshake error from 10.244.1.1:52202: EOF 2018/07/26 12:19:05 http: TLS handshake error from 10.244.1.1:52208: EOF 2018/07/26 12:19:15 http: TLS handshake error from 10.244.1.1:52214: EOF 2018/07/26 12:19:25 http: TLS handshake error from 10.244.1.1:52220: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:18:30 http: TLS handshake error from 10.244.0.1:36998: EOF level=info timestamp=2018-07-26T12:18:37.028458Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:18:37.163996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:18:40 http: TLS handshake error from 10.244.0.1:37022: EOF level=info timestamp=2018-07-26T12:18:47.096027Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:18:50 http: TLS handshake error from 10.244.0.1:37046: EOF level=info timestamp=2018-07-26T12:18:54.995488Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:18:54.999822Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:19:00 http: TLS handshake error from 10.244.0.1:37070: EOF level=info timestamp=2018-07-26T12:19:07.166645Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:19:07.278582Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:19:10 http: TLS handshake error from 10.244.0.1:37094: EOF level=info timestamp=2018-07-26T12:19:17.244083Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:19:20 http: TLS handshake error from 10.244.0.1:37118: EOF 2018/07/26 12:19:30 http: TLS handshake error from 10.244.0.1:37142: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:13:31.021877Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:13:31.069057Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi7cjg5\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi7cjg5" level=info timestamp=2018-07-26T12:13:31.207980Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmisb2sg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmisb2sg" level=info timestamp=2018-07-26T12:16:32.748190Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.753438Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:32.772620Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.772940Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:32.811735Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.811957Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:32.858013Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:16:32.858430Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:16:33.135176Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibcc8z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:33.568079Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" level=info timestamp=2018-07-26T12:16:33.732004Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibcc8z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:34.137105Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:17:00.681390Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:17:00.681885Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:17:00.682166Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:17:07.536585Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:07.578989Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:17:07.579374Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:07.579509Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbsfk kind= uid=bbd84cb9-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:17:09.817671Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:09.840930Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:17:09.844672Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:09.844928Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivnwdf kind= uid=bbe1fe6b-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:17:10.922930Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:10.946959Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:17:10.949920Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:10.950094Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibcc8z kind= uid=bbd33e09-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:16:49.924819Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:49.925117Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:51.205574Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:51.205816Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:51.205960Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:53.767075Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:53.769128Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:53.769961Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:16:58.891251Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:16:58.892759Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:16:58.893046Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmib8n9r" level=info timestamp=2018-07-26T12:17:09.133974Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:09.161978Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:17:09.165552Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:17:09.165743Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmib8n9r kind= uid=bbdd0c30-90cd-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmib8n9r-b4cmv Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201d20b0, 0xc420190c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191e040, 0xc42000e498, 0xc42021a8e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200961e0, 0xc42000e090, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4204745c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200961e0, 0xc42000e090, 0x13, 0xc421954718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420474380) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42031e0c0, 0xc4200ae370, 0xc4204a40e0, 0xc4204a40f0, 0xc42194a300, 0xc421912d00, 0x10f5e80, 0xc42000e498, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmibcc8z-kd688 Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d64b0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42064bbe0, 0xc420c7ce88, 0xc420cacba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420408180, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8100, 0x13, 0xc420afb718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c43c80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac3c0, 0xc4200ba960, 0xc420c8a4a0, 0xc420c8a4b0, 0xc420653d80, 0xc42065a0e0, 0x10f5e80, 0xc420c7ce88, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminbsfk-fhnjb Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42023e0b0, 0xc420120000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205b6080, 0xc42000e620, 0xc420c40f20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8180, 0xc4200b8118, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc42008a9c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8180, 0xc4200b8118, 0x13, 0xc421655f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420410040, 0xc4200ba5f0, 0xc42065d0c0, 0xc42065d0d0, 0xc42164a300, 0xc4205ad100, 0x10f5e80, 0xc42000e620, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivnwdf-swwnp Pod phase: Failed goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0d0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42166c040, 0xc420bf9138, 0xc420c254a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e1e0, 0xc4200b8090, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc42034a3c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e1e0, 0xc4200b8090, 0x13, 0xc42169ef18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c05ac0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc4200ba550, 0xc420bfb830, 0xc420bfb840, 0xc421692300, 0xc42165d940, 0x10f5e80, 0xc420bf9138, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.894 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on the same node from Node /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:16:34.143237Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmibcc8z-kd688" level=info timestamp=2018-07-26T12:16:51.066188Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmibcc8z-kd688" level=error timestamp=2018-07-26T12:16:51.359300Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:17:25 http: TLS handshake error from 10.244.1.1:52148: EOF 2018/07/26 12:17:35 http: TLS handshake error from 10.244.1.1:52154: EOF 2018/07/26 12:17:45 http: TLS handshake error from 10.244.1.1:52160: EOF 2018/07/26 12:17:55 http: TLS handshake error from 10.244.1.1:52166: EOF 2018/07/26 12:18:05 http: TLS handshake error from 10.244.1.1:52172: EOF 2018/07/26 12:18:15 http: TLS handshake error from 10.244.1.1:52178: EOF 2018/07/26 12:18:25 http: TLS handshake error from 10.244.1.1:52184: EOF 2018/07/26 12:18:35 http: TLS handshake error from 10.244.1.1:52190: EOF 2018/07/26 12:18:45 http: TLS handshake error from 10.244.1.1:52196: EOF 2018/07/26 12:18:55 http: TLS handshake error from 10.244.1.1:52202: EOF 2018/07/26 12:19:05 http: TLS handshake error from 10.244.1.1:52208: EOF 2018/07/26 12:19:15 http: TLS handshake error from 10.244.1.1:52214: EOF 2018/07/26 12:19:25 http: TLS handshake error from 10.244.1.1:52220: EOF 2018/07/26 12:19:35 http: TLS handshake error from 10.244.1.1:52226: EOF 2018/07/26 12:19:45 http: TLS handshake error from 10.244.1.1:52232: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T12:18:47.096027Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:18:50 http: TLS handshake error from 10.244.0.1:37046: EOF level=info timestamp=2018-07-26T12:18:54.995488Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:18:54.999822Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:19:00 http: TLS handshake error from 10.244.0.1:37070: EOF level=info timestamp=2018-07-26T12:19:07.166645Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:19:07.278582Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:19:10 http: TLS handshake error from 10.244.0.1:37094: EOF level=info timestamp=2018-07-26T12:19:17.244083Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:19:20 http: TLS handshake error from 10.244.0.1:37118: EOF 2018/07/26 12:19:30 http: TLS handshake error from 10.244.0.1:37142: EOF level=info timestamp=2018-07-26T12:19:37.316832Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:19:37.418846Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:19:40 http: TLS handshake error from 10.244.0.1:37166: EOF level=info timestamp=2018-07-26T12:19:47.125676Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:16:33.568079Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" level=info timestamp=2018-07-26T12:16:33.732004Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibcc8z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:34.137105Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" level=info timestamp=2018-07-26T12:19:34.458473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.462017Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.499682Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.499990Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.578091Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.578385Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.643934Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.644143Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.657947Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:34.911241Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:35.094049Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiw7445\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:35.896741Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmipl5jz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmipl5jz" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:19:49.561456Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.561649Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.561754Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:49.582129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.582353Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.582451Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:49.623058Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.623559Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.623869Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:49.704622Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.705013Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.705244Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:49.868013Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.868504Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.868785Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:19:49.496916Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.497143Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.497392Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:49.507426Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.507871Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.508126Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:49.625863Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.626264Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.642851Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:49.731131Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.737279Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.749116Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:49.910243Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:49.912531Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:49.912675Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" Pod name: virt-launcher-testvmi2c9rp-f4wvv Pod phase: Running level=info timestamp=2018-07-26T12:19:40.435828Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:19:40.436135Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:19:40.441154Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" Pod name: virt-launcher-testvmipl5jz-q4vkl Pod phase: Running level=info timestamp=2018-07-26T12:19:38.797899Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:19:38.798045Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:19:38.800833Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:19:49.094591Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:19:49.121713Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmipl5jz" level=info timestamp=2018-07-26T12:19:49.122419Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:19:49.122623Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmir9jbb-wmrtt Pod phase: Running goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0f0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420c5e040, 0xc42000f738, 0xc420189e00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c85a0, 0xc4200b80a0, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bc040, 0x6369766564222020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c85a0, 0xc4200b80a0, 0x13, 0xc420c87718, 0x3, 0x3, 0x5cc549, 0xc4203db080, 0x6e5522203a22656d) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c040, 0xc4200ba690, 0xc420230010, 0xc420230020, 0xc420c7c300, 0xc420c527a0, 0x10f5e80, 0xc42000f738, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiw7445-55cbm Pod phase: Running goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6380, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4215b98e0, 0xc420b52630, 0xc420b72860, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e5a0, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc420071f18, 0x3, 0x3, 0xc42008a040, 0x1d, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e5a0, 0xc42000e100, 0x13, 0xc4206df718, 0x3, 0x3, 0x1, 0x4, 0xc4203dbed0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a400, 0xc4200e6780, 0xc4215c7260, 0xc4215c7270, 0xc4215bdd80, 0xc4215c8080, 0x10f5e80, 0xc420b52630, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:20:05 http: TLS handshake error from 10.244.1.1:52244: EOF 2018/07/26 12:20:15 http: TLS handshake error from 10.244.1.1:52250: EOF 2018/07/26 12:20:25 http: TLS handshake error from 10.244.1.1:52256: EOF 2018/07/26 12:20:35 http: TLS handshake error from 10.244.1.1:52262: EOF 2018/07/26 12:20:45 http: TLS handshake error from 10.244.1.1:52268: EOF 2018/07/26 12:20:55 http: TLS handshake error from 10.244.1.1:52274: EOF 2018/07/26 12:21:05 http: TLS handshake error from 10.244.1.1:52280: EOF 2018/07/26 12:21:15 http: TLS handshake error from 10.244.1.1:52286: EOF 2018/07/26 12:21:25 http: TLS handshake error from 10.244.1.1:52292: EOF 2018/07/26 12:21:35 http: TLS handshake error from 10.244.1.1:52298: EOF 2018/07/26 12:21:45 http: TLS handshake error from 10.244.1.1:52304: EOF 2018/07/26 12:21:55 http: TLS handshake error from 10.244.1.1:52310: EOF 2018/07/26 12:22:05 http: TLS handshake error from 10.244.1.1:52316: EOF 2018/07/26 12:22:15 http: TLS handshake error from 10.244.1.1:52322: EOF 2018/07/26 12:22:25 http: TLS handshake error from 10.244.1.1:52328: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:21:30 http: TLS handshake error from 10.244.0.1:37430: EOF level=info timestamp=2018-07-26T12:21:37.867277Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:21:37.946957Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:21:40 http: TLS handshake error from 10.244.0.1:37454: EOF level=info timestamp=2018-07-26T12:21:47.127251Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:21:50 http: TLS handshake error from 10.244.0.1:37478: EOF level=info timestamp=2018-07-26T12:21:54.264164Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:21:54.270158Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:22:00 http: TLS handshake error from 10.244.0.1:37502: EOF level=info timestamp=2018-07-26T12:22:08.090729Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:22:08.093123Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:22:10 http: TLS handshake error from 10.244.0.1:37526: EOF level=info timestamp=2018-07-26T12:22:17.128050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:22:20 http: TLS handshake error from 10.244.0.1:37550: EOF 2018/07/26 12:22:30 http: TLS handshake error from 10.244.0.1:37574: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:16:33.568079Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" level=info timestamp=2018-07-26T12:16:33.732004Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibcc8z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibcc8z" level=info timestamp=2018-07-26T12:16:34.137105Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnwdf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnwdf" level=info timestamp=2018-07-26T12:19:34.458473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.462017Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.499682Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.499990Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.578091Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.578385Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.643934Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:19:34.644143Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.657947Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:34.911241Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:35.094049Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiw7445\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:35.896741Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmipl5jz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmipl5jz" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:20:01.566528Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:20:01.566979Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:20:01.567418Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmipl5jz" level=info timestamp=2018-07-26T12:20:10.039377Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:10.070469Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:20:10.070814Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:10.070972Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmir9jbb kind= uid=2820eefc-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:20:11.348622Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:11.379042Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:20:11.382754Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:11.382994Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi2c9rp kind= uid=2829c11d-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:20:11.808466Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:11.835877Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:20:11.836173Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:11.836407Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:19:50.876268Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:50.877116Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:52.164027Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:52.175708Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:52.176093Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:54.736659Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:54.737359Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:54.737496Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:59.858875Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:19:59.859931Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:19:59.860936Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:20:10.102456Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:10.155293Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:20:10.155685Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:20:10.155858Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiw7445 kind= uid=282f35ed-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi2c9rp-f4wvv Pod phase: Failed goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027c100, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420566040, 0xc4200b94e8, 0xc42193a200, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8480, 0xc4200b8098, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420406180, 0xc4204cc020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8480, 0xc4200b8098, 0x13, 0xc421953f18, 0x3, 0x3, 0xc420506c60, 0xc4203c4240, 0xc420314240) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac000, 0xc4200ba4b0, 0xc4208e5850, 0xc4208e5860, 0xc421931c00, 0xc42000c0a0, 0x10f5e80, 0xc4200b94e8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmipl5jz-q4vkl Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027c400, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205620a0, 0xc42000fda0, 0xc42173af80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e4e0, 0xc4200b8258, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc42008a040, 0x1e, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e4e0, 0xc4200b8258, 0x13, 0xc420191f18, 0x3, 0x3, 0x1, 0x4, 0xc420439258) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc840, 0xc4200baaa0, 0xc420025e30, 0xc420025e40, 0xc421967d80, 0xc42000c140, 0x10f5e80, 0xc42000fda0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmir9jbb-wmrtt Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a0f0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420c5e040, 0xc42000f738, 0xc420189e00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c85a0, 0xc4200b80a0, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc4200bc040, 0x6369766564222020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c85a0, 0xc4200b80a0, 0x13, 0xc420c87718, 0x3, 0x3, 0x5cc549, 0xc4203db080, 0x6e5522203a22656d) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c040, 0xc4200ba690, 0xc420230010, 0xc420230020, 0xc420c7c300, 0xc420c527a0, 0x10f5e80, 0xc42000f738, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiw7445-55cbm Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d6380, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4215b98e0, 0xc420b52630, 0xc420b72860, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e5a0, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc420071f18, 0x3, 0x3, 0xc42008a040, 0x1d, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e5a0, 0xc42000e100, 0x13, 0xc4206df718, 0x3, 0x3, 0x1, 0x4, 0xc4203dbed0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a400, 0xc4200e6780, 0xc4215c7260, 0xc4215c7270, 0xc4215bdd80, 0xc4215c8080, 0x10f5e80, 0xc420b52630, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.778 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on a different node from Node /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:19:35.816865Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmir9jbb-wmrtt" level=info timestamp=2018-07-26T12:19:50.144582Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmir9jbb-wmrtt" level=error timestamp=2018-07-26T12:19:50.527134Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:20:25 http: TLS handshake error from 10.244.1.1:52256: EOF 2018/07/26 12:20:35 http: TLS handshake error from 10.244.1.1:52262: EOF 2018/07/26 12:20:45 http: TLS handshake error from 10.244.1.1:52268: EOF 2018/07/26 12:20:55 http: TLS handshake error from 10.244.1.1:52274: EOF 2018/07/26 12:21:05 http: TLS handshake error from 10.244.1.1:52280: EOF 2018/07/26 12:21:15 http: TLS handshake error from 10.244.1.1:52286: EOF 2018/07/26 12:21:25 http: TLS handshake error from 10.244.1.1:52292: EOF 2018/07/26 12:21:35 http: TLS handshake error from 10.244.1.1:52298: EOF 2018/07/26 12:21:45 http: TLS handshake error from 10.244.1.1:52304: EOF 2018/07/26 12:21:55 http: TLS handshake error from 10.244.1.1:52310: EOF 2018/07/26 12:22:05 http: TLS handshake error from 10.244.1.1:52316: EOF 2018/07/26 12:22:15 http: TLS handshake error from 10.244.1.1:52322: EOF 2018/07/26 12:22:25 http: TLS handshake error from 10.244.1.1:52328: EOF 2018/07/26 12:22:35 http: TLS handshake error from 10.244.1.1:52334: EOF 2018/07/26 12:22:45 http: TLS handshake error from 10.244.1.1:52340: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:21:50 http: TLS handshake error from 10.244.0.1:37478: EOF level=info timestamp=2018-07-26T12:21:54.264164Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:21:54.270158Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:22:00 http: TLS handshake error from 10.244.0.1:37502: EOF level=info timestamp=2018-07-26T12:22:08.090729Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:22:08.093123Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:22:10 http: TLS handshake error from 10.244.0.1:37526: EOF level=info timestamp=2018-07-26T12:22:17.128050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:22:20 http: TLS handshake error from 10.244.0.1:37550: EOF 2018/07/26 12:22:30 http: TLS handshake error from 10.244.0.1:37574: EOF level=info timestamp=2018-07-26T12:22:38.255807Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:22:38.260862Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:22:40 http: TLS handshake error from 10.244.0.1:37598: EOF level=info timestamp=2018-07-26T12:22:47.125078Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:22:50 http: TLS handshake error from 10.244.0.1:37622: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:19:34.644143Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.657947Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:34.911241Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:35.094049Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiw7445\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:35.896741Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmipl5jz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmipl5jz" level=info timestamp=2018-07-26T12:22:36.442631Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.448410Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.483664Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.483992Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.525605Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.525909Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.634217Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.634519Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.669543Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikg55k\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikg55k" level=info timestamp=2018-07-26T12:22:37.278165Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicq6hd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:22:53.788221Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:53.788411Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.788537Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bsbv" level=info timestamp=2018-07-26T12:22:53.808897Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:53.809073Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.809206Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bsbv" level=info timestamp=2018-07-26T12:22:53.850533Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:53.850688Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.850771Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bsbv" level=info timestamp=2018-07-26T12:22:53.932595Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:53.932848Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.933009Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi9bsbv" level=info timestamp=2018-07-26T12:22:53.979926Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:53.980120Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.980248Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmil9r84" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:22:52.158714Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:52.159143Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:52.159460Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:22:52.240107Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:52.241057Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:52.247842Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:22:52.408737Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:52.409180Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:52.409542Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:22:52.730410Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:52.730881Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:52.731154Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:22:53.397186Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:53.397626Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.397910Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" Pod name: virt-launcher-testvmi9bsbv-89fvw Pod phase: Running goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d63e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420188360, 0xc420673018, 0xc4206a0ce0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8250, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bdb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8250, 0x13, 0xc421947718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac380, 0xc4200baa50, 0xc42042e9f0, 0xc42042ea00, 0xc42033bd80, 0xc4200c6140, 0x10f5e80, 0xc420673018, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmicq6hd-f2m4w Pod phase: Running goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200482d0, 0xc4201b0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192e040, 0xc4200b8528, 0xc4202ae400, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200ec1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bc080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200ec1e0, 0xc42000e078, 0x13, 0xc42195ff18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202c4140, 0xc4200d4370, 0xc4219e6d40, 0xc4219e6d50, 0xc421954300, 0xc421922d20, 0x10f5e80, 0xc4200b8528, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmikg55k-k7wvq Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 36 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203e43d0, 0xc420108280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420653c20, 0xc420c787d0, 0xc420ca8bc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8600, 0xc4200b80e8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc42008a740, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8600, 0xc4200b80e8, 0x13, 0xc420c8ef18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c25d40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a580, 0xc4200ba820, 0xc420c7a4b0, 0xc420c7a4c0, 0xc42065bd80, 0xc4206620e0, 0x10f5e80, 0xc420c787d0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmil9r84-m67w8 Pod phase: Running goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a3f0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42029c800, 0xc420684e60, 0xc4206b6040, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8308, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034c340, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8308, 0x13, 0xc421969718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae3c0, 0xc4200ba870, 0xc4204bf270, 0xc4204bf280, 0xc42023dd80, 0xc4202ee120, 0x10f5e80, 0xc420684e60, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:23:15 http: TLS handshake error from 10.244.1.1:52358: EOF 2018/07/26 12:23:25 http: TLS handshake error from 10.244.1.1:52364: EOF 2018/07/26 12:23:35 http: TLS handshake error from 10.244.1.1:52370: EOF 2018/07/26 12:23:45 http: TLS handshake error from 10.244.1.1:52376: EOF 2018/07/26 12:23:55 http: TLS handshake error from 10.244.1.1:52382: EOF 2018/07/26 12:24:05 http: TLS handshake error from 10.244.1.1:52388: EOF 2018/07/26 12:24:15 http: TLS handshake error from 10.244.1.1:52394: EOF 2018/07/26 12:24:25 http: TLS handshake error from 10.244.1.1:52400: EOF 2018/07/26 12:24:35 http: TLS handshake error from 10.244.1.1:52406: EOF 2018/07/26 12:24:45 http: TLS handshake error from 10.244.1.1:52412: EOF 2018/07/26 12:24:55 http: TLS handshake error from 10.244.1.1:52418: EOF 2018/07/26 12:25:05 http: TLS handshake error from 10.244.1.1:52424: EOF 2018/07/26 12:25:15 http: TLS handshake error from 10.244.1.1:52430: EOF 2018/07/26 12:25:25 http: TLS handshake error from 10.244.1.1:52436: EOF 2018/07/26 12:25:35 http: TLS handshake error from 10.244.1.1:52442: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T12:24:17.129733Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:24:20 http: TLS handshake error from 10.244.0.1:37838: EOF 2018/07/26 12:24:30 http: TLS handshake error from 10.244.0.1:37862: EOF level=info timestamp=2018-07-26T12:24:38.977079Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:24:38.978551Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:24:40 http: TLS handshake error from 10.244.0.1:37886: EOF level=info timestamp=2018-07-26T12:24:47.134951Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:24:50 http: TLS handshake error from 10.244.0.1:37910: EOF 2018/07/26 12:25:00 http: TLS handshake error from 10.244.0.1:37934: EOF level=info timestamp=2018-07-26T12:25:09.145836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:25:09.147131Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:25:10 http: TLS handshake error from 10.244.0.1:37958: EOF level=info timestamp=2018-07-26T12:25:17.192093Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:25:20 http: TLS handshake error from 10.244.0.1:37982: EOF 2018/07/26 12:25:30 http: TLS handshake error from 10.244.0.1:38006: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:19:34.644143Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmipl5jz kind= uid=283c2d55-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:19:34.657947Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:34.911241Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmir9jbb\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmir9jbb" level=info timestamp=2018-07-26T12:19:35.094049Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiw7445\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiw7445" level=info timestamp=2018-07-26T12:19:35.896741Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmipl5jz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmipl5jz" level=info timestamp=2018-07-26T12:22:36.442631Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.448410Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.483664Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.483992Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.525605Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.525909Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.634217Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.634519Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.669543Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikg55k\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikg55k" level=info timestamp=2018-07-26T12:22:37.278165Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicq6hd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:23:04.547980Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:23:04.548449Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:23:04.548754Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmikg55k" level=info timestamp=2018-07-26T12:23:13.828890Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:13.858459Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:23:13.858886Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:13.859022Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:23:14.262125Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:14.289371Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:23:14.289674Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:14.289812Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi9bsbv kind= uid=9499e196-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:23:14.789637Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:14.820605Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:23:14.820926Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:14.821056Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmikg55k kind= uid=94a0843b-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:22:53.397626Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:53.397910Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:22:54.678600Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:54.679206Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:54.680669Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:22:57.242197Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:22:57.243829Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:22:57.244141Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:23:02.365106Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:23:02.366795Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:23:02.367250Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:23:12.609910Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:12.665096Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:23:12.672636Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:23:12.672825Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi9bsbv-89fvw Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d63e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420188360, 0xc420673018, 0xc4206a0ce0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8250, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bdb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8250, 0x13, 0xc421947718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac380, 0xc4200baa50, 0xc42042e9f0, 0xc42042ea00, 0xc42033bd80, 0xc4200c6140, 0x10f5e80, 0xc420673018, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmicq6hd-f2m4w Pod phase: Failed goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200482d0, 0xc4201b0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192e040, 0xc4200b8528, 0xc4202ae400, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200ec1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006cf18, 0x3, 0x3, 0xc4200bc080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200ec1e0, 0xc42000e078, 0x13, 0xc42195ff18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202c4140, 0xc4200d4370, 0xc4219e6d40, 0xc4219e6d50, 0xc421954300, 0xc421922d20, 0x10f5e80, 0xc4200b8528, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmikg55k-k7wvq Pod phase: Failed goroutine 36 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203e43d0, 0xc420108280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420653c20, 0xc420c787d0, 0xc420ca8bc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8600, 0xc4200b80e8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc42008a740, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8600, 0xc4200b80e8, 0x13, 0xc420c8ef18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c25d40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42008a580, 0xc4200ba820, 0xc420c7a4b0, 0xc420c7a4c0, 0xc42065bd80, 0xc4206620e0, 0x10f5e80, 0xc420c787d0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmil9r84-m67w8 Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a3f0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42029c800, 0xc420684e60, 0xc4206b6040, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8d80, 0xc4200b8308, 0x13, 0x131ed4d, 0x4, 0xc42006bf18, 0x3, 0x3, 0xc42034c340, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8d80, 0xc4200b8308, 0x13, 0xc421969718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae3c0, 0xc4200ba870, 0xc4204bf270, 0xc4204bf280, 0xc42023dd80, 0xc4202ee120, 0x10f5e80, 0xc420684e60, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [182.072 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 with a service matching the vmi exposed [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:283 should be able to reach the vmi based on labels specified on the vmi /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:303 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:22:37.809465Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi9bsbv-89fvw" level=info timestamp=2018-07-26T12:22:54.530724Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi9bsbv-89fvw" level=error timestamp=2018-07-26T12:22:54.681939Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:23:35 http: TLS handshake error from 10.244.1.1:52370: EOF 2018/07/26 12:23:45 http: TLS handshake error from 10.244.1.1:52376: EOF 2018/07/26 12:23:55 http: TLS handshake error from 10.244.1.1:52382: EOF 2018/07/26 12:24:05 http: TLS handshake error from 10.244.1.1:52388: EOF 2018/07/26 12:24:15 http: TLS handshake error from 10.244.1.1:52394: EOF 2018/07/26 12:24:25 http: TLS handshake error from 10.244.1.1:52400: EOF 2018/07/26 12:24:35 http: TLS handshake error from 10.244.1.1:52406: EOF 2018/07/26 12:24:45 http: TLS handshake error from 10.244.1.1:52412: EOF 2018/07/26 12:24:55 http: TLS handshake error from 10.244.1.1:52418: EOF 2018/07/26 12:25:05 http: TLS handshake error from 10.244.1.1:52424: EOF 2018/07/26 12:25:15 http: TLS handshake error from 10.244.1.1:52430: EOF 2018/07/26 12:25:25 http: TLS handshake error from 10.244.1.1:52436: EOF 2018/07/26 12:25:35 http: TLS handshake error from 10.244.1.1:52442: EOF 2018/07/26 12:25:45 http: TLS handshake error from 10.244.1.1:52448: EOF 2018/07/26 12:25:55 http: TLS handshake error from 10.244.1.1:52454: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running 2018/07/26 12:24:50 http: TLS handshake error from 10.244.0.1:37910: EOF 2018/07/26 12:25:00 http: TLS handshake error from 10.244.0.1:37934: EOF level=info timestamp=2018-07-26T12:25:09.145836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:25:09.147131Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:25:10 http: TLS handshake error from 10.244.0.1:37958: EOF level=info timestamp=2018-07-26T12:25:17.192093Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:25:20 http: TLS handshake error from 10.244.0.1:37982: EOF 2018/07/26 12:25:30 http: TLS handshake error from 10.244.0.1:38006: EOF level=info timestamp=2018-07-26T12:25:39.323740Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:25:39.324228Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:25:40 http: TLS handshake error from 10.244.0.1:38030: EOF level=info timestamp=2018-07-26T12:25:47.349684Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:25:50 http: TLS handshake error from 10.244.0.1:38054: EOF level=info timestamp=2018-07-26T12:25:54.090569Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:25:54.093987Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:22:36.525605Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.525909Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.634217Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.634519Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.669543Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikg55k\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikg55k" level=info timestamp=2018-07-26T12:22:37.278165Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicq6hd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:25:38.309011Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.311901Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:38.322532Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.322689Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:38.350578Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.350772Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:38.382205Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.382578Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:39.220779Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij8nmg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij8nmg" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:25:55.576712Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.577139Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.577440Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminxw5f" level=info timestamp=2018-07-26T12:25:55.618199Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.618691Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.618927Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminxw5f" level=info timestamp=2018-07-26T12:25:55.701578Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.701858Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.702038Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminxw5f" level=info timestamp=2018-07-26T12:25:55.721213Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.721651Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.721920Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij8nmg" level=info timestamp=2018-07-26T12:25:55.797634Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.798029Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.798465Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijdb56" Pod name: virt-handler-7x9x5 Pod phase: Running level=info timestamp=2018-07-26T12:25:55.016714Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.018168Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.018825Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:25:55.059803Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.060235Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.061960Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:25:55.143805Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.144168Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.146070Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:25:55.307017Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.307693Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.307990Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:25:55.630445Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:55.630794Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:55.630948Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" Pod name: virt-launcher-testvmij8nmg-mg454 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a3e0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42029ece0, 0xc42000e480, 0xc42069ef80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8ae0, 0xc4200b83a8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bdb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8ae0, 0xc4200b83a8, 0x13, 0xc42005b718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc680, 0xc4200ba870, 0xc4206744c0, 0xc4206744d0, 0xc42032bd80, 0xc4202ec160, 0x10f5e80, 0xc42000e480, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmijdb56-vbkvz Pod phase: Running goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d60a0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4203400c0, 0xc4206385b0, 0xc420663060, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e180, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420071f18, 0x3, 0x3, 0xc4200bc040, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e180, 0xc42000e078, 0x13, 0xc4204b9f18, 0x3, 0x3, 0x105b2d1, 0x11a3360, 0xc4203de1b0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420408040, 0xc4200e6370, 0xc4219450c0, 0xc4219450d0, 0xc42087a300, 0xc4202ef220, 0x10f5e80, 0xc4206385b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmik9mqw-2ph7q Pod phase: Running goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a93e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421918cc0, 0xc42000f6d0, 0xc42029e380, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8e40, 0xc4200b8758, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420408d40, 0xc4204d5f80, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8e40, 0xc4200b8758, 0x13, 0xc421a8b718, 0x3, 0x3, 0xc4204d6480, 0xc4204d6500, 0xc4204d6580) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ad000, 0xc4200e67d0, 0xc42190fdb0, 0xc42190fdc0, 0xc42191be00, 0xc421961a80, 0x10f5e80, 0xc42000f6d0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminxw5f-r5278 Pod phase: Running goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202644d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204fd900, 0xc42000ecb0, 0xc420b94040, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e4e0, 0xc42000e0e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bca40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e4e0, 0xc42000e0e0, 0x13, 0xc42005bf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42065dc80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc640, 0xc4200f4820, 0xc420575e70, 0xc420575e80, 0xc420501e00, 0xc4200d60a0, 0x10f5e80, 0xc42000ecb0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-kcn9c Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-mbnmx Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-c2ffk Pod phase: Running 2018/07/26 12:26:15 http: TLS handshake error from 10.244.1.1:52466: EOF 2018/07/26 12:26:25 http: TLS handshake error from 10.244.1.1:52472: EOF 2018/07/26 12:26:35 http: TLS handshake error from 10.244.1.1:52478: EOF 2018/07/26 12:26:45 http: TLS handshake error from 10.244.1.1:52484: EOF 2018/07/26 12:26:55 http: TLS handshake error from 10.244.1.1:52490: EOF 2018/07/26 12:27:05 http: TLS handshake error from 10.244.1.1:52496: EOF 2018/07/26 12:27:15 http: TLS handshake error from 10.244.1.1:52502: EOF 2018/07/26 12:27:25 http: TLS handshake error from 10.244.1.1:52508: EOF 2018/07/26 12:27:35 http: TLS handshake error from 10.244.1.1:52514: EOF 2018/07/26 12:27:45 http: TLS handshake error from 10.244.1.1:52520: EOF 2018/07/26 12:27:55 http: TLS handshake error from 10.244.1.1:52526: EOF 2018/07/26 12:28:05 http: TLS handshake error from 10.244.1.1:52532: EOF 2018/07/26 12:28:15 http: TLS handshake error from 10.244.1.1:52538: EOF 2018/07/26 12:28:25 http: TLS handshake error from 10.244.1.1:52544: EOF 2018/07/26 12:28:35 http: TLS handshake error from 10.244.1.1:52550: EOF Pod name: virt-api-7d79764579-lpjfs Pod phase: Running level=info timestamp=2018-07-26T12:27:17.114514Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:27:20 http: TLS handshake error from 10.244.0.1:38270: EOF 2018/07/26 12:27:30 http: TLS handshake error from 10.244.0.1:38294: EOF level=info timestamp=2018-07-26T12:27:39.999981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:27:40.018174Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:27:40 http: TLS handshake error from 10.244.0.1:38318: EOF level=info timestamp=2018-07-26T12:27:47.238263Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:27:50 http: TLS handshake error from 10.244.0.1:38342: EOF 2018/07/26 12:28:00 http: TLS handshake error from 10.244.0.1:38366: EOF level=info timestamp=2018-07-26T12:28:10.141890Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:28:10.146490Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:28:10 http: TLS handshake error from 10.244.0.1:38390: EOF level=info timestamp=2018-07-26T12:28:17.259758Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:28:20 http: TLS handshake error from 10.244.0.1:38414: EOF 2018/07/26 12:28:30 http: TLS handshake error from 10.244.0.1:38438: EOF Pod name: virt-controller-7d57d96b65-8tb6l Pod phase: Running level=info timestamp=2018-07-26T12:22:36.525605Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.525909Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicq6hd kind= uid=94a7d1e1-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.634217Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:22:36.634519Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmil9r84 kind= uid=94ad9acb-90ce-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:22:36.669543Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmikg55k\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmikg55k" level=info timestamp=2018-07-26T12:22:37.278165Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicq6hd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicq6hd" level=info timestamp=2018-07-26T12:25:38.309011Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.311901Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:38.322532Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.322689Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:38.350578Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.350772Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:38.382205Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:25:38.382578Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:25:39.220779Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij8nmg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij8nmg" Pod name: virt-controller-7d57d96b65-bdcj9 Pod phase: Running level=info timestamp=2018-07-26T10:57:48.269938Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-handler-5pv4z Pod phase: Running level=info timestamp=2018-07-26T12:26:05.792366Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:26:05.792841Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:26:05.793136Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminxw5f" level=info timestamp=2018-07-26T12:26:15.008319Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:15.072338Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:26:15.072792Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:15.073005Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijdb56 kind= uid=01064950-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:26:15.895204Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:15.920258Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:26:15.920627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:15.920766Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij8nmg kind= uid=010d587c-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:26:16.034032Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:16.060626Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:26:16.063490Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:16.063711Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminxw5f kind= uid=01032114-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-7x9x5 Pod phase: Running level=error timestamp=2018-07-26T12:25:56.271687Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:56.271831Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:25:57.552660Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:25:57.553087Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:25:57.553206Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:26:00.113613Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:26:00.113945Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:26:00.114059Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:26:05.240639Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:26:05.246466Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:26:05.247087Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik9mqw" level=info timestamp=2018-07-26T12:26:15.488137Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:15.533163Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:26:15.536727Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:26:15.537881Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmik9mqw kind= uid=0108a747-90cf-11e8-992e-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmij8nmg-mg454 Pod phase: Failed goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42027a3e0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42029ece0, 0xc42000e480, 0xc42069ef80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8ae0, 0xc4200b83a8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bdb00, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8ae0, 0xc4200b83a8, 0x13, 0xc42005b718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc680, 0xc4200ba870, 0xc4206744c0, 0xc4206744d0, 0xc42032bd80, 0xc4202ec160, 0x10f5e80, 0xc42000e480, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmijdb56-vbkvz Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d60a0, 0xc420108000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4203400c0, 0xc4206385b0, 0xc420663060, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e180, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420071f18, 0x3, 0x3, 0xc4200bc040, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e180, 0xc42000e078, 0x13, 0xc4204b9f18, 0x3, 0x3, 0x105b2d1, 0x11a3360, 0xc4203de1b0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420408040, 0xc4200e6370, 0xc4219450c0, 0xc4219450d0, 0xc42087a300, 0xc4202ef220, 0x10f5e80, 0xc4206385b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmik9mqw-2ph7q Pod phase: Failed goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a93e0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421918cc0, 0xc42000f6d0, 0xc42029e380, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c8e40, 0xc4200b8758, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420408d40, 0xc4204d5f80, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c8e40, 0xc4200b8758, 0x13, 0xc421a8b718, 0x3, 0x3, 0xc4204d6480, 0xc4204d6500, 0xc4204d6580) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ad000, 0xc4200e67d0, 0xc42190fdb0, 0xc42190fdc0, 0xc42191be00, 0xc421961a80, 0x10f5e80, 0xc42000f6d0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminxw5f-r5278 Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202644d0, 0xc4201a2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204fd900, 0xc42000ecb0, 0xc420b94040, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008e4e0, 0xc42000e0e0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bca40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008e4e0, 0xc42000e0e0, 0x13, 0xc42005bf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42065dc80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200bc640, 0xc4200f4820, 0xc420575e70, 0xc420575e80, 0xc420501e00, 0xc4200d60a0, 0x10f5e80, 0xc42000ecb0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.717 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 with a service matching the vmi exposed [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:283 should fail to reach the vmi if an invalid servicename is used /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:314 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:25:39.639943Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvminxw5f-r5278" level=info timestamp=2018-07-26T12:25:56.309699Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvminxw5f-r5278" level=error timestamp=2018-07-26T12:25:56.479677Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" panic: test timed out after 1h30m0s goroutine 15653 [running]: testing.(*M).startAlarm.func1() /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1240 +0xfc created by time.goFunc /gimme/.gimme/versions/go1.10.linux.amd64/src/time/sleep.go:172 +0x44 goroutine 1 [chan receive, 90 minutes]: testing.(*T).Run(0xc42041eff0, 0x139b503, 0x9, 0x142d848, 0x4801e6) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:825 +0x301 testing.runTests.func1(0xc42041ef00) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1063 +0x64 testing.tRunner(0xc42041ef00, 0xc4208e9df8) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0xd0 testing.runTests(0xc42054f3e0, 0x1d2da50, 0x1, 0x1, 0x412009) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1061 +0x2c4 testing.(*M).Run(0xc4206e6800, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:978 +0x171 main.main() _testmain.go:44 +0x151 goroutine 5 [chan receive]: kubevirt.io/kubevirt/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x1d59280) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/golang/glog/glog.go:879 +0x8b created by kubevirt.io/kubevirt/vendor/github.com/golang/glog.init.0 /root/go/src/kubevirt.io/kubevirt/vendor/github.com/golang/glog/glog.go:410 +0x203 goroutine 6 [syscall, 90 minutes]: os/signal.signal_recv(0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/sigqueue.go:139 +0xa6 os/signal.loop() /gimme/.gimme/versions/go1.10.linux.amd64/src/os/signal/signal_unix.go:22 +0x22 created by os/signal.init.0 /gimme/.gimme/versions/go1.10.linux.amd64/src/os/signal/signal_unix.go:28 +0x41 goroutine 16 [select]: kubevirt.io/kubevirt/tests.(*ObjectEventWatcher).Watch(0xc420fbd020, 0xc420594240) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:288 +0x57e kubevirt.io/kubevirt/tests.(*ObjectEventWatcher).WaitFor(0xc420fbd020, 0x13972ab, 0x6, 0x11e03c0, 0x14ada60, 0x0) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:298 +0xba kubevirt.io/kubevirt/tests.waitForVMIStart(0x14b7060, 0xc420eac500, 0x5a, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1035 +0x4ea kubevirt.io/kubevirt/tests.WaitForSuccessfulVMIStart(0x14b7060, 0xc420eac500, 0xc420865f00, 0xc420fbd118) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1072 +0x43 kubevirt.io/kubevirt/tests_test.glob..func18.3(0xc420eac500, 0x142d820, 0x1d77938) /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:86 +0x4d kubevirt.io/kubevirt/tests_test.glob..func18.5() /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:150 +0x745 kubevirt.io/kubevirt/tests.BeforeAll.func1() /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1343 +0x3f kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc420424120, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:113 +0x9c kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0xc420424120, 0xc4207cf160, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64 +0x13e kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*SetupNode).Run(0xc42000e718, 0x14b3500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes/setup_nodes.go:15 +0x7f kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec.(*Spec).runSample(0xc4205a42d0, 0x0, 0x14b3500, 0xc4200c3500) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec/spec.go:181 +0x1f1 kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec.(*Spec).Run(0xc4205a42d0, 0x14b3500, 0xc4200c3500) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec/spec.go:138 +0xff kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec(0xc420351180, 0xc4205a42d0, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:200 +0x10d kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs(0xc420351180, 0x1) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:170 +0x329 kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0xc420351180, 0xb) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:66 +0x11b kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0xc4200eaaf0, 0x7fa659b85070, 0xc42041eff0, 0x139dae6, 0xb, 0xc42054f700, 0x2, 0x2, 0x14cfd80, 0xc4200c3500, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/suite/suite.go:62 +0x27c kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo.RunSpecsWithCustomReporters(0x14b4560, 0xc42041eff0, 0x139dae6, 0xb, 0xc42054f560, 0x2, 0x2, 0x2) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:221 +0x258 kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters(0x14b4560, 0xc42041eff0, 0x139dae6, 0xb, 0xc42051fe50, 0x1, 0x1, 0x1) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:209 +0xab kubevirt.io/kubevirt/tests_test.TestTests(0xc42041eff0) /root/go/src/kubevirt.io/kubevirt/tests/tests_suite_test.go:43 +0xaa testing.tRunner(0xc42041eff0, 0x142d848) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0xd0 created by testing.(*T).Run /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:824 +0x2e0 goroutine 50 [chan receive, 90 minutes]: kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).registerForInterrupts(0xc420351180, 0xc4206a5920) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:223 +0xd1 created by kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:60 +0x88 goroutine 51 [select, 90 minutes, locked to thread]: runtime.gopark(0x142fa20, 0x0, 0x1398025, 0x6, 0x18, 0x1) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/proc.go:291 +0x11a runtime.selectgo(0xc420491750, 0xc4206a59e0) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/select.go:392 +0xe50 runtime.ensureSigM.func1() /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/signal_unix.go:549 +0x1f4 runtime.goexit() /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/asm_amd64.s:2361 +0x1 goroutine 32 [IO wait]: internal/poll.runtime_pollWait(0x7fa659b64f00, 0x72, 0xc420b11850) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/netpoll.go:173 +0x57 internal/poll.(*pollDesc).wait(0xc4206c6318, 0x72, 0xffffffffffffff00, 0x14b5720, 0x1c447d0) /gimme/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(*pollDesc).waitRead(0xc4206c6318, 0xc420802000, 0x8000, 0x8000) /gimme/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(*FD).Read(0xc4206c6300, 0xc420802000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_unix.go:157 +0x17d net.(*netFD).Read(0xc4206c6300, 0xc420802000, 0x8000, 0x8000, 0x0, 0x8, 0x7ffb) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc42000e820, 0xc420802000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/net.go:176 +0x6a crypto/tls.(*block).readFromUntil(0xc4206dac30, 0x7fa65400d0b0, 0xc42000e820, 0x5, 0xc42000e820, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/crypto/tls/conn.go:493 +0x96 crypto/tls.(*Conn).readRecord(0xc4206cf500, 0x142fb17, 0xc4206cf620, 0x20) /gimme/.gimme/versions/go1.10.linux.amd64/src/crypto/tls/conn.go:595 +0xe0 crypto/tls.(*Conn).Read(0xc4206cf500, 0xc42078f000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/crypto/tls/conn.go:1156 +0x100 bufio.(*Reader).Read(0xc4205f1860, 0xc4205e02d8, 0x9, 0x9, 0xc420f78428, 0x1, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/bufio/bufio.go:216 +0x238 io.ReadAtLeast(0x14b2300, 0xc4205f1860, 0xc4205e02d8, 0x9, 0x9, 0x9, 0xc420a4b8a0, 0x43f2c1, 0xc420d66480) /gimme/.gimme/versions/go1.10.linux.amd64/src/io/io.go:309 +0x86 io.ReadFull(0x14b2300, 0xc4205f1860, 0xc4205e02d8, 0x9, 0x9, 0x142fb68, 0xc420b11d10, 0x462d33) /gimme/.gimme/versions/go1.10.linux.amd64/src/io/io.go:327 +0x58 kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.readFrameHeader(0xc4205e02d8, 0x9, 0x9, 0x14b2300, 0xc4205f1860, 0x0, 0xc400000000, 0x7efbe0, 0xc420350b68) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/frame.go:237 +0x7b kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc4205e02a0, 0xc4208da8a0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/frame.go:492 +0xa4 kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*clientConnReadLoop).run(0xc420b11fb0, 0x142e7a0, 0xc42048b7b0) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:1428 +0x8e kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*ClientConn).readLoop(0xc420249860) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:1354 +0x76 created by kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*Transport).newClientConn /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:579 +0x651 goroutine 7813 [chan send, 45 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4206bd1a0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 308 [chan send, 62 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4206d6000) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 3219 [chan send, 71 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420835980) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 401 [chan send, 89 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42067b1d0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 8824 [chan send, 39 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4207f83c0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 13556 [chan send, 12 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4202ccc30) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 8191 [chan send, 42 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4205945d0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 5451 [chan send, 58 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420559080) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 6251 [chan send, 52 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420559830) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 941 [chan send, 86 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4208563c0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 1510 [chan send, 83 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420595a70) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 10895 [chan send, 27 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420559920) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 4348 [chan send, 65 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4202cced0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 1892 [chan send, 68 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4206d6120) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 15648 [chan receive]: kubevirt.io/kubevirt/tests.(*ObjectEventWatcher).Watch.func3(0x14bc560, 0xc4209388d0, 0x0, 0xc4204dce20, 0xc4206e01e0) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:275 +0x93 created by kubevirt.io/kubevirt/tests.(*ObjectEventWatcher).Watch /root/go/src/kubevirt.io/kubevirt/tests/utils.go:273 +0x4b3 goroutine 7062 [chan send, 48 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420838a80) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 9369 [chan send, 36 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420972ea0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 14629 [chan send, 6 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42067a180) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 15151 [chan send, 3 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420980900) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 13031 [chan send, 15 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420939110) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 5976 [chan send, 55 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4205692f0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 11424 [chan send, 24 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4208c4a50) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 12515 [chan send, 18 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420558b10) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 15647 [semacquire]: sync.runtime_notifyListWait(0xc420350b80, 0xc400000001) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/sema.go:510 +0x10b sync.(*Cond).Wait(0xc420350b70) /gimme/.gimme/versions/go1.10.linux.amd64/src/sync/cond.go:56 +0x80 kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*pipe).Read(0xc420350b68, 0xc420c45201, 0x5ff, 0x5ff, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/pipe.go:64 +0x8f kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.transportResponseBody.Read(0xc420350b40, 0xc420c45201, 0x5ff, 0x5ff, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:1674 +0xa1 encoding/json.(*Decoder).refill(0xc4207ecf00, 0x835c0a, 0x9) /gimme/.gimme/versions/go1.10.linux.amd64/src/encoding/json/stream.go:159 +0x132 encoding/json.(*Decoder).readValue(0xc4207ecf00, 0x0, 0x0, 0x11f2320) /gimme/.gimme/versions/go1.10.linux.amd64/src/encoding/json/stream.go:134 +0x23d encoding/json.(*Decoder).Decode(0xc4207ecf00, 0x120e2c0, 0xc42084e220, 0x14ba320, 0xc42015c780) /gimme/.gimme/versions/go1.10.linux.amd64/src/encoding/json/stream.go:63 +0x78 kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/util/framer.(*jsonFrameReader).Read(0xc4209388a0, 0xc4208e5c00, 0x400, 0x400, 0xc420976340, 0x40, 0x38) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/util/framer/framer.go:150 +0x295 kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc420958000, 0x0, 0x14bbfe0, 0xc420976340, 0x380, 0x14ba320, 0xc4206e01d8, 0x456ae0, 0xc4206e0180) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/runtime/serializer/streaming/streaming.go:77 +0x95 kubevirt.io/kubevirt/vendor/k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc4207a5240, 0xc420b15fa8, 0x5, 0x14ba320, 0xc42015c780, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/client-go/rest/watch/decoder.go:49 +0x7c kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4209388d0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:93 +0x12e created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 11956 [chan send, 21 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4206bc0f0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 9894 [chan send, 33 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4209802a0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 10405 [chan send, 30 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4209387b0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 14061 [chan send, 9 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420687a40) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 make: *** [functest] Error 2 + make cluster-down ./cluster/down.sh