+ export WORKSPACE=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release + WORKSPACE=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release + [[ k8s-1.10.4-release =~ openshift-.* ]] + [[ k8s-1.10.4-release =~ .*-1.9.3-.* ]] + export KUBEVIRT_PROVIDER=k8s-1.10.3 + KUBEVIRT_PROVIDER=k8s-1.10.3 + export KUBEVIRT_NUM_NODES=2 + KUBEVIRT_NUM_NODES=2 + export NFS_WINDOWS_DIR=/home/nfs/images/windows2016 + NFS_WINDOWS_DIR=/home/nfs/images/windows2016 + export NAMESPACE=kube-system + NAMESPACE=kube-system + trap '{ make cluster-down; }' EXIT SIGINT SIGTERM SIGSTOP + make cluster-down ./cluster/down.sh + make cluster-up ./cluster/up.sh Downloading ....... Downloading ....... 2018/07/26 10:16:28 Waiting for host: 192.168.66.101:22 2018/07/26 10:16:31 Problem with dial: dial tcp 192.168.66.101:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:16:43 Connected to tcp://192.168.66.101:22 + kubeadm init --config /etc/kubernetes/kubeadm.conf [init] Using Kubernetes version: v1.10.3 [init] Using Authorization modes: [Node RBAC] [preflight] Running pre-flight checks. [WARNING FileExisting-crictl]: crictl not found in system path Suggestion: go get github.com/kubernetes-incubator/cri-tools/cmd/crictl Flag --admission-control has been deprecated, Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version. [certificates] Generated ca certificate and key. [certificates] Generated apiserver certificate and key. [certificates] apiserver serving cert is signed for DNS names [node01 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.66.101] [certificates] Generated apiserver-kubelet-client certificate and key. [certificates] Generated etcd/ca certificate and key. [certificates] Generated etcd/server certificate and key. [certificates] etcd/server serving cert is signed for DNS names [localhost] and IPs [127.0.0.1] [certificates] Generated etcd/peer certificate and key. [certificates] etcd/peer serving cert is signed for DNS names [node01] and IPs [192.168.66.101] [certificates] Generated etcd/healthcheck-client certificate and key. [certificates] Generated apiserver-etcd-client certificate and key. [certificates] Generated sa key and public key. [certificates] Generated front-proxy-ca certificate and key. [certificates] Generated front-proxy-client certificate and key. [certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf" [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf" [controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml" [controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/manifests/kube-controller-manager.yaml" [controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/manifests/kube-scheduler.yaml" [etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/manifests/etcd.yaml" [init] Waiting for the kubelet to boot up the control plane as Static Pods from directory "/etc/kubernetes/manifests". [init] This might take a minute or longer if the control plane images have to be pulled. [apiclient] All control plane components are healthy after 28.506446 seconds [uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [markmaster] Will mark node node01 as master by adding a label and a taint [markmaster] Master node01 tainted and labelled with key/value: node-role.kubernetes.io/master="" [bootstraptoken] Using token: abcdef.1234567890123456 [bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [addons] Applied essential addon: kube-dns [addons] Applied essential addon: kube-proxy Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join 192.168.66.101:6443 --token abcdef.1234567890123456 --discovery-token-ca-cert-hash sha256:0008d98262501a69ce7521e17593d1a71fb76eba3083f386c4cea22fe074b815 + kubectl --kubeconfig=/etc/kubernetes/admin.conf apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml clusterrole.rbac.authorization.k8s.io "flannel" created clusterrolebinding.rbac.authorization.k8s.io "flannel" created serviceaccount "flannel" created configmap "kube-flannel-cfg" created daemonset.extensions "kube-flannel-ds" created + kubectl --kubeconfig=/etc/kubernetes/admin.conf taint nodes node01 node-role.kubernetes.io/master:NoSchedule- node "node01" untainted 2018/07/26 10:17:29 Waiting for host: 192.168.66.102:22 2018/07/26 10:17:32 Problem with dial: dial tcp 192.168.66.102:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:17:40 Problem with dial: dial tcp 192.168.66.102:22: getsockopt: no route to host. Sleeping 5s 2018/07/26 10:17:45 Problem with dial: dial tcp 192.168.66.102:22: getsockopt: connection refused. Sleeping 5s 2018/07/26 10:17:50 Connected to tcp://192.168.66.102:22 + kubeadm join --token abcdef.1234567890123456 192.168.66.101:6443 --ignore-preflight-errors=all --discovery-token-unsafe-skip-ca-verification=true [preflight] Running pre-flight checks. [discovery] Trying to connect to API Server "192.168.66.101:6443" [WARNING FileExisting-crictl]: crictl not found in system path Suggestion: go get github.com/kubernetes-incubator/cri-tools/cmd/crictl [discovery] Created cluster-info discovery client, requesting info from "https://192.168.66.101:6443" [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "192.168.66.101:6443" [discovery] Successfully established connection with API Server "192.168.66.101:6443" This node has joined the cluster: * Certificate signing request was sent to master and a response was received. * The Kubelet was informed of the new secure connection details. Run 'kubectl get nodes' on the master to see this node join the cluster. Sending file modes: C0755 39588992 kubectl Sending file modes: C0600 5454 admin.conf Cluster "kubernetes" set. Cluster "kubernetes" set. + set +e + kubectl get nodes --no-headers + cluster/kubectl.sh get nodes --no-headers node01 Ready master 58s v1.10.3 node02 Ready 19s v1.10.3 + kubectl_rc=0 + '[' 0 -ne 0 ']' ++ kubectl get nodes --no-headers ++ cluster/kubectl.sh get nodes --no-headers ++ grep NotReady + '[' -n '' ']' + set -e + echo 'Nodes are ready:' Nodes are ready: + kubectl get nodes + cluster/kubectl.sh get nodes NAME STATUS ROLES AGE VERSION node01 Ready master 1m v1.10.3 node02 Ready 21s v1.10.3 + make cluster-sync ./cluster/build.sh Building ... sha256:b69a3f94b2043cd36cc41eb5d9446480e0a640962e468ab72c3cc51f2b89386a go version go1.10 linux/amd64 go version go1.10 linux/amd64 make[1]: Entering directory `/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt' hack/dockerized "./hack/check.sh && KUBEVIRT_VERSION= ./hack/build-go.sh install " && ./hack/build-copy-artifacts.sh sha256:b69a3f94b2043cd36cc41eb5d9446480e0a640962e468ab72c3cc51f2b89386a go version go1.10 linux/amd64 go version go1.10 linux/amd64 find: '/root/go/src/kubevirt.io/kubevirt/_out/cmd': No such file or directory Compiling tests... compiled tests.test hack/build-docker.sh build Sending build context to Docker daemon 40.37 MB Step 1/8 : FROM fedora:28 ---> cc510acfcd70 Step 2/8 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/8 : RUN useradd -u 1001 --create-home -s /bin/bash virt-controller ---> Using cache ---> 84570f0bf244 Step 4/8 : WORKDIR /home/virt-controller ---> Using cache ---> 4b8efcbf3461 Step 5/8 : USER 1001 ---> Using cache ---> c49257f2ff48 Step 6/8 : COPY virt-controller /usr/bin/virt-controller ---> cbdf56ba207a Removing intermediate container 3e0319b4d986 Step 7/8 : ENTRYPOINT /usr/bin/virt-controller ---> Running in 560f5061d9e1 ---> 95bd2e10f6f3 Removing intermediate container 560f5061d9e1 Step 8/8 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "virt-controller" '' ---> Running in 7c94bef302e4 ---> 007df2d0ffd0 Removing intermediate container 7c94bef302e4 Successfully built 007df2d0ffd0 Sending build context to Docker daemon 43.3 MB Step 1/10 : FROM kubevirt/libvirt:4.2.0 ---> 5f0bfe81a3e0 Step 2/10 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> c1e65e6c8241 Step 3/10 : RUN dnf -y install socat genisoimage util-linux libcgroup-tools ethtool net-tools sudo && dnf -y clean all && test $(id -u qemu) = 107 # make sure that the qemu user really is 107 ---> Using cache ---> 4c20d196c128 Step 4/10 : COPY virt-launcher /usr/bin/virt-launcher ---> 22be2ada4550 Removing intermediate container 41cc67259340 Step 5/10 : COPY kubevirt-sudo /etc/sudoers.d/kubevirt ---> c4ba6895a891 Removing intermediate container 3da24d274cae Step 6/10 : RUN setcap CAP_NET_BIND_SERVICE=+eip /usr/bin/qemu-system-x86_64 ---> Running in be7d5b82ca61  ---> 0e2cddbd1897 Removing intermediate container be7d5b82ca61 Step 7/10 : RUN mkdir -p /usr/share/kubevirt/virt-launcher ---> Running in 132edc209a16  ---> 2c9e1abcf8bc Removing intermediate container 132edc209a16 Step 8/10 : COPY entrypoint.sh libvirtd.sh sock-connector /usr/share/kubevirt/virt-launcher/ ---> a4ba335a949f Removing intermediate container a11edbf52998 Step 9/10 : ENTRYPOINT /usr/share/kubevirt/virt-launcher/entrypoint.sh ---> Running in 665fa160b477 ---> 89406b022f38 Removing intermediate container 665fa160b477 Step 10/10 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "virt-launcher" '' ---> Running in 1fd7afca4ee1 ---> 330f76be210b Removing intermediate container 1fd7afca4ee1 Successfully built 330f76be210b Sending build context to Docker daemon 41.67 MB Step 1/5 : FROM fedora:28 ---> cc510acfcd70 Step 2/5 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/5 : COPY virt-handler /usr/bin/virt-handler ---> 9ae68fbce6d1 Removing intermediate container ee686c93384c Step 4/5 : ENTRYPOINT /usr/bin/virt-handler ---> Running in e6f2e9a2e562 ---> af954c29a20b Removing intermediate container e6f2e9a2e562 Step 5/5 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "virt-handler" '' ---> Running in e59755b28a98 ---> 0e01fb8c2211 Removing intermediate container e59755b28a98 Successfully built 0e01fb8c2211 Sending build context to Docker daemon 38.81 MB Step 1/8 : FROM fedora:28 ---> cc510acfcd70 Step 2/8 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/8 : RUN useradd -u 1001 --create-home -s /bin/bash virt-api ---> Using cache ---> 6f2134b876af Step 4/8 : WORKDIR /home/virt-api ---> Using cache ---> d5ef0239bf68 Step 5/8 : USER 1001 ---> Using cache ---> 233000b2d9b5 Step 6/8 : COPY virt-api /usr/bin/virt-api ---> 3e2244259657 Removing intermediate container 1bb882765fb4 Step 7/8 : ENTRYPOINT /usr/bin/virt-api ---> Running in 4e7446c0b612 ---> 974ea66d1786 Removing intermediate container 4e7446c0b612 Step 8/8 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "virt-api" '' ---> Running in 52f4b7b42c23 ---> 10ac357249f1 Removing intermediate container 52f4b7b42c23 Successfully built 10ac357249f1 Sending build context to Docker daemon 4.096 kB Step 1/7 : FROM fedora:28 ---> cc510acfcd70 Step 2/7 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/7 : ENV container docker ---> Using cache ---> 3fe7db912524 Step 4/7 : RUN mkdir -p /images/custom /images/alpine && truncate -s 64M /images/custom/disk.img && curl http://dl-cdn.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso > /images/alpine/disk.img ---> Using cache ---> 06d762a67408 Step 5/7 : ADD entrypoint.sh / ---> Using cache ---> 3876d185cf84 Step 6/7 : CMD /entrypoint.sh ---> Using cache ---> 1fb50ce9b78f Step 7/7 : LABEL "disks-images-provider" '' "kubevirt-functional-tests-k8s-1.10.4-release2" '' ---> Running in aab879635b89 ---> 809173d04618 Removing intermediate container aab879635b89 Successfully built 809173d04618 Sending build context to Docker daemon 2.56 kB Step 1/5 : FROM fedora:28 ---> cc510acfcd70 Step 2/5 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/5 : ENV container docker ---> Using cache ---> 3fe7db912524 Step 4/5 : RUN dnf -y install procps-ng nmap-ncat && dnf -y clean all ---> Using cache ---> 6bc4f549313f Step 5/5 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "vm-killer" '' ---> Running in 02dd67e81363 ---> 74ffede570f3 Removing intermediate container 02dd67e81363 Successfully built 74ffede570f3 Sending build context to Docker daemon 5.12 kB Step 1/7 : FROM debian:sid ---> 68f33cf86aab Step 2/7 : MAINTAINER "David Vossel" \ ---> Using cache ---> 9ef1c0ce5d24 Step 3/7 : ENV container docker ---> Using cache ---> 9ad55e41ed61 Step 4/7 : RUN apt-get update && apt-get install -y bash curl bzip2 qemu-utils && mkdir -p /disk && rm -rf /var/lib/apt/lists/* ---> Using cache ---> 17a81fda7c2b Step 5/7 : ADD entry-point.sh / ---> Using cache ---> 681d01e165e6 Step 6/7 : CMD /entry-point.sh ---> Using cache ---> a79815fe82d9 Step 7/7 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "registry-disk-v1alpha" '' ---> Running in 62b49176427a ---> be30bba34bf4 Removing intermediate container 62b49176427a Successfully built be30bba34bf4 Sending build context to Docker daemon 2.56 kB Step 1/4 : FROM localhost:32858/kubevirt/registry-disk-v1alpha:devel ---> be30bba34bf4 Step 2/4 : MAINTAINER "David Vossel" \ ---> Running in 73e9a282c016 ---> c2e07818e476 Removing intermediate container 73e9a282c016 Step 3/4 : RUN curl https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img > /disk/cirros.img ---> Running in 56722df0c73f  % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0  0 0 0 --:--:-- --:--:-- --:--:-- 0 1 12.1M 1 128k 0 0 184k 0 0:01:07 --:--:-- 0:01:07 183k 100 12.1M 100 12.1M 0 0 7604k 0 0:00:01 0:00:01 --:--:-- 7604k  ---> 3aa1c3d0dfa2 Removing intermediate container 56722df0c73f Step 4/4 : LABEL "cirros-registry-disk-demo" '' "kubevirt-functional-tests-k8s-1.10.4-release2" '' ---> Running in 00e805090fa8 ---> 0504f767e30a Removing intermediate container 00e805090fa8 Successfully built 0504f767e30a Sending build context to Docker daemon 2.56 kB Step 1/4 : FROM localhost:32858/kubevirt/registry-disk-v1alpha:devel ---> be30bba34bf4 Step 2/4 : MAINTAINER "The KubeVirt Project" ---> Running in 50acd68181da ---> 0544873f9886 Removing intermediate container 50acd68181da Step 3/4 : RUN curl -g -L https://download.fedoraproject.org/pub/fedora/linux/releases/27/CloudImages/x86_64/images/Fedora-Cloud-Base-27-1.6.x86_64.qcow2 > /disk/fedora.qcow2 ---> Running in 0aac27a8fa2c   % Total % Received % Xferd Average Speed Time Time Time Current  Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0  0 221M 0 357k 0 0 584k 0 0:06:28 --:--:-- 0:06:28 584k 8 221M 8 18.4M 0 0 11.5M 0 0:00:19 0:00:01 0:00:18 18.3M 12 221M 12 28.1M 0 0 10.8M 0 0:00:20 0:00:02 0:00:18 13.9M 15 221M 15 35.2M 0 0 9.7M 0 0:00:22 0:00:03 0:00:19 11.6M 17 221M 17 38.4M 0 0 8543k 0 0:00:26 0:00:04  0:00:22 9763k 18 221M 18 41.0M 0 0 7487k 0 0:00:30 0:00:05 0:00:25 8333k 19 221M 19 43.6M 0 0 6770k 0  0:00:33 0:00:06 0:00:27 5159k 20 221M 20 45.9M 0 0 6179k 0 0:00:36 0:00:07 0:00:29 3632k 21 221M 21 48.4M 0 0 5760k 0 0:00:39 0:00:08 0:00:31 2698k 22 221M 22 50.6M 0 0 5399k 0 0:00:42 0:00:09 0:00:33 2494k 23 221M 23 52.7M 0 0 5096k 0 0:00:44 0:00:10 0:00:34 2411k 24 221M 24 55.0M 0 0 4856k 0 0:00:46 0:00:11 0:00:35 2324k 25 221M 25 57.2M 0 0 4655k 0 0:00:48 0:00:12 0:00:36 2331k 26 221M 26 59.6M 0 0 4488k 0 0:00:50 0:00:13 0:00:37 2294k 28 221M 28 62.0M 0 0 4351k 0 0:00:52 0:00:14 0:00:38 2336k 28  221M 28 64.2M 0 0 4215k 0 0:00:53 0:00:15 0:00:38 2344k 30 221M 30 66.6M 0 0 4111k 0 0:00:55 0:00:16 0:00:39 2383k 31 221M 31 69.2M 0 0 4031k 0 0:00:56 0:00:17 0:00:39 2458k 32 221M 32 71.9M 0 0 3962k 0 0:00:57 0:00:18 0:00:39 2534k 33 221M 33 74.6M 0 0 3900k 0 0:00:58 0:00:19 0:00:39 2584k 34 221M 34 77.3M 0 0 3844k 0 0:00:59 0:00:20 0:00:39 2688k 36 221M 36 80.3M 0 0 3807k 0 0:00:59 0:00:21  0:00:38 2799k 37 221M 37 83.8M 0 0 3798k 0 0:00:59  0:00:22 0:00:37 2976k 39 221M 39 88.1M 0 0 3826k 0 0:00:59 0:00:23 0:00:36 3317k 41 221M 41 92.7M 0 0 3861k 0 0:00:58 0:00:24 0:00:34 3706k 43 221M 43 96.2M 0 0 3850k  0 0:00:58 0:00:25 0:00:33 3878k 45 221M 45 99.9M 0 0 3847k 0  0:00:58 0:00:26 0:00:32 4018k 46 221M 46 103M 0 0 3847k 0 0:00:58 0:00:27 0:00:31 4072k 48 221M 48 107M 0 0 3846k 0 0:00:58 0:00:28 0:00:30 3942k 49 221M 49 110M 0 0 3822k 0 0:00:59  0:00:29 0:00:30 3629k 51 221M 51 114M 0 0 3814k 0 0:00:59 0:00:30 0:00:29 3630k 53 221M 53 117M 0 0 3814k 0 0:00:59 0:00:31 0:00:28 3641k 54 221M 54 121M 0 0 3816k 0 0:00:59 0:00:32 0:00:27 3645k 56 221M 56 125M 0 0 3818k   0 0:00:59 0:00:33 0:00:26 3660k 58 221M 58 129M 0 0 3822k 0 0:00:59 0:00:34 0:00:25 3820k 60 221M 60 133M 0 0 3830k 0 0:00:59 0:00:35  0:00:24 3928k 62 221M 62 137M 0 0 3851k 0 0:00:58 0:00:36 0:00:22 4080k 64 221M 64 142M 0 0 3892k 0 0:00:58 0:00:37 0:00:21 4386k 67 221M 67 148M 0 0 3950k 0 0:00:57 0:00:38 0:00:19 4837k 69 221M 69 154M 0 0 3995k 0 0:00:56 0:00:39 0:00:17 5199k 72 221M 72 160M 0 0 4055k 0 0:00:55 0:00:40 0:00:15 5652k 75 221M 75 167M 0 0 4112k 0 0:00:55 0:00:41 0:00:14 6029k 77 221M 77 170M  0 0 4105k 0 0:00:55 0:00:42 0:00:13 5704k 78 221M 78 174M 0 0 4104k  0 0:00:55 0:00:43 0:00:12 5294k 80 221M 80 178M 0 0 4104k 0 0:00:55 0:00:44 0:00:11 4965k 82 221M 82 182M 0 0 4092k  0 0:00:55  0:00:45 0:00:10 4396k 83 221M 83 184M 0 0 4056k 0 0:00:55 0:00:46 0:00:09 3591k 84 221M 84 186M 0 0 4014k  0 0:00:56 0:00:47 0:00:09 3238k 85 221M 85 188M 0 0  3975k 0 0:00:57 0:00:48 0:00:09 2845k 86 221M 86 190M 0 0 3938k 0 0:00:57 0:00:49 0:00:08 2453k 87 221M 87 192M 0 0 3903k   0 0:00:58 0:00:50 0:00:08 2181k 88 221M 88 195M 0 0 3873k 0 0:00:58 0:00:51 0:00:07 2165k 89 221M 89 197M 0 0 3847k 0 0:00:58 0:00:52 0:00:06 2268k 90 221M 90 200M 0 0  3835k 0 0:00:59 0:00:53 0:00:06 2486k 92 221M 92 204M 0 0 3828k 0  0:00:59 0:00:54 0:00:05 2738k 93 221M 93 207M 0 0 3817k 0 0:00:59 0:00:55 0:00:04 2949k 94 221M 94 209M 0 0 3788k  0 0:00:59 0:00:56 0:00:03 2909k 95 221M 95 211M 0 0 3758k 0 0:01:00 0:00:57 0:00:03 2812k 96 221M 96 213M 0 0 3725k 0 0:01:00 0:00:58 0:00:02 2545k 97 221M 97 214M 0 0 3692k  0 0:01:01 0:00:59 0:00:02 2210k 97 221M 97 216M 0 0 3660k  0 0:01:01 0:01:00 0:00:01 1914k 98 221M 98 218M 0 0 3630k 0 0:01:02 0:01:01 0:00:01 1850k 99 221M 99 220M 0 0 3603k 0  0:01:02 0:01:02 --:--:-- 1820k 100 221M 100 221M 0 0 3585k 0 0:01:03 0:01:03 --:--:-- 1828k  ---> f24cef0cba64 Removing intermediate container 0aac27a8fa2c Step 4/4 : LABEL "fedora-cloud-registry-disk-demo" '' "kubevirt-functional-tests-k8s-1.10.4-release2" '' ---> Running in 5c512eefc09d ---> 8d0e2100cb8e Removing intermediate container 5c512eefc09d Successfully built 8d0e2100cb8e Sending build context to Docker daemon 2.56 kB Step 1/4 : FROM localhost:32858/kubevirt/registry-disk-v1alpha:devel ---> be30bba34bf4 Step 2/4 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 0544873f9886 Step 3/4 : RUN curl http://dl-cdn.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso > /disk/alpine.iso ---> Running in 3ff3e9a75ff8  % Total % Received % Xferd Average Speed Time Time Time Current  Dload Upload Total Spent Left Speed 0 0 0 0  0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 37.0M 0 51984 0 0 42297 0 0:15:17 0:00:01 0:15:16 42263 1 37.0M 1 734k 0 0 331k 0 0:01:54 0:00:02 0:01:52 331k 12 37.0M 12 4690k 0 0 1457k 0 0:00:25 0:00:03 0:00:22 1457k 27 37.0M 27 10.2M 0 0 2486k 0 0:00:15 0:00:04 0:00:11 2486k 45 37.0M 45 16.7M 0 0 3288k 0 0:00:11 0:00:05 0:00:06 3462k 66 37.0M 66 24.7M 0 0 4073k 0 0:00:09 0:00:06 0:00:03 5065k 91 37.0M 91 33.9M 0 0 4812k 0 0:00:07 0:00:07 --:--:-- 6799k 100 37.0M 100 37.0M 0 0 5042k 0 0:00:07 0:00:07 --:--:-- 7731k  ---> 72f3020fe53d Removing intermediate container 3ff3e9a75ff8 Step 4/4 : LABEL "alpine-registry-disk-demo" '' "kubevirt-functional-tests-k8s-1.10.4-release2" '' ---> Running in e2f7c158d802 ---> bb97480b13d7 Removing intermediate container e2f7c158d802 Successfully built bb97480b13d7 Sending build context to Docker daemon 35.59 MB Step 1/8 : FROM fedora:28 ---> cc510acfcd70 Step 2/8 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/8 : RUN useradd -u 1001 --create-home -s /bin/bash virtctl ---> Using cache ---> deebe9dc06da Step 4/8 : WORKDIR /home/virtctl ---> Using cache ---> 4094ce77e412 Step 5/8 : USER 1001 ---> Using cache ---> ba694520e9a4 Step 6/8 : COPY subresource-access-test /subresource-access-test ---> 2265100d664d Removing intermediate container 022a7a3251ef Step 7/8 : ENTRYPOINT /subresource-access-test ---> Running in bbb86e7da554 ---> 7c3e5ce575a8 Removing intermediate container bbb86e7da554 Step 8/8 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "subresource-access-test" '' ---> Running in 5a28962a596d ---> 9e67eb0a079b Removing intermediate container 5a28962a596d Successfully built 9e67eb0a079b Sending build context to Docker daemon 3.072 kB Step 1/9 : FROM fedora:28 ---> cc510acfcd70 Step 2/9 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> 3265a3c6f899 Step 3/9 : ENV container docker ---> Using cache ---> 3fe7db912524 Step 4/9 : RUN dnf -y install make git gcc && dnf -y clean all ---> Using cache ---> e0cf52293e57 Step 5/9 : ENV GIMME_GO_VERSION 1.9.2 ---> Using cache ---> 8c031086e8cb Step 6/9 : RUN mkdir -p /gimme && curl -sL https://raw.githubusercontent.com/travis-ci/gimme/master/gimme | HOME=/gimme bash >> /etc/profile.d/gimme.sh ---> Using cache ---> 0f6dd31de4d3 Step 7/9 : ENV GOPATH "/go" GOBIN "/usr/bin" ---> Using cache ---> 6a702eb79a95 Step 8/9 : RUN mkdir -p /go && source /etc/profile.d/gimme.sh && go get github.com/masterzen/winrm-cli ---> Using cache ---> bed79012c9f3 Step 9/9 : LABEL "kubevirt-functional-tests-k8s-1.10.4-release2" '' "winrmcli" '' ---> Running in 7dc26da81071 ---> 2d878a9cea0a Removing intermediate container 7dc26da81071 Successfully built 2d878a9cea0a Sending build context to Docker daemon 36.78 MB Step 1/5 : FROM fedora:27 ---> 9110ae7f579f Step 2/5 : MAINTAINER "The KubeVirt Project" ---> Using cache ---> cc296a71da13 Step 3/5 : COPY example-hook-sidecar /example-hook-sidecar ---> 5ee97cf52533 Removing intermediate container 945348fba0c1 Step 4/5 : ENTRYPOINT /example-hook-sidecar ---> Running in fa2d0fb3ea10 ---> 61347415f721 Removing intermediate container fa2d0fb3ea10 Step 5/5 : LABEL "example-hook-sidecar" '' "kubevirt-functional-tests-k8s-1.10.4-release2" '' ---> Running in 4f0ae801d4fe ---> faafc3368150 Removing intermediate container 4f0ae801d4fe Successfully built faafc3368150 hack/build-docker.sh push The push refers to a repository [localhost:32858/kubevirt/virt-controller] bd908c1db48d: Preparing 915a0c3e3f5f: Preparing 891e1e4ef82a: Preparing 915a0c3e3f5f: Pushed bd908c1db48d: Pushed 891e1e4ef82a: Pushed devel: digest: sha256:6b5e95dde63c8ad5684efac8946aa0c5a41223e64e7c210108b8f801f0c4dc85 size: 949 The push refers to a repository [localhost:32858/kubevirt/virt-launcher] 5d787aaf6368: Preparing faa40b968efa: Preparing 9dafa800e226: Preparing 4fead516ae9f: Preparing 681ff7fad496: Preparing 5379fb5d8cce: Preparing da38cf808aa5: Preparing b83399358a92: Preparing 186d8b3e4fd8: Preparing fa6154170bf5: Preparing 5eefb9960a36: Preparing 891e1e4ef82a: Preparing 5379fb5d8cce: Waiting 681ff7fad496: Waiting da38cf808aa5: Waiting fa6154170bf5: Waiting 186d8b3e4fd8: Waiting b83399358a92: Waiting 891e1e4ef82a: Waiting faa40b968efa: Pushed 5d787aaf6368: Pushed 4fead516ae9f: Pushed da38cf808aa5: Pushed b83399358a92: Pushed 186d8b3e4fd8: Pushed fa6154170bf5: Pushed 9dafa800e226: Pushed 891e1e4ef82a: Mounted from kubevirt/virt-controller 5379fb5d8cce: Pushed 681ff7fad496: Pushed 5eefb9960a36: Pushed devel: digest: sha256:a6966c4b11c942aeeaed0e3e73ea7009c0b28f5b8d3669e840ba14d4b1f4f9f0 size: 2828 The push refers to a repository [localhost:32858/kubevirt/virt-handler] 6bec2b3294f5: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/virt-launcher 6bec2b3294f5: Pushed devel: digest: sha256:af261184950fb26e1c3c18ec976ca6db42dde5a8e0ccd9b982b40eb76ab40667 size: 741 The push refers to a repository [localhost:32858/kubevirt/virt-api] 7ac614f32043: Preparing 7cc07c574d2a: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Waiting 891e1e4ef82a: Mounted from kubevirt/virt-handler 7cc07c574d2a: Pushed 7ac614f32043: Pushed devel: digest: sha256:24eb308ee7ef516bb923898e8536c93f1e52eececffc5d31e11ab3dcf3726187 size: 948 The push refers to a repository [localhost:32858/kubevirt/disks-images-provider] 1548fa7b1c9e: Preparing a7621d2cf364: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/virt-api 1548fa7b1c9e: Pushed a7621d2cf364: Pushed devel: digest: sha256:d6d5c8c273165f456708a67ac286665f1cda39f3e438d5dc02bb245338e410b6 size: 948 The push refers to a repository [localhost:32858/kubevirt/vm-killer] 3c31f9f8d755: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Mounted from kubevirt/disks-images-provider 3c31f9f8d755: Pushed devel: digest: sha256:3a8e51df34c6b379c71c809acac28e3f3b8f06e5b5d60591630e4aa6acee8ffd size: 740 The push refers to a repository [localhost:32858/kubevirt/registry-disk-v1alpha] c66b9a220e25: Preparing 4662bbc21c2d: Preparing 25edbec0eaea: Preparing c66b9a220e25: Pushed 4662bbc21c2d: Pushed 25edbec0eaea: Pushed devel: digest: sha256:df300c62a2c575b0f8665efe8b0061a2d5c8db2d24571818076722f92be58cd8 size: 948 The push refers to a repository [localhost:32858/kubevirt/cirros-registry-disk-demo] 10c4b87673ad: Preparing c66b9a220e25: Preparing 4662bbc21c2d: Preparing 25edbec0eaea: Preparing 25edbec0eaea: Mounted from kubevirt/registry-disk-v1alpha c66b9a220e25: Mounted from kubevirt/registry-disk-v1alpha 4662bbc21c2d: Mounted from kubevirt/registry-disk-v1alpha 10c4b87673ad: Pushed devel: digest: sha256:31b00cc184199173e0a89e8fe39fa9d9ecbb7ac114c1847b18534771d1ecca90 size: 1160 The push refers to a repository [localhost:32858/kubevirt/fedora-cloud-registry-disk-demo] e6f75f2f6f0f: Preparing c66b9a220e25: Preparing 4662bbc21c2d: Preparing 25edbec0eaea: Preparing c66b9a220e25: Mounted from kubevirt/cirros-registry-disk-demo 25edbec0eaea: Mounted from kubevirt/cirros-registry-disk-demo 4662bbc21c2d: Mounted from kubevirt/cirros-registry-disk-demo e6f75f2f6f0f: Pushed devel: digest: sha256:44da7bbdaea104b5fbe44fd514357563d85e588d3747145da620e34183ce276e size: 1161 The push refers to a repository [localhost:32858/kubevirt/alpine-registry-disk-demo] 4db9acb48d52: Preparing c66b9a220e25: Preparing 4662bbc21c2d: Preparing 25edbec0eaea: Preparing 25edbec0eaea: Mounted from kubevirt/fedora-cloud-registry-disk-demo 4662bbc21c2d: Mounted from kubevirt/fedora-cloud-registry-disk-demo c66b9a220e25: Mounted from kubevirt/fedora-cloud-registry-disk-demo 4db9acb48d52: Pushed devel: digest: sha256:255c0f7d70fb53b776f35e6ab23e350b97b3173764035b2d75ad11e848260ba8 size: 1160 The push refers to a repository [localhost:32858/kubevirt/subresource-access-test] c91dd93ad17b: Preparing 7e69243e781e: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Waiting 891e1e4ef82a: Mounted from kubevirt/vm-killer 7e69243e781e: Pushed c91dd93ad17b: Pushed devel: digest: sha256:1f7c6f0855f5687d442357bfa7f71919c8086eabfba2b691b87397c9fda5d4c3 size: 948 The push refers to a repository [localhost:32858/kubevirt/winrmcli] a117c61a5658: Preparing c9df4405017d: Preparing 99bb32247f65: Preparing 891e1e4ef82a: Preparing 891e1e4ef82a: Waiting a117c61a5658: Pushed 891e1e4ef82a: Mounted from kubevirt/subresource-access-test 99bb32247f65: Pushed c9df4405017d: Pushed devel: digest: sha256:1b7a0c846fa3219a016280b56d9a6099366812334a0bea88cedf6e8e3b46f32f size: 1165 The push refers to a repository [localhost:32858/kubevirt/example-hook-sidecar] 257df5103079: Preparing 39bae602f753: Preparing 257df5103079: Pushed 39bae602f753: Pushed devel: digest: sha256:75fbc3cea14696ad0d50d1a607440e21805e79190980a45cdf29e0b1224535b6 size: 740 make[1]: Leaving directory `/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt' Done ./cluster/clean.sh + source hack/common.sh ++++ dirname 'hack/common.sh[0]' +++ cd hack/../ +++ pwd ++ KUBEVIRT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt ++ OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out ++ VENDOR_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/vendor ++ CMD_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/cmd ++ TESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/tests ++ APIDOCS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/apidocs ++ MANIFESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests ++ MANIFEST_TEMPLATES_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/templates/manifests ++ PYTHON_CLIENT_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/client-python ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_NUM_NODES=2 ++ '[' -z kubevirt-functional-tests-k8s-1.10.4-release ']' ++ provider_prefix=kubevirt-functional-tests-k8s-1.10.4-release2 ++ job_prefix=kubevirt-functional-tests-k8s-1.10.4-release2 +++ kubevirt_version +++ '[' -n '' ']' +++ '[' -d /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/.git ']' ++++ git describe --always --tags +++ echo v0.7.0-126-g81b07a4 ++ KUBEVIRT_VERSION=v0.7.0-126-g81b07a4 + source cluster/k8s-1.10.3/provider.sh ++ set -e ++ image=k8s-1.10.3@sha256:d6290260e7e6b84419984f12719cf592ccbe327373b8df76aa0481f8ec01d357 ++ source cluster/ephemeral-provider-common.sh +++ set -e +++ _cli='docker run --privileged --net=host --rm -v /var/run/docker.sock:/var/run/docker.sock kubevirtci/gocli@sha256:aa7f295a7908fa333ab5e98ef3af0bfafbabfd3cee2b83f9af47f722e3000f6a' + source hack/config.sh ++ unset binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig manifest_docker_prefix namespace ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ source hack/config-default.sh source hack/config-k8s-1.10.3.sh +++ binaries='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virtctl cmd/fake-qemu-process cmd/virt-api cmd/subresource-access-test cmd/example-hook-sidecar' +++ docker_images='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virt-api images/disks-images-provider images/vm-killer cmd/registry-disk-v1alpha images/cirros-registry-disk-demo images/fedora-cloud-registry-disk-demo images/alpine-registry-disk-demo cmd/subresource-access-test images/winrmcli cmd/example-hook-sidecar' +++ docker_prefix=kubevirt +++ docker_tag=latest +++ master_ip=192.168.200.2 +++ network_provider=flannel +++ namespace=kube-system ++ test -f hack/config-provider-k8s-1.10.3.sh ++ source hack/config-provider-k8s-1.10.3.sh +++ master_ip=127.0.0.1 +++ docker_tag=devel +++ kubeconfig=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubeconfig +++ kubectl=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubectl +++ docker_prefix=localhost:32858/kubevirt +++ manifest_docker_prefix=registry:5000/kubevirt ++ test -f hack/config-local.sh ++ export binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig namespace + echo 'Cleaning up ...' Cleaning up ... + cluster/kubectl.sh get vmis --all-namespaces -o=custom-columns=NAME:.metadata.name,NAMESPACE:.metadata.namespace,FINALIZERS:.metadata.finalizers --no-headers + grep foregroundDeleteVirtualMachine + read p error: the server doesn't have a resource type "vmis" + _kubectl delete ds -l kubevirt.io -n kube-system --cascade=false --grace-period 0 No resources found + _kubectl delete pods -n kube-system -l=kubevirt.io=libvirt --force --grace-period 0 No resources found + _kubectl delete pods -n kube-system -l=kubevirt.io=virt-handler --force --grace-period 0 No resources found + namespaces=(default ${namespace}) + for i in '${namespaces[@]}' + _kubectl -n default delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete apiservices -l kubevirt.io No resources found + _kubectl -n default delete deployment -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete deployment -l kubevirt.io No resources found + _kubectl -n default delete rs -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete rs -l kubevirt.io No resources found + _kubectl -n default delete services -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete services -l kubevirt.io No resources found + _kubectl -n default delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete apiservices -l kubevirt.io No resources found + _kubectl -n default delete validatingwebhookconfiguration -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete validatingwebhookconfiguration -l kubevirt.io No resources found + _kubectl -n default delete secrets -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete secrets -l kubevirt.io No resources found + _kubectl -n default delete pv -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete pv -l kubevirt.io No resources found + _kubectl -n default delete pvc -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete pvc -l kubevirt.io No resources found + _kubectl -n default delete ds -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete ds -l kubevirt.io No resources found + _kubectl -n default delete customresourcedefinitions -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete customresourcedefinitions -l kubevirt.io No resources found + _kubectl -n default delete pods -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete pods -l kubevirt.io No resources found + _kubectl -n default delete clusterrolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete clusterrolebinding -l kubevirt.io No resources found + _kubectl -n default delete rolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete rolebinding -l kubevirt.io No resources found + _kubectl -n default delete roles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete roles -l kubevirt.io No resources found + _kubectl -n default delete clusterroles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete clusterroles -l kubevirt.io No resources found + _kubectl -n default delete serviceaccounts -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n default delete serviceaccounts -l kubevirt.io No resources found ++ _kubectl -n default get crd offlinevirtualmachines.kubevirt.io ++ wc -l ++ export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ cluster/k8s-1.10.3/.kubectl -n default get crd offlinevirtualmachines.kubevirt.io Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "offlinevirtualmachines.kubevirt.io" not found + '[' 0 -gt 0 ']' + for i in '${namespaces[@]}' + _kubectl -n kube-system delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete apiservices -l kubevirt.io No resources found + _kubectl -n kube-system delete deployment -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete deployment -l kubevirt.io No resources found + _kubectl -n kube-system delete rs -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete rs -l kubevirt.io No resources found + _kubectl -n kube-system delete services -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete services -l kubevirt.io No resources found + _kubectl -n kube-system delete apiservices -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete apiservices -l kubevirt.io No resources found + _kubectl -n kube-system delete validatingwebhookconfiguration -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete validatingwebhookconfiguration -l kubevirt.io No resources found + _kubectl -n kube-system delete secrets -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete secrets -l kubevirt.io No resources found + _kubectl -n kube-system delete pv -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete pv -l kubevirt.io No resources found + _kubectl -n kube-system delete pvc -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete pvc -l kubevirt.io No resources found + _kubectl -n kube-system delete ds -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete ds -l kubevirt.io No resources found + _kubectl -n kube-system delete customresourcedefinitions -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete customresourcedefinitions -l kubevirt.io No resources found + _kubectl -n kube-system delete pods -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete pods -l kubevirt.io No resources found + _kubectl -n kube-system delete clusterrolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete clusterrolebinding -l kubevirt.io No resources found + _kubectl -n kube-system delete rolebinding -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete rolebinding -l kubevirt.io No resources found + _kubectl -n kube-system delete roles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete roles -l kubevirt.io No resources found + _kubectl -n kube-system delete clusterroles -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete clusterroles -l kubevirt.io No resources found + _kubectl -n kube-system delete serviceaccounts -l kubevirt.io + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl -n kube-system delete serviceaccounts -l kubevirt.io No resources found ++ _kubectl -n kube-system get crd offlinevirtualmachines.kubevirt.io ++ wc -l ++ export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig ++ cluster/k8s-1.10.3/.kubectl -n kube-system get crd offlinevirtualmachines.kubevirt.io Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "offlinevirtualmachines.kubevirt.io" not found + '[' 0 -gt 0 ']' + sleep 2 + echo Done Done ./cluster/deploy.sh + source hack/common.sh ++++ dirname 'hack/common.sh[0]' +++ cd hack/../ +++ pwd ++ KUBEVIRT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt ++ OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out ++ VENDOR_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/vendor ++ CMD_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/cmd ++ TESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/tests ++ APIDOCS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/apidocs ++ MANIFESTS_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests ++ MANIFEST_TEMPLATES_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/templates/manifests ++ PYTHON_CLIENT_OUT_DIR=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/client-python ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_NUM_NODES=2 ++ '[' -z kubevirt-functional-tests-k8s-1.10.4-release ']' ++ provider_prefix=kubevirt-functional-tests-k8s-1.10.4-release2 ++ job_prefix=kubevirt-functional-tests-k8s-1.10.4-release2 +++ kubevirt_version +++ '[' -n '' ']' +++ '[' -d /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/.git ']' ++++ git describe --always --tags +++ echo v0.7.0-126-g81b07a4 ++ KUBEVIRT_VERSION=v0.7.0-126-g81b07a4 + source cluster/k8s-1.10.3/provider.sh ++ set -e ++ image=k8s-1.10.3@sha256:d6290260e7e6b84419984f12719cf592ccbe327373b8df76aa0481f8ec01d357 ++ source cluster/ephemeral-provider-common.sh +++ set -e +++ _cli='docker run --privileged --net=host --rm -v /var/run/docker.sock:/var/run/docker.sock kubevirtci/gocli@sha256:aa7f295a7908fa333ab5e98ef3af0bfafbabfd3cee2b83f9af47f722e3000f6a' + source hack/config.sh ++ unset binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig manifest_docker_prefix namespace ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ KUBEVIRT_PROVIDER=k8s-1.10.3 ++ source hack/config-default.sh source hack/config-k8s-1.10.3.sh +++ binaries='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virtctl cmd/fake-qemu-process cmd/virt-api cmd/subresource-access-test cmd/example-hook-sidecar' +++ docker_images='cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virt-api images/disks-images-provider images/vm-killer cmd/registry-disk-v1alpha images/cirros-registry-disk-demo images/fedora-cloud-registry-disk-demo images/alpine-registry-disk-demo cmd/subresource-access-test images/winrmcli cmd/example-hook-sidecar' +++ docker_prefix=kubevirt +++ docker_tag=latest +++ master_ip=192.168.200.2 +++ network_provider=flannel +++ namespace=kube-system ++ test -f hack/config-provider-k8s-1.10.3.sh ++ source hack/config-provider-k8s-1.10.3.sh +++ master_ip=127.0.0.1 +++ docker_tag=devel +++ kubeconfig=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubeconfig +++ kubectl=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/cluster/k8s-1.10.3/.kubectl +++ docker_prefix=localhost:32858/kubevirt +++ manifest_docker_prefix=registry:5000/kubevirt ++ test -f hack/config-local.sh ++ export binaries docker_images docker_prefix docker_tag manifest_templates master_ip network_provider kubeconfig namespace + echo 'Deploying ...' Deploying ... + [[ -z k8s-1.10.4-release ]] + [[ k8s-1.10.4-release =~ .*-dev ]] + [[ k8s-1.10.4-release =~ .*-release ]] + for manifest in '${MANIFESTS_OUT_DIR}/release/*' + [[ /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/demo-content.yaml =~ .*demo.* ]] + continue + for manifest in '${MANIFESTS_OUT_DIR}/release/*' + [[ /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/kubevirt.yaml =~ .*demo.* ]] + _kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/kubevirt.yaml + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests/release/kubevirt.yaml clusterrole.rbac.authorization.k8s.io "kubevirt.io:admin" created clusterrole.rbac.authorization.k8s.io "kubevirt.io:edit" created clusterrole.rbac.authorization.k8s.io "kubevirt.io:view" created serviceaccount "kubevirt-apiserver" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-apiserver" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-apiserver-auth-delegator" created rolebinding.rbac.authorization.k8s.io "kubevirt-apiserver" created role.rbac.authorization.k8s.io "kubevirt-apiserver" created clusterrole.rbac.authorization.k8s.io "kubevirt-apiserver" created clusterrole.rbac.authorization.k8s.io "kubevirt-controller" created serviceaccount "kubevirt-controller" created serviceaccount "kubevirt-privileged" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-controller" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-controller-cluster-admin" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-privileged-cluster-admin" created clusterrole.rbac.authorization.k8s.io "kubevirt.io:default" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt.io:default" created service "virt-api" created deployment.extensions "virt-api" created deployment.extensions "virt-controller" created daemonset.extensions "virt-handler" created customresourcedefinition.apiextensions.k8s.io "virtualmachineinstances.kubevirt.io" created customresourcedefinition.apiextensions.k8s.io "virtualmachineinstancereplicasets.kubevirt.io" created customresourcedefinition.apiextensions.k8s.io "virtualmachineinstancepresets.kubevirt.io" created customresourcedefinition.apiextensions.k8s.io "virtualmachines.kubevirt.io" created + _kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests/testing -R + export KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + KUBECONFIG=cluster/k8s-1.10.3/.kubeconfig + cluster/k8s-1.10.3/.kubectl create -f /var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/go/src/kubevirt.io/kubevirt/_out/manifests/testing -R persistentvolumeclaim "disk-alpine" created persistentvolume "host-path-disk-alpine" created persistentvolumeclaim "disk-custom" created persistentvolume "host-path-disk-custom" created daemonset.extensions "disks-images-provider" created serviceaccount "kubevirt-testing" created clusterrolebinding.rbac.authorization.k8s.io "kubevirt-testing-cluster-admin" created + [[ k8s-1.10.3 =~ os-* ]] + echo Done Done + namespaces=(kube-system default) + [[ kube-system != \k\u\b\e\-\s\y\s\t\e\m ]] + timeout=300 + sample=30 + for i in '${namespaces[@]}' + current_time=0 ++ kubectl get pods -n kube-system --no-headers ++ cluster/kubectl.sh get pods -n kube-system --no-headers ++ grep -v Running + '[' -n 'virt-api-7d79764579-frkfq 0/1 ContainerCreating 0 3s virt-api-7d79764579-k9ldt 0/1 ContainerCreating 0 3s virt-controller-7d57d96b65-grz6n 0/1 ContainerCreating 0 3s virt-controller-7d57d96b65-z7nc8 0/1 ContainerCreating 0 3s virt-handler-rmqjz 0/1 ContainerCreating 0 3s virt-handler-xqtr8 0/1 ContainerCreating 0 3s' ']' + echo 'Waiting for kubevirt pods to enter the Running state ...' Waiting for kubevirt pods to enter the Running state ... + kubectl get pods -n kube-system --no-headers + cluster/kubectl.sh get pods -n kube-system --no-headers + grep -v Running disks-images-provider-lggnr 0/1 ContainerCreating 0 1s disks-images-provider-z45tv 0/1 ContainerCreating 0 1s virt-api-7d79764579-frkfq 0/1 ContainerCreating 0 4s virt-api-7d79764579-k9ldt 0/1 ContainerCreating 0 4s virt-controller-7d57d96b65-grz6n 0/1 ContainerCreating 0 4s virt-controller-7d57d96b65-z7nc8 0/1 ContainerCreating 0 4s virt-handler-rmqjz 0/1 ContainerCreating 0 4s virt-handler-xqtr8 0/1 ContainerCreating 0 4s + sleep 30 + current_time=30 + '[' 30 -gt 300 ']' ++ kubectl get pods -n kube-system --no-headers ++ grep -v Running ++ cluster/kubectl.sh get pods -n kube-system --no-headers + '[' -n '' ']' + current_time=0 ++ kubectl get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ grep false ++ cluster/kubectl.sh get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers + '[' -n false ']' + echo 'Waiting for KubeVirt containers to become ready ...' Waiting for KubeVirt containers to become ready ... + kubectl get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers + grep false + cluster/kubectl.sh get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers false + sleep 30 + current_time=30 + '[' 30 -gt 300 ']' ++ kubectl get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ cluster/kubectl.sh get pods -n kube-system '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ grep false + '[' -n '' ']' + kubectl get pods -n kube-system + cluster/kubectl.sh get pods -n kube-system NAME READY STATUS RESTARTS AGE disks-images-provider-lggnr 1/1 Running 0 1m disks-images-provider-z45tv 1/1 Running 0 1m etcd-node01 1/1 Running 0 16m kube-apiserver-node01 1/1 Running 0 16m kube-controller-manager-node01 1/1 Running 0 16m kube-dns-86f4d74b45-sxt2j 3/3 Running 0 17m kube-flannel-ds-kgjlp 1/1 Running 0 17m kube-flannel-ds-ltq26 1/1 Running 0 17m kube-proxy-x2q6z 1/1 Running 0 17m kube-proxy-zqkpb 1/1 Running 0 17m kube-scheduler-node01 1/1 Running 0 16m virt-api-7d79764579-frkfq 1/1 Running 0 1m virt-api-7d79764579-k9ldt 1/1 Running 0 1m virt-controller-7d57d96b65-grz6n 1/1 Running 0 1m virt-controller-7d57d96b65-z7nc8 1/1 Running 0 1m virt-handler-rmqjz 1/1 Running 0 1m virt-handler-xqtr8 1/1 Running 0 1m + for i in '${namespaces[@]}' + current_time=0 ++ kubectl get pods -n default --no-headers ++ cluster/kubectl.sh get pods -n default --no-headers ++ grep -v Running No resources found. + '[' -n '' ']' + current_time=0 ++ kubectl get pods -n default '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ cluster/kubectl.sh get pods -n default '-ocustom-columns=status:status.containerStatuses[*].ready' --no-headers ++ grep false + '[' -n '' ']' + kubectl get pods -n default + cluster/kubectl.sh get pods -n default No resources found. + kubectl version + cluster/kubectl.sh version Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:17:39Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:05:37Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} + ginko_params='--ginkgo.noColor --junit-output=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/junit.xml' + [[ k8s-1.10.4-release =~ windows.* ]] + FUNC_TEST_ARGS='--ginkgo.noColor --junit-output=/var/lib/swarm/workspace/kubevirt-functional-tests-k8s-1.10.4-release/junit.xml' + make functest hack/dockerized "hack/build-func-tests.sh" sha256:b69a3f94b2043cd36cc41eb5d9446480e0a640962e468ab72c3cc51f2b89386a go version go1.10 linux/amd64 Waiting for rsyncd to be ready........................ go version go1.10 linux/amd64 Compiling tests... compiled tests.test hack/functests.sh Running Suite: Tests Suite ========================== Random Seed: 1532601439 Will run 148 of 148 specs • ------------------------------ • [SLOW TEST:8.550 seconds] VirtualMachineInstanceReplicaSet /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:46 should scale /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 to five, to six and then to zero replicas /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 ------------------------------ •• Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:38:21.386832Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:38:29 http: TLS handshake error from 10.244.0.1:39328: EOF level=info timestamp=2018-07-26T10:38:32.985200Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:38:33.012231Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:38:39 http: TLS handshake error from 10.244.0.1:39352: EOF level=info timestamp=2018-07-26T10:38:48.626723Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:38:49 http: TLS handshake error from 10.244.0.1:39376: EOF level=info timestamp=2018-07-26T10:38:51.448846Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:38:59 http: TLS handshake error from 10.244.0.1:39400: EOF 2018/07/26 10:39:09 http: TLS handshake error from 10.244.0.1:39424: EOF level=info timestamp=2018-07-26T10:39:18.763751Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:39:19 http: TLS handshake error from 10.244.0.1:39448: EOF level=info timestamp=2018-07-26T10:39:21.915923Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:39:29 http: TLS handshake error from 10.244.0.1:39472: EOF 2018/07/26 10:39:39 http: TLS handshake error from 10.244.0.1:39496: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:37:50 http: TLS handshake error from 10.244.1.1:41574: EOF level=info timestamp=2018-07-26T10:37:51.703521Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:38:00 http: TLS handshake error from 10.244.1.1:41580: EOF 2018/07/26 10:38:10 http: TLS handshake error from 10.244.1.1:41586: EOF 2018/07/26 10:38:20 http: TLS handshake error from 10.244.1.1:41592: EOF level=info timestamp=2018-07-26T10:38:21.766541Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:38:30 http: TLS handshake error from 10.244.1.1:41598: EOF 2018/07/26 10:38:40 http: TLS handshake error from 10.244.1.1:41604: EOF 2018/07/26 10:38:50 http: TLS handshake error from 10.244.1.1:41610: EOF level=info timestamp=2018-07-26T10:38:51.773488Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:39:00 http: TLS handshake error from 10.244.1.1:41616: EOF 2018/07/26 10:39:10 http: TLS handshake error from 10.244.1.1:41622: EOF 2018/07/26 10:39:20 http: TLS handshake error from 10.244.1.1:41628: EOF level=info timestamp=2018-07-26T10:39:21.827834Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:39:30 http: TLS handshake error from 10.244.1.1:41634: EOF Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:38:51.816990Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm996rz4l kind= uid=16904d77-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:38:51.948396Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwnm996rz4l\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwnm996rz4l" level=info timestamp=2018-07-26T10:38:53.873905Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:38:53.874103Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmiwnm99l4l9m" level=info timestamp=2018-07-26T10:38:53.874347Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:38:53.874627Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:39:28.635036Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99nxng2 kind= uid=2c8483bb-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:39:28.637995Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99nxng2 kind= uid=2c8483bb-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmiwnm99nxng2" level=info timestamp=2018-07-26T10:39:28.638330Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99nxng2 kind= uid=2c8483bb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:39:28.638605Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99nxng2 kind= uid=2c8483bb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:39:29.655577Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99q7n72 kind= uid=2d1eb305-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:39:29.655905Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99q7n72 kind= uid=2d1eb305-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmiwnm99q7n72" level=info timestamp=2018-07-26T10:39:29.656052Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99q7n72 kind= uid=2d1eb305-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:39:29.656311Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwnm99q7n72 kind= uid=2d1eb305-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:39:29.906412Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwnm99q7n72\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwnm99q7n72" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:39:28.587410Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind= uid=16904d77-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:28.587737Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind= uid=16904d77-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:39:28.587853Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind= uid=16904d77-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:28.686638Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind= uid=16904d77-90c0-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:39:28.687471Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind= uid=16904d77-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:28.842277Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:39:28.842555Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm996rz4l kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:29.559485Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:39:29.596366Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:29.596834Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:39:29.596949Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:29.724034Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:39:29.724632Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind= uid=17cc57e3-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:39:30.014796Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:39:30.014988Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwnm99l4l9m kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmiwnm99nxng2-qhxh6 Pod phase: Running level=info timestamp=2018-07-26T10:39:32.943200Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T10:39:32.943406Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T10:39:32.945075Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" Pod name: virt-launcher-testvmiwnm99q7n72-fb7wj Pod phase: Running level=info timestamp=2018-07-26T10:39:33.973988Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T10:39:33.974188Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T10:39:33.982055Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" ------------------------------ • Failure [122.135 seconds] VirtualMachineInstanceReplicaSet /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:46 should update readyReplicas once VMIs are up [It] /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:157 Timed out after 120.001s. Expected : 0 to equal : 2 /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:166 ------------------------------ STEP: Create a new VirtualMachineInstance replica set STEP: Scaling to 2 STEP: Checking the number of replicas STEP: Checking the number of ready replicas • [SLOW TEST:12.859 seconds] VirtualMachineInstanceReplicaSet /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:46 should remove VMIs once it is marked for deletion /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:169 ------------------------------ • ------------------------------ • [SLOW TEST:5.570 seconds] VirtualMachineInstanceReplicaSet /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:46 should not scale when paused and scale when resume /root/go/src/kubevirt.io/kubevirt/tests/replicaset_test.go:223 ------------------------------ • Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:38:51.448846Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:38:59 http: TLS handshake error from 10.244.0.1:39400: EOF 2018/07/26 10:39:09 http: TLS handshake error from 10.244.0.1:39424: EOF level=info timestamp=2018-07-26T10:39:18.763751Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:39:19 http: TLS handshake error from 10.244.0.1:39448: EOF level=info timestamp=2018-07-26T10:39:21.915923Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:39:29 http: TLS handshake error from 10.244.0.1:39472: EOF 2018/07/26 10:39:39 http: TLS handshake error from 10.244.0.1:39496: EOF level=info timestamp=2018-07-26T10:39:48.840044Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:39:49 http: TLS handshake error from 10.244.0.1:39524: EOF level=info timestamp=2018-07-26T10:39:52.030214Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:39:59 http: TLS handshake error from 10.244.0.1:39548: EOF 2018/07/26 10:40:09 http: TLS handshake error from 10.244.0.1:39572: EOF level=info timestamp=2018-07-26T10:40:18.972265Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:40:19 http: TLS handshake error from 10.244.0.1:39596: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:38:30 http: TLS handshake error from 10.244.1.1:41598: EOF 2018/07/26 10:38:40 http: TLS handshake error from 10.244.1.1:41604: EOF 2018/07/26 10:38:50 http: TLS handshake error from 10.244.1.1:41610: EOF level=info timestamp=2018-07-26T10:38:51.773488Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:39:00 http: TLS handshake error from 10.244.1.1:41616: EOF 2018/07/26 10:39:10 http: TLS handshake error from 10.244.1.1:41622: EOF 2018/07/26 10:39:20 http: TLS handshake error from 10.244.1.1:41628: EOF level=info timestamp=2018-07-26T10:39:21.827834Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:39:30 http: TLS handshake error from 10.244.1.1:41634: EOF 2018/07/26 10:39:40 http: TLS handshake error from 10.244.1.1:41640: EOF 2018/07/26 10:39:50 http: TLS handshake error from 10.244.1.1:41646: EOF level=info timestamp=2018-07-26T10:39:51.769956Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:40:00 http: TLS handshake error from 10.244.1.1:41652: EOF 2018/07/26 10:40:10 http: TLS handshake error from 10.244.1.1:41658: EOF 2018/07/26 10:40:20 http: TLS handshake error from 10.244.1.1:41664: EOF Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:40:01.163901Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:01.163987Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:01.164095Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:01.164154Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:02.851526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi65gs4hv4pn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:03.268558Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:03.268738Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4wq2dc" level=info timestamp=2018-07-26T10:40:03.268846Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:03.268951Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T10:40:04.722442Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetfph5d kind= uid=3fe67653-90c0-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T10:40:04.722656Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetfph5d" level=info timestamp=2018-07-26T10:40:05.824740Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:05.824967Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:05.941099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:06.238282Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:40:21.504095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:21.507941Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:21.508336Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:21.514212Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:21.514422Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:21.514566Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:21.556564Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:21.557106Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:21.557419Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:21.638043Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:21.638405Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:21.638558Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:21.799573Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:21.800131Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:21.800484Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" Pod name: virt-launcher-testvmi4lbwc-rlbwf Pod phase: Running goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4204ef0d0, 0xc420114000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c6a0, 0xc42000e470, 0xc4205c3fe0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096fc0, 0xc4200b6818, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420088080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096fc0, 0xc4200b6818, 0x13, 0xc420059f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42071d000) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba600, 0xc4200b8cd0, 0xc420349270, 0xc420349280, 0xc420471d80, 0xc4200c4120, 0x10f5e80, 0xc42000e470, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi65gs4zxwpk-7l2hx Pod phase: Failed level=info timestamp=2018-07-26T10:40:05.614769Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T10:40:05.618530Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T10:40:05.620704Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" caught signal virt-launcher exited with code 127 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 10:41:49 http: TLS handshake error from 10.244.0.1:39812: EOF level=info timestamp=2018-07-26T10:41:52.751932Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:41:59 http: TLS handshake error from 10.244.0.1:39836: EOF 2018/07/26 10:42:09 http: TLS handshake error from 10.244.0.1:39860: EOF 2018/07/26 10:42:19 http: TLS handshake error from 10.244.0.1:39884: EOF level=info timestamp=2018-07-26T10:42:19.574698Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:42:22.869724Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:42:29 http: TLS handshake error from 10.244.0.1:39908: EOF level=info timestamp=2018-07-26T10:42:30.558157Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:42:30.566100Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:42:39 http: TLS handshake error from 10.244.0.1:39932: EOF 2018/07/26 10:42:49 http: TLS handshake error from 10.244.0.1:39956: EOF level=info timestamp=2018-07-26T10:42:49.926520Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:42:52.990431Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:42:59 http: TLS handshake error from 10.244.0.1:39980: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:41:20 http: TLS handshake error from 10.244.1.1:41700: EOF level=info timestamp=2018-07-26T10:41:21.770815Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:41:30 http: TLS handshake error from 10.244.1.1:41706: EOF 2018/07/26 10:41:40 http: TLS handshake error from 10.244.1.1:41712: EOF 2018/07/26 10:41:50 http: TLS handshake error from 10.244.1.1:41718: EOF level=info timestamp=2018-07-26T10:41:51.745449Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:42:00 http: TLS handshake error from 10.244.1.1:41724: EOF 2018/07/26 10:42:10 http: TLS handshake error from 10.244.1.1:41730: EOF 2018/07/26 10:42:20 http: TLS handshake error from 10.244.1.1:41736: EOF level=info timestamp=2018-07-26T10:42:21.932735Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:42:30 http: TLS handshake error from 10.244.1.1:41742: EOF 2018/07/26 10:42:40 http: TLS handshake error from 10.244.1.1:41748: EOF 2018/07/26 10:42:50 http: TLS handshake error from 10.244.1.1:41754: EOF level=info timestamp=2018-07-26T10:42:51.753666Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:43:00 http: TLS handshake error from 10.244.1.1:41760: EOF Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:40:01.163901Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:01.163987Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:01.164095Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:01.164154Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:02.851526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi65gs4hv4pn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:03.268558Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:03.268738Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4wq2dc" level=info timestamp=2018-07-26T10:40:03.268846Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:03.268951Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T10:40:04.722442Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetfph5d kind= uid=3fe67653-90c0-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T10:40:04.722656Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetfph5d" level=info timestamp=2018-07-26T10:40:05.824740Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:05.824967Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:05.941099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:06.238282Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:40:22.763086Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:22.763432Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:24.044694Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:24.045722Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:24.046003Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:26.606822Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:26.607424Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:26.607700Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:31.728481Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:40:31.728707Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:40:31.728837Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:41.969716Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:40:42.025378Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:40:42.025687Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:40:42.025858Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi4lbwc-rlbwf Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4204ef0d0, 0xc420114000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c6a0, 0xc42000e470, 0xc4205c3fe0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096fc0, 0xc4200b6818, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420088080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096fc0, 0xc4200b6818, 0x13, 0xc420059f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42071d000) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba600, 0xc4200b8cd0, 0xc420349270, 0xc420349280, 0xc420471d80, 0xc4200c4120, 0x10f5e80, 0xc42000e470, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.856 seconds] Health Monitoring /root/go/src/kubevirt.io/kubevirt/tests/vmi_monitoring_test.go:37 A VirtualMachineInstance with a watchdog device /root/go/src/kubevirt.io/kubevirt/tests/vmi_monitoring_test.go:56 should be shut down when the watchdog expires [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_monitoring_test.go:57 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting a VirtualMachineInstance level=info timestamp=2018-07-26T10:40:06.310304Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi4lbwc-rlbwf" level=info timestamp=2018-07-26T10:40:21.391704Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi4lbwc-rlbwf" level=error timestamp=2018-07-26T10:40:21.626784Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 10:42:19 http: TLS handshake error from 10.244.0.1:39884: EOF level=info timestamp=2018-07-26T10:42:19.574698Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:42:22.869724Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:42:29 http: TLS handshake error from 10.244.0.1:39908: EOF level=info timestamp=2018-07-26T10:42:30.558157Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:42:30.566100Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:42:39 http: TLS handshake error from 10.244.0.1:39932: EOF 2018/07/26 10:42:49 http: TLS handshake error from 10.244.0.1:39956: EOF level=info timestamp=2018-07-26T10:42:49.926520Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:42:52.990431Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:42:59 http: TLS handshake error from 10.244.0.1:39980: EOF 2018/07/26 10:43:09 http: TLS handshake error from 10.244.0.1:40004: EOF 2018/07/26 10:43:19 http: TLS handshake error from 10.244.0.1:40028: EOF level=info timestamp=2018-07-26T10:43:20.092552Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:43:23.100248Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:41:40 http: TLS handshake error from 10.244.1.1:41712: EOF 2018/07/26 10:41:50 http: TLS handshake error from 10.244.1.1:41718: EOF level=info timestamp=2018-07-26T10:41:51.745449Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:42:00 http: TLS handshake error from 10.244.1.1:41724: EOF 2018/07/26 10:42:10 http: TLS handshake error from 10.244.1.1:41730: EOF 2018/07/26 10:42:20 http: TLS handshake error from 10.244.1.1:41736: EOF level=info timestamp=2018-07-26T10:42:21.932735Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:42:30 http: TLS handshake error from 10.244.1.1:41742: EOF 2018/07/26 10:42:40 http: TLS handshake error from 10.244.1.1:41748: EOF 2018/07/26 10:42:50 http: TLS handshake error from 10.244.1.1:41754: EOF level=info timestamp=2018-07-26T10:42:51.753666Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:43:00 http: TLS handshake error from 10.244.1.1:41760: EOF 2018/07/26 10:43:10 http: TLS handshake error from 10.244.1.1:41766: EOF 2018/07/26 10:43:20 http: TLS handshake error from 10.244.1.1:41772: EOF level=info timestamp=2018-07-26T10:43:21.684067Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:40:01.164095Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:01.164154Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:02.851526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi65gs4hv4pn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:03.268558Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:03.268738Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4wq2dc" level=info timestamp=2018-07-26T10:40:03.268846Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:03.268951Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T10:40:04.722442Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetfph5d kind= uid=3fe67653-90c0-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T10:40:04.722656Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetfph5d" level=info timestamp=2018-07-26T10:40:05.824740Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:05.824967Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:05.941099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:06.238282Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:43:06.543346Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:43:06.547293Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:43:23.484756Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:23.485005Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:23.485255Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:23.496401Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:23.496807Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:23.497010Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:23.537581Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:23.537944Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:23.538485Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:23.621935Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:23.622556Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:23.622813Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:23.785539Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:23.786016Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:23.786316Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" Pod name: virt-launcher-testvmibq6kh-pjrj6 Pod phase: Running goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278420, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191ec80, 0xc4200b7a78, 0xc4202ebc80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6de0, 0xc42000e0e0, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200ba280, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6de0, 0xc42000e0e0, 0x13, 0xc421959718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba980, 0xc4200b8910, 0xc421a50910, 0xc421a50920, 0xc421921d80, 0xc42192a0e0, 0x10f5e80, 0xc4200b7a78, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:47:51.398132Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:47:54.217095Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:47:59 http: TLS handshake error from 10.244.0.1:40700: EOF 2018/07/26 10:48:09 http: TLS handshake error from 10.244.0.1:40724: EOF 2018/07/26 10:48:19 http: TLS handshake error from 10.244.0.1:40748: EOF level=info timestamp=2018-07-26T10:48:21.521643Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:48:24.326941Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:48:29 http: TLS handshake error from 10.244.0.1:40772: EOF level=info timestamp=2018-07-26T10:48:30.219846Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:48:30.222828Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:48:39 http: TLS handshake error from 10.244.0.1:40796: EOF 2018/07/26 10:48:49 http: TLS handshake error from 10.244.0.1:40820: EOF level=info timestamp=2018-07-26T10:48:51.663627Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:48:54.441591Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:48:59 http: TLS handshake error from 10.244.0.1:40844: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T10:47:21.633748Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:47:21.804562Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:47:30 http: TLS handshake error from 10.244.1.1:41922: EOF 2018/07/26 10:47:40 http: TLS handshake error from 10.244.1.1:41928: EOF 2018/07/26 10:47:50 http: TLS handshake error from 10.244.1.1:41934: EOF level=info timestamp=2018-07-26T10:47:51.804737Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:48:00 http: TLS handshake error from 10.244.1.1:41940: EOF 2018/07/26 10:48:10 http: TLS handshake error from 10.244.1.1:41946: EOF 2018/07/26 10:48:20 http: TLS handshake error from 10.244.1.1:41952: EOF level=info timestamp=2018-07-26T10:48:21.806856Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:48:30 http: TLS handshake error from 10.244.1.1:41958: EOF 2018/07/26 10:48:40 http: TLS handshake error from 10.244.1.1:41964: EOF 2018/07/26 10:48:50 http: TLS handshake error from 10.244.1.1:41970: EOF level=info timestamp=2018-07-26T10:48:51.719373Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:49:00 http: TLS handshake error from 10.244.1.1:41976: EOF Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:40:01.164095Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:01.164154Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4hv4pn kind= uid=3fec3bf9-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:02.851526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi65gs4hv4pn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:03.268558Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:03.268738Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4wq2dc" level=info timestamp=2018-07-26T10:40:03.268846Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:03.268951Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T10:40:04.722442Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetfph5d kind= uid=3fe67653-90c0-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T10:40:04.722656Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetfph5d" level=info timestamp=2018-07-26T10:40:05.824740Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:05.824967Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:05.941099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:06.238282Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:43:06.543346Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:43:06.547293Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:43:24.748948Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:24.749330Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:26.030068Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:26.030555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:26.030818Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:28.591863Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:28.592358Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:28.592666Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:33.713495Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:43:33.714029Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:43:33.714342Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibq6kh" level=info timestamp=2018-07-26T10:43:43.954954Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:43:44.018655Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:43:44.018960Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:43:44.019075Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmibq6kh-pjrj6 Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278420, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191ec80, 0xc4200b7a78, 0xc4202ebc80, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6de0, 0xc42000e0e0, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200ba280, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6de0, 0xc42000e0e0, 0x13, 0xc421959718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba980, 0xc4200b8910, 0xc421a50910, 0xc421a50920, 0xc421921d80, 0xc42192a0e0, 0x10f5e80, 0xc4200b7a78, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [360.726 seconds] RegistryDisk /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:41 Starting and stopping the same VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:90 with ephemeral registry disk /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:91 should success multiple times [It] /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:92 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting the VirtualMachineInstance level=info timestamp=2018-07-26T10:43:06.811333Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmibq6kh-pjrj6" level=info timestamp=2018-07-26T10:43:23.253812Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmibq6kh-pjrj6" level=error timestamp=2018-07-26T10:43:23.607467Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 10:48:09 http: TLS handshake error from 10.244.0.1:40724: EOF 2018/07/26 10:48:19 http: TLS handshake error from 10.244.0.1:40748: EOF level=info timestamp=2018-07-26T10:48:21.521643Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:48:24.326941Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:48:29 http: TLS handshake error from 10.244.0.1:40772: EOF level=info timestamp=2018-07-26T10:48:30.219846Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:48:30.222828Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:48:39 http: TLS handshake error from 10.244.0.1:40796: EOF 2018/07/26 10:48:49 http: TLS handshake error from 10.244.0.1:40820: EOF level=info timestamp=2018-07-26T10:48:51.663627Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:48:54.441591Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:48:59 http: TLS handshake error from 10.244.0.1:40844: EOF 2018/07/26 10:49:09 http: TLS handshake error from 10.244.0.1:40868: EOF 2018/07/26 10:49:19 http: TLS handshake error from 10.244.0.1:40892: EOF level=info timestamp=2018-07-26T10:49:21.780961Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:47:40 http: TLS handshake error from 10.244.1.1:41928: EOF 2018/07/26 10:47:50 http: TLS handshake error from 10.244.1.1:41934: EOF level=info timestamp=2018-07-26T10:47:51.804737Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:48:00 http: TLS handshake error from 10.244.1.1:41940: EOF 2018/07/26 10:48:10 http: TLS handshake error from 10.244.1.1:41946: EOF 2018/07/26 10:48:20 http: TLS handshake error from 10.244.1.1:41952: EOF level=info timestamp=2018-07-26T10:48:21.806856Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:48:30 http: TLS handshake error from 10.244.1.1:41958: EOF 2018/07/26 10:48:40 http: TLS handshake error from 10.244.1.1:41964: EOF 2018/07/26 10:48:50 http: TLS handshake error from 10.244.1.1:41970: EOF level=info timestamp=2018-07-26T10:48:51.719373Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:49:00 http: TLS handshake error from 10.244.1.1:41976: EOF 2018/07/26 10:49:10 http: TLS handshake error from 10.244.1.1:41982: EOF 2018/07/26 10:49:20 http: TLS handshake error from 10.244.1.1:41988: EOF level=info timestamp=2018-07-26T10:49:21.693782Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:40:02.851526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi65gs4hv4pn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:03.268558Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:03.268738Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4wq2dc" level=info timestamp=2018-07-26T10:40:03.268846Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:03.268951Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T10:40:04.722442Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetfph5d kind= uid=3fe67653-90c0-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T10:40:04.722656Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetfph5d" level=info timestamp=2018-07-26T10:40:05.824740Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:05.824967Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:05.941099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:06.238282Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:43:06.543346Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:43:06.547293Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:49:07.266838Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:49:07.275371Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:49:22.977597Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:22.977929Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:22.978146Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:22.998810Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:22.999277Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:22.999480Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:23.040054Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:23.040581Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:23.040782Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:23.121795Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:23.122321Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:23.122649Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:23.283523Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:23.283954Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:23.284380Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" Pod name: virt-launcher-testvmi5hqnm-r9hfb Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d20b0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420592040, 0xc4206aa5a8, 0xc42044ae20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e20c0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420405080, 0x43fe6c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e20c0, 0xc42000e078, 0x13, 0xc4201d4f18, 0x3, 0x3, 0x469301, 0x11a2dc0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420348040, 0xc4200e4500, 0xc42196f8b0, 0xc42196f8c0, 0xc421939300, 0xc420554180, 0x10f5e80, 0xc4206aa5a8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:49:51.895770Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:49:54.676745Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:49:59 http: TLS handshake error from 10.244.0.1:40988: EOF 2018/07/26 10:50:09 http: TLS handshake error from 10.244.0.1:41012: EOF 2018/07/26 10:50:19 http: TLS handshake error from 10.244.0.1:41036: EOF level=info timestamp=2018-07-26T10:50:22.034013Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:50:24.803783Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:50:29 http: TLS handshake error from 10.244.0.1:41060: EOF level=info timestamp=2018-07-26T10:50:30.227627Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:50:30.230152Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:50:39 http: TLS handshake error from 10.244.0.1:41084: EOF 2018/07/26 10:50:49 http: TLS handshake error from 10.244.0.1:41108: EOF level=info timestamp=2018-07-26T10:50:52.224704Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:50:54.941996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:50:59 http: TLS handshake error from 10.244.0.1:41132: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:49:20 http: TLS handshake error from 10.244.1.1:41988: EOF level=info timestamp=2018-07-26T10:49:21.693782Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:49:30 http: TLS handshake error from 10.244.1.1:41994: EOF 2018/07/26 10:49:40 http: TLS handshake error from 10.244.1.1:42000: EOF 2018/07/26 10:49:50 http: TLS handshake error from 10.244.1.1:42006: EOF level=info timestamp=2018-07-26T10:49:51.780885Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:50:00 http: TLS handshake error from 10.244.1.1:42012: EOF 2018/07/26 10:50:10 http: TLS handshake error from 10.244.1.1:42018: EOF 2018/07/26 10:50:20 http: TLS handshake error from 10.244.1.1:42024: EOF level=info timestamp=2018-07-26T10:50:21.764858Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:50:30 http: TLS handshake error from 10.244.1.1:42030: EOF 2018/07/26 10:50:40 http: TLS handshake error from 10.244.1.1:42036: EOF 2018/07/26 10:50:50 http: TLS handshake error from 10.244.1.1:42042: EOF level=info timestamp=2018-07-26T10:50:51.702833Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:51:00 http: TLS handshake error from 10.244.1.1:42048: EOF Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:40:02.851526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi65gs4hv4pn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi65gs4hv4pn" level=info timestamp=2018-07-26T10:40:03.268558Z pos=vm.go:459 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Looking for VirtualMachineInstance Ref" level=error timestamp=2018-07-26T10:40:03.268738Z pos=vm.go:462 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Cant find the matching VM for VirtualMachineInstance: testvmi65gs4wq2dc" level=info timestamp=2018-07-26T10:40:03.268846Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:03.268951Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi65gs4wq2dc kind= uid=412b1bda-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=error timestamp=2018-07-26T10:40:04.722442Z pos=replicaset.go:230 component=virt-controller service=http namespace=kubevirt-test-default name=replicasetfph5d kind= uid=3fe67653-90c0-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the replicaset status failed." level=info timestamp=2018-07-26T10:40:04.722656Z pos=replicaset.go:137 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstancereplicasets.kubevirt.io \"replicasetfph5d\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstanceReplicaSet kubevirt-test-default/replicasetfph5d" level=info timestamp=2018-07-26T10:40:05.824740Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:40:05.824967Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4lbwc kind= uid=42b2e8fb-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:40:05.941099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:40:06.238282Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4lbwc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4lbwc" level=info timestamp=2018-07-26T10:43:06.543346Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:43:06.547293Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibq6kh kind= uid=ae696fb2-90c0-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:49:07.266838Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:49:07.275371Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:34:00.550030Z pos=virt-handler.go:87 component=virt-handler hostname=node01 level=info timestamp=2018-07-26T10:34:00.555715Z pos=vm.go:210 component=virt-handler msg="Starting virt-handler controller." level=info timestamp=2018-07-26T10:34:00.556816Z pos=cache.go:151 component=virt-handler msg="Synchronizing domains" level=info timestamp=2018-07-26T10:34:00.663631Z pos=device_controller.go:133 component=virt-handler msg="Starting device plugin controller" level=info timestamp=2018-07-26T10:34:01.646310Z pos=device_controller.go:127 component=virt-handler msg="tun device plugin started" level=info timestamp=2018-07-26T10:34:01.770521Z pos=device_controller.go:127 component=virt-handler msg="kvm device plugin started" Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:49:24.248621Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:24.248823Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:25.529714Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:25.530268Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:25.530870Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:28.091943Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:28.092805Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:28.093090Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:33.214940Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:49:33.216273Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:49:33.216843Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5hqnm" level=info timestamp=2018-07-26T10:49:43.458002Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:49:43.493148Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:49:43.501140Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:49:43.501363Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5hqnm kind= uid=856a8cae-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi5hqnm-r9hfb Pod phase: Failed goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d20b0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420592040, 0xc4206aa5a8, 0xc42044ae20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e20c0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420405080, 0x43fe6c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e20c0, 0xc42000e078, 0x13, 0xc4201d4f18, 0x3, 0x3, 0x469301, 0x11a2dc0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420348040, 0xc4200e4500, 0xc42196f8b0, 0xc42196f8c0, 0xc421939300, 0xc420554180, 0x10f5e80, 0xc4206aa5a8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [120.696 seconds] RegistryDisk /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:41 Starting a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:111 with ephemeral registry disk /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:112 should not modify the spec on status update [It] /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:113 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting the VirtualMachineInstance level=info timestamp=2018-07-26T10:49:07.625584Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi5hqnm-r9hfb" level=info timestamp=2018-07-26T10:49:22.832026Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi5hqnm-r9hfb" level=error timestamp=2018-07-26T10:49:23.089462Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 10:50:39 http: TLS handshake error from 10.244.0.1:41084: EOF 2018/07/26 10:50:49 http: TLS handshake error from 10.244.0.1:41108: EOF level=info timestamp=2018-07-26T10:50:52.224704Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:50:54.941996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:50:59 http: TLS handshake error from 10.244.0.1:41132: EOF 2018/07/26 10:51:09 http: TLS handshake error from 10.244.0.1:41156: EOF 2018/07/26 10:51:19 http: TLS handshake error from 10.244.0.1:41180: EOF level=info timestamp=2018-07-26T10:51:22.313737Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:51:25.133010Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:51:29 http: TLS handshake error from 10.244.0.1:41204: EOF 2018/07/26 10:51:39 http: TLS handshake error from 10.244.0.1:41228: EOF 2018/07/26 10:51:49 http: TLS handshake error from 10.244.0.1:41252: EOF level=info timestamp=2018-07-26T10:51:52.436077Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:51:55.233963Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:51:59 http: TLS handshake error from 10.244.0.1:41276: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:50:20 http: TLS handshake error from 10.244.1.1:42024: EOF level=info timestamp=2018-07-26T10:50:21.764858Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:50:30 http: TLS handshake error from 10.244.1.1:42030: EOF 2018/07/26 10:50:40 http: TLS handshake error from 10.244.1.1:42036: EOF 2018/07/26 10:50:50 http: TLS handshake error from 10.244.1.1:42042: EOF level=info timestamp=2018-07-26T10:50:51.702833Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:51:00 http: TLS handshake error from 10.244.1.1:42048: EOF 2018/07/26 10:51:10 http: TLS handshake error from 10.244.1.1:42054: EOF 2018/07/26 10:51:20 http: TLS handshake error from 10.244.1.1:42060: EOF level=info timestamp=2018-07-26T10:51:21.768371Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:51:30 http: TLS handshake error from 10.244.1.1:42066: EOF 2018/07/26 10:51:40 http: TLS handshake error from 10.244.1.1:42072: EOF 2018/07/26 10:51:50 http: TLS handshake error from 10.244.1.1:42078: EOF level=info timestamp=2018-07-26T10:51:51.701904Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:52:00 http: TLS handshake error from 10.244.1.1:42084: EOF Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-z7nc8 Pod phase: Running level=info timestamp=2018-07-26T10:51:07.979446Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwbmn kind= uid=cd5ef688-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:51:07.982382Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwbmn kind= uid=cd5ef688-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:51:08.020297Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigtttn kind= uid=cd642e44-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:51:08.020619Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigtttn kind= uid=cd642e44-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:51:08.066312Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizh97j kind= uid=cd69b37b-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:51:08.066666Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizh97j kind= uid=cd69b37b-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:51:08.111938Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8md6z kind= uid=cd7206e6-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:51:08.112263Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8md6z kind= uid=cd7206e6-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:51:08.211820Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigtttn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigtttn" level=info timestamp=2018-07-26T10:51:08.218540Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:51:08.218780Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:51:08.260320Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwbmn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwbmn" level=info timestamp=2018-07-26T10:51:09.196980Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8md6z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8md6z" level=info timestamp=2018-07-26T10:51:09.391245Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4smw9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:09.999784Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8md6z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8md6z" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T10:51:25.219210Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:25.219997Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:26.500928Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:26.501707Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:51:49.653990Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizh97j kind= uid=cd69b37b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:49.657243Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizh97j kind= uid=cd69b37b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:49.657427Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizh97j kind= uid=cd69b37b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:50.087357Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi8md6z kind= uid=cd7206e6-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:50.112136Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi8md6z kind= uid=cd7206e6-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:50.113720Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi8md6z kind= uid=cd7206e6-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:50.113922Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi8md6z kind= uid=cd7206e6-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:50.228057Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigtttn kind= uid=cd642e44-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:50.260608Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigtttn kind= uid=cd642e44-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:50.260932Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigtttn kind= uid=cd642e44-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:50.261302Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigtttn kind= uid=cd642e44-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:51.697785Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitwbmn kind= uid=cd5ef688-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:51.736495Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitwbmn kind= uid=cd5ef688-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:51.736842Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitwbmn kind= uid=cd5ef688-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:51.737007Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitwbmn kind= uid=cd5ef688-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi4smw9-kr78k Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42024a0e0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191c040, 0xc42000e420, 0xc420212800, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b0120, 0xc42000e0a0, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4203d6180, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b0120, 0xc42000e0a0, 0x13, 0xc42193b718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42027d500) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420480040, 0xc4200a8410, 0xc421907dc0, 0xc421907dd0, 0xc421936300, 0xc421910d20, 0x10f5e80, 0xc42000e420, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmi8md6z-k5jrf Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40b0, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4215e2040, 0xc420c7ae28, 0xc420ca6f20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6120, 0xc4200b60a0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc420406140, 0x705f736922202020, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6120, 0xc4200b60a0, 0x13, 0xc420a41718, 0x3, 0x3, 0x2020202020202020, 0x20200a7b20202020, 0x2020202020202020) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406040, 0xc4200b84b0, 0xc420ab10c0, 0xc420ab10d0, 0xc420a36300, 0xc4215e8140, 0x10f5e80, 0xc420c7ae28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigtttn-29fhz Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202780f0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202ec080, 0xc420670128, 0xc4206ba920, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e0a8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200ba040, 0x43fe6c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e0a8, 0x13, 0xc420016f18, 0x3, 0x3, 0x469301, 0x11a2dc0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420348040, 0xc420194370, 0xc421933cb0, 0xc421933cc0, 0xc4207d0300, 0xc42029d980, 0x10f5e80, 0xc420670128, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmitwbmn-8r27b Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202783f0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420c28720, 0xc4200b64b8, 0xc4202ec3a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6cc0, 0xc4200b60d0, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4202aa7c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6cc0, 0xc4200b60d0, 0x13, 0xc4206d1f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4206dc040) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa4c0, 0xc4200b8870, 0xc420d28da0, 0xc420d28db0, 0xc420c27d80, 0xc420c300e0, 0x10f5e80, 0xc4200b64b8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmizh97j-k7lgm Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201ca0d0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420bb00a0, 0xc42000f6a0, 0xc420212a40, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c540, 0xc4200d2070, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420318940, 0x2020200a202c2272, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c540, 0xc4200d2070, 0x13, 0xc420bcff18, 0x3, 0x3, 0x202020200a7b2020, 0x6e65762220202020, 0x666572705f726f64) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420318140, 0xc4200a8410, 0xc421600b00, 0xc421600b10, 0xc420be4300, 0xc420ba6320, 0x10f5e80, 0xc42000f6a0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [61.690 seconds] RegistryDisk /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:41 Starting multiple VMIs /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:129 with ephemeral registry disk /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:130 should success [It] /root/go/src/kubevirt.io/kubevirt/tests/registry_disk_test.go:131 Timed out after 30.009s. Timed out waiting for VMI to enter Running phase Expected : false to equal : true /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1049 ------------------------------ STEP: Starting a VirtualMachineInstance STEP: Starting a VirtualMachineInstance STEP: Starting a VirtualMachineInstance STEP: Starting a VirtualMachineInstance STEP: Starting a VirtualMachineInstance level=info timestamp=2018-07-26T10:51:08.494394Z pos=utils.go:254 component=tests msg="Created virtual machine pod virt-launcher-testvmitwbmn-8r27b" level=info timestamp=2018-07-26T10:51:31.159184Z pos=utils.go:254 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmitwbmn-8r27b" level=error timestamp=2018-07-26T10:51:32.863969Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="unexpected EOF" level=error timestamp=2018-07-26T10:51:33.101867Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:33.268213Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:33.465906Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:33.663757Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:33.886786Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:34.064313Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:34.465377Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:35.076046Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:35.264430Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" level=error timestamp=2018-07-26T10:51:36.474572Z pos=utils.go:252 component=tests reason="unexpected warning event received" msg="connection is shut down" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:51:25.133010Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:51:29 http: TLS handshake error from 10.244.0.1:41204: EOF 2018/07/26 10:51:39 http: TLS handshake error from 10.244.0.1:41228: EOF 2018/07/26 10:51:49 http: TLS handshake error from 10.244.0.1:41252: EOF level=info timestamp=2018-07-26T10:51:52.436077Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:51:55.233963Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:51:59 http: TLS handshake error from 10.244.0.1:41276: EOF 2018/07/26 10:52:09 http: TLS handshake error from 10.244.0.1:41300: EOF 2018/07/26 10:52:19 http: TLS handshake error from 10.244.0.1:41324: EOF level=info timestamp=2018-07-26T10:52:22.530999Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:52:25.288519Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:52:29 http: TLS handshake error from 10.244.0.1:41348: EOF level=info timestamp=2018-07-26T10:52:33.287039Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:52:33.294873Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:52:39 http: TLS handshake error from 10.244.0.1:41372: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T10:50:51.702833Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:51:00 http: TLS handshake error from 10.244.1.1:42048: EOF 2018/07/26 10:51:10 http: TLS handshake error from 10.244.1.1:42054: EOF 2018/07/26 10:51:20 http: TLS handshake error from 10.244.1.1:42060: EOF level=info timestamp=2018-07-26T10:51:21.768371Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:51:30 http: TLS handshake error from 10.244.1.1:42066: EOF 2018/07/26 10:51:40 http: TLS handshake error from 10.244.1.1:42072: EOF 2018/07/26 10:51:50 http: TLS handshake error from 10.244.1.1:42078: EOF level=info timestamp=2018-07-26T10:51:51.701904Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:52:00 http: TLS handshake error from 10.244.1.1:42084: EOF 2018/07/26 10:52:10 http: TLS handshake error from 10.244.1.1:42090: EOF 2018/07/26 10:52:20 http: TLS handshake error from 10.244.1.1:42094: EOF level=info timestamp=2018-07-26T10:52:21.826058Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:52:30 http: TLS handshake error from 10.244.1.1:42096: EOF 2018/07/26 10:52:40 http: TLS handshake error from 10.244.1.1:42102: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 level=info timestamp=2018-07-26T10:52:30.672968Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmInformer" level=info timestamp=2018-07-26T10:52:30.679487Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmiInformer" level=info timestamp=2018-07-26T10:52:30.679571Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer kubeVirtPodInformer" level=info timestamp=2018-07-26T10:52:30.679602Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer kubeVirtNodeInformer" level=info timestamp=2018-07-26T10:52:30.679627Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmiPresetInformer" level=info timestamp=2018-07-26T10:52:30.679651Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmirsInformer" level=info timestamp=2018-07-26T10:52:30.679686Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer configMapInformer" level=info timestamp=2018-07-26T10:52:30.679833Z pos=vm.go:85 component=virt-controller service=http msg="Starting VirtualMachine controller." level=info timestamp=2018-07-26T10:52:30.704962Z pos=node.go:104 component=virt-controller service=http msg="Starting node controller." level=info timestamp=2018-07-26T10:52:30.705087Z pos=vmi.go:129 component=virt-controller service=http msg="Starting vmi controller." level=info timestamp=2018-07-26T10:52:30.705145Z pos=replicaset.go:111 component=virt-controller service=http msg="Starting VirtualMachineInstanceReplicaSet controller." level=info timestamp=2018-07-26T10:52:30.705193Z pos=preset.go:71 component=virt-controller service=http msg="Starting Virtual Machine Initializer." level=info timestamp=2018-07-26T10:52:31.063094Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:52:31.069099Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:52:46.760972Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:46.761510Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:46.761767Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:46.782607Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:46.783083Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:46.783551Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:46.824645Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:46.825087Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:46.825438Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:46.906350Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:46.906754Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:46.907094Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:47.067578Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:47.067757Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:47.067919Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" Pod name: virt-launcher-testvmijml29-l28gk Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203000c0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c080, 0xc42066ce20, 0xc42068eaa0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c3c0, 0xc42000e080, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200ba280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c3c0, 0xc42000e080, 0x13, 0xc420059f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420621d40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4204a2140, 0xc4203a6410, 0xc420672460, 0xc420672470, 0xc4203e0300, 0xc42000c180, 0x10f5e80, 0xc42066ce20, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 10:54:09 http: TLS handshake error from 10.244.0.1:41588: EOF 2018/07/26 10:54:19 http: TLS handshake error from 10.244.0.1:41612: EOF level=info timestamp=2018-07-26T10:54:23.124022Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:54:25.738991Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:54:29 http: TLS handshake error from 10.244.0.1:41636: EOF 2018/07/26 10:54:39 http: TLS handshake error from 10.244.0.1:41660: EOF 2018/07/26 10:54:49 http: TLS handshake error from 10.244.0.1:41684: EOF level=info timestamp=2018-07-26T10:54:53.252505Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:54:55.873640Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:54:59 http: TLS handshake error from 10.244.0.1:41708: EOF 2018/07/26 10:55:09 http: TLS handshake error from 10.244.0.1:41732: EOF 2018/07/26 10:55:19 http: TLS handshake error from 10.244.0.1:41756: EOF level=info timestamp=2018-07-26T10:55:23.439009Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:55:25.989503Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:55:29 http: TLS handshake error from 10.244.0.1:41780: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:53:50 http: TLS handshake error from 10.244.1.1:42144: EOF level=info timestamp=2018-07-26T10:53:51.704107Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:54:00 http: TLS handshake error from 10.244.1.1:42150: EOF 2018/07/26 10:54:10 http: TLS handshake error from 10.244.1.1:42156: EOF 2018/07/26 10:54:20 http: TLS handshake error from 10.244.1.1:42162: EOF level=info timestamp=2018-07-26T10:54:21.757421Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:54:30 http: TLS handshake error from 10.244.1.1:42168: EOF 2018/07/26 10:54:40 http: TLS handshake error from 10.244.1.1:42174: EOF 2018/07/26 10:54:50 http: TLS handshake error from 10.244.1.1:42180: EOF level=info timestamp=2018-07-26T10:54:51.803453Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:55:00 http: TLS handshake error from 10.244.1.1:42186: EOF 2018/07/26 10:55:10 http: TLS handshake error from 10.244.1.1:42192: EOF 2018/07/26 10:55:20 http: TLS handshake error from 10.244.1.1:42198: EOF level=info timestamp=2018-07-26T10:55:21.780996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:55:30 http: TLS handshake error from 10.244.1.1:42204: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:34:00.403166Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 level=info timestamp=2018-07-26T10:52:30.672968Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmInformer" level=info timestamp=2018-07-26T10:52:30.679487Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmiInformer" level=info timestamp=2018-07-26T10:52:30.679571Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer kubeVirtPodInformer" level=info timestamp=2018-07-26T10:52:30.679602Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer kubeVirtNodeInformer" level=info timestamp=2018-07-26T10:52:30.679627Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmiPresetInformer" level=info timestamp=2018-07-26T10:52:30.679651Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer vmirsInformer" level=info timestamp=2018-07-26T10:52:30.679686Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer configMapInformer" level=info timestamp=2018-07-26T10:52:30.679833Z pos=vm.go:85 component=virt-controller service=http msg="Starting VirtualMachine controller." level=info timestamp=2018-07-26T10:52:30.704962Z pos=node.go:104 component=virt-controller service=http msg="Starting node controller." level=info timestamp=2018-07-26T10:52:30.705087Z pos=vmi.go:129 component=virt-controller service=http msg="Starting vmi controller." level=info timestamp=2018-07-26T10:52:30.705145Z pos=replicaset.go:111 component=virt-controller service=http msg="Starting VirtualMachineInstanceReplicaSet controller." level=info timestamp=2018-07-26T10:52:30.705193Z pos=preset.go:71 component=virt-controller service=http msg="Starting Virtual Machine Initializer." level=info timestamp=2018-07-26T10:52:31.063094Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:52:31.069099Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:52:48.033371Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:48.033627Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:49.314546Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:49.315062Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:49.315385Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:51.876418Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:51.876866Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:51.877091Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:52:56.998033Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:52:56.998611Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:52:56.998918Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:53:07.240984Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:53:07.276871Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:53:07.277292Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:53:07.277413Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmijml29-l28gk Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203000c0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c080, 0xc42066ce20, 0xc42068eaa0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c3c0, 0xc42000e080, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200ba280, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c3c0, 0xc42000e080, 0x13, 0xc420059f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420621d40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4204a2140, 0xc4203a6410, 0xc420672460, 0xc420672470, 0xc4203e0300, 0xc42000c180, 0x10f5e80, 0xc42066ce20, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [201.644 seconds] LeaderElection /root/go/src/kubevirt.io/kubevirt/tests/controller_leader_election_test.go:43 Start a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/controller_leader_election_test.go:53 when the controller pod is not running /root/go/src/kubevirt.io/kubevirt/tests/controller_leader_election_test.go:54 should success [It] /root/go/src/kubevirt.io/kubevirt/tests/controller_leader_election_test.go:55 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Destroying the leading controller pod STEP: Starting a new VirtualMachineInstance level=info timestamp=2018-07-26T10:52:31.036223Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmijml29-l28gk" level=info timestamp=2018-07-26T10:52:46.585643Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmijml29-l28gk" level=error timestamp=2018-07-26T10:52:46.864607Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" •• ------------------------------ • [SLOW TEST:5.338 seconds] Subresource Api /root/go/src/kubevirt.io/kubevirt/tests/subresource_api_test.go:37 Rbac Authorization For Version Command /root/go/src/kubevirt.io/kubevirt/tests/subresource_api_test.go:63 with authenticated user /root/go/src/kubevirt.io/kubevirt/tests/subresource_api_test.go:66 should be allowed to access subresource version endpoint /root/go/src/kubevirt.io/kubevirt/tests/subresource_api_test.go:67 ------------------------------ •• Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 10:55:09 http: TLS handshake error from 10.244.0.1:41732: EOF 2018/07/26 10:55:19 http: TLS handshake error from 10.244.0.1:41756: EOF level=info timestamp=2018-07-26T10:55:23.439009Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:55:25.989503Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:55:29 http: TLS handshake error from 10.244.0.1:41780: EOF level=info timestamp=2018-07-26T10:55:33.339084Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:55:33.360274Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:55:35.550616Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2/namespaces/default/virtualmachineinstances/fake/test proto=HTTP/2.0 statusCode=200 contentLength=0 2018/07/26 10:55:39 http: TLS handshake error from 10.244.0.1:41804: EOF level=info timestamp=2018-07-26T10:55:43.955980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2/version proto=HTTP/2.0 statusCode=200 contentLength=247 level=info timestamp=2018-07-26T10:55:49.173664Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2/version proto=HTTP/2.0 statusCode=200 contentLength=247 2018/07/26 10:55:49 http: TLS handshake error from 10.244.0.1:41828: EOF level=info timestamp=2018-07-26T10:55:53.539283Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:55:56.090382Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:55:59 http: TLS handshake error from 10.244.0.1:41852: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:54:20 http: TLS handshake error from 10.244.1.1:42162: EOF level=info timestamp=2018-07-26T10:54:21.757421Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:54:30 http: TLS handshake error from 10.244.1.1:42168: EOF 2018/07/26 10:54:40 http: TLS handshake error from 10.244.1.1:42174: EOF 2018/07/26 10:54:50 http: TLS handshake error from 10.244.1.1:42180: EOF level=info timestamp=2018-07-26T10:54:51.803453Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:55:00 http: TLS handshake error from 10.244.1.1:42186: EOF 2018/07/26 10:55:10 http: TLS handshake error from 10.244.1.1:42192: EOF 2018/07/26 10:55:20 http: TLS handshake error from 10.244.1.1:42198: EOF level=info timestamp=2018-07-26T10:55:21.780996Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:55:30 http: TLS handshake error from 10.244.1.1:42204: EOF 2018/07/26 10:55:40 http: TLS handshake error from 10.244.1.1:42214: EOF 2018/07/26 10:55:50 http: TLS handshake error from 10.244.1.1:42224: EOF level=info timestamp=2018-07-26T10:55:51.817905Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:56:00 http: TLS handshake error from 10.244.1.1:42230: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:52:30.679686Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer configMapInformer" level=info timestamp=2018-07-26T10:52:30.679833Z pos=vm.go:85 component=virt-controller service=http msg="Starting VirtualMachine controller." level=info timestamp=2018-07-26T10:52:30.704962Z pos=node.go:104 component=virt-controller service=http msg="Starting node controller." level=info timestamp=2018-07-26T10:52:30.705087Z pos=vmi.go:129 component=virt-controller service=http msg="Starting vmi controller." level=info timestamp=2018-07-26T10:52:30.705145Z pos=replicaset.go:111 component=virt-controller service=http msg="Starting VirtualMachineInstanceReplicaSet controller." level=info timestamp=2018-07-26T10:52:30.705193Z pos=preset.go:71 component=virt-controller service=http msg="Starting Virtual Machine Initializer." level=info timestamp=2018-07-26T10:52:31.063094Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:52:31.069099Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:31.584591Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijml29\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmijml29, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: fe948d8f-90c1-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:55:50.753056Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:50.753344Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.072964Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4gt87\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi4gt87, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 75adfd4d-90c2-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4gt87" level=info timestamp=2018-07-26T10:55:51.170980Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:51.171125Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.254846Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:56:05.675561Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=error timestamp=2018-07-26T10:56:05.698601Z pos=vm.go:404 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="Updating the VirtualMachineInstance status failed." level=info timestamp=2018-07-26T10:56:05.698836Z pos=vm.go:251 component=virt-handler reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:05.699148Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:05.699504Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:05.699756Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:05.710050Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:05.710450Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:05.710697Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:05.751623Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:05.752001Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:05.752376Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:05.832954Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:05.833503Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:05.833772Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" Pod name: virt-launcher-testvmihhxg2-tls6h Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202763a0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42066fc00, 0xc4200b7ef8, 0xc420cf4ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e29c0, 0xc42000e0c0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4200ba080, 0x202c2234413a3036, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e29c0, 0xc42000e0c0, 0x13, 0xc420cdef18, 0x3, 0x3, 0x64744c2959545028, 0x202020200a202c22, 0x7665642220202020) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202a83c0, 0xc4200e4960, 0xc421619fb0, 0xc421619fc0, 0xc420677e00, 0xc42067e0e0, 0x10f5e80, 0xc4200b7ef8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:57:30.452890Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:57:39 http: TLS handshake error from 10.244.0.1:42092: EOF 2018/07/26 10:57:49 http: TLS handshake error from 10.244.0.1:42116: EOF level=info timestamp=2018-07-26T10:57:54.218133Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:57:56.513240Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:57:59 http: TLS handshake error from 10.244.0.1:42140: EOF 2018/07/26 10:58:09 http: TLS handshake error from 10.244.0.1:42164: EOF 2018/07/26 10:58:19 http: TLS handshake error from 10.244.0.1:42188: EOF level=info timestamp=2018-07-26T10:58:24.348203Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:58:26.609441Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:58:29 http: TLS handshake error from 10.244.0.1:42212: EOF level=info timestamp=2018-07-26T10:58:30.458753Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:58:30.462790Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:58:39 http: TLS handshake error from 10.244.0.1:42236: EOF 2018/07/26 10:58:49 http: TLS handshake error from 10.244.0.1:42260: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 10:57:10 http: TLS handshake error from 10.244.1.1:42272: EOF 2018/07/26 10:57:20 http: TLS handshake error from 10.244.1.1:42278: EOF level=info timestamp=2018-07-26T10:57:21.631928Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:57:21.951549Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:57:30 http: TLS handshake error from 10.244.1.1:42284: EOF 2018/07/26 10:57:40 http: TLS handshake error from 10.244.1.1:42290: EOF 2018/07/26 10:57:50 http: TLS handshake error from 10.244.1.1:42296: EOF level=info timestamp=2018-07-26T10:57:51.718227Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:58:00 http: TLS handshake error from 10.244.1.1:42302: EOF 2018/07/26 10:58:10 http: TLS handshake error from 10.244.1.1:42308: EOF 2018/07/26 10:58:20 http: TLS handshake error from 10.244.1.1:42314: EOF level=info timestamp=2018-07-26T10:58:21.818741Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:58:30 http: TLS handshake error from 10.244.1.1:42320: EOF 2018/07/26 10:58:40 http: TLS handshake error from 10.244.1.1:42326: EOF 2018/07/26 10:58:50 http: TLS handshake error from 10.244.1.1:42332: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:52:30.679686Z pos=virtinformers.go:104 component=virt-controller service=http msg="STARTING informer configMapInformer" level=info timestamp=2018-07-26T10:52:30.679833Z pos=vm.go:85 component=virt-controller service=http msg="Starting VirtualMachine controller." level=info timestamp=2018-07-26T10:52:30.704962Z pos=node.go:104 component=virt-controller service=http msg="Starting node controller." level=info timestamp=2018-07-26T10:52:30.705087Z pos=vmi.go:129 component=virt-controller service=http msg="Starting vmi controller." level=info timestamp=2018-07-26T10:52:30.705145Z pos=replicaset.go:111 component=virt-controller service=http msg="Starting VirtualMachineInstanceReplicaSet controller." level=info timestamp=2018-07-26T10:52:30.705193Z pos=preset.go:71 component=virt-controller service=http msg="Starting Virtual Machine Initializer." level=info timestamp=2018-07-26T10:52:31.063094Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:52:31.069099Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:31.584591Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijml29\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmijml29, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: fe948d8f-90c1-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:55:50.753056Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:50.753344Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.072964Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4gt87\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi4gt87, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 75adfd4d-90c2-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4gt87" level=info timestamp=2018-07-26T10:55:51.170980Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:51.171125Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.254846Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:56:06.959840Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:06.960236Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:08.241694Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:08.243045Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:08.243370Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:10.804456Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:10.805065Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:10.805510Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:15.926330Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:56:15.926810Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:56:15.927116Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:56:26.167878Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:56:26.220538Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:56:26.220822Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:56:26.220961Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmihhxg2-tls6h Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202763a0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42066fc00, 0xc4200b7ef8, 0xc420cf4ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e29c0, 0xc42000e0c0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4200ba080, 0x202c2234413a3036, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e29c0, 0xc42000e0c0, 0x13, 0xc420cdef18, 0x3, 0x3, 0x64744c2959545028, 0x202020200a202c22, 0x7665642220202020) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202a83c0, 0xc4200e4960, 0xc421619fb0, 0xc421619fc0, 0xc420677e00, 0xc42067e0e0, 0x10f5e80, 0xc4200b7ef8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.819 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 should start it [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:76 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T10:55:51.109874Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmihhxg2-tls6h" level=info timestamp=2018-07-26T10:56:05.475981Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmihhxg2-tls6h" level=error timestamp=2018-07-26T10:56:05.771459Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T10:57:54.218133Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:57:56.513240Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:57:59 http: TLS handshake error from 10.244.0.1:42140: EOF 2018/07/26 10:58:09 http: TLS handshake error from 10.244.0.1:42164: EOF 2018/07/26 10:58:19 http: TLS handshake error from 10.244.0.1:42188: EOF level=info timestamp=2018-07-26T10:58:24.348203Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:58:26.609441Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:58:29 http: TLS handshake error from 10.244.0.1:42212: EOF level=info timestamp=2018-07-26T10:58:30.458753Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:58:30.462790Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:58:39 http: TLS handshake error from 10.244.0.1:42236: EOF 2018/07/26 10:58:49 http: TLS handshake error from 10.244.0.1:42260: EOF level=info timestamp=2018-07-26T10:58:54.485629Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T10:58:56.731977Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 10:58:59 http: TLS handshake error from 10.244.0.1:42284: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T10:57:21.631928Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T10:57:21.951549Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:57:30 http: TLS handshake error from 10.244.1.1:42284: EOF 2018/07/26 10:57:40 http: TLS handshake error from 10.244.1.1:42290: EOF 2018/07/26 10:57:50 http: TLS handshake error from 10.244.1.1:42296: EOF level=info timestamp=2018-07-26T10:57:51.718227Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:58:00 http: TLS handshake error from 10.244.1.1:42302: EOF 2018/07/26 10:58:10 http: TLS handshake error from 10.244.1.1:42308: EOF 2018/07/26 10:58:20 http: TLS handshake error from 10.244.1.1:42314: EOF level=info timestamp=2018-07-26T10:58:21.818741Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:58:30 http: TLS handshake error from 10.244.1.1:42320: EOF 2018/07/26 10:58:40 http: TLS handshake error from 10.244.1.1:42326: EOF 2018/07/26 10:58:50 http: TLS handshake error from 10.244.1.1:42332: EOF level=info timestamp=2018-07-26T10:58:51.794766Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 10:59:00 http: TLS handshake error from 10.244.1.1:42338: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:52:30.705087Z pos=vmi.go:129 component=virt-controller service=http msg="Starting vmi controller." level=info timestamp=2018-07-26T10:52:30.705145Z pos=replicaset.go:111 component=virt-controller service=http msg="Starting VirtualMachineInstanceReplicaSet controller." level=info timestamp=2018-07-26T10:52:30.705193Z pos=preset.go:71 component=virt-controller service=http msg="Starting Virtual Machine Initializer." level=info timestamp=2018-07-26T10:52:31.063094Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:52:31.069099Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:31.584591Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijml29\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmijml29, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: fe948d8f-90c1-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:55:50.753056Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:50.753344Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.072964Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4gt87\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi4gt87, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 75adfd4d-90c2-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4gt87" level=info timestamp=2018-07-26T10:55:51.170980Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:51.171125Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.254846Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:58:52.024264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:52.024676Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:52.153572Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwnxq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T10:59:06.260441Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:06.260780Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:06.260979Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:06.281828Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:06.282125Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:06.282510Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:06.323110Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:06.324082Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:06.324405Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:06.405089Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:06.405507Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:06.405702Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:06.566369Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:06.566820Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:06.567069Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" Pod name: virt-launcher-testvmitwnxq-lsf94 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202760f0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421924040, 0xc42000fcb8, 0xc42029aaa0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2120, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4202a95c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2120, 0xc42000e078, 0x13, 0xc42005df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346dc0, 0xc4200e41e0, 0xc421996180, 0xc421996190, 0xc421940300, 0xc421916d40, 0x10f5e80, 0xc42000fcb8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:00:29 http: TLS handshake error from 10.244.0.1:42500: EOF level=info timestamp=2018-07-26T11:00:30.467867Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:00:30.470975Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:00:39 http: TLS handshake error from 10.244.0.1:42524: EOF 2018/07/26 11:00:49 http: TLS handshake error from 10.244.0.1:42548: EOF level=info timestamp=2018-07-26T11:00:55.279877Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:00:57.232921Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:00:59 http: TLS handshake error from 10.244.0.1:42572: EOF 2018/07/26 11:01:09 http: TLS handshake error from 10.244.0.1:42596: EOF 2018/07/26 11:01:19 http: TLS handshake error from 10.244.0.1:42620: EOF level=info timestamp=2018-07-26T11:01:25.583546Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:01:27.339670Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:29 http: TLS handshake error from 10.244.0.1:42644: EOF 2018/07/26 11:01:39 http: TLS handshake error from 10.244.0.1:42668: EOF 2018/07/26 11:01:49 http: TLS handshake error from 10.244.0.1:42692: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:00:10 http: TLS handshake error from 10.244.1.1:42380: EOF 2018/07/26 11:00:20 http: TLS handshake error from 10.244.1.1:42386: EOF level=info timestamp=2018-07-26T11:00:21.847592Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:00:30 http: TLS handshake error from 10.244.1.1:42392: EOF 2018/07/26 11:00:40 http: TLS handshake error from 10.244.1.1:42398: EOF 2018/07/26 11:00:50 http: TLS handshake error from 10.244.1.1:42404: EOF level=info timestamp=2018-07-26T11:00:51.837505Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:01:00 http: TLS handshake error from 10.244.1.1:42410: EOF 2018/07/26 11:01:10 http: TLS handshake error from 10.244.1.1:42416: EOF 2018/07/26 11:01:20 http: TLS handshake error from 10.244.1.1:42422: EOF level=info timestamp=2018-07-26T11:01:21.726634Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:01:30 http: TLS handshake error from 10.244.1.1:42428: EOF 2018/07/26 11:01:40 http: TLS handshake error from 10.244.1.1:42434: EOF 2018/07/26 11:01:50 http: TLS handshake error from 10.244.1.1:42440: EOF level=info timestamp=2018-07-26T11:01:51.894836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:52:30.705087Z pos=vmi.go:129 component=virt-controller service=http msg="Starting vmi controller." level=info timestamp=2018-07-26T10:52:30.705145Z pos=replicaset.go:111 component=virt-controller service=http msg="Starting VirtualMachineInstanceReplicaSet controller." level=info timestamp=2018-07-26T10:52:30.705193Z pos=preset.go:71 component=virt-controller service=http msg="Starting Virtual Machine Initializer." level=info timestamp=2018-07-26T10:52:31.063094Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:52:31.069099Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijml29 kind= uid=fe948d8f-90c1-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:31.584591Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijml29\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmijml29, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: fe948d8f-90c1-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijml29" level=info timestamp=2018-07-26T10:55:50.753056Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:50.753344Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi4gt87 kind= uid=75adfd4d-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.072964Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi4gt87\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi4gt87, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 75adfd4d-90c2-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi4gt87" level=info timestamp=2018-07-26T10:55:51.170980Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:55:51.171125Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.254846Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:58:52.024264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:52.024676Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:52.153572Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwnxq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T10:59:07.531257Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:07.531662Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:08.814007Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:08.814489Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:08.814785Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:11.376333Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:11.378927Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:11.379692Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:16.500649Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:59:16.501523Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:59:16.501945Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T10:59:26.742914Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:59:26.787024Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:59:26.787534Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:59:26.787680Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmitwnxq-lsf94 Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202760f0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421924040, 0xc42000fcb8, 0xc42029aaa0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2120, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4202a95c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2120, 0xc42000e078, 0x13, 0xc42005df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346dc0, 0xc4200e41e0, 0xc421996180, 0xc421996190, 0xc421940300, 0xc421916d40, 0x10f5e80, 0xc42000fcb8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.834 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 should attach virt-launcher to it [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:82 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T10:58:51.938386Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmitwnxq-lsf94" level=info timestamp=2018-07-26T10:59:06.079155Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmitwnxq-lsf94" level=error timestamp=2018-07-26T10:59:06.346755Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" •••• Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:00:55.279877Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:00:57.232921Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:00:59 http: TLS handshake error from 10.244.0.1:42572: EOF 2018/07/26 11:01:09 http: TLS handshake error from 10.244.0.1:42596: EOF 2018/07/26 11:01:19 http: TLS handshake error from 10.244.0.1:42620: EOF level=info timestamp=2018-07-26T11:01:25.583546Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:01:27.339670Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:29 http: TLS handshake error from 10.244.0.1:42644: EOF 2018/07/26 11:01:39 http: TLS handshake error from 10.244.0.1:42668: EOF 2018/07/26 11:01:49 http: TLS handshake error from 10.244.0.1:42692: EOF level=info timestamp=2018-07-26T11:01:53.034811Z pos=validating-webhook.go:84 component=virt-api msg="rejected vmi admission" level=info timestamp=2018-07-26T11:01:55.738629Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:01:57.485811Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:01:59 http: TLS handshake error from 10.244.0.1:42716: EOF 2018/07/26 11:02:09 http: TLS handshake error from 10.244.0.1:42740: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:00:20 http: TLS handshake error from 10.244.1.1:42386: EOF level=info timestamp=2018-07-26T11:00:21.847592Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:00:30 http: TLS handshake error from 10.244.1.1:42392: EOF 2018/07/26 11:00:40 http: TLS handshake error from 10.244.1.1:42398: EOF 2018/07/26 11:00:50 http: TLS handshake error from 10.244.1.1:42404: EOF level=info timestamp=2018-07-26T11:00:51.837505Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:01:00 http: TLS handshake error from 10.244.1.1:42410: EOF 2018/07/26 11:01:10 http: TLS handshake error from 10.244.1.1:42416: EOF 2018/07/26 11:01:20 http: TLS handshake error from 10.244.1.1:42422: EOF level=info timestamp=2018-07-26T11:01:21.726634Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:01:30 http: TLS handshake error from 10.244.1.1:42428: EOF 2018/07/26 11:01:40 http: TLS handshake error from 10.244.1.1:42434: EOF 2018/07/26 11:01:50 http: TLS handshake error from 10.244.1.1:42440: EOF level=info timestamp=2018-07-26T11:01:51.894836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:02:00 http: TLS handshake error from 10.244.1.1:42446: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:55:51.171125Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.254846Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:58:52.024264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:52.024676Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:52.153572Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwnxq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T11:01:53.331026Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.332028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.583298Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.607230Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.736916Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi69pt9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4dc68ceb-90c3-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.832611Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.833028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.896742Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicdp2b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicdp2b" level=info timestamp=2018-07-26T11:01:54.195711Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:02:09.409094Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:09.409296Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:09.409398Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:09.429948Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:09.430102Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:09.430244Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:09.470529Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:09.470723Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:09.470842Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:09.552387Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:09.552578Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:09.552728Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:09.713106Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:09.713312Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:09.713413Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" Pod name: virt-launcher-testvmit8jqw-cflw6 Pod phase: Running goroutine 9 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202dc3b0, 0xc4201b4c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192cc60, 0xc4200b7058, 0xc4202950e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200fe5a0, 0xc42000e238, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420567900, 0x1c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200fe5a0, 0xc42000e238, 0x13, 0xc420748f18, 0x3, 0x3, 0x1, 0x4, 0xc42052f378) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a5c0, 0xc4200e2820, 0xc421abb1c0, 0xc421abb1d0, 0xc42192fe00, 0xc4219380c0, 0x10f5e80, 0xc4200b7058, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:03:29 http: TLS handshake error from 10.244.0.1:42932: EOF level=info timestamp=2018-07-26T11:03:32.089923Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:03:32.093058Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:03:39 http: TLS handshake error from 10.244.0.1:42956: EOF 2018/07/26 11:03:49 http: TLS handshake error from 10.244.0.1:42980: EOF level=info timestamp=2018-07-26T11:03:56.269079Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:03:57.969968Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:03:59 http: TLS handshake error from 10.244.0.1:43004: EOF 2018/07/26 11:04:09 http: TLS handshake error from 10.244.0.1:43028: EOF 2018/07/26 11:04:19 http: TLS handshake error from 10.244.0.1:43052: EOF level=info timestamp=2018-07-26T11:04:26.420838Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:28.103167Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:29 http: TLS handshake error from 10.244.0.1:43076: EOF 2018/07/26 11:04:39 http: TLS handshake error from 10.244.0.1:43100: EOF 2018/07/26 11:04:49 http: TLS handshake error from 10.244.0.1:43124: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:03:10 http: TLS handshake error from 10.244.1.1:42488: EOF 2018/07/26 11:03:20 http: TLS handshake error from 10.244.1.1:42494: EOF level=info timestamp=2018-07-26T11:03:21.828716Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:03:30 http: TLS handshake error from 10.244.1.1:42500: EOF 2018/07/26 11:03:40 http: TLS handshake error from 10.244.1.1:42506: EOF 2018/07/26 11:03:50 http: TLS handshake error from 10.244.1.1:42512: EOF level=info timestamp=2018-07-26T11:03:51.739080Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:04:00 http: TLS handshake error from 10.244.1.1:42518: EOF 2018/07/26 11:04:10 http: TLS handshake error from 10.244.1.1:42524: EOF 2018/07/26 11:04:20 http: TLS handshake error from 10.244.1.1:42530: EOF level=info timestamp=2018-07-26T11:04:21.802276Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:04:30 http: TLS handshake error from 10.244.1.1:42536: EOF 2018/07/26 11:04:40 http: TLS handshake error from 10.244.1.1:42542: EOF 2018/07/26 11:04:50 http: TLS handshake error from 10.244.1.1:42548: EOF level=info timestamp=2018-07-26T11:04:51.834432Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:55:51.171125Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihhxg2 kind= uid=75f07dbd-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:55:51.254846Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihhxg2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihhxg2" level=info timestamp=2018-07-26T10:58:52.024264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:52.024676Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:52.153572Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwnxq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T11:01:53.331026Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.332028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.583298Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.607230Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.736916Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi69pt9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4dc68ceb-90c3-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.832611Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.833028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.896742Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicdp2b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicdp2b" level=info timestamp=2018-07-26T11:01:54.195711Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:02:10.676104Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:10.676523Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:11.957282Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:11.957702Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:11.958065Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:14.519497Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:14.520287Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:14.520598Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:19.642420Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:02:19.643732Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:02:19.644288Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit8jqw" level=info timestamp=2018-07-26T11:02:29.885361Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:02:29.937762Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:02:29.940535Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:02:29.940786Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmit8jqw-cflw6 Pod phase: Failed goroutine 9 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202dc3b0, 0xc4201b4c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192cc60, 0xc4200b7058, 0xc4202950e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200fe5a0, 0xc42000e238, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420567900, 0x1c, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200fe5a0, 0xc42000e238, 0x13, 0xc420748f18, 0x3, 0x3, 0x1, 0x4, 0xc42052f378) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a5c0, 0xc4200e2820, 0xc421abb1c0, 0xc421abb1d0, 0xc42192fe00, 0xc4219380c0, 0x10f5e80, 0xc4200b7058, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.758 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with boot order /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:170 should be able to boot from selected disk /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 Alpine as first boot [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: defining a VirtualMachineInstance with an Alpine disk STEP: adding a Cirros Disk STEP: setting boot order STEP: starting VirtualMachineInstance STEP: Waiting the VirtualMachineInstance start level=info timestamp=2018-07-26T11:01:54.157951Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmit8jqw-cflw6" level=info timestamp=2018-07-26T11:02:09.084561Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmit8jqw-cflw6" level=error timestamp=2018-07-26T11:02:09.493474Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:03:49 http: TLS handshake error from 10.244.0.1:42980: EOF level=info timestamp=2018-07-26T11:03:56.269079Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:03:57.969968Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:03:59 http: TLS handshake error from 10.244.0.1:43004: EOF 2018/07/26 11:04:09 http: TLS handshake error from 10.244.0.1:43028: EOF 2018/07/26 11:04:19 http: TLS handshake error from 10.244.0.1:43052: EOF level=info timestamp=2018-07-26T11:04:26.420838Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:28.103167Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:29 http: TLS handshake error from 10.244.0.1:43076: EOF 2018/07/26 11:04:39 http: TLS handshake error from 10.244.0.1:43100: EOF 2018/07/26 11:04:49 http: TLS handshake error from 10.244.0.1:43124: EOF level=info timestamp=2018-07-26T11:04:56.514819Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:04:58.227638Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:04:59 http: TLS handshake error from 10.244.0.1:43148: EOF 2018/07/26 11:05:09 http: TLS handshake error from 10.244.0.1:43172: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:03:21.828716Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:03:30 http: TLS handshake error from 10.244.1.1:42500: EOF 2018/07/26 11:03:40 http: TLS handshake error from 10.244.1.1:42506: EOF 2018/07/26 11:03:50 http: TLS handshake error from 10.244.1.1:42512: EOF level=info timestamp=2018-07-26T11:03:51.739080Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:04:00 http: TLS handshake error from 10.244.1.1:42518: EOF 2018/07/26 11:04:10 http: TLS handshake error from 10.244.1.1:42524: EOF 2018/07/26 11:04:20 http: TLS handshake error from 10.244.1.1:42530: EOF level=info timestamp=2018-07-26T11:04:21.802276Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:04:30 http: TLS handshake error from 10.244.1.1:42536: EOF 2018/07/26 11:04:40 http: TLS handshake error from 10.244.1.1:42542: EOF 2018/07/26 11:04:50 http: TLS handshake error from 10.244.1.1:42548: EOF level=info timestamp=2018-07-26T11:04:51.834432Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:05:00 http: TLS handshake error from 10.244.1.1:42554: EOF 2018/07/26 11:05:10 http: TLS handshake error from 10.244.1.1:42560: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:58:52.024264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:52.024676Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:52.153572Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwnxq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T11:01:53.331026Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.332028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.583298Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.607230Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.736916Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi69pt9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4dc68ceb-90c3-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.832611Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.833028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.896742Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicdp2b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicdp2b" level=info timestamp=2018-07-26T11:01:54.195711Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:55.061287Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:55.062193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:05:11.571140Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:11.571659Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:11.572119Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:11.593025Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:11.593631Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:11.593973Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:11.634740Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:11.635312Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:11.635642Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:11.725923Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:11.726487Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:11.727142Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:11.888450Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:11.890109Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:11.890411Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" Pod name: virt-launcher-testvmir8lgg-fs7rd Pod phase: Running goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4204a40b0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191e040, 0xc42000ee20, 0xc420268960, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096180, 0xc42000e088, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4203d8180, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096180, 0xc42000e088, 0x13, 0xc421949718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420280040, 0xc4200aa320, 0xc42037c0a0, 0xc42037c0d0, 0xc421944300, 0xc42191e140, 0x10f5e80, 0xc42000ee20, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:06:29 http: TLS handshake error from 10.244.0.1:43364: EOF 2018/07/26 11:06:39 http: TLS handshake error from 10.244.0.1:43388: EOF 2018/07/26 11:06:49 http: TLS handshake error from 10.244.0.1:43412: EOF level=info timestamp=2018-07-26T11:06:57.013971Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:06:58.681661Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:06:59 http: TLS handshake error from 10.244.0.1:43436: EOF 2018/07/26 11:07:09 http: TLS handshake error from 10.244.0.1:43460: EOF 2018/07/26 11:07:19 http: TLS handshake error from 10.244.0.1:43484: EOF level=info timestamp=2018-07-26T11:07:27.150056Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:07:28.951763Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:07:29 http: TLS handshake error from 10.244.0.1:43508: EOF level=info timestamp=2018-07-26T11:07:30.930648Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:07:30.933366Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:07:39 http: TLS handshake error from 10.244.0.1:43532: EOF 2018/07/26 11:07:49 http: TLS handshake error from 10.244.0.1:43556: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:06:21.791563Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:06:30 http: TLS handshake error from 10.244.1.1:42608: EOF 2018/07/26 11:06:40 http: TLS handshake error from 10.244.1.1:42614: EOF 2018/07/26 11:06:50 http: TLS handshake error from 10.244.1.1:42620: EOF level=info timestamp=2018-07-26T11:06:51.821151Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:07:00 http: TLS handshake error from 10.244.1.1:42626: EOF 2018/07/26 11:07:10 http: TLS handshake error from 10.244.1.1:42632: EOF 2018/07/26 11:07:20 http: TLS handshake error from 10.244.1.1:42638: EOF level=info timestamp=2018-07-26T11:07:21.635743Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:07:21.730605Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:07:21.948310Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:07:30 http: TLS handshake error from 10.244.1.1:42644: EOF 2018/07/26 11:07:40 http: TLS handshake error from 10.244.1.1:42650: EOF 2018/07/26 11:07:50 http: TLS handshake error from 10.244.1.1:42656: EOF level=info timestamp=2018-07-26T11:07:51.802833Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T10:58:52.024264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T10:58:52.024676Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitwnxq kind= uid=e1bb7f1c-90c2-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T10:58:52.153572Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitwnxq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitwnxq" level=info timestamp=2018-07-26T11:01:53.331026Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.332028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi69pt9 kind= uid=4dc68ceb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.583298Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.607230Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.736916Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi69pt9\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi69pt9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4dc68ceb-90c3-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi69pt9" level=info timestamp=2018-07-26T11:01:53.832611Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.833028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.896742Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicdp2b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicdp2b" level=info timestamp=2018-07-26T11:01:54.195711Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:55.061287Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:55.062193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:05:12.853553Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:12.853820Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:14.134714Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:14.135289Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:14.135588Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:16.696493Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:16.696935Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:16.697402Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:21.818455Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:05:21.819487Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:05:21.819793Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmir8lgg" level=info timestamp=2018-07-26T11:05:32.061811Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:05:32.112836Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:05:32.124259Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:05:32.124514Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmir8lgg-fs7rd Pod phase: Failed goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4204a40b0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191e040, 0xc42000ee20, 0xc420268960, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096180, 0xc42000e088, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4203d8180, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096180, 0xc42000e088, 0x13, 0xc421949718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420280040, 0xc4200aa320, 0xc42037c0a0, 0xc42037c0d0, 0xc421944300, 0xc42191e140, 0x10f5e80, 0xc42000ee20, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.856 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with boot order /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:170 should be able to boot from selected disk /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 Cirros as first boot [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: defining a VirtualMachineInstance with an Alpine disk STEP: adding a Cirros Disk STEP: setting boot order STEP: starting VirtualMachineInstance STEP: Waiting the VirtualMachineInstance start level=info timestamp=2018-07-26T11:04:54.949871Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmir8lgg-fs7rd" level=info timestamp=2018-07-26T11:05:11.227605Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmir8lgg-fs7rd" level=error timestamp=2018-07-26T11:05:11.653932Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" • [SLOW TEST:16.666 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with user-data /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:201 without k8s secret /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:202 should retry starting the VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:203 ------------------------------ • [SLOW TEST:45.995 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with user-data /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:201 without k8s secret /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:202 should log warning and proceed once the secret is there /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:233 ------------------------------ Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:07:59.086915Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:07:59 http: TLS handshake error from 10.244.0.1:43580: EOF 2018/07/26 11:08:09 http: TLS handshake error from 10.244.0.1:43604: EOF 2018/07/26 11:08:19 http: TLS handshake error from 10.244.0.1:43628: EOF level=info timestamp=2018-07-26T11:08:27.455902Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:08:29.197979Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:08:29 http: TLS handshake error from 10.244.0.1:43652: EOF level=info timestamp=2018-07-26T11:08:30.945036Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:08:30.952058Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:08:39 http: TLS handshake error from 10.244.0.1:43676: EOF 2018/07/26 11:08:49 http: TLS handshake error from 10.244.0.1:43700: EOF level=info timestamp=2018-07-26T11:08:57.644616Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:08:59.346842Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:08:59 http: TLS handshake error from 10.244.0.1:43724: EOF 2018/07/26 11:09:09 http: TLS handshake error from 10.244.0.1:43748: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:07:21.948310Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:07:30 http: TLS handshake error from 10.244.1.1:42644: EOF 2018/07/26 11:07:40 http: TLS handshake error from 10.244.1.1:42650: EOF 2018/07/26 11:07:50 http: TLS handshake error from 10.244.1.1:42656: EOF level=info timestamp=2018-07-26T11:07:51.802833Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:08:00 http: TLS handshake error from 10.244.1.1:42662: EOF 2018/07/26 11:08:10 http: TLS handshake error from 10.244.1.1:42668: EOF 2018/07/26 11:08:20 http: TLS handshake error from 10.244.1.1:42674: EOF level=info timestamp=2018-07-26T11:08:21.750250Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:08:30 http: TLS handshake error from 10.244.1.1:42680: EOF 2018/07/26 11:08:40 http: TLS handshake error from 10.244.1.1:42686: EOF 2018/07/26 11:08:50 http: TLS handshake error from 10.244.1.1:42692: EOF level=info timestamp=2018-07-26T11:08:51.751050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:09:00 http: TLS handshake error from 10.244.1.1:42698: EOF 2018/07/26 11:09:10 http: TLS handshake error from 10.244.1.1:42704: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:01:53.832611Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.833028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.896742Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicdp2b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicdp2b" level=info timestamp=2018-07-26T11:01:54.195711Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:55.061287Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:55.062193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:07:55.858283Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:07:55.859068Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:12.460264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:12.460700Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:09:13.379385Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:13.386924Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:13.387439Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:13.408376Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:13.408858Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:13.409440Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:13.450321Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:13.450700Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:13.451329Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:13.532660Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:13.533078Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:13.533675Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:13.694609Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:13.695015Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:13.695390Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" Pod name: virt-launcher-testvmip77fj-rzln8 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202e45e0, 0xc4201aac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204a6040, 0xc4200c6ce0, 0xc421941b20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e0b0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4203568c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e0b0, 0x13, 0xc42052bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420411640, 0xc4200a8410, 0xc420594690, 0xc4205946a0, 0xc420250300, 0xc4204fc840, 0x10f5e80, 0xc4200c6ce0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:10:30.957254Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:10:30.959973Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:10:39 http: TLS handshake error from 10.244.0.1:43964: EOF 2018/07/26 11:10:49 http: TLS handshake error from 10.244.0.1:43988: EOF level=info timestamp=2018-07-26T11:10:58.166495Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:10:59 http: TLS handshake error from 10.244.0.1:44012: EOF level=info timestamp=2018-07-26T11:10:59.840633Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:09 http: TLS handshake error from 10.244.0.1:44036: EOF 2018/07/26 11:11:19 http: TLS handshake error from 10.244.0.1:44060: EOF level=info timestamp=2018-07-26T11:11:28.309292Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:29 http: TLS handshake error from 10.244.0.1:44084: EOF level=info timestamp=2018-07-26T11:11:29.965564Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:39 http: TLS handshake error from 10.244.0.1:44108: EOF 2018/07/26 11:11:49 http: TLS handshake error from 10.244.0.1:44132: EOF level=info timestamp=2018-07-26T11:11:58.450408Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:10:10 http: TLS handshake error from 10.244.1.1:42740: EOF 2018/07/26 11:10:20 http: TLS handshake error from 10.244.1.1:42746: EOF level=info timestamp=2018-07-26T11:10:21.871853Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:10:30 http: TLS handshake error from 10.244.1.1:42752: EOF 2018/07/26 11:10:40 http: TLS handshake error from 10.244.1.1:42758: EOF 2018/07/26 11:10:50 http: TLS handshake error from 10.244.1.1:42764: EOF level=info timestamp=2018-07-26T11:10:52.069674Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:11:00 http: TLS handshake error from 10.244.1.1:42770: EOF 2018/07/26 11:11:10 http: TLS handshake error from 10.244.1.1:42776: EOF 2018/07/26 11:11:20 http: TLS handshake error from 10.244.1.1:42782: EOF level=info timestamp=2018-07-26T11:11:21.763078Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:11:30 http: TLS handshake error from 10.244.1.1:42788: EOF 2018/07/26 11:11:40 http: TLS handshake error from 10.244.1.1:42794: EOF 2018/07/26 11:11:50 http: TLS handshake error from 10.244.1.1:42800: EOF level=info timestamp=2018-07-26T11:11:51.760244Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:01:53.832611Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:53.833028Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicdp2b kind= uid=4e1b5feb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:01:53.896742Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmicdp2b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmicdp2b" level=info timestamp=2018-07-26T11:01:54.195711Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:55.061287Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:55.062193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:07:55.858283Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:07:55.859068Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:12.460264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:12.460700Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:09:14.659867Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:14.660233Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:15.941068Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:15.941570Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:15.942006Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:18.504651Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:18.506911Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:18.507317Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:23.628053Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:09:23.628893Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:09:23.629352Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:09:33.870273Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:09:33.916853Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:09:33.917256Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:09:33.917375Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmip77fj-rzln8 Pod phase: Failed goroutine 27 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202e45e0, 0xc4201aac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204a6040, 0xc4200c6ce0, 0xc421941b20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e0b0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4203568c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e0b0, 0x13, 0xc42052bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420411640, 0xc4200a8410, 0xc420594690, 0xc4205946a0, 0xc420250300, 0xc4204fc840, 0x10f5e80, 0xc4200c6ce0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.828 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-launcher crashes /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:281 should be stopped and have Failed phase [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:282 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:08:58.402722Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmip77fj-rzln8" level=info timestamp=2018-07-26T11:09:13.142057Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmip77fj-rzln8" level=error timestamp=2018-07-26T11:09:13.460584Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:10:49 http: TLS handshake error from 10.244.0.1:43988: EOF level=info timestamp=2018-07-26T11:10:58.166495Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:10:59 http: TLS handshake error from 10.244.0.1:44012: EOF level=info timestamp=2018-07-26T11:10:59.840633Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:09 http: TLS handshake error from 10.244.0.1:44036: EOF 2018/07/26 11:11:19 http: TLS handshake error from 10.244.0.1:44060: EOF level=info timestamp=2018-07-26T11:11:28.309292Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:29 http: TLS handshake error from 10.244.0.1:44084: EOF level=info timestamp=2018-07-26T11:11:29.965564Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:39 http: TLS handshake error from 10.244.0.1:44108: EOF 2018/07/26 11:11:49 http: TLS handshake error from 10.244.0.1:44132: EOF level=info timestamp=2018-07-26T11:11:58.450408Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:11:59 http: TLS handshake error from 10.244.0.1:44156: EOF level=info timestamp=2018-07-26T11:12:00.104277Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:12:09 http: TLS handshake error from 10.244.0.1:44180: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:10:21.871853Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:10:30 http: TLS handshake error from 10.244.1.1:42752: EOF 2018/07/26 11:10:40 http: TLS handshake error from 10.244.1.1:42758: EOF 2018/07/26 11:10:50 http: TLS handshake error from 10.244.1.1:42764: EOF level=info timestamp=2018-07-26T11:10:52.069674Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:11:00 http: TLS handshake error from 10.244.1.1:42770: EOF 2018/07/26 11:11:10 http: TLS handshake error from 10.244.1.1:42776: EOF 2018/07/26 11:11:20 http: TLS handshake error from 10.244.1.1:42782: EOF level=info timestamp=2018-07-26T11:11:21.763078Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:11:30 http: TLS handshake error from 10.244.1.1:42788: EOF 2018/07/26 11:11:40 http: TLS handshake error from 10.244.1.1:42794: EOF 2018/07/26 11:11:50 http: TLS handshake error from 10.244.1.1:42800: EOF level=info timestamp=2018-07-26T11:11:51.760244Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:12:00 http: TLS handshake error from 10.244.1.1:42806: EOF 2018/07/26 11:12:10 http: TLS handshake error from 10.244.1.1:42812: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:55.061287Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:55.062193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:07:55.858283Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:07:55.859068Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:12.460264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:12.460700Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:11:59.357960Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:59.358206Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:59.528721Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:11:59.572466Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:12:15.382128Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:15.382546Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:15.382942Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:15.403695Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:15.404117Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:15.404572Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:15.445452Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:15.446303Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:15.446554Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:15.527673Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:15.527961Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:15.528113Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:15.689005Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:15.689585Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:15.689885Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" Pod name: virt-launcher-testvminp9vd-st6mx Pod phase: Running goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42025c4d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421918c80, 0xc42037c100, 0xc4202ed6a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6d20, 0xc4200b6258, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200bac40, 0xc420404c00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6d20, 0xc4200b6258, 0x13, 0xc421a75f18, 0x3, 0x3, 0xc420405100, 0xc420405180, 0xc420405200) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba980, 0xc4200b8aa0, 0xc421a2fd80, 0xc421a2fd90, 0xc42191be00, 0xc4219240e0, 0x10f5e80, 0xc42037c100, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:13:31.009872Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:13:39 http: TLS handshake error from 10.244.0.1:44396: EOF 2018/07/26 11:13:49 http: TLS handshake error from 10.244.0.1:44420: EOF level=info timestamp=2018-07-26T11:13:59.116698Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:13:59 http: TLS handshake error from 10.244.0.1:44444: EOF level=info timestamp=2018-07-26T11:14:00.504816Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:09 http: TLS handshake error from 10.244.0.1:44468: EOF 2018/07/26 11:14:19 http: TLS handshake error from 10.244.0.1:44492: EOF level=info timestamp=2018-07-26T11:14:29.242050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:29 http: TLS handshake error from 10.244.0.1:44516: EOF level=info timestamp=2018-07-26T11:14:30.636051Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:39 http: TLS handshake error from 10.244.0.1:44540: EOF 2018/07/26 11:14:49 http: TLS handshake error from 10.244.0.1:44564: EOF level=info timestamp=2018-07-26T11:14:59.363512Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:59 http: TLS handshake error from 10.244.0.1:44588: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:13:10 http: TLS handshake error from 10.244.1.1:42848: EOF 2018/07/26 11:13:20 http: TLS handshake error from 10.244.1.1:42854: EOF level=info timestamp=2018-07-26T11:13:21.784270Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:13:30 http: TLS handshake error from 10.244.1.1:42860: EOF 2018/07/26 11:13:40 http: TLS handshake error from 10.244.1.1:42866: EOF 2018/07/26 11:13:50 http: TLS handshake error from 10.244.1.1:42872: EOF level=info timestamp=2018-07-26T11:13:51.817572Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:14:00 http: TLS handshake error from 10.244.1.1:42878: EOF 2018/07/26 11:14:10 http: TLS handshake error from 10.244.1.1:42884: EOF 2018/07/26 11:14:20 http: TLS handshake error from 10.244.1.1:42890: EOF level=info timestamp=2018-07-26T11:14:21.913583Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:14:30 http: TLS handshake error from 10.244.1.1:42896: EOF 2018/07/26 11:14:40 http: TLS handshake error from 10.244.1.1:42902: EOF 2018/07/26 11:14:50 http: TLS handshake error from 10.244.1.1:42908: EOF level=info timestamp=2018-07-26T11:14:51.769973Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:01:54.195968Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit8jqw kind= uid=4e4f6340-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:04:55.061287Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:04:55.062193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmir8lgg kind= uid=ba1e39bb-90c3-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:07:55.858283Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:07:55.859068Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:12.460264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:12.460700Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:11:59.357960Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:59.358206Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:59.528721Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:11:59.572466Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:12:16.652971Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:16.653380Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:17.934014Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:17.934577Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:17.934825Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:20.496142Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:20.496878Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:20.497369Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:25.618391Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:12:25.618860Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:12:25.619084Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:12:35.859948Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:12:35.901637Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:12:35.902012Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:12:35.902127Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvminp9vd-st6mx Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42025c4d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421918c80, 0xc42037c100, 0xc4202ed6a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6d20, 0xc4200b6258, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200bac40, 0xc420404c00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6d20, 0xc4200b6258, 0x13, 0xc421a75f18, 0x3, 0x3, 0xc420405100, 0xc420405180, 0xc420405200) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba980, 0xc4200b8aa0, 0xc421a2fd80, 0xc421a2fd90, 0xc42191be00, 0xc4219240e0, 0x10f5e80, 0xc42037c100, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.746 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-handler crashes /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:304 should recover and continue management [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:305 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:11:59.283332Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvminp9vd-st6mx" level=info timestamp=2018-07-26T11:12:15.111865Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvminp9vd-st6mx" level=error timestamp=2018-07-26T11:12:15.446736Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" • [SLOW TEST:16.690 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-handler is responsive /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:335 should indicate that a node is ready for vmis /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:336 ------------------------------ Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:14:29.242050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:29 http: TLS handshake error from 10.244.0.1:44516: EOF level=info timestamp=2018-07-26T11:14:30.636051Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:39 http: TLS handshake error from 10.244.0.1:44540: EOF 2018/07/26 11:14:49 http: TLS handshake error from 10.244.0.1:44564: EOF level=info timestamp=2018-07-26T11:14:59.363512Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:14:59 http: TLS handshake error from 10.244.0.1:44588: EOF level=info timestamp=2018-07-26T11:15:00.761137Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:15:09 http: TLS handshake error from 10.244.0.1:44612: EOF 2018/07/26 11:15:19 http: TLS handshake error from 10.244.0.1:44636: EOF 2018/07/26 11:15:29 http: TLS handshake error from 10.244.0.1:44660: EOF level=info timestamp=2018-07-26T11:15:29.493927Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:15:30.871229Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:15:31.013910Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:15:31.015722Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:13:50 http: TLS handshake error from 10.244.1.1:42872: EOF level=info timestamp=2018-07-26T11:13:51.817572Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:14:00 http: TLS handshake error from 10.244.1.1:42878: EOF 2018/07/26 11:14:10 http: TLS handshake error from 10.244.1.1:42884: EOF 2018/07/26 11:14:20 http: TLS handshake error from 10.244.1.1:42890: EOF level=info timestamp=2018-07-26T11:14:21.913583Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:14:30 http: TLS handshake error from 10.244.1.1:42896: EOF 2018/07/26 11:14:40 http: TLS handshake error from 10.244.1.1:42902: EOF 2018/07/26 11:14:50 http: TLS handshake error from 10.244.1.1:42908: EOF level=info timestamp=2018-07-26T11:14:51.769973Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:15:00 http: TLS handshake error from 10.244.1.1:42914: EOF 2018/07/26 11:15:10 http: TLS handshake error from 10.244.1.1:42920: EOF 2018/07/26 11:15:20 http: TLS handshake error from 10.244.1.1:42926: EOF level=info timestamp=2018-07-26T11:15:21.819651Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:15:30 http: TLS handshake error from 10.244.1.1:42932: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:07:55.858283Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:07:55.859068Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:12.460264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:12.460700Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:11:59.357960Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:59.358206Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:59.528721Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:11:59.572466Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:15:16.665017Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:15:16.665491Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:15:16.851185Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminbn4q\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:15:33.256916Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:33.257475Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:33.257757Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:33.278520Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:33.278929Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:33.279280Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:33.320054Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:33.320558Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:33.320902Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:33.402140Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:33.402629Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:33.402941Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:33.563617Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:33.564040Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:33.564435Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" Pod name: virt-launcher-testvminbn4q-lnrn9 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 37 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203144d0, 0xc420106280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42038eb80, 0xc420724e90, 0xc42075c780, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096d20, 0xc4200b6310, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200ba900, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096d20, 0xc4200b6310, 0x13, 0xc421587718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420726f80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba700, 0xc4200b8960, 0xc4207324b0, 0xc4207324c0, 0xc42018de00, 0xc420400100, 0x10f5e80, 0xc420724e90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:16:59.879869Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:17:01.350126Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:17:09 http: TLS handshake error from 10.244.0.1:44900: EOF 2018/07/26 11:17:19 http: TLS handshake error from 10.244.0.1:44924: EOF 2018/07/26 11:17:29 http: TLS handshake error from 10.244.0.1:44948: EOF level=info timestamp=2018-07-26T11:17:30.076098Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:17:31.483930Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:17:31.593478Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:17:31.596916Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:39 http: TLS handshake error from 10.244.0.1:44972: EOF 2018/07/26 11:17:49 http: TLS handshake error from 10.244.0.1:44996: EOF 2018/07/26 11:17:59 http: TLS handshake error from 10.244.0.1:45020: EOF level=info timestamp=2018-07-26T11:18:00.250380Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:18:01.631660Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:18:09 http: TLS handshake error from 10.244.0.1:45044: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:16:40 http: TLS handshake error from 10.244.1.1:42974: EOF 2018/07/26 11:16:50 http: TLS handshake error from 10.244.1.1:42980: EOF level=info timestamp=2018-07-26T11:16:51.838407Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:00 http: TLS handshake error from 10.244.1.1:42986: EOF 2018/07/26 11:17:10 http: TLS handshake error from 10.244.1.1:42992: EOF 2018/07/26 11:17:20 http: TLS handshake error from 10.244.1.1:42998: EOF level=info timestamp=2018-07-26T11:17:21.632757Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:17:21.731315Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:17:21.776431Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:30 http: TLS handshake error from 10.244.1.1:43004: EOF 2018/07/26 11:17:40 http: TLS handshake error from 10.244.1.1:43010: EOF 2018/07/26 11:17:50 http: TLS handshake error from 10.244.1.1:43016: EOF level=info timestamp=2018-07-26T11:17:51.872928Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:18:00 http: TLS handshake error from 10.244.1.1:43022: EOF 2018/07/26 11:18:10 http: TLS handshake error from 10.244.1.1:43028: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:07:55.858283Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:07:55.859068Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmih575g kind= uid=25e0ab14-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:12.460264Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:12.460700Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizv47b kind= uid=2fc83f1a-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:11:59.357960Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:59.358206Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:59.528721Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:11:59.572466Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:15:16.665017Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:15:16.665491Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:15:16.851185Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminbn4q\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T10:51:26.502083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:29.065129Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:29.065778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:29.066098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:34.187095Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T10:51:34.187921Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T10:51:34.189024Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi4smw9" level=info timestamp=2018-07-26T10:51:44.430949Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.457963Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:51:44.458627Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T10:51:44.458823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:09.845167Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:09.851016Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind= uid=cd7b486b-90c1-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T10:52:10.225896Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T10:52:10.226053Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi4smw9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:15:34.527990Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:34.528323Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:35.809048Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:35.809641Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:35.809976Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:38.370662Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:38.371314Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:38.371580Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:43.492336Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:43.494276Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:43.494551Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:53.735713Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:15:53.775755Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:15:53.776094Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:15:53.776338Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvminbn4q-lnrn9 Pod phase: Failed goroutine 37 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203144d0, 0xc420106280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42038eb80, 0xc420724e90, 0xc42075c780, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096d20, 0xc4200b6310, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200ba900, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096d20, 0xc4200b6310, 0x13, 0xc421587718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420726f80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba700, 0xc4200b8960, 0xc4207324b0, 0xc4207324c0, 0xc42018de00, 0xc420400100, 0x10f5e80, 0xc420724e90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [180.663 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 when virt-handler is not responsive /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:366 the node controller should react [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:405 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:15:16.583398Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvminbn4q-lnrn9" level=info timestamp=2018-07-26T11:15:32.906921Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvminbn4q-lnrn9" level=error timestamp=2018-07-26T11:15:33.302501Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:17:31.483930Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:17:31.593478Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:17:31.596916Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:39 http: TLS handshake error from 10.244.0.1:44972: EOF 2018/07/26 11:17:49 http: TLS handshake error from 10.244.0.1:44996: EOF 2018/07/26 11:17:59 http: TLS handshake error from 10.244.0.1:45020: EOF level=info timestamp=2018-07-26T11:18:00.250380Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:18:01.631660Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:18:09 http: TLS handshake error from 10.244.0.1:45044: EOF 2018/07/26 11:18:19 http: TLS handshake error from 10.244.0.1:45068: EOF 2018/07/26 11:18:29 http: TLS handshake error from 10.244.0.1:45092: EOF level=info timestamp=2018-07-26T11:18:30.327557Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:18:31.603659Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:18:31.606243Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:18:31.702186Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:17:00 http: TLS handshake error from 10.244.1.1:42986: EOF 2018/07/26 11:17:10 http: TLS handshake error from 10.244.1.1:42992: EOF 2018/07/26 11:17:20 http: TLS handshake error from 10.244.1.1:42998: EOF level=info timestamp=2018-07-26T11:17:21.632757Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:17:21.731315Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:17:21.776431Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:17:30 http: TLS handshake error from 10.244.1.1:43004: EOF 2018/07/26 11:17:40 http: TLS handshake error from 10.244.1.1:43010: EOF 2018/07/26 11:17:50 http: TLS handshake error from 10.244.1.1:43016: EOF level=info timestamp=2018-07-26T11:17:51.872928Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:18:00 http: TLS handshake error from 10.244.1.1:43022: EOF 2018/07/26 11:18:10 http: TLS handshake error from 10.244.1.1:43028: EOF 2018/07/26 11:18:20 http: TLS handshake error from 10.244.1.1:43034: EOF level=info timestamp=2018-07-26T11:18:21.839153Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:18:30 http: TLS handshake error from 10.244.1.1:43040: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:11:59.357960Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:59.358206Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:59.528721Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:11:59.572466Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:15:16.665017Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:15:16.665491Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:15:16.851185Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminbn4q\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:18:17.636845Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:18:17.638076Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:18:17.930538Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:17.993733Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:18:33.507663Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:33.507962Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:33.508088Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:33.517839Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:33.518122Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:33.518228Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:33.584575Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:33.584991Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:33.585306Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:33.678390Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:33.678966Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:33.679229Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:34.072666Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:34.073132Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:34.073755Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:15:35.809976Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:38.370662Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:38.371314Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:38.371580Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:43.492336Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:43.494276Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:43.494551Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:53.735713Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:15:53.775755Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:15:53.776094Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:15:53.776338Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:16.836718Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:18:16.838604Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:16.916899Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:18:16.917317Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmifzvc9-ksxjq Pod phase: Running goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203f00b0, 0xc4201aac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204fc040, 0xc42000f018, 0xc4206905a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200f60c0, 0xc4200c6088, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420410b40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200f60c0, 0xc4200c6088, 0x13, 0xc420277718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42064f240) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42035a040, 0xc4200c83c0, 0xc4209c7ce0, 0xc4209c7cf0, 0xc4202f4300, 0xc42054dc00, 0x10f5e80, 0xc42000f018, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:20:00.787525Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:20:02.040647Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:20:09 http: TLS handshake error from 10.244.0.1:45332: EOF 2018/07/26 11:20:19 http: TLS handshake error from 10.244.0.1:45356: EOF 2018/07/26 11:20:29 http: TLS handshake error from 10.244.0.1:45380: EOF level=info timestamp=2018-07-26T11:20:30.903950Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:20:31.613479Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:20:31.616106Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:20:32.161579Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:20:39 http: TLS handshake error from 10.244.0.1:45404: EOF 2018/07/26 11:20:49 http: TLS handshake error from 10.244.0.1:45428: EOF 2018/07/26 11:20:59 http: TLS handshake error from 10.244.0.1:45452: EOF level=info timestamp=2018-07-26T11:21:01.036300Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:21:02.288734Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:21:09 http: TLS handshake error from 10.244.0.1:45476: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:19:21.788731Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:19:30 http: TLS handshake error from 10.244.1.1:43076: EOF 2018/07/26 11:19:40 http: TLS handshake error from 10.244.1.1:43082: EOF 2018/07/26 11:19:50 http: TLS handshake error from 10.244.1.1:43088: EOF level=info timestamp=2018-07-26T11:19:51.900777Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:20:00 http: TLS handshake error from 10.244.1.1:43094: EOF 2018/07/26 11:20:10 http: TLS handshake error from 10.244.1.1:43100: EOF 2018/07/26 11:20:20 http: TLS handshake error from 10.244.1.1:43106: EOF level=info timestamp=2018-07-26T11:20:21.790123Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:20:30 http: TLS handshake error from 10.244.1.1:43112: EOF 2018/07/26 11:20:40 http: TLS handshake error from 10.244.1.1:43118: EOF 2018/07/26 11:20:50 http: TLS handshake error from 10.244.1.1:43124: EOF level=info timestamp=2018-07-26T11:20:51.874360Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:21:00 http: TLS handshake error from 10.244.1.1:43130: EOF 2018/07/26 11:21:10 http: TLS handshake error from 10.244.1.1:43136: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:08:58.430526Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizv47b\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmizv47b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2fc83f1a-90c4-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizv47b" level=info timestamp=2018-07-26T11:08:58.583757Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:08:58.583907Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmip77fj kind= uid=4b45a166-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:08:58.712847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmip77fj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmip77fj" level=info timestamp=2018-07-26T11:11:59.357960Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:11:59.358206Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminp9vd kind= uid=b7053875-90c4-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:11:59.528721Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:11:59.572466Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminp9vd\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminp9vd" level=info timestamp=2018-07-26T11:15:16.665017Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:15:16.665491Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:15:16.851185Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminbn4q\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:18:17.636845Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:18:17.638076Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:18:17.930538Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:17.993733Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:18:35.081903Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:35.083566Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:36.365242Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:36.365837Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:36.366673Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:38.927802Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:38.929299Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:38.934083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:44.055712Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:44.057570Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:44.057917Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:54.299429Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.347204Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:54.348640Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.348799Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:15:35.809976Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:38.370662Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:38.371314Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:38.371580Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:43.492336Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:15:43.494276Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:15:43.494551Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminbn4q" level=info timestamp=2018-07-26T11:15:53.735713Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:15:53.775755Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:15:53.776094Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:15:53.776338Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:16.836718Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:18:16.838604Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind= uid=2c9faf03-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:16.916899Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:18:16.917317Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminbn4q kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmifzvc9-ksxjq Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203f00b0, 0xc4201aac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204fc040, 0xc42000f018, 0xc4206905a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200f60c0, 0xc4200c6088, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420410b40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200f60c0, 0xc4200c6088, 0x13, 0xc420277718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42064f240) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42035a040, 0xc4200c83c0, 0xc4209c7ce0, 0xc4209c7cf0, 0xc4202f4300, 0xc42054dc00, 0x10f5e80, 0xc42000f018, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [181.162 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with node tainted /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:458 the vmi with tolerations should be scheduled [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:480 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:18:17.582533Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmifzvc9-ksxjq" level=info timestamp=2018-07-26T11:18:32.864383Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmifzvc9-ksxjq" level=error timestamp=2018-07-26T11:18:33.164596Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" • ------------------------------ S [SKIPPING] [0.405 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with non default namespace /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:530 should log libvirt start and stop lifecycle events of the domain /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 kubevirt-test-default [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Skip log query tests for JENKINS ci test environment /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:535 ------------------------------ S [SKIPPING] [0.265 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 with non default namespace /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:530 should log libvirt start and stop lifecycle events of the domain /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 kubevirt-test-alternative [It] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Skip log query tests for JENKINS ci test environment /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:535 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.156 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 VirtualMachineInstance Emulation Mode /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:591 should enable emulation in virt-launcher [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:603 Software emulation is not enabled on this cluster /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:599 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.179 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 VirtualMachineInstance Emulation Mode /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:591 should be reflected in domain XML [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:640 Software emulation is not enabled on this cluster /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:599 ------------------------------ S [SKIPPING] in Spec Setup (BeforeEach) [0.192 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Creating a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:70 VirtualMachineInstance Emulation Mode /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:591 should request a TUN device but not KVM [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:684 Software emulation is not enabled on this cluster /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:599 ------------------------------ •••• Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:20:30.903950Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:20:31.613479Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:20:31.616106Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:20:32.161579Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:20:39 http: TLS handshake error from 10.244.0.1:45404: EOF 2018/07/26 11:20:49 http: TLS handshake error from 10.244.0.1:45428: EOF 2018/07/26 11:20:59 http: TLS handshake error from 10.244.0.1:45452: EOF level=info timestamp=2018-07-26T11:21:01.036300Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:21:02.288734Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:21:09 http: TLS handshake error from 10.244.0.1:45476: EOF 2018/07/26 11:21:19 http: TLS handshake error from 10.244.0.1:45500: EOF 2018/07/26 11:21:29 http: TLS handshake error from 10.244.0.1:45524: EOF level=info timestamp=2018-07-26T11:21:31.163069Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:21:32.421896Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:21:39 http: TLS handshake error from 10.244.0.1:45548: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:19:50 http: TLS handshake error from 10.244.1.1:43088: EOF level=info timestamp=2018-07-26T11:19:51.900777Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:20:00 http: TLS handshake error from 10.244.1.1:43094: EOF 2018/07/26 11:20:10 http: TLS handshake error from 10.244.1.1:43100: EOF 2018/07/26 11:20:20 http: TLS handshake error from 10.244.1.1:43106: EOF level=info timestamp=2018-07-26T11:20:21.790123Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:20:30 http: TLS handshake error from 10.244.1.1:43112: EOF 2018/07/26 11:20:40 http: TLS handshake error from 10.244.1.1:43118: EOF 2018/07/26 11:20:50 http: TLS handshake error from 10.244.1.1:43124: EOF level=info timestamp=2018-07-26T11:20:51.874360Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:21:00 http: TLS handshake error from 10.244.1.1:43130: EOF 2018/07/26 11:21:10 http: TLS handshake error from 10.244.1.1:43136: EOF 2018/07/26 11:21:20 http: TLS handshake error from 10.244.1.1:43142: EOF level=info timestamp=2018-07-26T11:21:21.790215Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:21:30 http: TLS handshake error from 10.244.1.1:43148: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:18:17.930538Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:17.993733Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:21:18.647182Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwr2ff kind= uid=045d8d58-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:18.651193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwr2ff kind= uid=045d8d58-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:18.756199Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.810044Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.843809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:20.237021Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwr2ff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 045d8d58-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:18:36.366673Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:38.927802Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:38.929299Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:38.934083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:44.055712Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:44.057570Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:44.057917Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:54.299429Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.347204Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:54.348640Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.348799Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.442912Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.444122Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.454023Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.454143Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:21:39.979251Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:39.979762Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:39.980009Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:40.000899Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:40.001483Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:40.001833Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:40.042720Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:40.043371Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:40.043708Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:40.124540Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:40.124941Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:40.125346Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:40.286948Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:40.287384Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:40.287703Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" Pod name: virt-launcher-testvmit4scg-wzmhp Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421922040, 0xc420420128, 0xc4202ea420, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096540, 0xc4200b6590, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200bb080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096540, 0xc4200b6590, 0x13, 0xc421942f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420388080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420404080, 0xc4200b87d0, 0xc421a04820, 0xc421a04830, 0xc42193c300, 0xc421914d20, 0x10f5e80, 0xc420420128, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:23:02.758309Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:23:09 http: TLS handshake error from 10.244.0.1:45764: EOF 2018/07/26 11:23:19 http: TLS handshake error from 10.244.0.1:45788: EOF 2018/07/26 11:23:29 http: TLS handshake error from 10.244.0.1:45812: EOF level=info timestamp=2018-07-26T11:23:30.906608Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:23:30.910735Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:23:31.718691Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:23:32.864646Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:23:39 http: TLS handshake error from 10.244.0.1:45836: EOF 2018/07/26 11:23:49 http: TLS handshake error from 10.244.0.1:45860: EOF 2018/07/26 11:23:59 http: TLS handshake error from 10.244.0.1:45884: EOF level=info timestamp=2018-07-26T11:24:01.844765Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:24:02.990303Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:24:09 http: TLS handshake error from 10.244.0.1:45908: EOF 2018/07/26 11:24:19 http: TLS handshake error from 10.244.0.1:45932: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:22:40 http: TLS handshake error from 10.244.1.1:43190: EOF 2018/07/26 11:22:50 http: TLS handshake error from 10.244.1.1:43196: EOF level=info timestamp=2018-07-26T11:22:51.842214Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:23:00 http: TLS handshake error from 10.244.1.1:43202: EOF 2018/07/26 11:23:10 http: TLS handshake error from 10.244.1.1:43208: EOF 2018/07/26 11:23:20 http: TLS handshake error from 10.244.1.1:43214: EOF level=info timestamp=2018-07-26T11:23:21.802113Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:23:30 http: TLS handshake error from 10.244.1.1:43220: EOF 2018/07/26 11:23:40 http: TLS handshake error from 10.244.1.1:43226: EOF 2018/07/26 11:23:50 http: TLS handshake error from 10.244.1.1:43232: EOF level=info timestamp=2018-07-26T11:23:51.796465Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:24:00 http: TLS handshake error from 10.244.1.1:43238: EOF 2018/07/26 11:24:10 http: TLS handshake error from 10.244.1.1:43244: EOF 2018/07/26 11:24:20 http: TLS handshake error from 10.244.1.1:43250: EOF level=info timestamp=2018-07-26T11:24:21.875870Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:18:17.930538Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:17.993733Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifzvc9\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:21:18.647182Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwr2ff kind= uid=045d8d58-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:18.651193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwr2ff kind= uid=045d8d58-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:18.756199Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.810044Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.843809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:20.237021Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwr2ff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 045d8d58-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:18:36.366673Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:38.927802Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:38.929299Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:38.934083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:44.055712Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:44.057570Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:44.057917Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:54.299429Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.347204Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:54.348640Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.348799Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.442912Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.444122Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.454023Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.454143Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:21:41.266708Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:41.266994Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:42.547752Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:42.548403Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:42.548818Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:45.110764Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:45.112550Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:45.113301Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:21:50.235412Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:21:50.237124Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:21:50.237763Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmit4scg" level=info timestamp=2018-07-26T11:22:00.478770Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:22:00.531580Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:22:00.531890Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:22:00.532027Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmit4scg-wzmhp Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421922040, 0xc420420128, 0xc4202ea420, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096540, 0xc4200b6590, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200bb080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096540, 0xc4200b6590, 0x13, 0xc421942f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420388080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420404080, 0xc4200b87d0, 0xc421a04820, 0xc421a04830, 0xc42193c300, 0xc421914d20, 0x10f5e80, 0xc420420128, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 ------------------------------ • Failure [180.644 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Delete a VirtualMachineInstance's Pod /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:836 should result in the VirtualMachineInstance moving to a finalized state [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:837 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Creating the VirtualMachineInstance level=info timestamp=2018-07-26T11:21:22.578293Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmit4scg-wzmhp" level=info timestamp=2018-07-26T11:21:39.742271Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmit4scg-wzmhp" level=error timestamp=2018-07-26T11:21:40.026952Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:23:29 http: TLS handshake error from 10.244.0.1:45812: EOF level=info timestamp=2018-07-26T11:23:30.906608Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:23:30.910735Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:23:31.718691Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:23:32.864646Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:23:39 http: TLS handshake error from 10.244.0.1:45836: EOF 2018/07/26 11:23:49 http: TLS handshake error from 10.244.0.1:45860: EOF 2018/07/26 11:23:59 http: TLS handshake error from 10.244.0.1:45884: EOF level=info timestamp=2018-07-26T11:24:01.844765Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:24:02.990303Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:24:09 http: TLS handshake error from 10.244.0.1:45908: EOF 2018/07/26 11:24:19 http: TLS handshake error from 10.244.0.1:45932: EOF 2018/07/26 11:24:29 http: TLS handshake error from 10.244.0.1:45956: EOF level=info timestamp=2018-07-26T11:24:31.924273Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:24:33.097478Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:22:50 http: TLS handshake error from 10.244.1.1:43196: EOF level=info timestamp=2018-07-26T11:22:51.842214Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:23:00 http: TLS handshake error from 10.244.1.1:43202: EOF 2018/07/26 11:23:10 http: TLS handshake error from 10.244.1.1:43208: EOF 2018/07/26 11:23:20 http: TLS handshake error from 10.244.1.1:43214: EOF level=info timestamp=2018-07-26T11:23:21.802113Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:23:30 http: TLS handshake error from 10.244.1.1:43220: EOF 2018/07/26 11:23:40 http: TLS handshake error from 10.244.1.1:43226: EOF 2018/07/26 11:23:50 http: TLS handshake error from 10.244.1.1:43232: EOF level=info timestamp=2018-07-26T11:23:51.796465Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:24:00 http: TLS handshake error from 10.244.1.1:43238: EOF 2018/07/26 11:24:10 http: TLS handshake error from 10.244.1.1:43244: EOF 2018/07/26 11:24:20 http: TLS handshake error from 10.244.1.1:43250: EOF level=info timestamp=2018-07-26T11:24:21.875870Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:24:30 http: TLS handshake error from 10.244.1.1:43256: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:18.651193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwr2ff kind= uid=045d8d58-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:18.756199Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.810044Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.843809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:20.237021Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwr2ff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 045d8d58-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:18:36.366673Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:38.927802Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:38.929299Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:38.934083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:44.055712Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:44.057570Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:44.057917Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:54.299429Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.347204Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:54.348640Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.348799Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.442912Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.444122Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.454023Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.454143Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:24:37.572634Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:37.572910Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:37.573267Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:37.577924Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:37.578545Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:37.578809Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:37.599589Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:37.599929Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:37.600248Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:37.641016Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:37.641502Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:37.641716Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:37.722459Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:37.722995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:37.723411Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" Pod name: virt-launcher-testvmi6fvrg-hn7q6 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0b0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191e040, 0xc42000e398, 0xc4204039e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6480, 0xc4200b62a8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc421a03540, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6480, 0xc4200b62a8, 0x13, 0xc42193ef18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42021bc00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae3c0, 0xc4200b8870, 0xc42191c850, 0xc42191c860, 0xc421938300, 0xc421910d00, 0x10f5e80, 0xc42000e398, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:25:59 http: TLS handshake error from 10.244.0.1:46172: EOF level=info timestamp=2018-07-26T11:26:02.304492Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:26:03.440252Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:26:09 http: TLS handshake error from 10.244.0.1:46196: EOF 2018/07/26 11:26:19 http: TLS handshake error from 10.244.0.1:46220: EOF 2018/07/26 11:26:29 http: TLS handshake error from 10.244.0.1:46244: EOF level=info timestamp=2018-07-26T11:26:32.450835Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:26:33.558141Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:26:39 http: TLS handshake error from 10.244.0.1:46268: EOF 2018/07/26 11:26:49 http: TLS handshake error from 10.244.0.1:46292: EOF 2018/07/26 11:26:59 http: TLS handshake error from 10.244.0.1:46316: EOF level=info timestamp=2018-07-26T11:27:02.607591Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:27:03.690212Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:27:09 http: TLS handshake error from 10.244.0.1:46340: EOF 2018/07/26 11:27:19 http: TLS handshake error from 10.244.0.1:46364: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:25:52.015017Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:26:00 http: TLS handshake error from 10.244.1.1:43310: EOF 2018/07/26 11:26:10 http: TLS handshake error from 10.244.1.1:43316: EOF 2018/07/26 11:26:20 http: TLS handshake error from 10.244.1.1:43322: EOF level=info timestamp=2018-07-26T11:26:21.939322Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:26:30 http: TLS handshake error from 10.244.1.1:43328: EOF 2018/07/26 11:26:40 http: TLS handshake error from 10.244.1.1:43334: EOF 2018/07/26 11:26:50 http: TLS handshake error from 10.244.1.1:43340: EOF level=info timestamp=2018-07-26T11:26:51.914335Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:27:00 http: TLS handshake error from 10.244.1.1:43346: EOF 2018/07/26 11:27:10 http: TLS handshake error from 10.244.1.1:43352: EOF 2018/07/26 11:27:20 http: TLS handshake error from 10.244.1.1:43358: EOF level=info timestamp=2018-07-26T11:27:21.632630Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:27:21.732850Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:27:21.995914Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:18.651193Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwr2ff kind= uid=045d8d58-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:18.756199Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.810044Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:18.843809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:20.237021Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwr2ff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 045d8d58-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:18:36.366673Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:38.927802Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:38.929299Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:38.934083Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:44.055712Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:18:44.057570Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:18:44.057917Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifzvc9" level=info timestamp=2018-07-26T11:18:54.299429Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.347204Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:18:54.348640Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:18:54.348799Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.442912Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.444122Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind= uid=987829b5-90c5-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:21:18.454023Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:21:18.454143Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifzvc9 kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:24:38.847734Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:38.848011Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:40.128753Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:40.129323Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:40.129628Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:42.690460Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:42.690945Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:42.691393Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:47.812433Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:47.812914Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:47.813228Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:58.055123Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:24:58.111683Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:24:58.111999Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:24:58.112120Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi6fvrg-hn7q6 Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0b0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42191e040, 0xc42000e398, 0xc4204039e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6480, 0xc4200b62a8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc421a03540, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6480, 0xc4200b62a8, 0x13, 0xc42193ef18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42021bc00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ae3c0, 0xc4200b8870, 0xc42191c850, 0xc42191c860, 0xc421938300, 0xc421910d00, 0x10f5e80, 0xc42000e398, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.873 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Delete a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:868 with an active pod. /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:869 should result in pod being terminated [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:870 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Creating the VirtualMachineInstance level=info timestamp=2018-07-26T11:24:23.329490Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmi6fvrg-hn7q6" level=info timestamp=2018-07-26T11:24:37.317799Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmi6fvrg-hn7q6" level=error timestamp=2018-07-26T11:24:37.623149Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:26:32.450835Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:26:33.558141Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:26:39 http: TLS handshake error from 10.244.0.1:46268: EOF 2018/07/26 11:26:49 http: TLS handshake error from 10.244.0.1:46292: EOF 2018/07/26 11:26:59 http: TLS handshake error from 10.244.0.1:46316: EOF level=info timestamp=2018-07-26T11:27:02.607591Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:27:03.690212Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:27:09 http: TLS handshake error from 10.244.0.1:46340: EOF 2018/07/26 11:27:19 http: TLS handshake error from 10.244.0.1:46364: EOF 2018/07/26 11:27:29 http: TLS handshake error from 10.244.0.1:46388: EOF level=info timestamp=2018-07-26T11:27:32.683871Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:27:33.037950Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:27:33.043801Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:27:33.754771Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:27:39 http: TLS handshake error from 10.244.0.1:46412: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:26:10 http: TLS handshake error from 10.244.1.1:43316: EOF 2018/07/26 11:26:20 http: TLS handshake error from 10.244.1.1:43322: EOF level=info timestamp=2018-07-26T11:26:21.939322Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:26:30 http: TLS handshake error from 10.244.1.1:43328: EOF 2018/07/26 11:26:40 http: TLS handshake error from 10.244.1.1:43334: EOF 2018/07/26 11:26:50 http: TLS handshake error from 10.244.1.1:43340: EOF level=info timestamp=2018-07-26T11:26:51.914335Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:27:00 http: TLS handshake error from 10.244.1.1:43346: EOF 2018/07/26 11:27:10 http: TLS handshake error from 10.244.1.1:43352: EOF 2018/07/26 11:27:20 http: TLS handshake error from 10.244.1.1:43358: EOF level=info timestamp=2018-07-26T11:27:21.632630Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:27:21.732850Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:27:21.995914Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:27:30 http: TLS handshake error from 10.244.1.1:43364: EOF 2018/07/26 11:27:40 http: TLS handshake error from 10.244.1.1:43370: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:18.843809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:20.237021Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwr2ff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 045d8d58-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:27:24.644977Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:27:24.645574Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:27:24.805604Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij9tdc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:27:43.371596Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:43.371822Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:43.371964Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:43.376729Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:43.377164Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:43.378052Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:43.401723Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:43.402985Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:43.403300Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:43.447540Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:43.447997Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:43.449299Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:43.530146Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:43.530999Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:43.531449Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:24:40.129628Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:42.690460Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:42.690945Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:42.691393Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:47.812433Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:47.812914Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:47.813228Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:58.055123Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:24:58.111683Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:24:58.111999Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:24:58.112120Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:27:23.763400Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:27:23.765114Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:27:23.823081Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:27:23.823465Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmij9tdc-qbsfb Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926060, 0xc4200b65c8, 0xc4202eaa60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c240, 0xc42000e080, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4202ac4c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c240, 0xc42000e080, 0x13, 0xc421958f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420404040, 0xc4200e4320, 0xc4206d2c00, 0xc4206d2c10, 0xc421952300, 0xc421918d40, 0x10f5e80, 0xc4200b65c8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:28:59 http: TLS handshake error from 10.244.0.1:46604: EOF level=info timestamp=2018-07-26T11:29:03.115907Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:29:04.209909Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:29:09 http: TLS handshake error from 10.244.0.1:46628: EOF 2018/07/26 11:29:19 http: TLS handshake error from 10.244.0.1:46652: EOF 2018/07/26 11:29:29 http: TLS handshake error from 10.244.0.1:46676: EOF level=info timestamp=2018-07-26T11:29:33.229995Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:29:34.325265Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:29:39 http: TLS handshake error from 10.244.0.1:46700: EOF 2018/07/26 11:29:49 http: TLS handshake error from 10.244.0.1:46724: EOF 2018/07/26 11:29:59 http: TLS handshake error from 10.244.0.1:46748: EOF level=info timestamp=2018-07-26T11:30:03.392706Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:30:04.435730Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:30:09 http: TLS handshake error from 10.244.0.1:46772: EOF 2018/07/26 11:30:19 http: TLS handshake error from 10.244.0.1:46796: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:28:40 http: TLS handshake error from 10.244.1.1:43406: EOF 2018/07/26 11:28:50 http: TLS handshake error from 10.244.1.1:43412: EOF level=info timestamp=2018-07-26T11:28:51.912445Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:29:00 http: TLS handshake error from 10.244.1.1:43418: EOF 2018/07/26 11:29:10 http: TLS handshake error from 10.244.1.1:43424: EOF 2018/07/26 11:29:20 http: TLS handshake error from 10.244.1.1:43430: EOF level=info timestamp=2018-07-26T11:29:21.883298Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:29:30 http: TLS handshake error from 10.244.1.1:43436: EOF 2018/07/26 11:29:40 http: TLS handshake error from 10.244.1.1:43442: EOF 2018/07/26 11:29:50 http: TLS handshake error from 10.244.1.1:43448: EOF level=info timestamp=2018-07-26T11:29:51.897820Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:30:00 http: TLS handshake error from 10.244.1.1:43454: EOF 2018/07/26 11:30:10 http: TLS handshake error from 10.244.1.1:43460: EOF 2018/07/26 11:30:20 http: TLS handshake error from 10.244.1.1:43466: EOF level=info timestamp=2018-07-26T11:30:21.914837Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:18.843809Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:20.237021Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwr2ff\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwr2ff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 045d8d58-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwr2ff" level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:27:24.644977Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:27:24.645574Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:27:24.805604Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij9tdc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:27:44.656165Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:44.656677Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:45.937752Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:45.938205Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:45.939950Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:48.500925Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:48.501555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:48.501923Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:53.623061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:53.623958Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:53.624099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:28:03.865230Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.923560Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:28:03.926475Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.926893Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:24:40.129628Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:42.690460Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:42.690945Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:42.691393Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:47.812433Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:24:47.812914Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:24:47.813228Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:24:58.055123Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:24:58.111683Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:24:58.111999Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:24:58.112120Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:27:23.763400Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:27:23.765114Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:27:23.823081Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:27:23.823465Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi6fvrg kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmij9tdc-qbsfb Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926060, 0xc4200b65c8, 0xc4202eaa60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c240, 0xc42000e080, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4202ac4c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c240, 0xc42000e080, 0x13, 0xc421958f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420404040, 0xc4200e4320, 0xc4206d2c00, 0xc4206d2c10, 0xc421952300, 0xc421918d40, 0x10f5e80, 0xc4200b65c8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [181.098 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Delete a VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:868 with grace period greater than 0 /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:893 should run graceful shutdown [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:894 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Setting a VirtualMachineInstance termination grace period to 5 STEP: Creating the VirtualMachineInstance level=info timestamp=2018-07-26T11:27:24.508829Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmij9tdc-qbsfb" level=info timestamp=2018-07-26T11:27:42.738489Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmij9tdc-qbsfb" level=error timestamp=2018-07-26T11:27:43.019947Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:29:33.229995Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:29:34.325265Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:29:39 http: TLS handshake error from 10.244.0.1:46700: EOF 2018/07/26 11:29:49 http: TLS handshake error from 10.244.0.1:46724: EOF 2018/07/26 11:29:59 http: TLS handshake error from 10.244.0.1:46748: EOF level=info timestamp=2018-07-26T11:30:03.392706Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:30:04.435730Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:30:09 http: TLS handshake error from 10.244.0.1:46772: EOF 2018/07/26 11:30:19 http: TLS handshake error from 10.244.0.1:46796: EOF 2018/07/26 11:30:29 http: TLS handshake error from 10.244.0.1:46820: EOF level=info timestamp=2018-07-26T11:30:33.067204Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:30:33.081038Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:30:33.522765Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:30:34.509113Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:30:39 http: TLS handshake error from 10.244.0.1:46844: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:28:51.912445Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:29:00 http: TLS handshake error from 10.244.1.1:43418: EOF 2018/07/26 11:29:10 http: TLS handshake error from 10.244.1.1:43424: EOF 2018/07/26 11:29:20 http: TLS handshake error from 10.244.1.1:43430: EOF level=info timestamp=2018-07-26T11:29:21.883298Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:29:30 http: TLS handshake error from 10.244.1.1:43436: EOF 2018/07/26 11:29:40 http: TLS handshake error from 10.244.1.1:43442: EOF 2018/07/26 11:29:50 http: TLS handshake error from 10.244.1.1:43448: EOF level=info timestamp=2018-07-26T11:29:51.897820Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:30:00 http: TLS handshake error from 10.244.1.1:43454: EOF 2018/07/26 11:30:10 http: TLS handshake error from 10.244.1.1:43460: EOF 2018/07/26 11:30:20 http: TLS handshake error from 10.244.1.1:43466: EOF level=info timestamp=2018-07-26T11:30:21.914837Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:30:30 http: TLS handshake error from 10.244.1.1:43472: EOF 2018/07/26 11:30:40 http: TLS handshake error from 10.244.1.1:43478: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:27:24.644977Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:27:24.645574Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:27:24.805604Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij9tdc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:30:25.433031Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:25.434800Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:27:45.939950Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:48.500925Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:48.501555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:48.501923Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:53.623061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:53.623958Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:53.624099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:28:03.865230Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.923560Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:28:03.926475Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.926893Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.224419Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.225296Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.250869Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.251094Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:30:41.052939Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:41.053479Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:41.053985Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:41.058285Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:41.058708Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:41.059150Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:41.081017Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:41.081467Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:41.081972Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:41.122656Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:41.123146Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:41.123699Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:41.204508Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:41.204855Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:41.205094Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" Pod name: virt-launcher-testvmisgbjr-hqmrj Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 8 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278880, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204013e0, 0xc42000e448, 0xc4205641a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096f60, 0xc4200b2318, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200b6040, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096f60, 0xc4200b2318, 0x13, 0xc42159d718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4218f4300) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200b6640, 0xc4200b4960, 0xc420681260, 0xc420681270, 0xc42064dd80, 0xc42044c100, 0x10f5e80, 0xc42000e448, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:32:04.864987Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:32:09 http: TLS handshake error from 10.244.0.1:47060: EOF 2018/07/26 11:32:19 http: TLS handshake error from 10.244.0.1:47084: EOF 2018/07/26 11:32:29 http: TLS handshake error from 10.244.0.1:47108: EOF level=info timestamp=2018-07-26T11:32:31.063121Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:32:31.071642Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:32:34.014832Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:32:34.982647Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:32:39 http: TLS handshake error from 10.244.0.1:47132: EOF 2018/07/26 11:32:49 http: TLS handshake error from 10.244.0.1:47156: EOF 2018/07/26 11:32:59 http: TLS handshake error from 10.244.0.1:47180: EOF level=info timestamp=2018-07-26T11:33:04.149680Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:33:05.138957Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:33:09 http: TLS handshake error from 10.244.0.1:47204: EOF 2018/07/26 11:33:19 http: TLS handshake error from 10.244.0.1:47228: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:31:40 http: TLS handshake error from 10.244.1.1:43514: EOF 2018/07/26 11:31:50 http: TLS handshake error from 10.244.1.1:43520: EOF level=info timestamp=2018-07-26T11:31:51.950863Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:00 http: TLS handshake error from 10.244.1.1:43526: EOF 2018/07/26 11:32:10 http: TLS handshake error from 10.244.1.1:43532: EOF 2018/07/26 11:32:20 http: TLS handshake error from 10.244.1.1:43538: EOF level=info timestamp=2018-07-26T11:32:22.021017Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:30 http: TLS handshake error from 10.244.1.1:43544: EOF 2018/07/26 11:32:40 http: TLS handshake error from 10.244.1.1:43550: EOF 2018/07/26 11:32:50 http: TLS handshake error from 10.244.1.1:43556: EOF level=info timestamp=2018-07-26T11:32:51.970539Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:33:00 http: TLS handshake error from 10.244.1.1:43562: EOF 2018/07/26 11:33:10 http: TLS handshake error from 10.244.1.1:43568: EOF 2018/07/26 11:33:20 http: TLS handshake error from 10.244.1.1:43574: EOF level=info timestamp=2018-07-26T11:33:21.961830Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:21.596265Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.597858Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi565wt kind= uid=061ff8a1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:21.908856Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi565wt\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi565wt, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 061ff8a1-90c6-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi565wt" level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:27:24.644977Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:27:24.645574Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:27:24.805604Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij9tdc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:30:25.433031Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:25.434800Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:27:45.939950Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:48.500925Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:48.501555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:48.501923Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:53.623061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:53.623958Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:53.624099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:28:03.865230Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.923560Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:28:03.926475Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.926893Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.224419Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.225296Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.250869Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.251094Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:30:42.329995Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:42.330334Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:43.611127Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:43.611747Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:43.612049Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:46.172779Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:46.173802Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:46.174143Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:30:51.295355Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:30:51.296287Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:30:51.296565Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisgbjr" level=info timestamp=2018-07-26T11:31:01.538336Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:31:01.603889Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:31:01.604937Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:31:01.605582Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmisgbjr-hqmrj Pod phase: Failed goroutine 8 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278880, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4204013e0, 0xc42000e448, 0xc4205641a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096f60, 0xc4200b2318, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4200b6040, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096f60, 0xc4200b2318, 0x13, 0xc42159d718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4218f4300) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200b6640, 0xc4200b4960, 0xc420681260, 0xc420681270, 0xc42064dd80, 0xc42044c100, 0x10f5e80, 0xc42000e448, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.673 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Killed VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:945 should be in Failed phase [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:946 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting a VirtualMachineInstance level=info timestamp=2018-07-26T11:30:25.274877Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmisgbjr-hqmrj" level=info timestamp=2018-07-26T11:30:40.808004Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmisgbjr-hqmrj" level=error timestamp=2018-07-26T11:30:41.080310Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:32:34.014832Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:32:34.982647Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:32:39 http: TLS handshake error from 10.244.0.1:47132: EOF 2018/07/26 11:32:49 http: TLS handshake error from 10.244.0.1:47156: EOF 2018/07/26 11:32:59 http: TLS handshake error from 10.244.0.1:47180: EOF level=info timestamp=2018-07-26T11:33:04.149680Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:33:05.138957Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:33:09 http: TLS handshake error from 10.244.0.1:47204: EOF 2018/07/26 11:33:19 http: TLS handshake error from 10.244.0.1:47228: EOF 2018/07/26 11:33:29 http: TLS handshake error from 10.244.0.1:47252: EOF level=info timestamp=2018-07-26T11:33:31.077491Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:33:31.081750Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:33:34.274708Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:33:35.249268Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:33:39 http: TLS handshake error from 10.244.0.1:47276: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:31:51.950863Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:00 http: TLS handshake error from 10.244.1.1:43526: EOF 2018/07/26 11:32:10 http: TLS handshake error from 10.244.1.1:43532: EOF 2018/07/26 11:32:20 http: TLS handshake error from 10.244.1.1:43538: EOF level=info timestamp=2018-07-26T11:32:22.021017Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:32:30 http: TLS handshake error from 10.244.1.1:43544: EOF 2018/07/26 11:32:40 http: TLS handshake error from 10.244.1.1:43550: EOF 2018/07/26 11:32:50 http: TLS handshake error from 10.244.1.1:43556: EOF level=info timestamp=2018-07-26T11:32:51.970539Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:33:00 http: TLS handshake error from 10.244.1.1:43562: EOF 2018/07/26 11:33:10 http: TLS handshake error from 10.244.1.1:43568: EOF 2018/07/26 11:33:20 http: TLS handshake error from 10.244.1.1:43574: EOF level=info timestamp=2018-07-26T11:33:21.961830Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:33:30 http: TLS handshake error from 10.244.1.1:43580: EOF 2018/07/26 11:33:40 http: TLS handshake error from 10.244.1.1:43586: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:27:24.644977Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:27:24.645574Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:27:24.805604Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij9tdc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:30:25.433031Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:25.434800Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.192262Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:26.192965Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.445446Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihtrfr\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:27:45.939950Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:48.500925Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:48.501555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:48.501923Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:53.623061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:53.623958Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:53.624099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:28:03.865230Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.923560Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:28:03.926475Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.926893Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.224419Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.225296Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.250869Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.251094Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:33:42.400543Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:42.400894Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:42.401098Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:42.421873Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:42.422599Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:42.422887Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:42.463604Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:42.464074Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:42.464401Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:42.545012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:42.545553Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:42.545857Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:42.706936Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:42.707595Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:42.707866Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" Pod name: virt-launcher-testvmihtrfr-h644f Pod phase: Running goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201cc0d0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926040, 0xc4203ea100, 0xc420214240, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b2060, 0xc4200d4078, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420088cc0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b2060, 0xc4200d4078, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420318040, 0xc4200d2320, 0xc421a33d70, 0xc421a33d80, 0xc421952300, 0xc42191ad20, 0x10f5e80, 0xc4203ea100, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:35:05.601755Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:35:09 http: TLS handshake error from 10.244.0.1:47492: EOF 2018/07/26 11:35:19 http: TLS handshake error from 10.244.0.1:47516: EOF 2018/07/26 11:35:29 http: TLS handshake error from 10.244.0.1:47540: EOF level=info timestamp=2018-07-26T11:35:31.097086Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:35:31.108074Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:35:34.790627Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:35:35.722169Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:35:39 http: TLS handshake error from 10.244.0.1:47564: EOF 2018/07/26 11:35:49 http: TLS handshake error from 10.244.0.1:47588: EOF 2018/07/26 11:35:59 http: TLS handshake error from 10.244.0.1:47612: EOF level=info timestamp=2018-07-26T11:36:04.919182Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:36:05.853052Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:36:09 http: TLS handshake error from 10.244.0.1:47636: EOF 2018/07/26 11:36:19 http: TLS handshake error from 10.244.0.1:47660: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:34:40 http: TLS handshake error from 10.244.1.1:43622: EOF 2018/07/26 11:34:50 http: TLS handshake error from 10.244.1.1:43628: EOF level=info timestamp=2018-07-26T11:34:51.913904Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:35:00 http: TLS handshake error from 10.244.1.1:43634: EOF 2018/07/26 11:35:10 http: TLS handshake error from 10.244.1.1:43640: EOF 2018/07/26 11:35:20 http: TLS handshake error from 10.244.1.1:43646: EOF level=info timestamp=2018-07-26T11:35:21.955916Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:35:30 http: TLS handshake error from 10.244.1.1:43652: EOF 2018/07/26 11:35:40 http: TLS handshake error from 10.244.1.1:43658: EOF 2018/07/26 11:35:50 http: TLS handshake error from 10.244.1.1:43664: EOF level=info timestamp=2018-07-26T11:35:51.967412Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:36:00 http: TLS handshake error from 10.244.1.1:43670: EOF 2018/07/26 11:36:10 http: TLS handshake error from 10.244.1.1:43676: EOF 2018/07/26 11:36:20 http: TLS handshake error from 10.244.1.1:43682: EOF level=info timestamp=2018-07-26T11:36:21.867720Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:21:21.994112Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:21.994243Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiprppg kind= uid=066246b9-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:21:22.706972Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:21:22.707208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmit4scg kind= uid=06cc9ef1-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.481473Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:24:23.482022Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi6fvrg kind= uid=728e1b4a-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:24:23.544787Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi6fvrg\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi6fvrg" level=info timestamp=2018-07-26T11:27:24.644977Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:27:24.645574Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:27:24.805604Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmij9tdc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:30:25.433031Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:25.434800Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.192262Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:26.192965Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.445446Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihtrfr\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:27:45.939950Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:48.500925Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:48.501555Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:48.501923Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:27:53.623061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:27:53.623958Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:27:53.624099Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmij9tdc" level=info timestamp=2018-07-26T11:28:03.865230Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.923560Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:28:03.926475Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:28:03.926893Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.224419Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.225296Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind= uid=de8600ce-90c6-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:30:25.250869Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:30:25.251094Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmij9tdc kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:33:43.670586Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:43.670909Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:44.958457Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:44.958934Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:44.959272Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:47.520046Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:47.520582Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:47.520807Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:33:52.641558Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:33:52.642032Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:33:52.642401Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:34:02.883594Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:34:02.914409Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:34:02.918438Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:34:02.918680Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmihtrfr-h644f Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201cc0d0, 0xc42018cc80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926040, 0xc4203ea100, 0xc420214240, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b2060, 0xc4200d4078, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420088cc0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b2060, 0xc4200d4078, 0x13, 0xc421957f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420318040, 0xc4200d2320, 0xc421a33d70, 0xc421a33d80, 0xc421952300, 0xc42191ad20, 0x10f5e80, 0xc4203ea100, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure [180.783 seconds] VMIlifecycle /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:48 Killed VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:945 should be left alone by virt-handler [It] /root/go/src/kubevirt.io/kubevirt/tests/vmi_lifecycle_test.go:973 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ STEP: Starting a VirtualMachineInstance level=info timestamp=2018-07-26T11:33:26.039209Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmihtrfr-h644f" level=info timestamp=2018-07-26T11:33:42.099594Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmihtrfr-h644f" level=error timestamp=2018-07-26T11:33:42.403659Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:35:31.097086Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:35:31.108074Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:35:34.790627Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:35:35.722169Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:35:39 http: TLS handshake error from 10.244.0.1:47564: EOF 2018/07/26 11:35:49 http: TLS handshake error from 10.244.0.1:47588: EOF 2018/07/26 11:35:59 http: TLS handshake error from 10.244.0.1:47612: EOF level=info timestamp=2018-07-26T11:36:04.919182Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:36:05.853052Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:36:09 http: TLS handshake error from 10.244.0.1:47636: EOF 2018/07/26 11:36:19 http: TLS handshake error from 10.244.0.1:47660: EOF 2018/07/26 11:36:29 http: TLS handshake error from 10.244.0.1:47684: EOF level=info timestamp=2018-07-26T11:36:35.051185Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:36:35.930432Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:36:39 http: TLS handshake error from 10.244.0.1:47708: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:34:51.913904Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:35:00 http: TLS handshake error from 10.244.1.1:43634: EOF 2018/07/26 11:35:10 http: TLS handshake error from 10.244.1.1:43640: EOF 2018/07/26 11:35:20 http: TLS handshake error from 10.244.1.1:43646: EOF level=info timestamp=2018-07-26T11:35:21.955916Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:35:30 http: TLS handshake error from 10.244.1.1:43652: EOF 2018/07/26 11:35:40 http: TLS handshake error from 10.244.1.1:43658: EOF 2018/07/26 11:35:50 http: TLS handshake error from 10.244.1.1:43664: EOF level=info timestamp=2018-07-26T11:35:51.967412Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:36:00 http: TLS handshake error from 10.244.1.1:43670: EOF 2018/07/26 11:36:10 http: TLS handshake error from 10.244.1.1:43676: EOF 2018/07/26 11:36:20 http: TLS handshake error from 10.244.1.1:43682: EOF level=info timestamp=2018-07-26T11:36:21.867720Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:36:30 http: TLS handshake error from 10.244.1.1:43688: EOF 2018/07/26 11:36:40 http: TLS handshake error from 10.244.1.1:43694: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:30:25.433031Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:25.434800Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.192262Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:26.192965Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.445446Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihtrfr\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:36:27.046826Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.048030Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.129294Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.129756Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.142197Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.144049Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.215818Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.216127Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.494387Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmimm6kz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:27.858925Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminpctf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:36:43.386642Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:43.386865Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:43.386978Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:43.467466Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:43.467778Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:43.467929Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:43.628994Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:43.629917Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:43.630163Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:43.950852Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:43.951311Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:43.951725Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:44.592530Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:44.593488Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.594031Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T11:36:44.113096Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.113449Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:44.155591Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:44.156067Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.156412Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:44.236970Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:44.237460Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.237778Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:44.393044Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=info timestamp=2018-07-26T11:36:44.400606Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:44.401026Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.401300Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:44.444676Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:44.445041Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.445417Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmicp25v" Pod name: virt-launcher-testvmicp25v-xmn4b Pod phase: Running goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202a43c0, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192aca0, 0xc42000ed50, 0xc420169de0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42020ec60, 0xc42000e108, 0x13, 0x131ed4d, 0x4, 0xc421942f18, 0x3, 0x3, 0xc420316800, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42020ec60, 0xc42000e108, 0x13, 0xc42197cf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420316480, 0xc4200a8910, 0xc421967a10, 0xc421967a20, 0xc42192de00, 0xc421936780, 0x10f5e80, 0xc42000ed50, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmidzkxc-rckdg Pod phase: Running level=info timestamp=2018-07-26T11:36:31.932339Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:36:31.932461Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:36:31.937071Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:36:42.583414Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:36:42.621599Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmidzkxc" level=info timestamp=2018-07-26T11:36:42.622522Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:36:42.622724Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" level=info timestamp=2018-07-26T11:36:44.435824Z pos=cloud-init.go:254 component=virt-launcher msg="generated nocloud iso file /var/run/libvirt/kubevirt-ephemeral-disk/cloud-init-data/kubevirt-test-default/testvmidzkxc/noCloud.iso" level=error timestamp=2018-07-26T11:36:44.452761Z pos=common.go:126 component=virt-launcher msg="updated MAC for interface: eth0 - 0a:58:0a:13:48:fc" level=info timestamp=2018-07-26T11:36:44.462675Z pos=converter.go:751 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \n`\u0000\n" level=info timestamp=2018-07-26T11:36:44.462848Z pos=converter.go:752 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local" level=info timestamp=2018-07-26T11:36:44.463228Z pos=dhcp.go:62 component=virt-launcher msg="Starting SingleClientDHCPServer" level=info timestamp=2018-07-26T11:36:44.665769Z pos=manager.go:157 component=virt-launcher namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Domain defined." Pod name: virt-launcher-testvmimm6kz-9mtnc Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0b0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42033e080, 0xc4200b6ce8, 0xc42067ad60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e088, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200bbb00, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e088, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ae480) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4204060c0, 0xc42012e4b0, 0xc42064e690, 0xc42064e6a0, 0xc42085c300, 0xc4202ec760, 0x10f5e80, 0xc4200b6ce8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvminpctf-nvsnf Pod phase: Running goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192e040, 0xc42000e8d0, 0xc4203f4540, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4202b4240, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e070, 0x13, 0xc421950f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202b4040, 0xc4200e4190, 0xc421a4ee60, 0xc421a4ee70, 0xc42194a300, 0xc421922d40, 0x10f5e80, 0xc42000e8d0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:38:06.299188Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:38:09 http: TLS handshake error from 10.244.0.1:47924: EOF 2018/07/26 11:38:19 http: TLS handshake error from 10.244.0.1:47948: EOF 2018/07/26 11:38:29 http: TLS handshake error from 10.244.0.1:47972: EOF level=info timestamp=2018-07-26T11:38:31.208050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:38:31.212981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:38:35.583048Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:38:36.424723Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:38:39 http: TLS handshake error from 10.244.0.1:47996: EOF 2018/07/26 11:38:49 http: TLS handshake error from 10.244.0.1:48020: EOF 2018/07/26 11:38:59 http: TLS handshake error from 10.244.0.1:48044: EOF level=info timestamp=2018-07-26T11:39:05.727605Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:06.544046Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:39:09 http: TLS handshake error from 10.244.0.1:48068: EOF 2018/07/26 11:39:19 http: TLS handshake error from 10.244.0.1:48092: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:37:40 http: TLS handshake error from 10.244.1.1:43730: EOF 2018/07/26 11:37:50 http: TLS handshake error from 10.244.1.1:43736: EOF level=info timestamp=2018-07-26T11:37:51.840876Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:38:00 http: TLS handshake error from 10.244.1.1:43742: EOF 2018/07/26 11:38:10 http: TLS handshake error from 10.244.1.1:43748: EOF 2018/07/26 11:38:20 http: TLS handshake error from 10.244.1.1:43754: EOF level=info timestamp=2018-07-26T11:38:21.966985Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:38:30 http: TLS handshake error from 10.244.1.1:43760: EOF 2018/07/26 11:38:40 http: TLS handshake error from 10.244.1.1:43766: EOF 2018/07/26 11:38:50 http: TLS handshake error from 10.244.1.1:43772: EOF level=info timestamp=2018-07-26T11:38:51.960659Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:39:00 http: TLS handshake error from 10.244.1.1:43778: EOF 2018/07/26 11:39:10 http: TLS handshake error from 10.244.1.1:43784: EOF 2018/07/26 11:39:20 http: TLS handshake error from 10.244.1.1:43790: EOF level=info timestamp=2018-07-26T11:39:21.940869Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:30:25.433031Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:30:25.434800Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisgbjr kind= uid=4a4a33ac-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.192262Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:33:26.192965Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihtrfr kind= uid=b6066f65-90c7-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:33:26.445446Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihtrfr\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihtrfr" level=info timestamp=2018-07-26T11:36:27.046826Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.048030Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.129294Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.129756Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.142197Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.144049Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.215818Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.216127Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.494387Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmimm6kz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:27.858925Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminpctf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:36:44.593488Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:44.594031Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:45.874747Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:45.875251Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:45.876238Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:48.437963Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:48.438652Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:48.439005Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:36:53.560289Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:53.562003Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:53.562675Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:37:03.804197Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:03.862616Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:03.863543Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:03.864028Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:36:54.958143Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:36:54.958763Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:36:54.958995Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmidzkxc" level=info timestamp=2018-07-26T11:37:03.652929Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:03.682297Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:03.682588Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:03.682706Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:04.571553Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:04.614456Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:04.614733Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:04.614882Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmimm6kz kind= uid=21d04d0f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:05.200584Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:05.236625Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:37:05.236958Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:37:05.237077Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmidzkxc kind= uid=21d7f482-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmicp25v-xmn4b Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202a43c0, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192aca0, 0xc42000ed50, 0xc420169de0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42020ec60, 0xc42000e108, 0x13, 0x131ed4d, 0x4, 0xc421942f18, 0x3, 0x3, 0xc420316800, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42020ec60, 0xc42000e108, 0x13, 0xc42197cf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420316480, 0xc4200a8910, 0xc421967a10, 0xc421967a20, 0xc42192de00, 0xc421936780, 0x10f5e80, 0xc42000ed50, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmidzkxc-rckdg Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d4380, 0xc420104280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421942ca0, 0xc42000fd80, 0xc420187520, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c660, 0xc42000e530, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc42034a800, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c660, 0xc42000e530, 0x13, 0xc421a06f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421b35bc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a580, 0xc42015ca50, 0xc421a2ad30, 0xc421a2ad40, 0xc421945d80, 0xc421987a60, 0x10f5e80, 0xc42000fd80, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmimm6kz-9mtnc Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0b0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42033e080, 0xc4200b6ce8, 0xc42067ad60, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e088, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200bbb00, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e088, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ae480) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4204060c0, 0xc42012e4b0, 0xc42064e690, 0xc42064e6a0, 0xc42085c300, 0xc4202ec760, 0x10f5e80, 0xc4200b6ce8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminpctf-nvsnf Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a64b0, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42192e040, 0xc42000e8d0, 0xc4203f4540, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e070, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4202b4240, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e070, 0x13, 0xc421950f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202b4040, 0xc4200e4190, 0xc421a4ee60, 0xc421a4ee70, 0xc42194a300, 0xc421922d40, 0x10f5e80, 0xc42000e8d0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.399 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the Inbound VirtualMachineInstance /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:36:27.224968Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmimm6kz-9mtnc" level=info timestamp=2018-07-26T11:36:43.896356Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmimm6kz-9mtnc" level=error timestamp=2018-07-26T11:36:44.098313Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:38:31.208050Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:38:31.212981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:38:35.583048Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:38:36.424723Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:38:39 http: TLS handshake error from 10.244.0.1:47996: EOF 2018/07/26 11:38:49 http: TLS handshake error from 10.244.0.1:48020: EOF 2018/07/26 11:38:59 http: TLS handshake error from 10.244.0.1:48044: EOF level=info timestamp=2018-07-26T11:39:05.727605Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:06.544046Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:39:09 http: TLS handshake error from 10.244.0.1:48068: EOF 2018/07/26 11:39:19 http: TLS handshake error from 10.244.0.1:48092: EOF 2018/07/26 11:39:29 http: TLS handshake error from 10.244.0.1:48116: EOF level=info timestamp=2018-07-26T11:39:35.827129Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:39:36.613520Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:39:39 http: TLS handshake error from 10.244.0.1:48140: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:37:51.840876Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:38:00 http: TLS handshake error from 10.244.1.1:43742: EOF 2018/07/26 11:38:10 http: TLS handshake error from 10.244.1.1:43748: EOF 2018/07/26 11:38:20 http: TLS handshake error from 10.244.1.1:43754: EOF level=info timestamp=2018-07-26T11:38:21.966985Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:38:30 http: TLS handshake error from 10.244.1.1:43760: EOF 2018/07/26 11:38:40 http: TLS handshake error from 10.244.1.1:43766: EOF 2018/07/26 11:38:50 http: TLS handshake error from 10.244.1.1:43772: EOF level=info timestamp=2018-07-26T11:38:51.960659Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:39:00 http: TLS handshake error from 10.244.1.1:43778: EOF 2018/07/26 11:39:10 http: TLS handshake error from 10.244.1.1:43784: EOF 2018/07/26 11:39:20 http: TLS handshake error from 10.244.1.1:43790: EOF level=info timestamp=2018-07-26T11:39:21.940869Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:39:30 http: TLS handshake error from 10.244.1.1:43796: EOF 2018/07/26 11:39:40 http: TLS handshake error from 10.244.1.1:43802: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:36:27.142197Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.144049Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.215818Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.216127Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.494387Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmimm6kz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:27.858925Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminpctf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:39:28.923720Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:28.932153Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:28.966850Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:28.967170Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.049660Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.050033Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.101162Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.103215Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.249515Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmid6xbq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:39:44.943217Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:44.943741Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:44.944017Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:45.024787Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:45.025215Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:45.025598Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:45.186276Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:45.187276Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:45.189874Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:45.510624Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:45.511060Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:45.511396Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:46.154425Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:46.156069Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.169231Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:39:46.157720Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:46.158089Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.158409Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" level=info timestamp=2018-07-26T11:39:46.178990Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:46.179384Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.179622Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" level=info timestamp=2018-07-26T11:39:46.220117Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:46.220492Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.220737Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" level=info timestamp=2018-07-26T11:39:46.301321Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:46.301701Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.302000Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" level=info timestamp=2018-07-26T11:39:46.462659Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:46.463119Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.463489Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" Pod name: virt-launcher-testvmi7hxbk-mvcvh Pod phase: Running level=info timestamp=2018-07-26T11:39:33.376063Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:39:33.376298Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:39:33.386353Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:39:45.706045Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:39:45.742829Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi7hxbk" level=info timestamp=2018-07-26T11:39:45.745433Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:39:45.745909Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" level=info timestamp=2018-07-26T11:39:46.628915Z pos=cloud-init.go:254 component=virt-launcher msg="generated nocloud iso file /var/run/libvirt/kubevirt-ephemeral-disk/cloud-init-data/kubevirt-test-default/testvmi7hxbk/noCloud.iso" level=error timestamp=2018-07-26T11:39:46.646554Z pos=common.go:126 component=virt-launcher msg="updated MAC for interface: eth0 - 0a:58:0a:34:19:fa" level=info timestamp=2018-07-26T11:39:46.654587Z pos=converter.go:751 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \n`\u0000\n" level=info timestamp=2018-07-26T11:39:46.654745Z pos=converter.go:752 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local" level=info timestamp=2018-07-26T11:39:46.656676Z pos=dhcp.go:62 component=virt-launcher msg="Starting SingleClientDHCPServer" Pod name: virt-launcher-testvmid6xbq-xmjr7 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42036e3a0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205a2060, 0xc4206b2788, 0xc42044b0a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6cc0, 0xc42000e0d8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bac80, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6cc0, 0xc42000e0d8, 0x13, 0xc420059f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568000, 0xc4200e4820, 0xc42196ea70, 0xc42196ea80, 0xc421965e00, 0xc421970140, 0x10f5e80, 0xc4206b2788, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmirrt7b-rfb8d Pod phase: Running goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42025e0b0, 0xc420166780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc42000ea28, 0xc4203ba860, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6420, 0xc4200b6410, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc421a19f80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6420, 0xc4200b6410, 0x13, 0xc421957f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a8b940) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba2c0, 0xc4200b87d0, 0xc4219f3b50, 0xc4219f3b60, 0xc421952300, 0xc421926d00, 0x10f5e80, 0xc42000ea28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmizxddl-vbrjm Pod phase: Running level=info timestamp=2018-07-26T11:39:33.975307Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:39:33.975444Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:39:33.985511Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:39:44.874575Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:39:44.965623Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmizxddl" level=info timestamp=2018-07-26T11:39:44.981398Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:39:44.981730Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" level=info timestamp=2018-07-26T11:39:46.927528Z pos=cloud-init.go:254 component=virt-launcher msg="generated nocloud iso file /var/run/libvirt/kubevirt-ephemeral-disk/cloud-init-data/kubevirt-test-default/testvmizxddl/noCloud.iso" level=error timestamp=2018-07-26T11:39:46.943779Z pos=common.go:126 component=virt-launcher msg="updated MAC for interface: eth0 - 0a:58:0a:6e:66:c3" level=info timestamp=2018-07-26T11:39:46.955963Z pos=converter.go:751 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \n`\u0000\n" level=info timestamp=2018-07-26T11:39:46.956114Z pos=converter.go:752 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local" level=info timestamp=2018-07-26T11:39:46.957793Z pos=dhcp.go:62 component=virt-launcher msg="Starting SingleClientDHCPServer" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:40:59 http: TLS handshake error from 10.244.0.1:48332: EOF level=info timestamp=2018-07-26T11:41:06.281758Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:41:07.010096Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:41:09 http: TLS handshake error from 10.244.0.1:48356: EOF 2018/07/26 11:41:19 http: TLS handshake error from 10.244.0.1:48380: EOF 2018/07/26 11:41:29 http: TLS handshake error from 10.244.0.1:48404: EOF level=info timestamp=2018-07-26T11:41:36.413683Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:41:37.146303Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:41:39 http: TLS handshake error from 10.244.0.1:48428: EOF 2018/07/26 11:41:49 http: TLS handshake error from 10.244.0.1:48452: EOF 2018/07/26 11:41:59 http: TLS handshake error from 10.244.0.1:48476: EOF level=info timestamp=2018-07-26T11:42:06.545981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:07.221191Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:42:09 http: TLS handshake error from 10.244.0.1:48500: EOF 2018/07/26 11:42:19 http: TLS handshake error from 10.244.0.1:48524: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:40:40 http: TLS handshake error from 10.244.1.1:43838: EOF 2018/07/26 11:40:50 http: TLS handshake error from 10.244.1.1:43844: EOF level=info timestamp=2018-07-26T11:40:51.875942Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:41:00 http: TLS handshake error from 10.244.1.1:43850: EOF 2018/07/26 11:41:10 http: TLS handshake error from 10.244.1.1:43856: EOF 2018/07/26 11:41:20 http: TLS handshake error from 10.244.1.1:43862: EOF level=info timestamp=2018-07-26T11:41:21.882610Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:41:30 http: TLS handshake error from 10.244.1.1:43868: EOF 2018/07/26 11:41:40 http: TLS handshake error from 10.244.1.1:43874: EOF 2018/07/26 11:41:50 http: TLS handshake error from 10.244.1.1:43880: EOF level=info timestamp=2018-07-26T11:41:51.940118Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:42:00 http: TLS handshake error from 10.244.1.1:43886: EOF 2018/07/26 11:42:10 http: TLS handshake error from 10.244.1.1:43892: EOF 2018/07/26 11:42:20 http: TLS handshake error from 10.244.1.1:43898: EOF level=info timestamp=2018-07-26T11:42:21.970388Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:36:27.142197Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.144049Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmicp25v kind= uid=21de724e-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.215818Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:36:27.216127Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminpctf kind= uid=21e36d14-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:36:27.494387Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmimm6kz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmimm6kz" level=info timestamp=2018-07-26T11:36:27.858925Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvminpctf\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvminpctf" level=info timestamp=2018-07-26T11:39:28.923720Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:28.932153Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:28.966850Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:28.967170Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.049660Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.050033Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.101162Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.103215Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.249515Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmid6xbq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:39:46.156069Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:46.169231Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:47.452200Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:47.452999Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:47.453511Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:50.023152Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:50.039628Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:50.040150Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:39:55.161573Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:55.162051Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:55.162835Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmirrt7b" level=info timestamp=2018-07-26T11:40:05.403714Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:05.454726Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:05.455034Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:05.455164Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:39:57.472060Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:39:57.472972Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:39:57.473315Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizxddl" level=info timestamp=2018-07-26T11:40:06.635945Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:06.678454Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:06.678803Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:06.678937Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmid6xbq kind= uid=8e3bc05a-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:07.434989Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:07.454306Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:07.455302Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:07.455492Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:07.714456Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:07.737254Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:40:07.737896Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:40:07.738112Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi7hxbk-mvcvh Pod phase: Failed goroutine 26 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202789f0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4202ebee0, 0xc4200b73a8, 0xc420779980, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008cae0, 0xc42000e428, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc420348ec0, 0x1b, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008cae0, 0xc42000e428, 0x13, 0xc42056e718, 0x3, 0x3, 0x1, 0x4, 0xc420043350) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aad00, 0xc4200e4b40, 0xc42158f8a0, 0xc42158f8b0, 0xc4215f1e00, 0xc42033e7e0, 0x10f5e80, 0xc4200b73a8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmid6xbq-xmjr7 Pod phase: Failed goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42036e3a0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205a2060, 0xc4206b2788, 0xc42044b0a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6cc0, 0xc42000e0d8, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bac80, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6cc0, 0xc42000e0d8, 0x13, 0xc420059f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420568000, 0xc4200e4820, 0xc42196ea70, 0xc42196ea80, 0xc421965e00, 0xc421970140, 0x10f5e80, 0xc4206b2788, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmirrt7b-rfb8d Pod phase: Failed goroutine 14 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42025e0b0, 0xc420166780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932040, 0xc42000ea28, 0xc4203ba860, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6420, 0xc4200b6410, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc421a19f80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6420, 0xc4200b6410, 0x13, 0xc421957f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421a8b940) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba2c0, 0xc4200b87d0, 0xc4219f3b50, 0xc4219f3b60, 0xc421952300, 0xc421926d00, 0x10f5e80, 0xc42000ea28, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmizxddl-vbrjm Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0d0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420c50040, 0xc4200b7240, 0xc42000dd00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c180, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420348080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c180, 0xc42000e078, 0x13, 0xc420057f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202ac040, 0xc4201f85a0, 0xc420cf5bb0, 0xc420cf5bc0, 0xc420c6a300, 0xc420c467c0, 0x10f5e80, 0xc4200b7240, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.898 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the Inbound VirtualMachineInstance with pod network connectivity explicitly set /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:39:28.991772Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmid6xbq-xmjr7" level=info timestamp=2018-07-26T11:39:45.919413Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmid6xbq-xmjr7" level=error timestamp=2018-07-26T11:39:46.145902Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:41:36.413683Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:41:37.146303Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:41:39 http: TLS handshake error from 10.244.0.1:48428: EOF 2018/07/26 11:41:49 http: TLS handshake error from 10.244.0.1:48452: EOF 2018/07/26 11:41:59 http: TLS handshake error from 10.244.0.1:48476: EOF level=info timestamp=2018-07-26T11:42:06.545981Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:07.221191Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:42:09 http: TLS handshake error from 10.244.0.1:48500: EOF 2018/07/26 11:42:19 http: TLS handshake error from 10.244.0.1:48524: EOF 2018/07/26 11:42:29 http: TLS handshake error from 10.244.0.1:48548: EOF level=info timestamp=2018-07-26T11:42:32.674980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:42:32.680693Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:42:36.671275Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:42:37.370665Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:42:39 http: TLS handshake error from 10.244.0.1:48572: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:40:51.875942Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:41:00 http: TLS handshake error from 10.244.1.1:43850: EOF 2018/07/26 11:41:10 http: TLS handshake error from 10.244.1.1:43856: EOF 2018/07/26 11:41:20 http: TLS handshake error from 10.244.1.1:43862: EOF level=info timestamp=2018-07-26T11:41:21.882610Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:41:30 http: TLS handshake error from 10.244.1.1:43868: EOF 2018/07/26 11:41:40 http: TLS handshake error from 10.244.1.1:43874: EOF 2018/07/26 11:41:50 http: TLS handshake error from 10.244.1.1:43880: EOF level=info timestamp=2018-07-26T11:41:51.940118Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:42:00 http: TLS handshake error from 10.244.1.1:43886: EOF 2018/07/26 11:42:10 http: TLS handshake error from 10.244.1.1:43892: EOF 2018/07/26 11:42:20 http: TLS handshake error from 10.244.1.1:43898: EOF level=info timestamp=2018-07-26T11:42:21.970388Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:42:30 http: TLS handshake error from 10.244.1.1:43904: EOF 2018/07/26 11:42:40 http: TLS handshake error from 10.244.1.1:43910: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:39:28.967170Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.049660Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.050033Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.101162Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.103215Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.249515Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmid6xbq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" level=info timestamp=2018-07-26T11:42:31.239949Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.254897Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.246272Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.256050Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.269887Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.287229Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.339795Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.340067Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.792132Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitxchx\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitxchx" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:42:48.012659Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:48.012824Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:48.012935Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:48.018567Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:48.019457Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:48.019574Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:48.039855Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:48.040151Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:48.040291Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:48.084569Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:48.084896Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:48.085094Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:48.165645Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:48.165999Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:48.166176Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:42:47.671367Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:47.671708Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:47.671970Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigcrsh" level=info timestamp=2018-07-26T11:42:47.676731Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:47.677059Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:47.677348Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigcrsh" level=info timestamp=2018-07-26T11:42:47.697892Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:47.698143Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:47.698461Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigcrsh" level=info timestamp=2018-07-26T11:42:47.738984Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:47.739355Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:47.739600Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigcrsh" level=info timestamp=2018-07-26T11:42:47.822443Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:47.822750Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:47.822941Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmigcrsh" Pod name: virt-launcher-testvmifkbcn-k8smw Pod phase: Running level=info timestamp=2018-07-26T11:42:36.273402Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:42:36.274149Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:42:36.276526Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:42:47.330339Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:42:47.353780Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmifkbcn" level=info timestamp=2018-07-26T11:42:47.354506Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:42:47.354875Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmigcrsh-kdmnt Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202480e0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930080, 0xc42000e420, 0xc4202125c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e0b8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200882c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e0b8, 0x13, 0xc42195af18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4203d6040, 0xc4200a85f0, 0xc42191ddb0, 0xc42191ddc0, 0xc421956300, 0xc421926d80, 0x10f5e80, 0xc42000e420, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmitxchx-4gmsv Pod phase: Running level=info timestamp=2018-07-26T11:42:36.044342Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:42:36.045077Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:42:36.049135Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:42:46.982800Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:42:47.002101Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmitxchx" level=info timestamp=2018-07-26T11:42:47.005832Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:42:47.006104Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" level=info timestamp=2018-07-26T11:42:48.221360Z pos=cloud-init.go:254 component=virt-launcher msg="generated nocloud iso file /var/run/libvirt/kubevirt-ephemeral-disk/cloud-init-data/kubevirt-test-default/testvmitxchx/noCloud.iso" level=error timestamp=2018-07-26T11:42:48.250400Z pos=common.go:126 component=virt-launcher msg="updated MAC for interface: eth0 - 0a:58:0a:64:0a:09" level=info timestamp=2018-07-26T11:42:48.262023Z pos=converter.go:751 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \n`\u0000\n" level=info timestamp=2018-07-26T11:42:48.263528Z pos=converter.go:752 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local" level=info timestamp=2018-07-26T11:42:48.264380Z pos=dhcp.go:62 component=virt-launcher msg="Starting SingleClientDHCPServer" Pod name: virt-launcher-testvmizd2dv-zflbc Pod phase: Running goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202a40d0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4216bc080, 0xc420c9c4a8, 0xc420cc73a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b0180, 0xc42000e098, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420316240, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b0180, 0xc42000e098, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420316080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420480040, 0xc4200a8410, 0xc420c142b0, 0xc420c142c0, 0xc4216de300, 0xc4216adc60, 0x10f5e80, 0xc420c9c4a8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:44:07.146216Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:44:07.702245Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:44:09 http: TLS handshake error from 10.244.0.1:48788: EOF 2018/07/26 11:44:19 http: TLS handshake error from 10.244.0.1:48812: EOF 2018/07/26 11:44:29 http: TLS handshake error from 10.244.0.1:48836: EOF level=info timestamp=2018-07-26T11:44:37.266913Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:44:37.811392Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:44:39 http: TLS handshake error from 10.244.0.1:48860: EOF 2018/07/26 11:44:49 http: TLS handshake error from 10.244.0.1:48884: EOF 2018/07/26 11:44:59 http: TLS handshake error from 10.244.0.1:48908: EOF level=info timestamp=2018-07-26T11:45:07.492310Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:45:07.927218Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:45:09 http: TLS handshake error from 10.244.0.1:48932: EOF 2018/07/26 11:45:19 http: TLS handshake error from 10.244.0.1:48956: EOF 2018/07/26 11:45:29 http: TLS handshake error from 10.244.0.1:48980: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:43:50 http: TLS handshake error from 10.244.1.1:43952: EOF level=info timestamp=2018-07-26T11:43:51.943884Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:44:00 http: TLS handshake error from 10.244.1.1:43958: EOF 2018/07/26 11:44:10 http: TLS handshake error from 10.244.1.1:43964: EOF 2018/07/26 11:44:20 http: TLS handshake error from 10.244.1.1:43970: EOF level=info timestamp=2018-07-26T11:44:21.853034Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:44:30 http: TLS handshake error from 10.244.1.1:43976: EOF 2018/07/26 11:44:40 http: TLS handshake error from 10.244.1.1:43982: EOF 2018/07/26 11:44:50 http: TLS handshake error from 10.244.1.1:43988: EOF level=info timestamp=2018-07-26T11:44:51.957908Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:00 http: TLS handshake error from 10.244.1.1:43994: EOF 2018/07/26 11:45:10 http: TLS handshake error from 10.244.1.1:44000: EOF 2018/07/26 11:45:20 http: TLS handshake error from 10.244.1.1:44006: EOF level=info timestamp=2018-07-26T11:45:21.919586Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:30 http: TLS handshake error from 10.244.1.1:44012: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:39:28.967170Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi7hxbk kind= uid=8e4100b4-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.049660Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.050033Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizxddl kind= uid=8e45cb42-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.101162Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:39:29.103215Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmirrt7b kind= uid=8e4d9144-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:39:29.249515Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmid6xbq\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmid6xbq" level=info timestamp=2018-07-26T11:42:31.239949Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.254897Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.246272Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.256050Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.269887Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.287229Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.339795Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.340067Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.792132Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitxchx\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitxchx" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:42:49.292545Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:49.293620Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:50.583533Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:50.584166Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:50.585739Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:53.146964Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:53.148051Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:53.149189Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:58.283701Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:58.286703Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:58.287251Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:43:08.529140Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.589834Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:08.590176Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.607773Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:42:59.199327Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:59.199792Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:59.200054Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifkbcn" level=info timestamp=2018-07-26T11:43:08.161954Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.214375Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:08.214797Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.215044Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigcrsh kind= uid=fab0e41d-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:08.957991Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.982104Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:08.982420Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.982558Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmitxchx kind= uid=faecb9ac-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:09.440927Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:09.485732Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:09.486011Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:09.486254Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifkbcn kind= uid=fae9cd50-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmifkbcn-k8smw Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202f4680, 0xc420116000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000d8c0, 0xc42000e2e0, 0xc420665c40, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6cc0, 0xc4200b6878, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc42036bfc0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6cc0, 0xc4200b6878, 0x13, 0xc420059f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207dbd40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202cc440, 0xc4200b8820, 0xc4204869d0, 0xc4204869e0, 0xc420427d80, 0xc4200c40e0, 0x10f5e80, 0xc42000e2e0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigcrsh-kdmnt Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202480e0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930080, 0xc42000e420, 0xc4202125c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e0b8, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200882c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e0b8, 0x13, 0xc42195af18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4203d6040, 0xc4200a85f0, 0xc42191ddb0, 0xc42191ddc0, 0xc421956300, 0xc421926d80, 0x10f5e80, 0xc42000e420, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmitxchx-4gmsv Pod phase: Failed goroutine 36 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203e2440, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42168c280, 0xc420c23198, 0xc420c5f760, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c5a0, 0xc4200b6300, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4200bbb80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c5a0, 0xc4200b6300, 0x13, 0xc420059f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420c33f00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a400, 0xc4200b8af0, 0xc420c2b630, 0xc420c2b640, 0xc42168be00, 0xc4216cd020, 0x10f5e80, 0xc420c23198, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmizd2dv-zflbc Pod phase: Failed goroutine 16 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202a40d0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4216bc080, 0xc420c9c4a8, 0xc420cc73a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200b0180, 0xc42000e098, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420316240, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200b0180, 0xc42000e098, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420316080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420480040, 0xc4200a8410, 0xc420c142b0, 0xc420c142c0, 0xc4216de300, 0xc4216adc60, 0x10f5e80, 0xc420c9c4a8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [182.216 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the Inbound VirtualMachineInstance with custom MAC address /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:42:31.390386Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmigcrsh-kdmnt" level=info timestamp=2018-07-26T11:42:47.461597Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmigcrsh-kdmnt" level=error timestamp=2018-07-26T11:42:47.670111Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:44:37.811392Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:44:39 http: TLS handshake error from 10.244.0.1:48860: EOF 2018/07/26 11:44:49 http: TLS handshake error from 10.244.0.1:48884: EOF 2018/07/26 11:44:59 http: TLS handshake error from 10.244.0.1:48908: EOF level=info timestamp=2018-07-26T11:45:07.492310Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:45:07.927218Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:45:09 http: TLS handshake error from 10.244.0.1:48932: EOF 2018/07/26 11:45:19 http: TLS handshake error from 10.244.0.1:48956: EOF 2018/07/26 11:45:29 http: TLS handshake error from 10.244.0.1:48980: EOF level=info timestamp=2018-07-26T11:45:32.702826Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:45:32.704813Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:45:37.698963Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:45:38.013087Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:45:39 http: TLS handshake error from 10.244.0.1:49004: EOF 2018/07/26 11:45:49 http: TLS handshake error from 10.244.0.1:49028: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:43:51.943884Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:44:00 http: TLS handshake error from 10.244.1.1:43958: EOF 2018/07/26 11:44:10 http: TLS handshake error from 10.244.1.1:43964: EOF 2018/07/26 11:44:20 http: TLS handshake error from 10.244.1.1:43970: EOF level=info timestamp=2018-07-26T11:44:21.853034Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:44:30 http: TLS handshake error from 10.244.1.1:43976: EOF 2018/07/26 11:44:40 http: TLS handshake error from 10.244.1.1:43982: EOF 2018/07/26 11:44:50 http: TLS handshake error from 10.244.1.1:43988: EOF level=info timestamp=2018-07-26T11:44:51.957908Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:00 http: TLS handshake error from 10.244.1.1:43994: EOF 2018/07/26 11:45:10 http: TLS handshake error from 10.244.1.1:44000: EOF 2018/07/26 11:45:20 http: TLS handshake error from 10.244.1.1:44006: EOF level=info timestamp=2018-07-26T11:45:21.919586Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:45:30 http: TLS handshake error from 10.244.1.1:44012: EOF 2018/07/26 11:45:40 http: TLS handshake error from 10.244.1.1:44018: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:42:31.339795Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.340067Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.792132Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitxchx\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitxchx" level=info timestamp=2018-07-26T11:45:33.065078Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.068088Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.109888Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.110235Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.142192Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.143663Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.250690Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.253200Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.302674Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiccdzh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:33.392984Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiccdzh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:34.082238Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigct5m\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigct5m" level=info timestamp=2018-07-26T11:45:34.477592Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:42:53.146964Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:53.148051Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:53.149189Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:42:58.283701Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:42:58.286703Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:42:58.287251Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizd2dv" level=info timestamp=2018-07-26T11:43:08.529140Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.589834Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:43:08.590176Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:43:08.607773Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:45:32.453470Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:45:32.455183Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:45:32.614215Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T11:45:32.615543Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizd2dv kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:45:50.360594Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:45:49.784774Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:49.784980Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:49.785261Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:49.799685Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:49.800122Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:49.800486Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:49.841623Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:49.842049Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:49.842449Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:49.925637Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:49.926044Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:49.926189Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:50.087399Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:50.087661Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:50.087781Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" Pod name: virt-launcher-testvmi8n4qm-f62tt Pod phase: Running level=info timestamp=2018-07-26T11:45:37.705341Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:45:37.705563Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:45:37.708308Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:45:49.951408Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:45:49.975527Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi8n4qm" level=info timestamp=2018-07-26T11:45:49.976361Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:45:49.976593Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmiccdzh-cb2vn Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0b0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4206ea040, 0xc420c625b0, 0xc420c88fc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6540, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420088180, 0x1f, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6540, 0xc42000e078, 0x13, 0xc420583f18, 0x3, 0x3, 0x1, 0x4, 0xc4203aa128) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420404040, 0xc4200e4320, 0xc4206250d0, 0xc4206250e0, 0xc42070e300, 0xc4206dba40, 0x10f5e80, 0xc420c625b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmigct5m-tqqhl Pod phase: Running level=info timestamp=2018-07-26T11:45:38.940065Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:45:38.940252Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:45:38.942306Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:45:50.125062Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:45:50.199905Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmigct5m" level=info timestamp=2018-07-26T11:45:50.201830Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:45:50.202348Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmivdkn7-pqh45 Pod phase: Running goroutine 9 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278120, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420504040, 0xc42000e880, 0xc42195f4e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096180, 0xc4200b20e0, 0x13, 0x131ed4d, 0x4, 0xc42196df18, 0x3, 0x3, 0xc420089580, 0xc4202d6240, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096180, 0xc4200b20e0, 0x13, 0xc42195df18, 0x3, 0x3, 0xc42033e700, 0xc4202e6ea0, 0xc42026c000) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200b6000, 0xc4200b46e0, 0xc421630410, 0xc421630420, 0xc421945b80, 0xc42000c0c0, 0x10f5e80, 0xc42000e880, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:47:29 http: TLS handshake error from 10.244.0.1:49268: EOF level=info timestamp=2018-07-26T11:47:31.458871Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:47:31.464023Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:47:38.250043Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:47:38.533442Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:47:39 http: TLS handshake error from 10.244.0.1:49292: EOF 2018/07/26 11:47:49 http: TLS handshake error from 10.244.0.1:49316: EOF 2018/07/26 11:47:59 http: TLS handshake error from 10.244.0.1:49340: EOF level=info timestamp=2018-07-26T11:48:08.375275Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:48:08.688780Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:48:09 http: TLS handshake error from 10.244.0.1:49364: EOF 2018/07/26 11:48:19 http: TLS handshake error from 10.244.0.1:49388: EOF 2018/07/26 11:48:29 http: TLS handshake error from 10.244.0.1:49412: EOF level=info timestamp=2018-07-26T11:48:31.473713Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:31.479743Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:47:00 http: TLS handshake error from 10.244.1.1:44066: EOF 2018/07/26 11:47:10 http: TLS handshake error from 10.244.1.1:44072: EOF 2018/07/26 11:47:20 http: TLS handshake error from 10.244.1.1:44078: EOF level=info timestamp=2018-07-26T11:47:21.639040Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:47:21.734773Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:47:21.986759Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:47:30 http: TLS handshake error from 10.244.1.1:44084: EOF 2018/07/26 11:47:40 http: TLS handshake error from 10.244.1.1:44090: EOF 2018/07/26 11:47:50 http: TLS handshake error from 10.244.1.1:44096: EOF level=info timestamp=2018-07-26T11:47:51.938928Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:00 http: TLS handshake error from 10.244.1.1:44102: EOF 2018/07/26 11:48:10 http: TLS handshake error from 10.244.1.1:44108: EOF 2018/07/26 11:48:20 http: TLS handshake error from 10.244.1.1:44114: EOF level=info timestamp=2018-07-26T11:48:21.949540Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:30 http: TLS handshake error from 10.244.1.1:44120: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:42:31.339795Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:42:31.340067Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizd2dv kind= uid=faf0609f-90c8-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:42:31.792132Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmitxchx\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmitxchx" level=info timestamp=2018-07-26T11:45:33.065078Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.068088Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.109888Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.110235Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.142192Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.143663Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.250690Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:45:33.253200Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:45:33.302674Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiccdzh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:33.392984Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiccdzh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiccdzh" level=info timestamp=2018-07-26T11:45:34.082238Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigct5m\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigct5m" level=info timestamp=2018-07-26T11:45:34.477592Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:45:51.971772Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:51.974367Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivdkn7" level=info timestamp=2018-07-26T11:45:53.255219Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:53.257274Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:53.257846Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivdkn7" level=info timestamp=2018-07-26T11:45:55.818750Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:45:55.819869Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:45:55.820534Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivdkn7" level=info timestamp=2018-07-26T11:46:00.942038Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:00.944818Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:00.949242Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivdkn7" level=info timestamp=2018-07-26T11:46:11.190241Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:11.215752Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:11.216968Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:11.217270Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivdkn7 kind= uid=67585d02-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:46:01.708723Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:46:01.709384Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:46:01.709685Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:46:10.257925Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:10.326263Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:10.326624Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:10.326767Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiccdzh kind= uid=67463122-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:11.259684Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:11.284018Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:11.284405Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:11.284539Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmigct5m kind= uid=6751ea2b-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:11.952703Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:11.977461Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:46:11.977772Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:46:11.977930Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi8n4qm kind= uid=674c072d-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi8n4qm-f62tt Pod phase: Failed goroutine 35 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fe0d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420566020, 0xc42162b130, 0xc42192e720, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6540, 0xc4200b60a8, 0x13, 0x131ed4d, 0x4, 0xc42195bf18, 0x3, 0x3, 0xc4202a8300, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6540, 0xc4200b60a8, 0x13, 0xc420058718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202a8080, 0xc4200b85a0, 0xc42164e010, 0xc42164e020, 0xc421931c00, 0xc420187580, 0x10f5e80, 0xc42162b130, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiccdzh-cb2vn Pod phase: Failed goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4201fc0b0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4206ea040, 0xc420c625b0, 0xc420c88fc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6540, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420088180, 0x1f, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6540, 0xc42000e078, 0x13, 0xc420583f18, 0x3, 0x3, 0x1, 0x4, 0xc4203aa128) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420404040, 0xc4200e4320, 0xc4206250d0, 0xc4206250e0, 0xc42070e300, 0xc4206dba40, 0x10f5e80, 0xc420c625b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmigct5m-tqqhl Pod phase: Failed goroutine 29 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42031e0c0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c0e0, 0xc4200b74b0, 0xc4208bbb00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc4200b60a8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200bac80, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc4200b60a8, 0x13, 0xc420218f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420348080, 0xc4200b8500, 0xc420849720, 0xc420849730, 0xc4203b2300, 0xc4205655e0, 0x10f5e80, 0xc4200b74b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivdkn7-pqh45 Pod phase: Failed goroutine 9 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278120, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420504040, 0xc42000e880, 0xc42195f4e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096180, 0xc4200b20e0, 0x13, 0x131ed4d, 0x4, 0xc42196df18, 0x3, 0x3, 0xc420089580, 0xc4202d6240, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096180, 0xc4200b20e0, 0x13, 0xc42195df18, 0x3, 0x3, 0xc42033e700, 0xc4202e6ea0, 0xc42026c000) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200b6000, 0xc4200b46e0, 0xc421630410, 0xc421630420, 0xc421945b80, 0xc42000c0c0, 0x10f5e80, 0xc42000e880, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.953 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be able to reach [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 the internet /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:45:33.079250Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmiccdzh-cb2vn" level=info timestamp=2018-07-26T11:45:49.323006Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmiccdzh-cb2vn" level=error timestamp=2018-07-26T11:45:49.777774Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:47:38.533442Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:47:39 http: TLS handshake error from 10.244.0.1:49292: EOF 2018/07/26 11:47:49 http: TLS handshake error from 10.244.0.1:49316: EOF 2018/07/26 11:47:59 http: TLS handshake error from 10.244.0.1:49340: EOF level=info timestamp=2018-07-26T11:48:08.375275Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:48:08.688780Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:48:09 http: TLS handshake error from 10.244.0.1:49364: EOF 2018/07/26 11:48:19 http: TLS handshake error from 10.244.0.1:49388: EOF 2018/07/26 11:48:29 http: TLS handshake error from 10.244.0.1:49412: EOF level=info timestamp=2018-07-26T11:48:31.473713Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:31.479743Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:48:38.466978Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:48:38.749917Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:48:39 http: TLS handshake error from 10.244.0.1:49436: EOF 2018/07/26 11:48:49 http: TLS handshake error from 10.244.0.1:49460: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:47:21.639040Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:47:21.734773Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:47:21.986759Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:47:30 http: TLS handshake error from 10.244.1.1:44084: EOF 2018/07/26 11:47:40 http: TLS handshake error from 10.244.1.1:44090: EOF 2018/07/26 11:47:50 http: TLS handshake error from 10.244.1.1:44096: EOF level=info timestamp=2018-07-26T11:47:51.938928Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:00 http: TLS handshake error from 10.244.1.1:44102: EOF 2018/07/26 11:48:10 http: TLS handshake error from 10.244.1.1:44108: EOF 2018/07/26 11:48:20 http: TLS handshake error from 10.244.1.1:44114: EOF level=info timestamp=2018-07-26T11:48:21.949540Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:48:30 http: TLS handshake error from 10.244.1.1:44120: EOF 2018/07/26 11:48:40 http: TLS handshake error from 10.244.1.1:44126: EOF 2018/07/26 11:48:50 http: TLS handshake error from 10.244.1.1:44132: EOF level=info timestamp=2018-07-26T11:48:51.976631Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:45:34.082238Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigct5m\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigct5m" level=info timestamp=2018-07-26T11:45:34.477592Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:48:34.534885Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8n4qm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 674c072d-90c9-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:48:34.570691Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8n4qm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 674c072d-90c9-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:48:34.595971Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigct5m\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmigct5m, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6751ea2b-90c9-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigct5m" level=info timestamp=2018-07-26T11:48:35.107841Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.108434Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.237312Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.240189Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.278783Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.279109Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.344446Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.344830Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.467194Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizmk5b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizmk5b" level=info timestamp=2018-07-26T11:48:36.147830Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi86zch\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi86zch" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:48:49.835119Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:49.836504Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:49.840900Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:50.001661Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:50.001885Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:50.001991Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:50.322930Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:50.323490Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:50.323927Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:50.964838Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:50.965277Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:50.969548Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:52.264535Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.264975Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.267725Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:48:52.589521Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.589793Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.589997Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmim4z4x" level=info timestamp=2018-07-26T11:48:52.605244Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.605423Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.605742Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi86zch" level=info timestamp=2018-07-26T11:48:52.630393Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.630669Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.630875Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmim4z4x" level=info timestamp=2018-07-26T11:48:52.711680Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.712134Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.712678Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmim4z4x" level=info timestamp=2018-07-26T11:48:52.766426Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.766839Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.767149Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi86zch" Pod name: virt-launcher-testvmi86zch-8jvrt Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40d0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c0e0, 0xc4206aa5b0, 0xc4206c7340, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4202adb40, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc4204d7718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a040, 0xc420100370, 0xc421903090, 0xc4219030a0, 0xc4202ca300, 0xc42050b680, 0x10f5e80, 0xc4206aa5b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmim4z4x-cfg7l Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202c8530, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205620a0, 0xc421642d08, 0xc421943680, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc42029a6c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420534f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207ebfc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42029a540, 0xc4200b8960, 0xc42164c4b0, 0xc42164c4c0, 0xc421945d80, 0xc42000c160, 0x10f5e80, 0xc421642d08, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmivtwz4-4w2rn Pod phase: Running goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203f4350, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421912cc0, 0xc4204e0298, 0xc42025c880, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4204e4c00, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc420088cc0, 0xc4201fd180, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4204e4c00, 0xc42000e100, 0x13, 0xc421abef18, 0x3, 0x3, 0xc4201fd680, 0xc4201fd700, 0xc4201fd780) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420272040, 0xc4202648c0, 0xc4203fa1b0, 0xc4203fa1e0, 0xc421915d80, 0xc42191e0a0, 0x10f5e80, 0xc4204e0298, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmizmk5b-8vwds Pod phase: Running level=info timestamp=2018-07-26T11:48:39.912871Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:48:39.913033Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:48:39.915298Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:48:51.895004Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:48:51.955451Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmizmk5b" level=info timestamp=2018-07-26T11:48:51.957120Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:48:51.957694Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:50:09 http: TLS handshake error from 10.244.0.1:49652: EOF 2018/07/26 11:50:19 http: TLS handshake error from 10.244.0.1:49676: EOF 2018/07/26 11:50:29 http: TLS handshake error from 10.244.0.1:49700: EOF level=info timestamp=2018-07-26T11:50:31.490064Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:50:31.494986Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:50:38.975819Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:50:39.206750Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:50:39 http: TLS handshake error from 10.244.0.1:49724: EOF 2018/07/26 11:50:49 http: TLS handshake error from 10.244.0.1:49748: EOF 2018/07/26 11:50:59 http: TLS handshake error from 10.244.0.1:49772: EOF level=info timestamp=2018-07-26T11:51:09.109905Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:51:09.382413Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:09 http: TLS handshake error from 10.244.0.1:49796: EOF 2018/07/26 11:51:19 http: TLS handshake error from 10.244.0.1:49820: EOF 2018/07/26 11:51:29 http: TLS handshake error from 10.244.0.1:49844: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:49:50 http: TLS handshake error from 10.244.1.1:44168: EOF level=info timestamp=2018-07-26T11:49:51.930715Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:50:00 http: TLS handshake error from 10.244.1.1:44174: EOF 2018/07/26 11:50:10 http: TLS handshake error from 10.244.1.1:44180: EOF 2018/07/26 11:50:20 http: TLS handshake error from 10.244.1.1:44186: EOF level=info timestamp=2018-07-26T11:50:21.890001Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:50:30 http: TLS handshake error from 10.244.1.1:44192: EOF 2018/07/26 11:50:40 http: TLS handshake error from 10.244.1.1:44198: EOF 2018/07/26 11:50:50 http: TLS handshake error from 10.244.1.1:44204: EOF level=info timestamp=2018-07-26T11:50:51.888691Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:00 http: TLS handshake error from 10.244.1.1:44210: EOF 2018/07/26 11:51:10 http: TLS handshake error from 10.244.1.1:44216: EOF 2018/07/26 11:51:20 http: TLS handshake error from 10.244.1.1:44222: EOF level=info timestamp=2018-07-26T11:51:21.955777Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:30 http: TLS handshake error from 10.244.1.1:44228: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:45:34.082238Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigct5m\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigct5m" level=info timestamp=2018-07-26T11:45:34.477592Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:48:34.534885Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8n4qm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 674c072d-90c9-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:48:34.570691Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8n4qm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmi8n4qm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 674c072d-90c9-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8n4qm" level=info timestamp=2018-07-26T11:48:34.595971Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmigct5m\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmigct5m, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6751ea2b-90c9-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmigct5m" level=info timestamp=2018-07-26T11:48:35.107841Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.108434Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.237312Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.240189Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.278783Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.279109Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.344446Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.344830Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.467194Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizmk5b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizmk5b" level=info timestamp=2018-07-26T11:48:36.147830Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi86zch\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi86zch" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:48:50.965277Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:50.969548Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:52.264535Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:52.264975Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:52.267725Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:54.828618Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:54.828896Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:54.829201Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:48:59.949837Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:48:59.950236Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:48:59.950621Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivtwz4" level=info timestamp=2018-07-26T11:49:10.192142Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:10.238805Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:10.239204Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:10.240817Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:49:04.565441Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:49:04.566013Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:49:04.566390Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmizmk5b" level=info timestamp=2018-07-26T11:49:12.940092Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:12.962628Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:12.962993Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:12.963114Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:13.053141Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:13.074137Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:13.077452Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:13.077696Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmim4z4x kind= uid=d3b9658a-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:14.807080Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:14.831684Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:49:14.831991Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:49:14.832126Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmizmk5b kind= uid=d3d1534f-90c9-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi86zch-8jvrt Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40d0, 0xc420106000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c0e0, 0xc4206aa5b0, 0xc4206c7340, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4202adb40, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc4204d7718, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a040, 0xc420100370, 0xc421903090, 0xc4219030a0, 0xc4202ca300, 0xc42050b680, 0x10f5e80, 0xc4206aa5b0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmim4z4x-cfg7l Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202c8530, 0xc4201a8c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4205620a0, 0xc421642d08, 0xc421943680, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc42029a6c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b60f8, 0x13, 0xc420534f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4207ebfc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42029a540, 0xc4200b8960, 0xc42164c4b0, 0xc42164c4c0, 0xc421945d80, 0xc42000c160, 0x10f5e80, 0xc421642d08, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivtwz4-4w2rn Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203f4350, 0xc420180c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421912cc0, 0xc4204e0298, 0xc42025c880, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4204e4c00, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc420088cc0, 0xc4201fd180, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4204e4c00, 0xc42000e100, 0x13, 0xc421abef18, 0x3, 0x3, 0xc4201fd680, 0xc4201fd700, 0xc4201fd780) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420272040, 0xc4202648c0, 0xc4203fa1b0, 0xc4203fa1e0, 0xc421915d80, 0xc42191e0a0, 0x10f5e80, 0xc4204e0298, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmizmk5b-8vwds Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202783f0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42060dc00, 0xc420c62dd8, 0xc420c8ef40, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6d20, 0xc42000e0d8, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc420088440, 0x24, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6d20, 0xc42000e0d8, 0x13, 0xc420642f18, 0x3, 0x3, 0x1, 0x4, 0xc420043500) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba9c0, 0xc42015e7d0, 0xc4206bf0e0, 0xc4206bf0f0, 0xc420615d80, 0xc42060dd00, 0x10f5e80, 0xc420c62dd8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [182.080 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on the same node from Pod /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:48:35.236736Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmim4z4x-cfg7l" level=info timestamp=2018-07-26T11:48:52.227185Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmim4z4x-cfg7l" level=error timestamp=2018-07-26T11:48:52.621698Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:50:31.494986Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:50:38.975819Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:50:39.206750Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:50:39 http: TLS handshake error from 10.244.0.1:49724: EOF 2018/07/26 11:50:49 http: TLS handshake error from 10.244.0.1:49748: EOF 2018/07/26 11:50:59 http: TLS handshake error from 10.244.0.1:49772: EOF level=info timestamp=2018-07-26T11:51:09.109905Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:51:09.382413Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:09 http: TLS handshake error from 10.244.0.1:49796: EOF 2018/07/26 11:51:19 http: TLS handshake error from 10.244.0.1:49820: EOF 2018/07/26 11:51:29 http: TLS handshake error from 10.244.0.1:49844: EOF level=info timestamp=2018-07-26T11:51:39.228069Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:39 http: TLS handshake error from 10.244.0.1:49868: EOF level=info timestamp=2018-07-26T11:51:39.506183Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:51:49 http: TLS handshake error from 10.244.0.1:49892: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:50:10 http: TLS handshake error from 10.244.1.1:44180: EOF 2018/07/26 11:50:20 http: TLS handshake error from 10.244.1.1:44186: EOF level=info timestamp=2018-07-26T11:50:21.890001Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:50:30 http: TLS handshake error from 10.244.1.1:44192: EOF 2018/07/26 11:50:40 http: TLS handshake error from 10.244.1.1:44198: EOF 2018/07/26 11:50:50 http: TLS handshake error from 10.244.1.1:44204: EOF level=info timestamp=2018-07-26T11:50:51.888691Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:00 http: TLS handshake error from 10.244.1.1:44210: EOF 2018/07/26 11:51:10 http: TLS handshake error from 10.244.1.1:44216: EOF 2018/07/26 11:51:20 http: TLS handshake error from 10.244.1.1:44222: EOF level=info timestamp=2018-07-26T11:51:21.955777Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:51:30 http: TLS handshake error from 10.244.1.1:44228: EOF 2018/07/26 11:51:40 http: TLS handshake error from 10.244.1.1:44234: EOF 2018/07/26 11:51:50 http: TLS handshake error from 10.244.1.1:44240: EOF level=info timestamp=2018-07-26T11:51:52.008519Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:48:35.279109Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.344446Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.344830Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.467194Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizmk5b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizmk5b" level=info timestamp=2018-07-26T11:48:36.147830Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi86zch\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi86zch" level=info timestamp=2018-07-26T11:51:37.117710Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.121707Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.152316Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.152692Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.208839Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.209164Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.296598Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.296716Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.378581Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnr6n\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnr6n" level=info timestamp=2018-07-26T11:51:37.933213Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi5ww42\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:51:53.526576Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.526734Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.526816Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:51:53.547884Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.551572Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.551806Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:51:53.592198Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.592475Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.592601Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:51:53.673043Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.673316Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.673459Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:51:53.836756Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.837704Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.837967Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:51:53.473540Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.473682Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.473856Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilz4lc" level=info timestamp=2018-07-26T11:51:53.479637Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.479895Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.479999Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilz4lc" level=info timestamp=2018-07-26T11:51:53.508521Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.508701Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.508831Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilz4lc" level=info timestamp=2018-07-26T11:51:53.550389Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.550619Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.550721Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilz4lc" level=info timestamp=2018-07-26T11:51:53.631072Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:53.631304Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:53.631437Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmilz4lc" Pod name: virt-launcher-testvmi5ww42-k57hm Pod phase: Running goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420046440, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190ac60, 0xc42000e990, 0xc420390d20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6f60, 0xc4200b6760, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc42049e480, 0xc, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6f60, 0xc4200b6760, 0x13, 0xc4206b5f18, 0x3, 0x3, 0x1, 0x4, 0xc4202f66a8) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa580, 0xc4200b8e10, 0xc421a57710, 0xc421a57720, 0xc42190dd80, 0xc4219160e0, 0x10f5e80, 0xc42000e990, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmilz4lc-2k56h Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d4390, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420244460, 0xc4200b6cb8, 0xc420672160, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c600, 0xc42000e0e0, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4202aa2c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c600, 0xc42000e0e0, 0x13, 0xc420058f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a400, 0xc42015c5f0, 0xc4206424b0, 0xc4206424c0, 0xc42007de80, 0xc42029a0c0, 0x10f5e80, 0xc4200b6cb8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmivnr6n-pz4lx Pod phase: Running level=info timestamp=2018-07-26T11:51:41.312890Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:51:41.313688Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:51:41.315128Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:51:53.308626Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:51:53.355051Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmivnr6n" level=info timestamp=2018-07-26T11:51:53.363290Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:51:53.363599Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmiwsjmz-mvkrv Pod phase: Running level=info timestamp=2018-07-26T11:51:41.983475Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:51:41.983613Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:51:41.985324Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:51:53.569580Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:51:53.631564Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmiwsjmz" level=info timestamp=2018-07-26T11:51:53.632998Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:51:53.633275Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:53:09.883676Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:53:19 http: TLS handshake error from 10.244.0.1:50108: EOF 2018/07/26 11:53:29 http: TLS handshake error from 10.244.0.1:50132: EOF level=info timestamp=2018-07-26T11:53:31.567002Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:53:31.575027Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:53:39 http: TLS handshake error from 10.244.0.1:50156: EOF level=info timestamp=2018-07-26T11:53:39.738644Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:53:40.018775Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:53:49 http: TLS handshake error from 10.244.0.1:50180: EOF 2018/07/26 11:53:59 http: TLS handshake error from 10.244.0.1:50204: EOF 2018/07/26 11:54:09 http: TLS handshake error from 10.244.0.1:50228: EOF level=info timestamp=2018-07-26T11:54:09.866901Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:54:10.161659Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:19 http: TLS handshake error from 10.244.0.1:50252: EOF 2018/07/26 11:54:29 http: TLS handshake error from 10.244.0.1:50276: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:52:50 http: TLS handshake error from 10.244.1.1:44276: EOF level=info timestamp=2018-07-26T11:52:51.976894Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:53:00 http: TLS handshake error from 10.244.1.1:44282: EOF 2018/07/26 11:53:10 http: TLS handshake error from 10.244.1.1:44288: EOF 2018/07/26 11:53:20 http: TLS handshake error from 10.244.1.1:44294: EOF level=info timestamp=2018-07-26T11:53:21.879667Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:53:30 http: TLS handshake error from 10.244.1.1:44300: EOF 2018/07/26 11:53:40 http: TLS handshake error from 10.244.1.1:44306: EOF 2018/07/26 11:53:50 http: TLS handshake error from 10.244.1.1:44312: EOF level=info timestamp=2018-07-26T11:53:51.871711Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:00 http: TLS handshake error from 10.244.1.1:44318: EOF 2018/07/26 11:54:10 http: TLS handshake error from 10.244.1.1:44324: EOF 2018/07/26 11:54:20 http: TLS handshake error from 10.244.1.1:44330: EOF level=info timestamp=2018-07-26T11:54:22.001293Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:30 http: TLS handshake error from 10.244.1.1:44336: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:48:35.279109Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi86zch kind= uid=d3dbd8df-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.344446Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:48:35.344830Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivtwz4 kind= uid=d3e64f38-90c9-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:48:35.467194Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmizmk5b\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmizmk5b" level=info timestamp=2018-07-26T11:48:36.147830Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi86zch\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi86zch" level=info timestamp=2018-07-26T11:51:37.117710Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.121707Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.152316Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.152692Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.208839Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.209164Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.296598Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.296716Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.378581Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnr6n\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnr6n" level=info timestamp=2018-07-26T11:51:37.933213Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi5ww42\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:51:54.812188Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:54.812549Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:51:56.102773Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:56.105224Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:56.124151Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:51:58.687152Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:51:58.687787Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:51:58.688012Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:52:03.809456Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:03.809969Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:03.810433Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:52:14.052608Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:14.112823Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:14.125311Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:14.125870Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:52:06.303349Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:52:06.303808Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:52:06.304053Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiwsjmz" level=info timestamp=2018-07-26T11:52:13.968870Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:14.011792Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:14.012136Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:14.012399Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmilz4lc kind= uid=40429a02-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:15.877673Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:15.903477Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:15.903761Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:15.903923Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivnr6n kind= uid=4049b12e-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:16.546516Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:16.582204Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:52:16.582544Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:52:16.582694Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiwsjmz kind= uid=404fc7ec-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi5ww42-k57hm Pod phase: Failed goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420046440, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42190ac60, 0xc42000e990, 0xc420390d20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6f60, 0xc4200b6760, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc42049e480, 0xc, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6f60, 0xc4200b6760, 0x13, 0xc4206b5f18, 0x3, 0x3, 0x1, 0x4, 0xc4202f66a8) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa580, 0xc4200b8e10, 0xc421a57710, 0xc421a57720, 0xc42190dd80, 0xc4219160e0, 0x10f5e80, 0xc42000e990, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmilz4lc-2k56h Pod phase: Failed goroutine 25 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d4390, 0xc4201a0780, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420244460, 0xc4200b6cb8, 0xc420672160, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c600, 0xc42000e0e0, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4202aa2c0, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c600, 0xc42000e0e0, 0x13, 0xc420058f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a400, 0xc42015c5f0, 0xc4206424b0, 0xc4206424c0, 0xc42007de80, 0xc42029a0c0, 0x10f5e80, 0xc4200b6cb8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivnr6n-pz4lx Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42030e120, 0xc420110000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc4203e8800, 0xc4206b0cc0, 0xc4206e7a40, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c060, 0xc42000e320, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc420088040, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c060, 0xc42000e320, 0x13, 0xc42049bf18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420334540, 0xc4200d42d0, 0xc4217590e0, 0xc4217590f0, 0xc420962300, 0xc42037a8e0, 0x10f5e80, 0xc4206b0cc0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiwsjmz-mvkrv Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202783f0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421932c60, 0xc4200b6fa0, 0xc4202ec7c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6de0, 0xc4200b60f8, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4202aa700, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6de0, 0xc4200b60f8, 0x13, 0xc4219fbf18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421900c00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa440, 0xc4200b8960, 0xc421968770, 0xc421968780, 0xc421935d80, 0xc42193e0e0, 0x10f5e80, 0xc4200b6fa0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.960 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on a different node from Pod /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:51:37.004166Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmilz4lc-2k56h" level=info timestamp=2018-07-26T11:51:53.148210Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmilz4lc-2k56h" level=error timestamp=2018-07-26T11:51:53.455617Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:53:31.575027Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:53:39 http: TLS handshake error from 10.244.0.1:50156: EOF level=info timestamp=2018-07-26T11:53:39.738644Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:53:40.018775Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:53:49 http: TLS handshake error from 10.244.0.1:50180: EOF 2018/07/26 11:53:59 http: TLS handshake error from 10.244.0.1:50204: EOF 2018/07/26 11:54:09 http: TLS handshake error from 10.244.0.1:50228: EOF level=info timestamp=2018-07-26T11:54:09.866901Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:54:10.161659Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:19 http: TLS handshake error from 10.244.0.1:50252: EOF 2018/07/26 11:54:29 http: TLS handshake error from 10.244.0.1:50276: EOF 2018/07/26 11:54:39 http: TLS handshake error from 10.244.0.1:50300: EOF level=info timestamp=2018-07-26T11:54:39.993068Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:54:40.217308Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:54:49 http: TLS handshake error from 10.244.0.1:50324: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:53:10 http: TLS handshake error from 10.244.1.1:44288: EOF 2018/07/26 11:53:20 http: TLS handshake error from 10.244.1.1:44294: EOF level=info timestamp=2018-07-26T11:53:21.879667Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:53:30 http: TLS handshake error from 10.244.1.1:44300: EOF 2018/07/26 11:53:40 http: TLS handshake error from 10.244.1.1:44306: EOF 2018/07/26 11:53:50 http: TLS handshake error from 10.244.1.1:44312: EOF level=info timestamp=2018-07-26T11:53:51.871711Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:00 http: TLS handshake error from 10.244.1.1:44318: EOF 2018/07/26 11:54:10 http: TLS handshake error from 10.244.1.1:44324: EOF 2018/07/26 11:54:20 http: TLS handshake error from 10.244.1.1:44330: EOF level=info timestamp=2018-07-26T11:54:22.001293Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:54:30 http: TLS handshake error from 10.244.1.1:44336: EOF 2018/07/26 11:54:40 http: TLS handshake error from 10.244.1.1:44342: EOF 2018/07/26 11:54:50 http: TLS handshake error from 10.244.1.1:44348: EOF level=info timestamp=2018-07-26T11:54:51.901606Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:51:37.296598Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.296716Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.378581Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnr6n\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnr6n" level=info timestamp=2018-07-26T11:51:37.933213Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi5ww42\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:54:38.535687Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwsjmz\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwsjmz, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 404fc7ec-90ca-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwsjmz" level=info timestamp=2018-07-26T11:54:39.091081Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.092208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.145689Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.146060Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.195615Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.196157Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.226245Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.227138Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.823143Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiftsjp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiftsjp" level=info timestamp=2018-07-26T11:54:40.048180Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmindznh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:54:54.826061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:54.826306Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:54.828171Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:54:54.988606Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:54.988820Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:54.988948Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:54:55.309660Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:55.309877Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:55.310038Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:54:55.951974Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:55.958017Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:55.961691Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:54:57.243268Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.244065Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.247884Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:54:57.319742Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.319891Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.319990Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" level=info timestamp=2018-07-26T11:54:57.331663Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.331805Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.331877Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisncth" level=info timestamp=2018-07-26T11:54:57.400870Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.401064Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.401202Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" level=info timestamp=2018-07-26T11:54:57.412246Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.412422Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.412586Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisncth" level=info timestamp=2018-07-26T11:54:57.561704Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.562030Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.562266Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" level=info timestamp=2018-07-26T11:54:57.576564Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.576856Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.577053Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisncth" Pod name: virt-launcher-testvmiftsjp-2dvcn Pod phase: Running goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203f84d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420400b20, 0xc4205cce90, 0xc4205f9fc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6cc0, 0xc4200b6250, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bacc0, 0xb, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6cc0, 0xc4200b6250, 0x13, 0xc4215d5718, 0x3, 0x3, 0x1, 0x4, 0xc420437fa0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba680, 0xc4200b8a50, 0xc4206f3260, 0xc4206f3270, 0xc4204ede00, 0xc42044c100, 0x10f5e80, 0xc4205cce90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmindznh-ghzrh Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b54040, 0xc42000ead0, 0xc42160c3a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba600, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc420b77f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420089a40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa0c0, 0xc4201a85a0, 0xc420b53700, 0xc420b53710, 0xc420b6e300, 0xc420b45940, 0x10f5e80, 0xc42000ead0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmisncth-thll2 Pod phase: Running goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42193a040, 0xc4200b7490, 0xc420186600, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c240, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c240, 0xc42000e078, 0x13, 0xc421961f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa040, 0xc4201105f0, 0xc421939240, 0xc421939250, 0xc42195c300, 0xc42192cd20, 0x10f5e80, 0xc4200b7490, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiswg89-nclgg Pod phase: Running goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278430, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930cc0, 0xc4200b6468, 0xc42000da00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6de0, 0xc4200b6240, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4200bab80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6de0, 0xc4200b6240, 0x13, 0xc42199ff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42094a040) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa480, 0xc4200e4910, 0xc421a529d0, 0xc421a529e0, 0xc421933d80, 0xc4219460a0, 0x10f5e80, 0xc4200b6468, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:56:19 http: TLS handshake error from 10.244.0.1:50540: EOF 2018/07/26 11:56:29 http: TLS handshake error from 10.244.0.1:50564: EOF 2018/07/26 11:56:39 http: TLS handshake error from 10.244.0.1:50588: EOF level=info timestamp=2018-07-26T11:56:40.636218Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:56:40.738472Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:56:49 http: TLS handshake error from 10.244.0.1:50612: EOF 2018/07/26 11:56:59 http: TLS handshake error from 10.244.0.1:50636: EOF 2018/07/26 11:57:09 http: TLS handshake error from 10.244.0.1:50660: EOF level=info timestamp=2018-07-26T11:57:10.828121Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:57:10.916801Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:57:19 http: TLS handshake error from 10.244.0.1:50684: EOF 2018/07/26 11:57:29 http: TLS handshake error from 10.244.0.1:50708: EOF level=info timestamp=2018-07-26T11:57:33.365980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:57:33.371247Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:39 http: TLS handshake error from 10.244.0.1:50732: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:56:00 http: TLS handshake error from 10.244.1.1:44390: EOF 2018/07/26 11:56:10 http: TLS handshake error from 10.244.1.1:44396: EOF 2018/07/26 11:56:20 http: TLS handshake error from 10.244.1.1:44402: EOF level=info timestamp=2018-07-26T11:56:22.010953Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:56:30 http: TLS handshake error from 10.244.1.1:44408: EOF 2018/07/26 11:56:40 http: TLS handshake error from 10.244.1.1:44414: EOF 2018/07/26 11:56:50 http: TLS handshake error from 10.244.1.1:44420: EOF level=info timestamp=2018-07-26T11:56:51.943836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:00 http: TLS handshake error from 10.244.1.1:44426: EOF 2018/07/26 11:57:10 http: TLS handshake error from 10.244.1.1:44432: EOF 2018/07/26 11:57:20 http: TLS handshake error from 10.244.1.1:44438: EOF level=info timestamp=2018-07-26T11:57:21.636484Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:57:21.735356Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:57:21.954971Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:30 http: TLS handshake error from 10.244.1.1:44444: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:51:37.296598Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:51:37.296716Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi5ww42 kind= uid=405769a0-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:51:37.378581Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivnr6n\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivnr6n" level=info timestamp=2018-07-26T11:51:37.933213Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi5ww42\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi5ww42" level=info timestamp=2018-07-26T11:54:38.535687Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiwsjmz\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmiwsjmz, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 404fc7ec-90ca-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiwsjmz" level=info timestamp=2018-07-26T11:54:39.091081Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.092208Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.145689Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.146060Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.195615Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.196157Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.226245Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:54:39.227138Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.823143Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiftsjp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiftsjp" level=info timestamp=2018-07-26T11:54:40.048180Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmindznh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:54:55.958017Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:55.961691Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:54:57.243268Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:57.244065Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:57.247884Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:54:59.808996Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:54:59.809599Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:54:59.809943Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:55:04.931098Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:04.933696Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:04.934059Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiswg89" level=info timestamp=2018-07-26T11:55:15.175611Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:15.234724Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:15.235007Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:15.235164Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:55:07.503758Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:55:07.504321Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:55:07.504556Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmisncth" level=info timestamp=2018-07-26T11:55:15.826868Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:15.851353Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:15.851690Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:15.851835Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiftsjp kind= uid=acc21d90-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:17.736221Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:17.745289Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:17.762697Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:17.762957Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:17.763103Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmindznh kind= uid=acc7701d-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:17.783708Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:55:17.784024Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:55:17.784297Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmisncth kind= uid=acbbb2da-90ca-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmiftsjp-2dvcn Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203f84d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420400b20, 0xc4205cce90, 0xc4205f9fc0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6cc0, 0xc4200b6250, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc4200bacc0, 0xb, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6cc0, 0xc4200b6250, 0x13, 0xc4215d5718, 0x3, 0x3, 0x1, 0x4, 0xc420437fa0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba680, 0xc4200b8a50, 0xc4206f3260, 0xc4206f3270, 0xc4204ede00, 0xc42044c100, 0x10f5e80, 0xc4205cce90, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmindznh-ghzrh Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b54040, 0xc42000ead0, 0xc42160c3a0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba600, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc420b77f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420089a40) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa0c0, 0xc4201a85a0, 0xc420b53700, 0xc420b53710, 0xc420b6e300, 0xc420b45940, 0x10f5e80, 0xc42000ead0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmisncth-thll2 Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42193a040, 0xc4200b7490, 0xc420186600, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c240, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c240, 0xc42000e078, 0x13, 0xc421961f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa040, 0xc4201105f0, 0xc421939240, 0xc421939250, 0xc42195c300, 0xc42192cd20, 0x10f5e80, 0xc4200b7490, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiswg89-nclgg Pod phase: Failed goroutine 28 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278430, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421930cc0, 0xc4200b6468, 0xc42000da00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6de0, 0xc4200b6240, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4200bab80, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6de0, 0xc4200b6240, 0x13, 0xc42199ff18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42094a040) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa480, 0xc4200e4910, 0xc421a529d0, 0xc421a529e0, 0xc421933d80, 0xc4219460a0, 0x10f5e80, 0xc4200b6468, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.985 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on the same node from Node /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:54:39.140349Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmisncth-thll2" level=info timestamp=2018-07-26T11:54:56.817404Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmisncth-thll2" level=error timestamp=2018-07-26T11:54:57.248059Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:56:40.636218Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:56:40.738472Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:56:49 http: TLS handshake error from 10.244.0.1:50612: EOF 2018/07/26 11:56:59 http: TLS handshake error from 10.244.0.1:50636: EOF 2018/07/26 11:57:09 http: TLS handshake error from 10.244.0.1:50660: EOF level=info timestamp=2018-07-26T11:57:10.828121Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:57:10.916801Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:57:19 http: TLS handshake error from 10.244.0.1:50684: EOF 2018/07/26 11:57:29 http: TLS handshake error from 10.244.0.1:50708: EOF level=info timestamp=2018-07-26T11:57:33.365980Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:57:33.371247Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:39 http: TLS handshake error from 10.244.0.1:50732: EOF level=info timestamp=2018-07-26T11:57:40.973879Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:57:41.021720Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:57:49 http: TLS handshake error from 10.244.0.1:50756: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:56:22.010953Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:56:30 http: TLS handshake error from 10.244.1.1:44408: EOF 2018/07/26 11:56:40 http: TLS handshake error from 10.244.1.1:44414: EOF 2018/07/26 11:56:50 http: TLS handshake error from 10.244.1.1:44420: EOF level=info timestamp=2018-07-26T11:56:51.943836Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:00 http: TLS handshake error from 10.244.1.1:44426: EOF 2018/07/26 11:57:10 http: TLS handshake error from 10.244.1.1:44432: EOF 2018/07/26 11:57:20 http: TLS handshake error from 10.244.1.1:44438: EOF level=info timestamp=2018-07-26T11:57:21.636484Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:57:21.735356Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T11:57:21.954971Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:57:30 http: TLS handshake error from 10.244.1.1:44444: EOF 2018/07/26 11:57:40 http: TLS handshake error from 10.244.1.1:44450: EOF 2018/07/26 11:57:50 http: TLS handshake error from 10.244.1.1:44456: EOF level=info timestamp=2018-07-26T11:57:52.128738Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:54:39.227138Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.823143Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiftsjp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiftsjp" level=info timestamp=2018-07-26T11:54:40.048180Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmindznh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" level=info timestamp=2018-07-26T11:57:41.052143Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.055257Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.072026Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.072238Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.120776Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.121111Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.148135Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.148449Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.247482Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibpjhk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:41.559651Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmixm8zj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmixm8zj" level=info timestamp=2018-07-26T11:57:41.763783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijmjjz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:42.166099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijg52z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijg52z" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T11:57:57.757705Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.757904Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.758002Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:57.798670Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.799287Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.800259Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:57.881061Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.881453Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.881588Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:58.042631Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:58.043048Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:58.043297Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:58.366980Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:58.367164Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:58.367552Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:57:57.594246Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.594795Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.595468Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:57.608297Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.608737Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.609259Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmixm8zj" level=info timestamp=2018-07-26T11:57:57.636404Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.636817Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.637096Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:57.717988Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.718494Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.718741Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:57.886133Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:57:57.886702Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:57.886914Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" Pod name: virt-launcher-testvmibpjhk-xwqqb Pod phase: Running goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d43a0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b19580, 0xc42000e430, 0xc420db3ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6d80, 0xc4200b62a8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6d80, 0xc4200b62a8, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420348080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa480, 0xc420140960, 0xc420d964c0, 0xc420d964d0, 0xc420b1dd80, 0xc420b240e0, 0x10f5e80, 0xc42000e430, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmijg52z-2frnt Pod phase: Running level=info timestamp=2018-07-26T11:57:45.422369Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T11:57:45.422489Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T11:57:45.424476Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T11:57:57.579444Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T11:57:57.603795Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmijg52z" level=info timestamp=2018-07-26T11:57:57.607028Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T11:57:57.607560Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmijmjjz-gjwnl Pod phase: Running goroutine 36 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202763a0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420242620, 0xc420744640, 0xc420765ca0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2420, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420404480, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2420, 0xc42000e100, 0x13, 0xc42158f718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420701bc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420088500, 0xc4200e4780, 0xc4207424b0, 0xc4207424c0, 0xc4202f3d80, 0xc4202980e0, 0x10f5e80, 0xc420744640, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmixm8zj-lmgfg Pod phase: Running goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40c0, 0xc420106280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420244040, 0xc42000e1a0, 0xc420688740, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c240, 0xc4200b60a0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc42034a600, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c240, 0xc4200b60a0, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42034a580) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406040, 0xc4200b84b0, 0xc42064f830, 0xc42064f840, 0xc4204b8300, 0xc420186c80, 0x10f5e80, 0xc42000e1a0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 11:59:19 http: TLS handshake error from 10.244.0.1:50972: EOF 2018/07/26 11:59:29 http: TLS handshake error from 10.244.0.1:50996: EOF 2018/07/26 11:59:39 http: TLS handshake error from 10.244.0.1:51020: EOF level=info timestamp=2018-07-26T11:59:41.686173Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T11:59:41.685048Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:59:49 http: TLS handshake error from 10.244.0.1:51044: EOF 2018/07/26 11:59:59 http: TLS handshake error from 10.244.0.1:51068: EOF 2018/07/26 12:00:09 http: TLS handshake error from 10.244.0.1:51092: EOF level=info timestamp=2018-07-26T12:00:11.866479Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:00:11.866482Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:00:19 http: TLS handshake error from 10.244.0.1:51116: EOF 2018/07/26 12:00:29 http: TLS handshake error from 10.244.0.1:51140: EOF level=info timestamp=2018-07-26T12:00:33.394010Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:00:33.397213Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:39 http: TLS handshake error from 10.244.0.1:51164: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T11:58:51.984559Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:59:00 http: TLS handshake error from 10.244.1.1:44498: EOF 2018/07/26 11:59:10 http: TLS handshake error from 10.244.1.1:44504: EOF 2018/07/26 11:59:20 http: TLS handshake error from 10.244.1.1:44510: EOF level=info timestamp=2018-07-26T11:59:21.969264Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:59:30 http: TLS handshake error from 10.244.1.1:44516: EOF 2018/07/26 11:59:40 http: TLS handshake error from 10.244.1.1:44522: EOF 2018/07/26 11:59:50 http: TLS handshake error from 10.244.1.1:44528: EOF level=info timestamp=2018-07-26T11:59:51.947283Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:00 http: TLS handshake error from 10.244.1.1:44534: EOF 2018/07/26 12:00:10 http: TLS handshake error from 10.244.1.1:44540: EOF 2018/07/26 12:00:20 http: TLS handshake error from 10.244.1.1:44546: EOF level=info timestamp=2018-07-26T12:00:21.933661Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:30 http: TLS handshake error from 10.244.1.1:44552: EOF 2018/07/26 12:00:40 http: TLS handshake error from 10.244.1.1:44558: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:54:39.227138Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiswg89 kind= uid=accec88e-90ca-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:54:39.823143Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiftsjp\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiftsjp" level=info timestamp=2018-07-26T11:54:40.048180Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmindznh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmindznh" level=info timestamp=2018-07-26T11:57:41.052143Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.055257Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.072026Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.072238Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.120776Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.121111Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.148135Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T11:57:41.148449Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.247482Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibpjhk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:41.559651Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmixm8zj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmixm8zj" level=info timestamp=2018-07-26T11:57:41.763783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijmjjz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:42.166099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijg52z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijg52z" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T11:57:59.008273Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:57:59.008597Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:58:00.337960Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:00.338211Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:00.338542Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:58:02.899078Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:02.899350Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:02.899498Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:58:08.020125Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:08.021060Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:08.022652Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:58:18.264789Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:18.314063Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:18.315043Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:18.315266Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T11:58:09.753382Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T11:58:09.754130Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T11:58:09.754551Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmijg52z" level=info timestamp=2018-07-26T11:58:17.456397Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:17.511763Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:17.512062Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:17.512297Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmixm8zj kind= uid=19366645-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:18.057687Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:18.082296Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:18.082650Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:18.082802Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmibpjhk kind= uid=19324b6d-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:19.995469Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:20.020358Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T11:58:20.021287Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T11:58:20.021645Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmijg52z kind= uid=1939bc3f-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmibpjhk-xwqqb Pod phase: Failed goroutine 10 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d43a0, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420b19580, 0xc42000e430, 0xc420db3ba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6d80, 0xc4200b62a8, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200ba080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6d80, 0xc4200b62a8, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420348080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202aa480, 0xc420140960, 0xc420d964c0, 0xc420d964d0, 0xc420b1dd80, 0xc420b240e0, 0x10f5e80, 0xc42000e430, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmijg52z-2frnt Pod phase: Failed goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202e40b0, 0xc4201aac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c100, 0xc42000e4d8, 0xc420698760, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc4200c6070, 0x13, 0x131ed4d, 0x4, 0xc420068f18, 0x3, 0x3, 0xc4202ba600, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc4200c6070, 0x13, 0xc420057718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ba4c0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420358040, 0xc4200c8410, 0xc420660690, 0xc4206606a0, 0xc42032a300, 0xc420575d80, 0x10f5e80, 0xc42000e4d8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmijmjjz-gjwnl Pod phase: Failed goroutine 36 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202763a0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420242620, 0xc420744640, 0xc420765ca0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200e2420, 0xc42000e100, 0x13, 0x131ed4d, 0x4, 0xc42006ff18, 0x3, 0x3, 0xc420404480, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200e2420, 0xc42000e100, 0x13, 0xc42158f718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420701bc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420088500, 0xc4200e4780, 0xc4207424b0, 0xc4207424c0, 0xc4202f3d80, 0xc4202980e0, 0x10f5e80, 0xc420744640, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmixm8zj-lmgfg Pod phase: Failed goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202d40c0, 0xc420106280, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420244040, 0xc42000e1a0, 0xc420688740, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c240, 0xc4200b60a0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc42034a600, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c240, 0xc4200b60a0, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc42034a580) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420406040, 0xc4200b84b0, 0xc42064f830, 0xc42064f840, 0xc4204b8300, 0xc420186c80, 0x10f5e80, 0xc42000e1a0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.808 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 should be reachable via the propagated IP from a Pod [BeforeEach] /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table.go:92 on a different node from Node /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/extensions/table/table_entry.go:46 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T11:57:41.120410Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmibpjhk-xwqqb" level=info timestamp=2018-07-26T11:57:57.264961Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmibpjhk-xwqqb" level=error timestamp=2018-07-26T11:57:57.544188Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T11:59:41.685048Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 11:59:49 http: TLS handshake error from 10.244.0.1:51044: EOF 2018/07/26 11:59:59 http: TLS handshake error from 10.244.0.1:51068: EOF 2018/07/26 12:00:09 http: TLS handshake error from 10.244.0.1:51092: EOF level=info timestamp=2018-07-26T12:00:11.866479Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:00:11.866482Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:00:19 http: TLS handshake error from 10.244.0.1:51116: EOF 2018/07/26 12:00:29 http: TLS handshake error from 10.244.0.1:51140: EOF level=info timestamp=2018-07-26T12:00:33.394010Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:00:33.397213Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:39 http: TLS handshake error from 10.244.0.1:51164: EOF level=info timestamp=2018-07-26T12:00:42.067102Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:00:42.062244Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:00:49 http: TLS handshake error from 10.244.0.1:51188: EOF 2018/07/26 12:00:59 http: TLS handshake error from 10.244.0.1:51212: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 11:59:20 http: TLS handshake error from 10.244.1.1:44510: EOF level=info timestamp=2018-07-26T11:59:21.969264Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 11:59:30 http: TLS handshake error from 10.244.1.1:44516: EOF 2018/07/26 11:59:40 http: TLS handshake error from 10.244.1.1:44522: EOF 2018/07/26 11:59:50 http: TLS handshake error from 10.244.1.1:44528: EOF level=info timestamp=2018-07-26T11:59:51.947283Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:00 http: TLS handshake error from 10.244.1.1:44534: EOF 2018/07/26 12:00:10 http: TLS handshake error from 10.244.1.1:44540: EOF 2018/07/26 12:00:20 http: TLS handshake error from 10.244.1.1:44546: EOF level=info timestamp=2018-07-26T12:00:21.933661Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:00:30 http: TLS handshake error from 10.244.1.1:44552: EOF 2018/07/26 12:00:40 http: TLS handshake error from 10.244.1.1:44558: EOF 2018/07/26 12:00:50 http: TLS handshake error from 10.244.1.1:44564: EOF level=info timestamp=2018-07-26T12:00:51.895538Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:01:00 http: TLS handshake error from 10.244.1.1:44570: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:57:41.148449Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.247482Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibpjhk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:41.559651Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmixm8zj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmixm8zj" level=info timestamp=2018-07-26T11:57:41.763783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijmjjz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:42.166099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijg52z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijg52z" level=info timestamp=2018-07-26T12:00:42.888562Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:42.891972Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:42.926032Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:42.932669Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:42.982870Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:42.983211Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:43.075041Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:43.077668Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:43.263759Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifpzrv\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:00:43.513938Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik2b6w\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T12:00:59.598013Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:00:59.598261Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:00:59.599111Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:00:59.679806Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:00:59.681180Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:00:59.681938Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:00:59.843482Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:00:59.844055Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:00:59.844903Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:01:00.165532Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.165822Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.166142Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:01:00.806737Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.807843Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.807992Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T12:01:00.581801Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.582273Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.582502Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:01:00.603375Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.603769Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.604030Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:01:00.644785Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.645354Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.645608Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:01:00.726428Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.726870Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.727089Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:01:00.887823Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:00.888366Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.888605Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" Pod name: virt-launcher-testvmi22lc6-5rd5t Pod phase: Running level=info timestamp=2018-07-26T12:00:48.508088Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:00:48.513352Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:00:48.516050Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:00:59.963971Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:00:59.997035Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi22lc6" level=info timestamp=2018-07-26T12:00:59.997824Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:00:59.998005Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: virt-launcher-testvmifpzrv-skgfp Pod phase: Running goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203183f0, 0xc4201b0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42193cc60, 0xc4200b6460, 0xc4205097c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b6168, 0x13, 0x131ed4d, 0x4, 0xc42194ef18, 0x3, 0x3, 0xc420088080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b6168, 0x13, 0xc42196ef18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420558540, 0xc4200b8730, 0xc42059a010, 0xc42059a020, 0xc42193fe00, 0xc421989500, 0x10f5e80, 0xc4200b6460, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmik2b6w-zfbz6 Pod phase: Running goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420372370, 0xc4201a6c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421904c60, 0xc42000fa80, 0xc420397d20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c7020, 0xc4200b6318, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bb0c0, 0x1b, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c7020, 0xc4200b6318, 0x13, 0xc4206bcf18, 0x3, 0x3, 0x1, 0x4, 0xc4202bdaf8) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034e740, 0xc4200b8870, 0xc421aa4680, 0xc421aa4690, 0xc421907d80, 0xc4219100e0, 0x10f5e80, 0xc42000fa80, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminnqgg-nlrpt Pod phase: Running level=info timestamp=2018-07-26T12:00:48.132218Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:00:48.132869Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:00:48.135440Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:01:00.117580Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:01:00.149987Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvminnqgg" level=info timestamp=2018-07-26T12:01:00.150953Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:01:00.151215Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 12:02:39 http: TLS handshake error from 10.244.0.1:51452: EOF level=info timestamp=2018-07-26T12:02:42.754867Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:02:42.762090Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:02:49 http: TLS handshake error from 10.244.0.1:51476: EOF 2018/07/26 12:02:59 http: TLS handshake error from 10.244.0.1:51500: EOF 2018/07/26 12:03:09 http: TLS handshake error from 10.244.0.1:51524: EOF level=info timestamp=2018-07-26T12:03:12.945383Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:03:12.952430Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:03:19 http: TLS handshake error from 10.244.0.1:51548: EOF 2018/07/26 12:03:29 http: TLS handshake error from 10.244.0.1:51572: EOF level=info timestamp=2018-07-26T12:03:31.166869Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:03:31.171871Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:39 http: TLS handshake error from 10.244.0.1:51596: EOF level=info timestamp=2018-07-26T12:03:43.134993Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:03:43.140021Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T12:01:51.966282Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:02:00 http: TLS handshake error from 10.244.1.1:44606: EOF 2018/07/26 12:02:10 http: TLS handshake error from 10.244.1.1:44612: EOF 2018/07/26 12:02:20 http: TLS handshake error from 10.244.1.1:44618: EOF level=info timestamp=2018-07-26T12:02:22.085956Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:02:30 http: TLS handshake error from 10.244.1.1:44624: EOF 2018/07/26 12:02:40 http: TLS handshake error from 10.244.1.1:44630: EOF 2018/07/26 12:02:50 http: TLS handshake error from 10.244.1.1:44636: EOF level=info timestamp=2018-07-26T12:02:51.949498Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:00 http: TLS handshake error from 10.244.1.1:44642: EOF 2018/07/26 12:03:10 http: TLS handshake error from 10.244.1.1:44648: EOF 2018/07/26 12:03:20 http: TLS handshake error from 10.244.1.1:44654: EOF level=info timestamp=2018-07-26T12:03:22.039618Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:30 http: TLS handshake error from 10.244.1.1:44660: EOF 2018/07/26 12:03:40 http: TLS handshake error from 10.244.1.1:44666: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T11:57:41.148449Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmijmjjz kind= uid=193dde5d-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T11:57:41.247482Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmibpjhk\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmibpjhk" level=info timestamp=2018-07-26T11:57:41.559651Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmixm8zj\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmixm8zj" level=info timestamp=2018-07-26T11:57:41.763783Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijmjjz\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijmjjz" level=info timestamp=2018-07-26T11:57:42.166099Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmijg52z\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmijg52z" level=info timestamp=2018-07-26T12:00:42.888562Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:42.891972Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:42.926032Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:42.932669Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:42.982870Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:42.983211Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:43.075041Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:00:43.077668Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:43.263759Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifpzrv\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:00:43.513938Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik2b6w\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T12:01:00.807843Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:00.807992Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:01:02.088817Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:02.090568Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:02.091181Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:01:04.652597Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:04.653466Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:04.653843Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:01:09.775029Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:09.777066Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:09.777967Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:01:20.020009Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:20.100457Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:20.100729Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:20.100880Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T12:01:12.616574Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:01:12.617013Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:01:12.617327Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi22lc6" level=info timestamp=2018-07-26T12:01:21.061870Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:21.094459Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:21.094774Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:21.094905Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmifpzrv kind= uid=8592dddd-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:22.318774Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:22.340924Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:22.342588Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:22.342697Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvminnqgg kind= uid=859d15fe-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:22.858320Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:22.904054Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:01:22.904392Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:01:22.904537Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi22lc6 kind= uid=8598c15c-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi22lc6-5rd5t Pod phase: Failed goroutine 34 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202900d0, 0xc4201b0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420bda040, 0xc4200b7860, 0xc4216855e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc4200bb8c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc420057f18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc421648800) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202cc040, 0xc4200d45f0, 0xc420ca7cc0, 0xc420ca7cd0, 0xc420bf4300, 0xc420bcd680, 0x10f5e80, 0xc4200b7860, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmifpzrv-skgfp Pod phase: Failed goroutine 33 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203183f0, 0xc4201b0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42193cc60, 0xc4200b6460, 0xc4205097c0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c6c60, 0xc4200b6168, 0x13, 0x131ed4d, 0x4, 0xc42194ef18, 0x3, 0x3, 0xc420088080, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c6c60, 0xc4200b6168, 0x13, 0xc42196ef18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420558540, 0xc4200b8730, 0xc42059a010, 0xc42059a020, 0xc42193fe00, 0xc421989500, 0x10f5e80, 0xc4200b6460, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmik2b6w-zfbz6 Pod phase: Failed goroutine 12 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420372370, 0xc4201a6c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421904c60, 0xc42000fa80, 0xc420397d20, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c7020, 0xc4200b6318, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4200bb0c0, 0x1b, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c7020, 0xc4200b6318, 0x13, 0xc4206bcf18, 0x3, 0x3, 0x1, 0x4, 0xc4202bdaf8) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034e740, 0xc4200b8870, 0xc421aa4680, 0xc421aa4690, 0xc421907d80, 0xc4219100e0, 0x10f5e80, 0xc42000fa80, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvminnqgg-nlrpt Pod phase: Failed goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202da3d0, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42023b240, 0xc4200b7398, 0xc4206daee0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c5a0, 0xc4200b6118, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200ba040, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c5a0, 0xc4200b6118, 0x13, 0xc420059f18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202b45c0, 0xc4200b8a50, 0xc4216e78f0, 0xc4216e7900, 0xc42037fd80, 0xc420292160, 0x10f5e80, 0xc4200b7398, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.944 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 with a service matching the vmi exposed [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:283 should be able to reach the vmi based on labels specified on the vmi /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:303 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:00:42.907218Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmifpzrv-skgfp" level=info timestamp=2018-07-26T12:01:00.082828Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmifpzrv-skgfp" level=error timestamp=2018-07-26T12:01:00.526960Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T12:02:42.762090Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:02:49 http: TLS handshake error from 10.244.0.1:51476: EOF 2018/07/26 12:02:59 http: TLS handshake error from 10.244.0.1:51500: EOF 2018/07/26 12:03:09 http: TLS handshake error from 10.244.0.1:51524: EOF level=info timestamp=2018-07-26T12:03:12.945383Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:03:12.952430Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:03:19 http: TLS handshake error from 10.244.0.1:51548: EOF 2018/07/26 12:03:29 http: TLS handshake error from 10.244.0.1:51572: EOF level=info timestamp=2018-07-26T12:03:31.166869Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:03:31.171871Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:39 http: TLS handshake error from 10.244.0.1:51596: EOF level=info timestamp=2018-07-26T12:03:43.134993Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:03:43.140021Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:03:49 http: TLS handshake error from 10.244.0.1:51620: EOF 2018/07/26 12:03:59 http: TLS handshake error from 10.244.0.1:51644: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 12:02:20 http: TLS handshake error from 10.244.1.1:44618: EOF level=info timestamp=2018-07-26T12:02:22.085956Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:02:30 http: TLS handshake error from 10.244.1.1:44624: EOF 2018/07/26 12:02:40 http: TLS handshake error from 10.244.1.1:44630: EOF 2018/07/26 12:02:50 http: TLS handshake error from 10.244.1.1:44636: EOF level=info timestamp=2018-07-26T12:02:51.949498Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:00 http: TLS handshake error from 10.244.1.1:44642: EOF 2018/07/26 12:03:10 http: TLS handshake error from 10.244.1.1:44648: EOF 2018/07/26 12:03:20 http: TLS handshake error from 10.244.1.1:44654: EOF level=info timestamp=2018-07-26T12:03:22.039618Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:03:30 http: TLS handshake error from 10.244.1.1:44660: EOF 2018/07/26 12:03:40 http: TLS handshake error from 10.244.1.1:44666: EOF 2018/07/26 12:03:50 http: TLS handshake error from 10.244.1.1:44672: EOF level=info timestamp=2018-07-26T12:03:51.968480Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:04:00 http: TLS handshake error from 10.244.1.1:44678: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T12:00:43.077668Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:43.263759Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifpzrv\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:00:43.513938Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik2b6w\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:03:44.832875Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.834910Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:44.854781Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.855003Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:44.886745Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.886910Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:44.918566Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.918947Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:45.006918Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmif4qjn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmif4qjn" level=info timestamp=2018-07-26T12:03:45.426984Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiczgn7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiczgn7" level=info timestamp=2018-07-26T12:03:45.851007Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivcqrm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:03:46.223047Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivcqrm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T12:04:00.818704Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:00.819221Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:00.819743Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:00.903290Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:00.904291Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:00.906108Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:01.066920Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:01.067787Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:01.068244Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:01.389314Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:01.390098Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:01.390535Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:02.031019Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:02.031286Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.031494Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T12:04:02.435532Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:02.435862Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.436136Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiczgn7" level=info timestamp=2018-07-26T12:04:02.467554Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:02.467886Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.468051Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi8l86h" level=info timestamp=2018-07-26T12:04:02.510735Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:02.511325Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.511689Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmif4qjn" level=info timestamp=2018-07-26T12:04:02.789012Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:02.790510Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.790764Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmi8l86h" level=info timestamp=2018-07-26T12:04:02.833292Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:02.833896Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.834253Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmif4qjn" Pod name: virt-launcher-testvmi8l86h-6q9r2 Pod phase: Running goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42058c0f0, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421944040, 0xc42000edf8, 0xc4200c5f00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c60c0, 0xc4200b60f0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4202fa080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c60c0, 0xc4200b60f0, 0x13, 0xc42196e718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202fbe80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c0c0, 0xc4200e44b0, 0xc421a58b50, 0xc421a58b60, 0xc421960300, 0xc4219396a0, 0x10f5e80, 0xc42000edf8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiczgn7-gtjfs Pod phase: Running goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202780e0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42193a080, 0xc4200b6708, 0xc421be4680, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc420088140, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ac480) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc42018a5f0, 0xc421ba2f20, 0xc421ba2f30, 0xc421958300, 0xc42192ecc0, 0x10f5e80, 0xc4200b6708, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmif4qjn-4sfg9 Pod phase: Running goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203b03e0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421918c80, 0xc42000e368, 0xc420212060, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42014eae0, 0xc42000e0f0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204807c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42014eae0, 0xc42000e0f0, 0x13, 0xc42198f718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420480000) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4204805c0, 0xc4200a8820, 0xc4202c40e0, 0xc4202c4100, 0xc42191be00, 0xc4219240e0, 0x10f5e80, 0xc42000e368, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivcqrm-zwprg Pod phase: Running goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926040, 0xc421ba82f0, 0xc420187500, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096600, 0xc4200b6708, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420088480, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096600, 0xc4200b6708, 0x13, 0xc42005df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346040, 0xc4200b8640, 0xc4202f40d0, 0xc4202f4110, 0xc421940300, 0xc421918d00, 0x10f5e80, 0xc421ba82f0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running level=info timestamp=2018-07-26T12:05:31.180001Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/openapi/v2 proto=HTTP/2.0 statusCode=404 contentLength=19 level=info timestamp=2018-07-26T12:05:31.183588Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/swagger.json proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:05:39 http: TLS handshake error from 10.244.0.1:51884: EOF level=info timestamp=2018-07-26T12:05:43.971673Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:05:43.996639Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:05:49 http: TLS handshake error from 10.244.0.1:51908: EOF 2018/07/26 12:05:59 http: TLS handshake error from 10.244.0.1:51932: EOF 2018/07/26 12:06:09 http: TLS handshake error from 10.244.0.1:51956: EOF level=info timestamp=2018-07-26T12:06:14.135001Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:06:14.151191Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:06:19 http: TLS handshake error from 10.244.0.1:51980: EOF 2018/07/26 12:06:29 http: TLS handshake error from 10.244.0.1:52004: EOF 2018/07/26 12:06:39 http: TLS handshake error from 10.244.0.1:52028: EOF level=info timestamp=2018-07-26T12:06:44.343107Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:06:44.356051Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 Pod name: virt-api-7d79764579-k9ldt Pod phase: Running level=info timestamp=2018-07-26T12:04:52.059511Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:05:00 http: TLS handshake error from 10.244.1.1:44714: EOF 2018/07/26 12:05:10 http: TLS handshake error from 10.244.1.1:44720: EOF 2018/07/26 12:05:20 http: TLS handshake error from 10.244.1.1:44726: EOF level=info timestamp=2018-07-26T12:05:21.901415Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:05:30 http: TLS handshake error from 10.244.1.1:44732: EOF 2018/07/26 12:05:40 http: TLS handshake error from 10.244.1.1:44738: EOF 2018/07/26 12:05:50 http: TLS handshake error from 10.244.1.1:44744: EOF level=info timestamp=2018-07-26T12:05:52.062608Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:06:00 http: TLS handshake error from 10.244.1.1:44750: EOF 2018/07/26 12:06:10 http: TLS handshake error from 10.244.1.1:44756: EOF 2018/07/26 12:06:20 http: TLS handshake error from 10.244.1.1:44762: EOF level=info timestamp=2018-07-26T12:06:21.990320Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:06:30 http: TLS handshake error from 10.244.1.1:44768: EOF 2018/07/26 12:06:40 http: TLS handshake error from 10.244.1.1:44774: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T12:00:43.077668Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmik2b6w kind= uid=85a55e05-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:00:43.263759Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmifpzrv\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmifpzrv" level=info timestamp=2018-07-26T12:00:43.513938Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmik2b6w\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmik2b6w" level=info timestamp=2018-07-26T12:03:44.832875Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.834910Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:44.854781Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.855003Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:44.886745Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.886910Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:44.918566Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:03:44.918947Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:03:45.006918Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmif4qjn\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmif4qjn" level=info timestamp=2018-07-26T12:03:45.426984Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmiczgn7\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmiczgn7" level=info timestamp=2018-07-26T12:03:45.851007Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivcqrm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:03:46.223047Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivcqrm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" Pod name: virt-handler-rmqjz Pod phase: Running level=error timestamp=2018-07-26T12:04:02.031286Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:02.031494Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:03.327093Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:03.328041Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:03.337977Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:05.899295Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:05.899889Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:05.900251Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:11.021637Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:11.022113Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:11.023308Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:04:21.264858Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:21.310295Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:04:21.310877Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:21.311073Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-handler-xqtr8 Pod phase: Running level=info timestamp=2018-07-26T12:04:12.439443Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:04:12.439685Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:04:12.439791Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmif4qjn" level=info timestamp=2018-07-26T12:04:22.290717Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:22.321448Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:04:22.321837Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:22.322001Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmiczgn7 kind= uid=f20ad52a-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:04:22.638393Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:22.675610Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:04:22.676038Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:22.676415Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmi8l86h kind= uid=f20db28b-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:04:22.680471Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:22.707324Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:04:22.707575Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:22.707711Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmif4qjn kind= uid=f2059018-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." Pod name: virt-launcher-testvmi8l86h-6q9r2 Pod phase: Failed goroutine 15 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc42058c0f0, 0xc4201b2c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421944040, 0xc42000edf8, 0xc4200c5f00, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200c60c0, 0xc4200b60f0, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc4202fa080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200c60c0, 0xc4200b60f0, 0x13, 0xc42196e718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202fbe80) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034c0c0, 0xc4200e44b0, 0xc421a58b50, 0xc421a58b60, 0xc421960300, 0xc4219396a0, 0x10f5e80, 0xc42000edf8, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiczgn7-gtjfs Pod phase: Failed goroutine 13 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202780e0, 0xc4201a0c80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42193a080, 0xc4200b6708, 0xc421be4680, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c1e0, 0xc42000e078, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc420088140, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c1e0, 0xc42000e078, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ac480) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc42034a080, 0xc42018a5f0, 0xc421ba2f20, 0xc421ba2f30, 0xc421958300, 0xc42192ecc0, 0x10f5e80, 0xc4200b6708, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmif4qjn-4sfg9 Pod phase: Failed goroutine 30 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4203b03e0, 0xc42018ac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421918c80, 0xc42000e368, 0xc420212060, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42014eae0, 0xc42000e0f0, 0x13, 0x131ed4d, 0x4, 0xc42006af18, 0x3, 0x3, 0xc4204807c0, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42014eae0, 0xc42000e0f0, 0x13, 0xc42198f718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc420480000) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4204805c0, 0xc4200a8820, 0xc4202c40e0, 0xc4202c4100, 0xc42191be00, 0xc4219240e0, 0x10f5e80, 0xc42000e368, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmivcqrm-zwprg Pod phase: Failed goroutine 32 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4200a65d0, 0xc42019ec80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc421926040, 0xc421ba82f0, 0xc420187500, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc420096600, 0xc4200b6708, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc420088480, 0x0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc420096600, 0xc4200b6708, 0x13, 0xc42005df18, 0x3, 0x3, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc420346040, 0xc4200b8640, 0xc4202f40d0, 0xc4202f4110, 0xc421940300, 0xc421918d00, 0x10f5e80, 0xc421ba82f0, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 • Failure in Spec Setup (BeforeEach) [181.936 seconds] Networking /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:48 with a service matching the vmi exposed [BeforeEach] /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:283 should fail to reach the vmi if an invalid servicename is used /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:314 Unexpected Warning event received. Expected : Warning not to equal : Warning /root/go/src/kubevirt.io/kubevirt/tests/utils.go:245 ------------------------------ level=info timestamp=2018-07-26T12:03:44.821174Z pos=utils.go:243 component=tests msg="Created virtual machine pod virt-launcher-testvmif4qjn-4sfg9" level=info timestamp=2018-07-26T12:04:01.919886Z pos=utils.go:243 component=tests msg="Pod owner ship transferred to the node virt-launcher-testvmif4qjn-4sfg9" level=error timestamp=2018-07-26T12:04:02.234910Z pos=utils.go:241 component=tests reason="unexpected warning event received" msg="unexpected EOF" Pod name: disks-images-provider-lggnr Pod phase: Running copy all images to host mount directory Pod name: disks-images-provider-z45tv Pod phase: Running copy all images to host mount directory Pod name: virt-api-7d79764579-frkfq Pod phase: Running 2018/07/26 12:05:39 http: TLS handshake error from 10.244.0.1:51884: EOF level=info timestamp=2018-07-26T12:05:43.971673Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:05:43.996639Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:05:49 http: TLS handshake error from 10.244.0.1:51908: EOF 2018/07/26 12:05:59 http: TLS handshake error from 10.244.0.1:51932: EOF 2018/07/26 12:06:09 http: TLS handshake error from 10.244.0.1:51956: EOF level=info timestamp=2018-07-26T12:06:14.135001Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:06:14.151191Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:06:19 http: TLS handshake error from 10.244.0.1:51980: EOF 2018/07/26 12:06:29 http: TLS handshake error from 10.244.0.1:52004: EOF 2018/07/26 12:06:39 http: TLS handshake error from 10.244.0.1:52028: EOF level=info timestamp=2018-07-26T12:06:44.343107Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 level=info timestamp=2018-07-26T12:06:44.356051Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.1 username=- method=GET url=/apis/subresources.kubevirt.io/v1alpha2 proto=HTTP/2.0 statusCode=200 contentLength=136 2018/07/26 12:06:49 http: TLS handshake error from 10.244.0.1:52052: EOF 2018/07/26 12:06:59 http: TLS handshake error from 10.244.0.1:52076: EOF Pod name: virt-api-7d79764579-k9ldt Pod phase: Running 2018/07/26 12:05:20 http: TLS handshake error from 10.244.1.1:44726: EOF level=info timestamp=2018-07-26T12:05:21.901415Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:05:30 http: TLS handshake error from 10.244.1.1:44732: EOF 2018/07/26 12:05:40 http: TLS handshake error from 10.244.1.1:44738: EOF 2018/07/26 12:05:50 http: TLS handshake error from 10.244.1.1:44744: EOF level=info timestamp=2018-07-26T12:05:52.062608Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:06:00 http: TLS handshake error from 10.244.1.1:44750: EOF 2018/07/26 12:06:10 http: TLS handshake error from 10.244.1.1:44756: EOF 2018/07/26 12:06:20 http: TLS handshake error from 10.244.1.1:44762: EOF level=info timestamp=2018-07-26T12:06:21.990320Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:06:30 http: TLS handshake error from 10.244.1.1:44768: EOF 2018/07/26 12:06:40 http: TLS handshake error from 10.244.1.1:44774: EOF 2018/07/26 12:06:50 http: TLS handshake error from 10.244.1.1:44782: EOF level=info timestamp=2018-07-26T12:06:51.996510Z pos=filter.go:46 component=virt-api remoteAddress=10.244.0.0 username=- method=GET url=/ proto=HTTP/2.0 statusCode=404 contentLength=19 2018/07/26 12:07:00 http: TLS handshake error from 10.244.1.1:44786: EOF Pod name: virt-controller-7d57d96b65-dl6qt Pod phase: Running level=info timestamp=2018-07-26T10:52:14.045222Z pos=application.go:174 component=virt-controller service=http action=listening interface=0.0.0.0 port=8182 Pod name: virt-controller-7d57d96b65-grz6n Pod phase: Running level=info timestamp=2018-07-26T12:03:46.223047Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivcqrm\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:06:46.302847Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivcqrm\": StorageError: invalid object, Code: 4, Key: /registry/kubevirt.io/virtualmachineinstances/kubevirt-test-default/testvmivcqrm, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f211baf2-90cb-11e8-b0c0-525500d15501, UID in object meta: " msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivcqrm" level=info timestamp=2018-07-26T12:06:46.844943Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr7zh kind= uid=5e7a0fa2-90cc-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:06:46.846852Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmihr7zh kind= uid=5e7a0fa2-90cc-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:06:46.895176Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:06:46.902959Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:06:46.946063Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8srr2 kind= uid=5e8e8c14-90cc-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:06:46.946266Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmi8srr2 kind= uid=5e8e8c14-90cc-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:06:47.167657Z pos=preset.go:139 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 msg="Initializing VirtualMachineInstance" level=info timestamp=2018-07-26T12:06:47.167810Z pos=preset.go:165 component=virt-controller service=http namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 msg="Marking VirtualMachineInstance as initialized" level=info timestamp=2018-07-26T12:06:47.169951Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmihr7zh\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmihr7zh" level=info timestamp=2018-07-26T12:06:47.200101Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivmlkc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivmlkc" level=info timestamp=2018-07-26T12:06:47.328686Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8srr2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8srr2" level=info timestamp=2018-07-26T12:06:47.460815Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmivmlkc\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmivmlkc" level=info timestamp=2018-07-26T12:06:48.062260Z pos=vmi.go:157 component=virt-controller service=http reason="Operation cannot be fulfilled on virtualmachineinstances.kubevirt.io \"testvmi8srr2\": the object has been modified; please apply your changes to the latest version and try again" msg="reenqueuing VirtualMachineInstance kubevirt-test-default/testvmi8srr2" Pod name: virt-handler-rmqjz Pod phase: Running level=info timestamp=2018-07-26T12:04:21.310877Z pos=vm.go:392 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="No update processing required" level=info timestamp=2018-07-26T12:04:21.311073Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:06:46.169938Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T12:06:46.187714Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind= uid=f211baf2-90cb-11e8-b0c0-525500d15501 msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:06:46.305588Z pos=vm.go:386 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind=VirtualMachineInstance uid= msg="Processing local ephemeral data cleanup for shutdown domain." level=info timestamp=2018-07-26T12:06:46.306166Z pos=vm.go:413 component=virt-handler namespace=kubevirt-test-default name=testvmivcqrm kind=VirtualMachineInstance uid= msg="Synchronization loop succeeded." level=info timestamp=2018-07-26T12:07:04.065893Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:04.260620Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 reason="unexpected EOF" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:04.279927Z pos=vm.go:251 component=virt-handler reason="unexpected EOF" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiv7pcf" level=info timestamp=2018-07-26T12:07:04.280191Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:04.280574Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:04.280688Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiv7pcf" level=info timestamp=2018-07-26T12:07:04.285447Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:04.285643Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmiv7pcf kind= uid=5e951e91-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:04.285759Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmiv7pcf" Pod name: virt-handler-xqtr8 Pod phase: Running level=error timestamp=2018-07-26T12:07:03.647141Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:03.647451Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivmlkc" level=info timestamp=2018-07-26T12:07:03.688526Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:03.688899Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:03.689127Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivmlkc" level=info timestamp=2018-07-26T12:07:03.769691Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:03.769929Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:03.770064Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivmlkc" level=info timestamp=2018-07-26T12:07:03.794662Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmihr7zh kind= uid=5e7a0fa2-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:03.794928Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmihr7zh kind= uid=5e7a0fa2-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:03.795072Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmihr7zh" level=info timestamp=2018-07-26T12:07:03.853462Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmi8srr2 kind= uid=5e8e8c14-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=info timestamp=2018-07-26T12:07:03.934438Z pos=vm.go:389 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 msg="Processing vmi update" level=error timestamp=2018-07-26T12:07:03.934662Z pos=vm.go:397 component=virt-handler namespace=kubevirt-test-default name=testvmivmlkc kind= uid=5e899de6-90cc-11e8-b0c0-525500d15501 reason="connection is shut down" msg="Synchronizing the VirtualMachineInstance failed." level=info timestamp=2018-07-26T12:07:03.934761Z pos=vm.go:251 component=virt-handler reason="connection is shut down" msg="re-enqueuing VirtualMachineInstance kubevirt-test-default/testvmivmlkc" Pod name: virt-launcher-testvmi8srr2-78cts Pod phase: Running level=info timestamp=2018-07-26T12:06:52.298123Z pos=manager.go:69 component=virt-launcher msg="Collected all requested hook sidecar sockets" level=info timestamp=2018-07-26T12:06:52.299247Z pos=manager.go:72 component=virt-launcher msg="Sorted all collected sidecar sockets per hook point based on their priority and name: map[]" level=info timestamp=2018-07-26T12:06:52.308354Z pos=libvirt.go:261 component=virt-launcher msg="Connecting to libvirt daemon: qemu:///system" level=info timestamp=2018-07-26T12:07:02.925785Z pos=libvirt.go:276 component=virt-launcher msg="Connected to libvirt daemon" level=info timestamp=2018-07-26T12:07:02.945855Z pos=virt-launcher.go:143 component=virt-launcher msg="Watchdog file created at /var/run/kubevirt/watchdog-files/kubevirt-test-default_testvmi8srr2" level=info timestamp=2018-07-26T12:07:02.947083Z pos=client.go:152 component=virt-launcher msg="Registered libvirt event notify callback" level=info timestamp=2018-07-26T12:07:02.947286Z pos=virt-launcher.go:60 component=virt-launcher msg="Marked as ready" level=info timestamp=2018-07-26T12:07:03.904562Z pos=cloud-init.go:254 component=virt-launcher msg="generated nocloud iso file /var/run/libvirt/kubevirt-ephemeral-disk/cloud-init-data/kubevirt-test-default/testvmi8srr2/noCloud.iso" level=error timestamp=2018-07-26T12:07:03.919960Z pos=common.go:126 component=virt-launcher msg="updated MAC for interface: eth0 - 0a:58:0a:d3:00:2e" level=info timestamp=2018-07-26T12:07:03.921797Z pos=converter.go:751 component=virt-launcher msg="Found nameservers in /etc/resolv.conf: \n`\u0000\n" level=info timestamp=2018-07-26T12:07:03.921869Z pos=converter.go:752 component=virt-launcher msg="Found search domains in /etc/resolv.conf: kubevirt-test-default.svc.cluster.local svc.cluster.local cluster.local" level=info timestamp=2018-07-26T12:07:03.926463Z pos=dhcp.go:62 component=virt-launcher msg="Starting SingleClientDHCPServer" Pod name: virt-launcher-testvmihr7zh-f7rwl Pod phase: Running goroutine 26 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc420278550, 0xc420104000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420245180, 0xc4200b6540, 0xc420628580, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c5a0, 0xc42000e0d8, 0x13, 0x131ed4d, 0x4, 0xc420069f18, 0x3, 0x3, 0xc4200bb080, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c5a0, 0xc42000e0d8, 0x13, 0xc42005df18, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4202ac080) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba840, 0xc4201ea8c0, 0xc420600d50, 0xc420600d60, 0xc420017e00, 0xc42029a120, 0x10f5e80, 0xc4200b6540, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 Pod name: virt-launcher-testvmiv7pcf-9fzs4 Pod phase: Running [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1061516] goroutine 11 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202e6390, 0xc4201aac80, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc42000c020, 0xc420650600, 0xc4206649e0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc4200f80c0, 0xc42000e108, 0x13, 0x131ed4d, 0x4, 0xc42006ef18, 0x3, 0x3, 0xc42035a2c0, 0xc4204db400, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc4200f80c0, 0xc42000e108, 0x13, 0xc42192c718, 0x3, 0x3, 0xc4204db900, 0xc4204db980, 0xc4204dba00) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4202bc000, 0xc4200ae8c0, 0xc421955ee0, 0xc421955ef0, 0xc421927d00, 0xc4200c0060, 0x10f5e80, 0xc420650600, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a Pod name: virt-launcher-testvmivmlkc-c9pnd Pod phase: Running goroutine 31 [running]: kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap.(*LibvirtDomainManager).SyncVMI(0xc4202680d0, 0xc420110000, 0x0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/manager.go:163 +0x506 kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server.(*Launcher).Sync(0xc420288040, 0xc42066a768, 0xc42069aba0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/pkg/virt-launcher/virtwrap/cmd-server/server.go:66 +0xe7 reflect.Value.call(0xc42008c060, 0xc42000e098, 0x13, 0x131ed4d, 0x4, 0xc42006df18, 0x3, 0x3, 0xc420088c40, 0x114acc0, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:447 +0x969 reflect.Value.Call(0xc42008c060, 0xc42000e098, 0x13, 0xc42055d718, 0x3, 0x3, 0x126c760, 0x12b7901, 0xc4206634c0) /gimme/.gimme/versions/go1.10.linux.amd64/src/reflect/value.go:308 +0xa4 net/rpc.(*service).call(0xc4200ba040, 0xc4200d4410, 0xc420666650, 0xc420666660, 0xc420254300, 0xc420231200, 0x10f5e80, 0xc42066a768, 0x16, 0x10f5f00, ...) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:384 +0x14e created by net/rpc.(*Server).ServeCodec /gimme/.gimme/versions/go1.10.linux.amd64/src/net/rpc/server.go:480 +0x43a virt-launcher exited with code 2 panic: test timed out after 1h30m0s goroutine 15802 [running]: testing.(*M).startAlarm.func1() /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1240 +0xfc created by time.goFunc /gimme/.gimme/versions/go1.10.linux.amd64/src/time/sleep.go:172 +0x44 goroutine 1 [chan receive, 90 minutes]: testing.(*T).Run(0xc4203d7e00, 0x139b503, 0x9, 0x142d848, 0x4801e6) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:825 +0x301 testing.runTests.func1(0xc4203d7d10) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1063 +0x64 testing.tRunner(0xc4203d7d10, 0xc4205c7df8) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0xd0 testing.runTests(0xc4203fec80, 0x1d2da50, 0x1, 0x1, 0x412009) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1061 +0x2c4 testing.(*M).Run(0xc420702c80, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:978 +0x171 main.main() _testmain.go:44 +0x151 goroutine 20 [chan receive]: kubevirt.io/kubevirt/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x1d59280) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/golang/glog/glog.go:879 +0x8b created by kubevirt.io/kubevirt/vendor/github.com/golang/glog.init.0 /root/go/src/kubevirt.io/kubevirt/vendor/github.com/golang/glog/glog.go:410 +0x203 goroutine 21 [syscall, 90 minutes]: os/signal.signal_recv(0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/sigqueue.go:139 +0xa6 os/signal.loop() /gimme/.gimme/versions/go1.10.linux.amd64/src/os/signal/signal_unix.go:22 +0x22 created by os/signal.init.0 /gimme/.gimme/versions/go1.10.linux.amd64/src/os/signal/signal_unix.go:28 +0x41 goroutine 10 [select]: kubevirt.io/kubevirt/tests.(*ObjectEventWatcher).Watch(0xc420ec5020, 0xc420a72240) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:288 +0x57e kubevirt.io/kubevirt/tests.(*ObjectEventWatcher).WaitFor(0xc420ec5020, 0x13972ab, 0x6, 0x11e03c0, 0x14ada60, 0x0) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:298 +0xba kubevirt.io/kubevirt/tests.waitForVMIStart(0x14b7060, 0xc420cc4000, 0x5a, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1035 +0x4ea kubevirt.io/kubevirt/tests.WaitForSuccessfulVMIStart(0x14b7060, 0xc420cc4000, 0xc420c65b00, 0xc420ec5118) /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1072 +0x43 kubevirt.io/kubevirt/tests_test.glob..func18.3(0xc420cc4000, 0x142d820, 0x1d77938) /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:86 +0x4d kubevirt.io/kubevirt/tests_test.glob..func18.5() /root/go/src/kubevirt.io/kubevirt/tests/vmi_networking_test.go:150 +0x745 kubevirt.io/kubevirt/tests.BeforeAll.func1() /root/go/src/kubevirt.io/kubevirt/tests/utils.go:1343 +0x3f kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc420845860, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:113 +0x9c kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).run(0xc420845860, 0xc4209213f0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64 +0x13e kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*SetupNode).Run(0xc4204d26a8, 0x14b3500, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/leafnodes/setup_nodes.go:15 +0x7f kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec.(*Spec).runSample(0xc420549b30, 0x0, 0x14b3500, 0xc4200fb4c0) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec/spec.go:181 +0x1f1 kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec.(*Spec).Run(0xc420549b30, 0x14b3500, 0xc4200fb4c0) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/spec/spec.go:138 +0xff kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpec(0xc420379e00, 0xc420549b30, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:200 +0x10d kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).runSpecs(0xc420379e00, 0x1) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:170 +0x329 kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run(0xc420379e00, 0xb) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:66 +0x11b kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/suite.(*Suite).Run(0xc4200f8af0, 0x7f9ebef20b50, 0xc4203d7e00, 0x139dae6, 0xb, 0xc4203fecc0, 0x2, 0x2, 0x14cfd80, 0xc4200fb4c0, ...) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/suite/suite.go:62 +0x27c kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo.RunSpecsWithCustomReporters(0x14b4560, 0xc4203d7e00, 0x139dae6, 0xb, 0xc4203feca0, 0x2, 0x2, 0x2) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:221 +0x258 kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo.RunSpecsWithDefaultAndCustomReporters(0x14b4560, 0xc4203d7e00, 0x139dae6, 0xb, 0xc4204c6920, 0x1, 0x1, 0x1) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:209 +0xab kubevirt.io/kubevirt/tests_test.TestTests(0xc4203d7e00) /root/go/src/kubevirt.io/kubevirt/tests/tests_suite_test.go:43 +0xaa testing.tRunner(0xc4203d7e00, 0x142d848) /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0xd0 created by testing.(*T).Run /gimme/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:824 +0x2e0 goroutine 11 [chan receive, 90 minutes]: kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).registerForInterrupts(0xc420379e00, 0xc4200de960) /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:223 +0xd1 created by kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner.(*SpecRunner).Run /root/go/src/kubevirt.io/kubevirt/vendor/github.com/onsi/ginkgo/internal/specrunner/spec_runner.go:60 +0x88 goroutine 12 [select, 90 minutes, locked to thread]: runtime.gopark(0x142fa20, 0x0, 0x1398025, 0x6, 0x18, 0x1) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/proc.go:291 +0x11a runtime.selectgo(0xc4208f7f50, 0xc4200dea20) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/select.go:392 +0xe50 runtime.ensureSigM.func1() /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/signal_unix.go:549 +0x1f4 runtime.goexit() /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/asm_amd64.s:2361 +0x1 goroutine 50 [IO wait]: internal/poll.runtime_pollWait(0x7f9ebefadf00, 0x72, 0xc420b11850) /gimme/.gimme/versions/go1.10.linux.amd64/src/runtime/netpoll.go:173 +0x57 internal/poll.(*pollDesc).wait(0xc4202d5718, 0x72, 0xffffffffffffff00, 0x14b5720, 0x1c447d0) /gimme/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9b internal/poll.(*pollDesc).waitRead(0xc4202d5718, 0xc420b68000, 0x8000, 0x8000) /gimme/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d internal/poll.(*FD).Read(0xc4202d5700, 0xc420b68000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_unix.go:157 +0x17d net.(*netFD).Read(0xc4202d5700, 0xc420b68000, 0x8000, 0x8000, 0x0, 0x8, 0x7ffb) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc4204d2aa8, 0xc420b68000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/net/net.go:176 +0x6a crypto/tls.(*block).readFromUntil(0xc420538960, 0x7f9ebef20c20, 0xc4204d2aa8, 0x5, 0xc4204d2aa8, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/crypto/tls/conn.go:493 +0x96 crypto/tls.(*Conn).readRecord(0xc4203de000, 0x142fb17, 0xc4203de120, 0x180) /gimme/.gimme/versions/go1.10.linux.amd64/src/crypto/tls/conn.go:595 +0xe0 crypto/tls.(*Conn).Read(0xc4203de000, 0xc4205a4000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /gimme/.gimme/versions/go1.10.linux.amd64/src/crypto/tls/conn.go:1156 +0x100 bufio.(*Reader).Read(0xc4206f6420, 0xc420980e38, 0x9, 0x9, 0xc420cb84a8, 0x10, 0xc0) /gimme/.gimme/versions/go1.10.linux.amd64/src/bufio/bufio.go:216 +0x238 io.ReadAtLeast(0x14b2300, 0xc4206f6420, 0xc420980e38, 0x9, 0x9, 0x9, 0xc4208beba0, 0x10, 0x400) /gimme/.gimme/versions/go1.10.linux.amd64/src/io/io.go:309 +0x86 io.ReadFull(0x14b2300, 0xc4206f6420, 0xc420980e38, 0x9, 0x9, 0x142fb68, 0xc420b11d10, 0x462d33) /gimme/.gimme/versions/go1.10.linux.amd64/src/io/io.go:327 +0x58 kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.readFrameHeader(0xc420980e38, 0x9, 0x9, 0x14b2300, 0xc4206f6420, 0x0, 0xc400000000, 0x7efbe0, 0xc420ef0168) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/frame.go:237 +0x7b kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc420980e00, 0xc4209493b0, 0x0, 0x0, 0x0) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/frame.go:492 +0xa4 kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*clientConnReadLoop).run(0xc420b11fb0, 0x142e7a0, 0xc4208f87b0) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:1428 +0x8e kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*ClientConn).readLoop(0xc4202b5520) /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:1354 +0x76 created by kubevirt.io/kubevirt/vendor/golang.org/x/net/http2.(*Transport).newClientConn /root/go/src/kubevirt.io/kubevirt/vendor/golang.org/x/net/http2/transport.go:579 +0x651 goroutine 588 [chan send, 87 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420a72c00) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 3892 [chan send, 68 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4208a7f50) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 1124 [chan send, 84 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4207c0780) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 9470 [chan send, 37 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420020420) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 11566 [chan send, 24 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42062c450) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 14233 [chan send, 9 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4206b1320) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 9997 [chan send, 34 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42041d5f0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 2116 [chan send, 78 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420a60c30) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 14757 [chan send, 6 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420021050) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 12682 [chan send, 18 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4207f9350) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 2702 [chan send, 74 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4207780c0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 4536 [chan send, 65 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42084b4d0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 3398 [chan send, 71 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42066c090) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 11037 [chan send, 27 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4208d3a10) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 13180 [chan send, 15 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4205f3830) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 13711 [chan send, 12 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42071eba0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 5023 [chan send, 62 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420539b00) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 15811 [chan send]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4209a31a0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 5639 [chan send, 59 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4208aa930) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 6155 [chan send, 55 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4208326f0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 6698 [chan send, 52 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420986720) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 7971 [chan send, 45 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420949aa0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 7195 [chan send, 49 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc42062dd10) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 12112 [chan send, 21 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420909b90) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 8457 [chan send, 42 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4206b10b0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 7697 [chan send, 39 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc420538f30) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 10550 [chan send, 30 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4207b7740) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 goroutine 15279 [chan send, 3 minutes]: kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc4207ad0b0) /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0x114 created by kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch.NewStreamWatcher /root/go/src/kubevirt.io/kubevirt/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:60 +0xa8 make: *** [functest] Error 2 + make cluster-down ./cluster/down.sh