amuck-landowner

Critical OpenVZ/Virtuozzo Updates [CVE-2016-5696]

Geek

Technolojesus
Verified Provider
* Rebase to RHEL6u8 kernel 2.6.32-642.el6
* kvm: reporting emulation failures to userspace. (CVE-2010-5313, CVE-2014-7842)
* File descriptors passed over unix sockets are not properly accounted. (CVE-2013-4312)
* x86: espfix not working for 32-bit KVM paravirt guests. (CVE-2014-8134)
* Buffer overflow with fraglist larger than MAX_SKB_FRAGS + 2 in virtio-net. (CVE-2015-5156)
* Mounting ext2 fs e2fsprogs/tests/f_orphan as ext4 crashes system. (CVE-2015-7509)
* MTU value is not validated in IPv6 stack causing packet loss. (CVE-2015-8215)
* Null pointer dereference when mounting ext4. (CVE-2015-8324)
* IPv6 connect causes DoS via NULL pointer dereference. (CVE-2015-8543)
* An attacker with knowledge of a connections client IP, server IP, and server port can abuse the challenge ACK mechanism and remotely inject or control a TCP stream contents in a connection between a Linux device and its connected client/server. (CVE-2016-5696)
* Numabalanced acquire cgroup_mutex for a long time. (PSBM-26897)
* CPU hotplug improvements (PSBM-46773).
* cpt: incorrect restore of SKB resulting in warnings in tcp_recvmsg(). (PSBM-39332, PSBM-46741)
* cpt: crash in nfs_fscache_dup_uniq_id on dump of container with NFS mounts inside. (PSBM-47216)
* cpt: crash in svc_age_temp_xprts_now() on stop of container with NFS mount. (PSBM-47515)
* cpt: crash on closing restored Unix sockets. (PSBM-47529)
* cpt: fixed restore of shared mounts. (PSBM-47639, OVZ-6779)
* cpt: crash after restore of Unix sockets with in-flight file descriptors. (PSBM-51254, PSBM-51351)
* ext4: crash in ext4_kill_sb() on mount of non-EXT4 filesystems (042stab114.2+ are affected) (PSBM-47782).
* swap: forbid exceeding ub swappages limit on global memory pressure. (PSBM-47836).
* 25-second delays can happen while logging in to systemd-based containers after container migration or host vzreboot. (PSBM-47889)
* CISCO UCS eNIC driver wraps untagged traffic into vlan0. (PSBM-51149)
* aacraid: Crash in aac_intr_normal(). (042stab112.15+ are affected) PSBM-49814)
* Fixed operation of iputils-ping-20150815 (debian-9) inside containers. (OVZ-6744)
* module: removed warning about waiting module removal. (OVZ-6748)
* fs.mqueue.* sysctls can be changed inside containers. (OVZ-6757)


See also
========
https://access.redhat.com/security/cve/CVE-2010-5313
https://access.redhat.com/security/cve/CVE-2014-7842
https://access.redhat.com/security/cve/CVE-2013-4312
https://access.redhat.com/security/cve/CVE-2014-8134
https://access.redhat.com/security/cve/CVE-2015-5156
https://access.redhat.com/security/cve/CVE-2015-7509
https://access.redhat.com/security/cve/CVE-2015-8215
https://access.redhat.com/security/cve/CVE-2015-8324
https://access.redhat.com/security/cve/CVE-2016-5696
https://help.virtuozzo.com/customer/portal/articles/2549710

Code:
[B]Testing Kernel w/ add'l details:
[/B](since 042stab117.6)

* The fix for CVE-2016-5696. 
An attacker with knowledge of a connections client IP, server IP, and server port can abuse the challenge ACK mechanism and remotely inject or control a TCP stream contents in a connection between a Linux device and its connected client/server. All 042stab kernels are affected. (PSBM-50954)
* Node can crash and reboot due to a crash in nfsd_inetaddr_event on container stop if the NFS server has been started on host. 042stab117.x kernels are affected. (PSBM-49999)
* cpt: Crash after restore of Unix sockets with in-flight file descriptors. The issue can result in node crash after suspended containers are resumed. All 042stab kernels are affected. (PSBM-51254, PSBM-51351)
* CISCO UCS eNIC driver wraps untagged traffic into vlan0. Kernels 042stab112.15 and newer are affected. (PSBM-51149)
* Memory corruption during dump of containers with shared tmpfs mounts can lead to node crash or soft lockup on any mount-related operation on node or inside containers. This is a special case of a more global issue with similar symptoms (PSBM-47639) that was fixed in kernel 042stab117.2. All 042stab kernels are affected. (OVZ-6779)
* aacraid: Crash in aac_intr_normal(). Kernels 042stab112.15 and newer are affected. PSBM-49814)
* cpt: Unable to restore container with a bridge inside. All 042stab117.x kernels up to 042stab117.10 are affected (PSBM-50893)
* Ext4: a bug in the extent tree height calculation code could lead to a file system corruption on particular workloads. 042stab117.x kernels up to 042stab117.8 are affected (PSBM-50339)

See also
========
[URL="https://access.redhat.com/security/cve/CVE-2016-5696"]https://access.redhat.com/security/cve/CVE-2016-5696[/URL]
[URL="https://help.virtuozzo.com/customer/portal/articles/2549710"]https://help.virtuozzo.com/customer/portal/articles/2549710[/URL]





Last Updated: Sep 05, 2016 02:47PM UTC





Symptoms



After rebooting the system to the kernel vzkernel-2.6.32-042stab112.15 or newer, hardware node and virtual environments might experience network connectivity issues. Also, this issue might happen after upgrading Cisco UCS firmware if your servers were already running on this kernel.


Issue occurs when following conditions are met:

  1. Cisco UCS hardware
  2. enic driver version 2.1.1.67 is in use by one of the adapters VMs are bridged toNote! This driver is shipped by default since vzkernel-2.6.32-042stab112.15

Known symptoms caused by this problem:


  • arp response received on node's network adapter is not being forwarded to the virtual machine:



    Tcpdump from vme461ed477.0:


    09:27:35.455384 00:1c:42:63:42:79 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.66.21 tell 192.168.66.241, length 46
    09:27:36.454298 00:1c:42:63:42:79 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.66.21 tell 192.168.66.241, length 46


  • Tcpdump from node's physical adapter:


    09:27:35.455397 00:1c:42:63:42:79 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 192.168.66.21 tell 192.168.66.241, length 46
    09:27:35.455480 00:25:b5:01:b1:3a > 00:1c:42:63:42:79, ethertype ARP (0x0806), length 56: Reply 192.168.66.21 is-at 00:25:b5:01:b1:3a, length 42


As a result Virtual machines with bridged interfaces are not accessible.


[*]DHCP replies/offers are not forwarded to guest VMs.

[*]Hardware node itself might experience connectivity issues - inability to reach neighbor hosts

[*]
Severe performance degradation for network related operations (VM migration, VM backup, etc...) or even client's traffic


Tcpdump from HW node where destination VM is stored shows problems with sending back ack's:


14:11:32.108127 IP 172.20.13.54.64000 > 172.20.13.20.47187: Flags [.], ack 202720, win 501, options [nop,nop,TS val 49374790 ecr 921631588,nop,nop,sack 2 {205616:236024}{262088:264984}], length 0
14:11:32.108170 IP 172.20.13.20.47187 > 172.20.13.54.64000: Flags [.], seq 236024:237472, ack 349, win 191, options [nop,nop,TS val 921631590 ecr 49374790], length 1448
14:11:32.108265 IP 172.20.13.54.64000 > 172.20.13.20.47187: Flags [.], ack 202720, win 501, options [nop,nop,TS val 49374790 ecr 921631588,nop,nop,sack 2 {205616:237472}{262088:264984}], length 0
14:11:32.108306 IP 172.20.13.20.47187 > 172.20.13.54.64000: Flags [.], seq 237472:238920, ack 349, win 191, options [nop,nop,TS val 921631591 ecr 49374790], length 1448
14:11:32.108392 IP 172.20.13.54.64000 > 172.20.13.20.47187: Flags [.], ack 202720, win 501, options [nop,nop,TS val 49374790 ecr 921631588,nop,nop,sack 2 {205616:238920}{262088:264984}], length 0
14:11:32.108434 IP 172.20.13.20.47187 > 172.20.13.54.64000: Flags [.], seq 238920:240368, ack 349, win 191, options [nop,nop,TS val 921631591 ecr 49374790], length 1448
14:11:32.108532 IP 172.20.13.54.64000 > 172.20.13.20.47187: Flags [.], ack 202720, win 501, options [nop,nop,TS val 49374791 ecr 921631588,nop,nop,sack 2 {205616:240368}{262088:264984}], length 0
14:11:32.108572 IP 172.20.13.20.47187 > 172.20.13.54.64000: Flags [.], seq 240368:241816, ack 349, win 191, options [nop,nop,TS val 921631591 ecr 49374791], length 1448



Cause



RedHat elaborates this issue in the following article in more details. Plain RHEL machines are affected as well.


In short - the root cause is a bug in Cisco UCS firmware which incorrectly induces the enic driver to mark untagged packets (no 802.1q PDU) as tagged with VID 0. issue is triggered by enic driver starting from version 2.1.1.67.


Workaround



Starting from kernel vzkernel-2.6.32-042stab117.14 there is a workaround shipped with the kernel. It's a load option for enic module which allows one to force the old behavior. This option is disabled by default and should be enabled manually only if you suffer from this issue.


How to turn on the workaround:


  1. Make sure you're either running on kernel vzkernel-2.6.32-042stab117.14 or newer:


    # uname -r
    2.6.32-042stab117.14



    OR it is set up to be used upon the next reboot:


    # rpm -q vzkernel | grep 117.14
    vzkernel-2.6.32-042stab117.14.x86_64
    # egrep 'title|default' /boot/grub/grub.conf
    default=0
    title Virtuozzo (2.6.32-042stab117.14)
    title Virtuozzo (2.6.32-042stab116.1)
    title Virtuozzo (2.6.32-042stab113.21)



    If correct kernel version is not yet installed - install pending updates with yum.

  2. Create modprobe config for enic module:


    # echo "options enic no_vlan0=1" > /etc/modprobe.d/enic.conf
    # cat /etc/modprobe.d/enic.conf
    options enic no_vlan0=1

  3. Reboot the server to apply new module option




  4. [Release notes] CU-2.6.32-042stab117.11 Virtuozzo Containers 4.7 Core Update








    Article ID: 129522, created on Aug 16, 2016, last review on Aug 17, 2016







    • Applies to:
    • Virtuozzo containers for Linux 4.7






     



    Issue date: 2016-08-17


    1. What's Included in This Update



    This update includes a new Virtuozzo Containers for Linux 4.7 kernel 2.6.32-042stab117.11 based on the Red Hat Enterprise Linux 6.8 kernel 2.6.32-642.el6. The new kernel introduces stability fixes.


    2. Bug Fixes

    • aacraid: Crash in aac_intr_normal(). (PSBM-49814)
    • Crash in nfsd_inetaddr_event in containers with NFS server inside. (PSBM-50257)
    • cpt: Unable to restore container with a bridge inside. (PSBM-50893)

    3. Obtaining the Update



    You can download and install the update using the vzup2date utility included in the Virtuozzo Containers for Linux 4.7 distribution.




WARNING: Once modprobe config for enic is created be careful while booting into kernels older than 2.6.32-042stab117.14 - enic module might fail to load due to "unknown module option" error because module shipped with the old kernel did not have this option. Therefore, if you're loading into an old kernel make sure to remove this modprobe config before the reboot.
 
Last edited by a moderator:
Top
amuck-landowner