Hyper-V Server, Guest OS Ubuntu, Network Problems

[HalfEatenPie]

Howdy folks,

I've recently been having problems with my Ubuntu VPS on my Windows server.  I have access to both the Host node and the guest node (since this is a multi-use server and I wanted to use both Windows and Linux services), and I guess I thought I set it up properly but it seems there's a networking problem on the server.

There's periods when the Ubuntu VM has major packet loss while the actual host server is mostly fine.  Sometimes when I'm rsyncing large files (12 GB worth of files), the network gets dropped on the entire Ubuntu VM and I get the error "ping: sendmsg: No buffer space available".  This was reproducible whenever I'd rsync out larger files. 

Basic research shows this is usually due to an incompatibility of the on-board NIC on the Server (of course this is assuming Ubuntu was installed on the base hardware, not within a VM).  I'm assuming this is a problem with my bridge setup. 

Anyways, this is a Hyper-V Generation 1 VM.  The bridge is basically all default variables requested when setting it up through the Wizard available on the Windows Server 2012 R2 (Datacenter Edition).  Upon further Google Searching, I've found that we have to increase the memory allocation for the packets, and was suggested I add the following lines into sysctl:

net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
In addition, the following command was executed: echo 83886080 > /proc/sys/net/core/wmem_max

Anyways, even with these changes I'm still having some problems.  Anyone know of a good way to start troubleshooting this issue?  

Thanks!

 

[wlanboy]

Do you have installed the Hyper-V Integration Services on the guest?

And added the lines to /etc/initramfs-tools/modules ?

hv_vmbus
hv_storvsc
hv_blkvsc
hv_netvsc

 

[atho]

Was it a "clone" of another VM or a fresh install? if it is a clone i've seen that happen when you have two MAC's that are the same.

 

[HalfEatenPie]

Was it a "clone" of another VM or a fresh install? if it is a clone i've seen that happen when you have two MAC's that are the same.

Fresh install.

Do you have installed the Hyper-V Integration Services on the guest?

And added the lines to /etc/initramfs-tools/modules ?

hv_vmbus
hv_storvsc
hv_blkvsc
hv_netvsc

Yep.  Still not doing good though :/

 

[mitgib]

What kernel are you running? I had this problem before while running an OpenVZ kernel

 

[HalfEatenPie]

What kernel are you running? I had this problem before while running an OpenVZ kernel

root@sg02:~# uname -a
Linux sg02 3.16.0-48-generic #64~14.04.1-Ubuntu SMP Thu Aug 20 23:03:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Ubuntu 14.04 64-bit server default. 

 

[mitgib]

If you ifdown eth0;ifup eth0 does the issue resolve?

 

[Hxxx]

If you are referring to Microsoft Hyper V, man that is garbage if you are going to run Linux. Issues here and there. At least that's my experience. If is possible to replace that HyperV with a KVM, go for it.

 

[HalfEatenPie]

If you ifdown eth0;ifup eth0 does the issue resolve?

Nope that doesn't resolve it.  I've tried that but it didn't work.

If you are referring to Microsoft Hyper V, man that is garbage if you are going to run Linux. Issues here and there. At least that's my experience. If is possible to replace that HyperV with a KVM, go for it.

There's a purpose I went with Microsoft Hyper-V.  I needed a Windows Server + a Linux Server.  Therefore I decided to go with Windows Host and Linux Guest.  It's not just because I wanted to run Linux VMs I used Microsoft Hyper-V...  The host OS has to be Windows.  Trust me.  If I could have gone with Linux as the base OS I would have since that's what I'm more used to.  Unfortunately the conditions required Windows server to be the host OS.  

 

[rmlhhd]

I'll see if I can reproduce this issue tomorrow on both 2008 and 2012 Hyper-V environments.

Any reason why your using Gen 1?

 

[HalfEatenPie]

If you are referring to Microsoft Hyper V, man that is garbage if you are going to run Linux. Issues here and there. At least that's my experience. If is possible to replace that HyperV with a KVM, go for it.

This... Doesn't really help me.  I appreciate it but I'm in a situation where I need both Linux and Windows on a single dedicated machine.  Since I already had the Windows Server 2012 R2 license I decided to use that as the host node (since the Windows node is the more important part anyways).  It's something unavoidable and something I need in this configuration.

I'll see if I can reproduce this issue tomorrow on both 2008 and 2012 Hyper-V environments.

Any reason why your using Gen 1?

Mostly since it was easier for me to get setup.  I've currently moved everything off of the server so I guess I might as well try it again with Gen 2 and see if I'm still getting the issues.  

 

[maaarkus]

I have got the same issue. Ubuntu 14.04.3 LTS on a Hyper-V Server 2012 R2. The same issue on my test environment as well as on the production machine. The two are with completely diffent hardware but an identical copy of the VHD. In total three different NICs.
Today I checked using different Kernels:

The issue appears when loading: 3.16.0-48-generic

It DOESNT seem to appear when loading: 3.16.0-46-generic or 3.16.0-45-generic

 

[noen]

Do your Ubuntu VM have 4GB+ memory?

Can it be the same bug,in a newer kernel? -->https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1338185  

Try setting the ram to 2GB, and see..

 

[HalfEatenPie]

Do your Ubuntu VM have 4GB+ memory?

Can it be the same bug,in a newer kernel? -->https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1338185  

Try setting the ram to 2GB, and see..

Thanks for that link.  However this has always been 14.04 64-bit installation on top of Hyper-V, not 32-bit as described in the issue.  Another detail that rules out this bug is that this installation was made in August, 2015, many months after they've patched and released a fix.  Thanks though.

I guess I should update this thread.  I went with a reinstallation of Ubuntu 14.04 64-bit on a Generation 2 (instead of Generation 1 as stated before).  I want to say now it's much better and stable, however I haven't tested it long enough to confirm it.  All I'll say now is that it seems to be holding up now and this networking issue isn't a problem anymore.  I'll update this later if I remember after I do some more extensive testing and data collection.  

 

[pcan]

Glad to see that you solved the issue. Hyper-V is picky on the NICs; I had to exchange cards and update firmware several times on production servers to solve all kinds of strange networking behaviour. My Windows developement servers are mostly old hardware with random NICs, so I install VmWare workstation (or even VmWare player) on the host Windows server OS. It works perfectly and I can easily move VM to and from ESXi. I install the Hyper-V role on brand new hardware only, and a carefully selected configuration (Intel server NICs seems to be the way to go for Hyper-V).

 

[HalfEatenPie]

Glad to see that you solved the issue. Hyper-V is picky on the NICs; I had to exchange cards and update firmware several times on production servers to solve all kinds of strange networking behaviour. My Windows developement servers are mostly old hardware with random NICs, so I install VmWare workstation (or even VmWare player) on the host Windows server OS. It works perfectly and I can easily move VM to and from ESXi. I install the Hyper-V role on brand new hardware only, and a carefully selected configuration (Intel server NICs seems to be the way to go for Hyper-V).

It's still a bit finicky, but it's working much better now (I think it's because the network is so oversold and crap).  Good thing I'm moving out of this DC. 

Thanks a ton for that information.