Hyper-V on Nutanix

This article explains briefly about Hyper-V on Nutanix virtual computing platform. The below figure shows a 'N' node Nutanix architecture, where each node is an independent server unit with a hypervisor, processor, memory and local storage (combination of SSD and HDD). Along with this there is a CVM (controller VM) through which storage resources are accessed.

'N' node Hyper-V over Nutanix architecture

Local storage from all the nodes are combined together to form a virtualized and unified storage pool by the Nutanix Distributed File System (NDFS). This can be considered as an advanced NAS which delivers a unified storage pool across the cluster and having features like striping, replication, error detection, failover, automatic recovery etc. From this pool shared storage resources are presented to the hypervisors using containers.



NDFS - Logical view of storage pool and containers

As mentioned above, each Hyper-V node has its own CVM which is shown below.

Hyper-V node with a CVM

PRISM console

View of storage pool in PRISM

View of containers in PRISM

Containers are mounted to Hyper-V as SMB 3.0 based file shares where virtual machine files are stored. This is shown below.

SMB 3.0 share path to store VHDs and VM configuration files

Once share path is given properly as mentioned above, you can create virtual machines on your Hyper-V server.

Zoning and LUN masking

Zoning and LUN masking are used to isolate SAN traffic and to restrict access to storage devices. For example you might manage different zones separately for testing and production environment, so that they will not interfere. If you want to restrict certain hosts from accessing the storage devices then you have to setup zoning. This is generally done at FC switch level. Zoning are of two types : soft zoning and hard zoning.

Soft zoning is based on WWN name of the device and hard zoning is configured at FC switch port level. Soft zoning offers a greater range of flexibility. That means even if you move a device from one port to another on the FC switch, it will have the same access rights as the restriction is based on WWN name of the device. But the down side of this is using WWN spoofing you can gain access to zones that you aren't supposed to see. In case of hard zoning at switch port level, you will get a tighter access control but with less flexibility compared to soft zoning. Now if you change the device from one port to another as we done before, it won't be able to see its partner. In this case you can't spoof a physical port unless you are standing in the same room at the switch.

Once zoning is done you can further restrict access to SAN LUNs by using LUN masking. This will prevent certain devices from seeing specific LUNs hosted in the storage device. LUN masking is done at storage controller level or OS level of the storage device. It is recommended to use zoning and LUN masking together for securing storage traffic.

Installing Nutanix CE on nested ESXI 5.5 without using SSD

This is a great way to test/ learn Nutanix using the community edition (CE) which is totally free with Nutanix software based controller VM (CVM) and Acropolis hypervisor. This article will help you install Nutanix CE on a nested ESXI 5.5 without using SSD drives.

-Download links :

Version 2015.06.08: http://download.nutanix.com/ce/2015.06.08-beta/ce-2015.06.08-beta.img.gz

Version 2015.07.16: http://download.nutanix.com/ce/2015.07.16/ce-2015.07.16-beta.img.gz

-Extract the downloaded *.gz file and you will get a *.img file. Rename that file to ce-flat.vmdk.


Files

-Now create a descriptor file as given below :

# Disk DescriptorFile

version=4

encoding="UTF-8"

CID=a63adc2a

parentCID=ffffffff

isNativeSnapshot="no"

createType="vmfs"

# Extent description

RW 14540800 VMFS "ce-flat.vmdk"

# The Disk Data Base

#DDB

ddb.adapterType = "lsilogic"

ddb.geometry.cylinders = "905"

ddb.geometry.heads = "255"

ddb.geometry.sectors = "63"

ddb.longContentID = "2e046b033cecaa929776efb0a63adc2a"

ddb.uuid = "60 00 C2 9b 69 2f c9 76-74 c4 07 9e 10 87 3b f9"

ddb.virtualHWVersion = "10" 

-Copy the above content, paste it in a text file and save it as ce.vmdk.

-Create a VM on ESXI 5.5


VM properties

-RAM (min) 16 GB
-CPU (min) 4 (here I used 2 virtual sockets * 6 cores per socket)
-Network adapter E1000
-VM hardware version 8
-Guest OS selected as Linux CentOS 4/5/6 (64-bit)
-Initially the VM is created without a hard disk
-Enable CPU/ MMU virtualization

CPU/ MMU Virtualization

-Upload ce.vmdk and ce-flat.vmdk to the datastore

Datastore





-Add hard disk (ce.vmdk) to the VM

 
-Add 2 more hard disks to the VM of size 500 GB each. So now we have total 3 hard disks.

Hard disk 1 - ce.vmdk - SCSI (0:0)

Hard disk 2 - 500 GB - SCSI (0:1) (this drive will be emulated as SSD drive)

Hard disk 3 - 500 GB - SCSI (0:2)

-Download *.vmx file from the datastore and add line vhv.enable = "TRUE" to it and upload it back to the datastore. This is to enable nested hypervisor installation over ESXI 5.5.

-To emulate SSD, edit VM settings - options - advanced - general - configuration parameters - add row and add name scsi0:1.virtualSSD with value 1

-Power up the VM and Nutanix will start the installation wizard

-Login as root and password : nutanix/4u

-vi /home/install/phx_iso/phoenix/sysUtil.py

-Click insert and then edit SSD_rdIOPS_thresh = 50 and SSD_wrIOPS_thresh = 50

-And save it (press Esc button then :wq! to save the file) and reboot

-Login in with 'install' and hit enter (no password)

-Click proceed

-The installer will now verify all prerequisites and then you can enter the following details

-Host IP, subnet mask, gateway

-CVM IP, subnet mask, gateway

-Select create single-node cluster and enter DNS IP (8.8.8.8)

-Scroll down through the license agreement and select accept and click proceed

-Set promiscuous  mode in vSwitch to Accept

-Once the installation is complete, you can access the PRISM dashboard with the CVM IP address and NEXT account credentials



PRISM dashboard

Nutanix : a web-scale hyper converged infrastructure solution for enterprise datacenters

Nutanix is an industry leader in hyper converged infrastructure and software defined storage that is optimized for virtual workloads. You can even think it as a cluster-in-a-box solution with compute, storage and hypervisor consolidated together into a 1U or 2U enclosure. And its interesting that in a Nutanix architecture there is no RAID and no need of a SAN storage too. Storage is totally local and they are using direct attached local disks (combination of both SSD and SAS disks) for storing data.

How does it look like ?



Front and rear side of a Nutanix appliance (eg : NX-1000)

Each Nutanix box contains 4 independent nodes with are clustered together. This is shown in the figure below.

Nutanix box with 4 nodes

Each of these nodes operate independently, it has its own CPU, RAM, HDDs etc and all those nodes are clustered together. so each time you want to increase the compute and storage capacity, you can add more boxes (with 1, 2 or 4 nodes depending on the need) to the cluster. Detailed logical architecture of a single node is given below.

Single Nutanix node architecture

You can see in each node, there are SSDs as well as SAS HDDs for storage. And there is a controller VM, which is actually a virtual storage controller that runs on each and every node for improving scalability and resiliency while preventing performance bottlenecks. This controller VM is something like a VSA, but it does more than that. It is intelligent than a traditional VSA and is capable of  functionalities like automated tiering, data locality, de-duplication etc and much more. All storage controllers in a cluster communicates with each other forming Nutanix distributed file system. For each read, there are 3 levels of cache. An in-memory cache within each node, then a hot tier (SSDs) and finally cold tier (SAS HDDs). Here the hypervisor communicates with the controller VM just like it would communicate to a physical storage controller. When a write operation happens, the VM will contact the virtual storage controller and then it is written first to the local SSDs. To ensure the protection data is then replicated to multiple nodes in the cluster, so that it is always available even if a node fails. We can have RF2 (2 way replication) or RF3 (3 way replication). It is an auto healing system, so that if a node fails and if it has only one copy of data left, then the system will automatically identify it using map reduce or those type of analytics and then it will be replicated to another nodes.

If you want to add more nodes, all you have to do is to connect it to the network and power it on, the system will be auto discovered using a auto discovery protocol which runs on top of IPV6. So its very easy to add a new node to a cluster. You can dynamically expand your cluster resources by adding more boxes without shutting down the cluster. Rolling upgrades can be done with out downtime by updating the controller VM one by one in a cluster. Now, each node is clustered at the Nutanix architecture level and you can cluster it at the hypervisor level too (say, VMware ESXI cluster using vCenter server) and providing a highly available web-scale hyper converged solution.

DELL and Nutanix partnered together and they have introduced DELL XC Series appliances optimized for virtual workloads.

References :
www.nutanix.com

Shared Nothing Live Migration

Shared Nothing Live Migration is a Hyper-V 3.0 feature that help us live migrate virtual machines from one Hyper-V server to another without a shared storage and cluster membership.

 

Note : Failover clustering provides HA, but shared nothing live migration is a mobility solution that gives flexibility in a planned movement of VMs between Hyper-V hosts without downtime.

Hyper-V settings for Live migrations 

As a prerequisite for this, we need to standardize network connectivity on Hyper-V host machines (eg : vSwitches should have same names for VM traffic, iSCSI traffic etc). And for this shared nothing Live Migration traffic we can use a separate VLAN (say, VLAN 90) so that it won’t affect local LAN.

Separate VLAN for live migration traffic

Also we need to configure constrained delegation on Hyper-V servers to use Kerberos authentication protocol when managing the servers remotely. This is shown below.

Use Kerberos



Delegation to specified services

 

Best practice recommendations for iSCSI network adapters

Best practice recommendations for iSCSI network adapters

Note : all those settings are enabled by default, we need to disable it as best practice on all iSCSI NICs

Also, if your network/ network devices supports jumbo frames, then that should be enabled too on the network adapters.

Recommended BIOS settings for DELL PowerEdge 12G servers

BIOS settings for optimal performance

Memory mode : Optimizer
Node interleave : Disabled
Logical processor : Enabled
QPI frequency : Maximum frequency
CPU power management : Maximum performance
Turbo boost : Enabled
C1E : Disabled
C-states : Disabled
Memory frequency : Maximum performance