Saturday, December 5, 2015

Assign VLAN Tag for Nutanix Hardware using HyperV

You can use the following commands to tag VLAN for IPMI Traffic from HyperV host on Nutanix platform.

To set VLAN ID

ipmiutil -e -I "IP Address" -G "Gateway" -S "SubnetMask"  -h "VLAN ID"

To Remove VLAN ID

ipmiutil -e I "IP Address" -G "Gateway" -S "SubnetMask"  -h 4099

Note: Remove the quotes while entering command

Acropolis Hypervisor Configuration

The following commands help to configure Factory (Nutanix) loaded hypervisor called as Acropolis without reinstalling using Foundation process:

To configure IP Address:  root@ahv# vi /etc/sysconfig/network-scripts/ifcfg-br0 
BOOTPROTO="none“  (set as “static”)

DNS Entry:  root@ahv# vi /etc/resolv.conf

To List Ethernet Interfaces:  manage_ovs show_interfaces 

To verify the interfaces are assigned to br0:  manage_ovs --bridge_name br0 show_uplinks 
To Add 1gb Interface into br0:  manage_ovs --bond_name bond1 --interfaces 1g update_uplinks 

To Configure IP Address in CVM: ssh to CVM (ssh nutanix@ and use sudo vi /etc/sysconfig/network-scripts/ifcfg-eth0, finally reboot the CVM

To Create cluster:  1. cluster -s <cvm_ipaddress> (- - redundancy_factor=3 -> Only if RF3) create 
2.      cluster start

eg: cluster -s,, create

To Set Cluster Name:  # ncli cluster edit-params new-name=cluster_name 
To Add DNS Server:  # ncli cluster add-to-name-servers servers="dns_server

To Add NTP Server: # ncli cluster add-to-ntp-servers servers="ntp_server

To Add External IP Address: # ncli cluster set-external-ip-address external-ip-address=‘Cluster IP Address”

To change CVM VLAN: change_cvm_vlan (SSH to CVM and run this command)

To Set Hostname: # vi /etc/sysconfig/network 
and /etc/hosts (Edit HOSTNAME entry)

To Configure IPMI for AHV:

root@ahv# ipmitool -U ADMIN -P ADMIN lan set 1 ipsrc static
root@ahv# ipmitool -U ADMIN -P ADMIN lan set 1 ipaddr mgmt_interface_ip_addr 
root@ahv# ipmitool -U ADMIN -P ADMIN lan set 1 netmask mgmt_interface_subnet_addr 
root@ahv# ipmitool -U ADMIN -P ADMIN lan set 1 defgw ipaddr mgmt_interface_gateway 
ipmitool lan set 1 plan id lan_id_number

To set VLAN ID for Acropolis Hypervisor (AHV)

ovs-vsctl set port br0 tag=host_vlan_tag 
Confirm Tagging by entering:

ovs-vsctl list port br0 

Cluster Initialization:
  1. http://cvm_ipv6_addr:2100/cluster_init.html.

    . http://NTNX-12AM3K520060-A-CVM.local.:2100/cluster_init.html

Wednesday, May 20, 2015

My Nutanix Notes

IPMI Tool 
~ # /ipmitool lan print1 (To Print the IP Address Info. command must run thru ESXi console)
~ # /ipmitool lan set 1 vlan id 333 (To set the VLAN ID)
~ # /ipmitool lan set 1 vlan id off (To remove the VLAN or disable)
Set time zone for CVMs:

SSH to any CVM

# ncli cluster set-timezone timezone=America\New_York

Test E-Mail config:

#ncli cluster send-test-email recipient=“” subject=“test”

To Stop/Start PRISM Service:
curl http://localhost/h/exit - Prism restart from zeusleader node

curl http://localhost/prism/leader   —> Prism start

Commands to verify the Block-Awareness

  • ncli cluster get-fault-tolerance-state
  • ncli cluster get-domain-fault-tolerance-status  type=rackable_unit
  • ncli cluster get-domain-fault-tolerance-status type=node
  • ncli rack ls - how many blocks are in the cluster

Expand Cluster thru nCLI
# ncli cluster discover-nodes (First Discover the nodes)

nutanix@NTNX-14SM36410061-A-CVM:$ ncli cluster discover-nodes
    Cluster Id                :
    Controller Vm Address     :
    Hypervisor Address        :
    Ip                        : fe80::20c:29ff:fe00:a1e4%eth0
    Ipmi Address              :
    Model String              : NX-3060
    Node Position             : B
    Node Uuid                 : 942205fd-f14c-4540-846b-b80b30eecb40
    Rackable Unit Model       : NX-3060
    Rackable Unit Serial      : 14SM36430046
    Sed Node                  : false

    Svm Id                    :
ncli cluster add-node node-uuid=942205fd-f14c-4540-846b-b80b30eecb40 (Add nodes into cluster using Node UUID)

Shutdown a Nutanix Block/Nodes:

Step 1: Shutdown all VMs, not CVMs

Step 2: SSH to any CVM

Step 3: Issue “cluster stop”, make sure cluster stopped and no errors reported

Step 4: Issue “cluster status”, all processes must be DOWN state

Step 5: Shutdown CVMs one by one . Use “shutdown -P now” command

Step 6: Shutdown ESXi/Hyper-V/KVM Nodes

Start or Power On a Nutanix Block:

Step 1: Power On a node at a time

Step 2: Wait for 10-20 seconds to power on every nodes

Step 3: Make sure all CVMs are powered on

Step 4: SSH to any CVM

Step 5: Type “cluster status” if all process are showing down

Step 6: Type “cluster start”

Step 7: Type “cluster status” to see all process are showing Up

Step 8: Make sure you’re able to access Prism Web Interface

Step 9: Make sure NFS mounts are accessible

Thursday, February 26, 2015

Nutanix Metro Availability - Setup Guide

Nutanix Metro Availability (MA):

This new feature introduced on NOS 4.1 (Nutanix Operating System) which is really useful for the customers who is looking inexpensive DR solution especially VDI use case.  This feature allows replicate data synchronously between two sites.

Lets see requirements for implementing this feature:

NOS Version:

NOS 4.1 or later

1.     <=5ms RTT latency
2.     <400 between="" both="" distance="" kms="" o:p="" sites="">
3.     Bandwidth depends on “data change rate”
4.     Redundant  physical network connection


1.     2 Nutanix clusters at each site
2.     mix/match models (eg: NX3000 at primary site and NX6000 at DR Site)


1.     ESXi 5.x supported
2.     KVM/Hyper-V not now, maybe later

Nutanix License:

1.     Ultimate

MA supports compression-over-the-wire which reduces bandwidth utilization significantly during replication.

Though it is only few clicks to configure the Metro Availability feature, but it requires few other configurations. Technically, the container is getting replicated to the DR site. So, the container name must be same for both sites (reason behind this the same datastore name maintained in VM configuration file to avoid re-registering the VMs in the DR site).

Step 1:

Create the Container on the both sites using the same name.  Select the “Storage” from Prism UI, click the Container option.  The Data de-duplication and compression supported on the containers for MA.

Step 2:

Create the Remote Site, Select the “Data Protection” from the Prism UI, and then click “Remote Site”

Enter the name for Remote site and enter IP Address of CVMs of the target site.

Select the Use SSH Tunnel if you are planning to use the SSH tunnel between both sites.  Select the “Disaster Recovery”  without fail. VStore name mapping not required.

Repeat the same setup on the DR site, but enter IP address of Primary site CVMs. After the configuration, test the connection between both sites using Test Connection option

Step 3:

Time to configure the Metro Availability (MA), select the Data Protection from Prism UI, click the “Protection Domain” and select “Metro Availability”

Enter name for MA, press Next.

Select the container to replication,  you cannot select multiple containers.

Then select the remote site from compatible Remote site (Upper window)

In the VM Availability screen, select the method to disable the MA either manually or automatically after specified time when a network or any other problem interrupts the connection between both sites.

Press Next, in the “Review” screen verify the all the configuration and click the “Create” button.

Step 4:

Activate the Metro Availability , click the “Metro Availability” in the “Protection Domain”, select the target container, Click the “Activate” button. Make sure that target container is empty, else MA will delete the data from the container.

Tuesday, February 24, 2015

vSphere Network Port Diagram

VMware has published a detailed network port diagram and list of TCP/UDP ports used in various products. It is available as PDF file for anyone to download.

Thursday, September 11, 2014

Set VLAN ID to IPMI Address in Nutanix

The IPMI is remote management interface like as HP iLO, DELL DRAC. During the Nutanix cluster deployment, it is not possible to mention the VLAN ID of the IPMI's IP Address. This need to be entered thru command line via ESXi host or Windows Hyper-V. This article talks about ESXi host only.

Login to ESXi thru SSH client. Run the following command to enter the VLAN ID (we assume that you already configured IP Address)

~ # /ipmitool lan set 1 vlan id 333

~ # /ipmitool lan print1
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD 
Auth Type Enable        : Callback : MD2 MD5 PASSWORD 
                        : User     : MD2 MD5 PASSWORD 
                        : Operator : MD2 MD5 PASSWORD 
                        : Admin    : MD2 MD5 PASSWORD 
                        : OEM      : MD2 MD5 PASSWORD 
IP Address Source       : Static Address
IP Address              : xx.xx.xx.xx
Subnet Mask             :
MAC Address             : xx:xx:xx:xx:xx:
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Default Gateway IP      : xx.xx.xx.xx
Default Gateway MAC     : 00:00:00:00:00:00
Backup Gateway IP       :
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : 333
802.1q VLAN Priority    : 0
RMCP+ Cipher Suites     : 1,2,3,6,7,8,11,12
Cipher Suite Priv Max   : aaaaXXaaaXXaaXX
                        :     X=Cipher Suite Unused
                        :     c=CALLBACK
                        :     u=USER
                        :     o=OPERATOR
                        :     a=ADMIN
                        :     O=OEM

To remove the VLAN ID:

~#ipmitool lan set 1 vlan id off

~ # /ipmitool lan print1
Set in Progress         : Set Complete
Auth Type Support       : NONE MD2 MD5 PASSWORD 
Auth Type Enable        : Callback : MD2 MD5 PASSWORD 
                        : User     : MD2 MD5 PASSWORD 
                        : Operator : MD2 MD5 PASSWORD 
                        : Admin    : MD2 MD5 PASSWORD 
                        : OEM      : MD2 MD5 PASSWORD 
IP Address Source       : Static Address
IP Address              : xx.xx.xx.xx
Subnet Mask             :
MAC Address             : xx:xx:xx:xx:xx
SNMP Community String   : public
IP Header               : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00
BMC ARP Control         : ARP Responses Enabled, Gratuitous ARP Disabled
Default Gateway IP      : xx.xx.xx.xx
Default Gateway MAC     : 
Backup Gateway IP       :
Backup Gateway MAC      : 00:00:00:00:00:00
802.1q VLAN ID          : Disabled
802.1q VLAN Priority    : 0

Monday, May 14, 2012

HP ProLiant DL360p Gen8

I got 3 new toys in my server build room for a virtualization project. They are HP DL360p Gen8 (G8 became Gen8) servers. It is a 1U with elegant view. This server powered up with 2xIntel Xeon process and can be supported up to 768GB RAM (DDR3 with Load Reduced DIMM). HP introduced iLO ver4.0 in this servers and FlexibleLOM (4x 1Gb ports and optional 10 Gb port).  It has new slide-out SID (System Insight Display), the most useful stuff to find out the faulty components immediately. I watch this display screen often to see warning/critical errors of a server.
HP DL360p Gen8 introduced “Intelligent Provisioning” where you can configure and install the OS and perform maintenance activities such as Array Configuration utility, Firmware Update, iLO configuration and other tasks.
This feature really eliminates the SmartCD requirement.


Tasks listed under Perform Maintenance:

The internal storage with Hot Plug SFF SAS/SATA can be extended upto 8TB and 12TB with LFF SAS/SATA
No front Video interface, it requires Front Video adapter kit.. Only Rear video.
1 internal SD Slot
1 internal USB port and 6 external (4 Rear, 2 Front)
New version iLO4.0 and introduces iLO 4 management processor. ”Active Health System” is essential component of the HP iLO management engine and continuously monitoring over 1600 parameters. This log can be downloaded and forward to HP Support team for further investigations.


DL360p Specs:

Memory Configuration Tool:

128 GB
Qty 8 x HP Option Part# 672631-B21 HP 16GB 2Rx4 PC3-12800R-11 Kit (HP Module Part# 672612-081) STD

Thursday, December 15, 2011

How to reset HP iLO Password (DL 585 G2)

Today, I got into trouble with a HP Server. I need to hold remote console of the HP DL 585 G2 server. When I tried to access the console, prompted me to enter "Administrator" credential...Oops... no idea what is the password. I have not been known about this while KT from previous administrator. Now, I have to find a way to reset this password and access the server remotely. While searching, I ended up with 3 different ways to reset the password. All the 3 ways given below, you choose which one good for you:

Resetting using SmartStart CD
1.        Boot the server using HP SmartStart CD
2.       Press F8 to goto iLO Configuration console
3.       Select the HP iLO setting
4.      Goto User setting
5.       Select the “Administrator” and change the password or Create new user with administrator privileges

Resetting using Security Override Switch
1.        Look system board diagram server top bonnet
2.       Look for the system management switch (mostly no:11 on DL 585 G2 servers)
3.       Turn off the iLO Security switch (SW5)
iLO security override
The override allows the administrator full access to the iLO processor.
  • Open (off) = Normal
  • Closed (on) = Override
Redundant bootblock select
Forces booting from the redundant bootblock
  • Open (off) = Normal
  • Closed (on) = Redundant bootblock (ROM)

Resetting the Password using HPONCFG command:
                Using the command needs some prerequisites. You can use this command on the same server where you need to reset the password. The server can be loaded with either Windows or Linux OS. You have to run the SmartUpdate CD again to choose and install the HP iLO configuration utility.
1.        Install the SNMP (pre-requisite, if you already have installed this, ignore)
2.       Install  the HP Insight Management
3.       Install the HP iLO configuration utility/driver as highlight above picture
4.      Create a XML file as follow and save it as “Administrator_reset_pw.xml”
(Unable to publish XML codes here, thatswhat, screeshot)

5.       Save this file where the iLO Configuration utility. The common location of the utility is “C:\Program Files\HP\HPONCFG”
6.      Then, run the following command in RUN command
                                                               i.      HPONCFG.exe /f Administrator_reset_pw.xml /l log.txt >output.txt
7.       Open the iLO page in IE or Firefox. You should able to login using “Administrator”

Wednesday, April 20, 2011

Office 365 - Microsoft offers MS-Office on Cloud

Finally, Microsoft put their Ms-Office suite on the cloud and started as cloud service ( ).They hosted important application such as Email (Outlook) and Calendar. Office 365 offers 25GB mailbox for each user and they can send email messages upto 25 MB.  Other important MS-Office application like MS-Word, Excel, Powerpoint and OneNote are also available under Office365. Users can view,edit and simultaneously access and edit the same document with other users. The same feature available in Google docs and referred as “Real-time Collaboration”( ).  I generally use the Google docs in home computer for documentation work. I noticed that Google docs automatically saves the document and maintain version. Microsoft offers the office365 as Beta version only. I have not yet used office365 and unable to conclude the feature comparison between Office 365 and Google docs.

Amazon Cloud Drive

Amazon offers free 5GB cloud drive at free of cost . You can save your documents, music and video files. Addition to this, you can directly play your music files without downloading. This feature is currently available only for US customer. Amazon may start this service for world wide.  If you are willing to use more than 5 GB, Amazon will charge you as per storage choice. I remember few years back I used to keep my documents in  What is difference between them and Amazon? Just few features, more memory, cloud (music) player and you can directly purchase the music albums from Amazon store and copy them to cloud drive.  Some old wine in new sexy bottle. That’s all!