I came across a strange issue where 2 blades was unable to ping I could get 2 ping and then Request time outs
Problem : 2 Servers( in the same chassis server 3,4 ) unable to ping their gateway. Ping drops are after 1-2 packets. We cannot ping/ssh to them from outside and they cannot be added to vCenter. Other blades in the same chassis are working ( 1,2)
Steps Taken :
- Confirmed that we do not have any IP conflicts
- Made sure that we do not use the same subnet anywhere else in the network
- Also checked and confirmed that the Mgmt(CIMC) subnet is different
- Started a ping from inside a Host ( Server-3 ) to check if you are able to get to the gateway
- Checked vobd logs to find if you can find any entrie for duplicate ip/mac
- Checked the Mgmt Interface : vmk0
- It is connected to Port Group : Management Network on VLAN ID, Check if we are using the right vlan
- Found that the MAC Address of the vmk0 is same as vnic (vmnic0) : This is a known issue VMware KB article here. This is the vnic mac address of the service profile which is attached to the esx blade.
- Deleted and re-created the vmk0 interface
- Confirmed that the MAC Address has changed
- Started a continuous ping which work fine now
- Host can now be added to the vCenter
So let me go through of the process of how we found the duplicate mac address.
The vnic mac address was taken by vmknic as you might know vmknic mad starts with 00:50:56 but in my case it was same as the mac address of vnic from the service profile. Continue reading “UCS Blade unable to ping or connect to vCenter”
This week I have seen an interesting issue on Windows Servers 2008 /2012 . Just to let you know these are VM’s and I was unable to connect to these server. So I login on vCenter to check the windows host and found the server was having network error.
After checking the event viewer i found this error
The system detected an address conflict for IP address 0.0.0.0 with the system having network hardware address xx-xx-xx-CE-44-3F. Network operations on this system may be disrupted as a result. Time stamp 28/10/2014 hh:mm:ss.
Reboot the server and it got the network back. I have to mention giving ipconfig was showing the correct IP address even when it was having the network error.
Further investigation reviled few interesting and worrying facts, apparently this issue is only affecting Windows Vista and above, we also found that it is a known issue, However it will arise only when a windows server is rebooted.
The root cause is part of the detection flows defined by the RFC 5227 (IPv4 Address Conflict Detection).
The error is caused by the method used by Windows to detect an address conflict (http://tools.ietf.org/html/rfc5227#section-2.1.1) and one of the packets used by the cisco security feature called “ip device tracking”, used for the NAC Layer 2 validation.
Unfortunately the IOS version (15.2) used by the most of the Cisco core switches, enables this feature by default and there is no way to disable it. The only options are: downgrade the IOS or tuning some parameters.
a. downgrade the IOS
b. modify a parameter of Cisco ‘IP device tracking’ feature in order to potentially solve this issue.
On each interfaces
ip device tracking maximum 0
I am writing this post to show the process on how to provision LUN’s from Cisco UCS Invicta to ESX host in 3 steps hope you enjoy the post.
- Create a default LUN and add MAP ID 0 ‘zero’ when ever you are provisioning a new ESX rule of thumb for Invicta is add a default lun lun0_ default with MAP ID 0
- Keep all the LUN’s with the same MAP id on each host when provisioning them. ( Caution : If you do not give the same MAP ID for a LUN which is already provisioned and the lun has VM running. The LUN comes up as a new lun to be added to VMware so the Administrator if he is not aware of this issue may end up formating it assuming its a new LUN to provision using VMFS 5.
- You need to remember that you can right click and drag and drop luns to map them.
In our environment I have created a default lun with 10 GB and never provision this on VMware, however always add this default lun first with MAP ID 0 to any ESX host you want to provision.
Login to the invicta with admin or superuser account.
Step 1: Creating a LUN
Click on LUN Configuration and LUNs and Click on Create LUN
As I have mentioned in the Notes at the top that we need to first create a default lun which is what I am creating below. just fill in the details I call it lun0_default
The size of this lun can even be 1 GB as you are not going to use this lun . I have put 10 GB a I have 65 TB of thin provisioned disk and I am not going to provision this lun its just what is recommended by Cisco documentation to have a default lun 0 provisioned on each host with MAP ID 0
I have created another LUN of 1 TB calling it lun1 as shown below
Step 2. Adding a Host Initiator Group
Click on LUN Configuration and Initiator Groups and Click on Create New Group
Continue reading “Cisco UCS Invicta Storage – How to Provision a LUN to ESX Host”