IBM Storwize v7000 - MTU setting

MTU Settings

On the IBM Storwize v7000 I was told the default settings of the MTU was 9000 Jumbo Frames, while dealing with a problem of one of the VMware hosts dropping one of the iSCSI paths, Support suggested I check the MTU setting on the v7000.

After spending a fair bit of time trying to find out how to check the MTU setting and being previously told by IBM you can't change it I found the following.

Command to view settings

lsportip:

Use the lsportip command to list the Internet Small Computer Systems Interface (iSCSI) Internet Protocol (IP) addresses assigned for each port on each node in the clustered system.

Example command: svcinfo lsportip this will display the following.

I used the -delim to seperate it with :

full command:  svcinfo lsportip -delim :

id:node_id:node_name:IP_address:mask:gateway:IP_address_6:prefix_6:gateway_6:MAC:duplex:state:speed:failover:link_state

1:1:node1:::::::e4:1f:13:75:81:xx:Full:unconfigured:1Gb/s:no:active

1:1:node1:::::::e4:1f:13:75:81:xx:Full:unconfigured:1Gb/s:yes:active

2:1:node1:::::::e4:1f:13:75:81:xx:Full:unconfigured:1Gb/s:no:active

2:1:node1:::::::e4:1f:13:75:81:xx:Full:unconfigured:1Gb/s:yes:active

3:1:node1:10.10.11.1:255.255.255.0:10.10.11.254::::00:00:c9:d2:ed:xx:Full:configured:10Gb/s:no:active

3:1:node1:::::::00:00:c9:d2:ed:xx:Full:configured:10Gb/s:yes:active

4:1:node1:10.10.12.1:255.255.255.0:10.10.12.254::::00:00:c9:d2:ed:xx:Full:configured:10Gb/s:no:active

4:1:node1:::::::00:00:c9:d2:ed:xx:Full:configured:10Gb/s:yes:active

1:2:node2:::::::e4:1f:13:75:b0:xx:Full:unconfigured:1Gb/s:no:active

1:2:node2:::::::e4:1f:13:75:b0:xx:Full:unconfigured:1Gb/s:yes:active

2:2:node2:::::::e4:1f:13:75:b0:xx:Full:unconfigured:1Gb/s:no:active

2:2:node2:::::::e4:1f:13:75:b0:xx:Full:unconfigured:1Gb/s:yes:active

3:2:node2:10.10.11.2:255.255.255.0:10.10.11.254::::00:00:c9:d9:4a:xx:Full:configured:10Gb/s:no:active

3:2:node2:::::::00:00:c9:d9:4a:xx:Full:configured:10Gb/s:yes:active

4:2:node2:10..10.12..2:255.255.255.0:10.10.12.254::::00:00:c9:d9:4a:xx:Full:configured:10Gb/s:no:active

4:2:node2:::::::00:00:c9:d9:4a:xx:Full:configured:10Gb/s:yes:active

to see the NIC setting select the appropriate id this can be done by doing the following command:

full command: svcinfo lsportip 1 where 1 is the ethernet port id this will display something similar to the following.

id 1

node_id 2

node_name node2

IP_address

mask

gateway

IP_address_6

prefix_6

gateway_6

MAC e4:1f:13:75:xx:xx

duplex Full

state unconfigured

speed 1Gb/s

failover no

mtu 1500

link_state active

You can see that the MTU is 1500 on the 1Gbs interfaces it is also 1500 on the 10Gbs interfaces 

id 3

node_id 2

node_name node2

IP_address 10.10.11.2

mask 255.255.255.0

gateway 10.10.11.254

IP_address_6

prefix_6

gateway_6

MAC 00:00:c9:d9:xx:xx

duplex Full

state configured

speed 10Gb/s

failover no

mtu 1500

link_state active

To make changes to these setting use svctask cfgportip command.

You can find more details in the IBM v7000 InfoCenter and in the IBM Storwize v7000 Command-Line Interface Users Guide 6.4.1, If you google these you will find them on line.

IBM Storwize v7000 Software v7.2.0

Overview

As of the 8th October 2013 IBM have released details of the new software for the IBM Storwize family, In addition to the software there are new models appearing in the range as well.

The key point for the work I do is that the v7000 now suports IP Replication without employing FCIP (Fibre Channel) IP Routers (FCIP Gateways) such as the IBM SAN06B-R.

IBM are expecting to release this version of software on 29th November 2013.

Functionality

IBM Storwize Family Software V7.2 provides software-defined storage capabilities
across a variety of platforms including SAN Volume Controller, Storwize V7000
(Unified), Flex System V7000, Storwize V5000, and Storwize V3700 and V3500.
These offerings help customers reduce the complexities and cost of managing their
storage through:

• Centralizing management of storage volumes, enabling administrators to manage storage volumes from a single point

• Improving utilization of storage capacity with virtual volumes, enabling businesses to tap into previously unused disk capacity

• Avoiding downtime for backups, maintenance, and upgrades

• Performing data migration without disruption to applications

• Enabling all storage devices to be organized into storage pools from which virtual volumes, whether standard, compressed, or thin-provisioned, are created with the desired characteristics

• Delivering automation of storage management with the Virtual Storage Center, Tivoli® Storage Productivity Center (as applicable by platform), and Tivoli Storage FlashCopy Manager (as applicable by platform)

• Increasing the performance efficiency of storage pools that use SSDs with Easy Tier

• Restoring data access quickly with near and remote copy capabilities across Fibre Channel, Fibre Channel over Ethernet (FCoE), and IP networksIBM Europe, Middle East, and Africa Software Announcement ZP13-0608

More information can be found on the IBM website this is a link to the document.

VMware 5.0u1, IBM v7000 - iSCSI Performance

Overview

I am performance tuning the IBM hosts today as the iSCSI has not been performing as it should, it performs lots better after my first round of tuning, the first set of upgrade I have done which can be found here. To expand on this I have come across new drivers and firmware which I am benchmarking at the moment using the same tests that I have performed in my previous post.

Tests

Using IO meter with the workloads I have, which are available here, I have merged the 2 workloads together which can be downloaded from here. These can be used to test the SAN performance with various workloads and block sizes.

The results can be found at this page VMware 5.0u1, IBM v7000, iSCSI Performance - Results.

Configuration Changes

I have applied an IBM fix pack for VMware, this can only be added to a customised version of VMware for IBM servers. Also I have applied new system firmware and new Emulex drivers downloaded from VMware.

The updates were applied in the following order

IBM Fix Pack
Emulex Drivers
IBM System Firmware (x3550M4)

IBM Fix pack the official name of the patch is:

VMware vSphere ESXi 5.0.x and 5.1.x with IBM Customization Patch 7, this can be downloaded from the IBM website. This pack contains the following updates.

fupb                             5.1-7.450
concretejob                   500-2ACE19BUS
filetrans                        500-2ACE19BUS
fwupdate                      500-2ACE19BUS
immpassthru                 500-2ACE19BUS
pciinfo                          500-ESXI05ACN
hwckvm                        500-ESXI04ACN
soibms                           500-ESXI04ACN
Brcmfwup (facade)         500-2.00.20130305
Brcm                            500.2.0.3-000000
ilfu (facade)                   500-1.01.20130201
lsiprovider                     500.04.v0.38-0006
qlcnic                          5.0.750-1OEM.500.0.0.472560
scsi-qla2xxx                  934.5.6.0-1OEM.500.0.0.472560
emulex-cim-provider       3.8.15.1-01
net-be2net (5.0.x)           4.6.142.10-1OEM.500.0.0.472560
net-be2net (5.1.x)           4.6.142.10-1OEM.500.0.0.802205
be2iscsi                         4.6.142.2-1OEM.500.0.0.472629
ima-be2isci                    4.6.142.2-1OEM.500.0.0.472629
provider-cna                  1.4.7-469512
ibmpowercim                5.0-00ACN

Note: you need to be running an IBM customised install for VMware wiith the following versions, before applying the above patch.

    *VMware ESXi 5.0 build 474610
    *VMware ESXi 5.0u1 build 623860
    *VMware ESXi 5.0u2 build 914586
    *VMware ESXi 5.1 build 799733
    *VMware ESXi 5.1u1 build 1065491

Emulex Drivers

I also applied Emulex driver version be2net-4.6.247.7 this can be downloaded from VMware.

This a newer driver than the one included in the IBM Fix Pack.

Firmware changes.

These firmware updates can be download from IBM as a pack for VMware.

IMM2               3.10
IBM uEFI          1.50
IBM DSA          9.4.1

VMware Errors

If you experience any alerts or problems while doing these upgrades have a look at the page I have started, which has problems I have come accross and also how to fix. This can be found on my

VMware Errors page.

VMware Environment

I am running 2 sites with with VMware 5.0 Update 1 Enterprise Plus Edition, The details of the setup and problems I have over come are available at the link bellow.


VMware Environment

I have just started this page so I will update as I go.

Extend SAN Fabric for Replication

As an addition to the IBM Storwize V7000 replication I have created a new page about the extending of the Fabric which can be reached at the link bellow


Extend SAN Fabric for Replication


This is not my strong point but it might help others, as I find out more I will update this section.

IBM Storwize V7000 Replication

I have started to add a page about my experiences of replication with the IBM Storwize V7000, this will be about Global Mirror (Asynchronous Replication) as I haven't used Metro Mirror (Synchronous Replication). The link to this page is bellow.

This is a work in progress at the moment.


IBM Storwize V7000 Replication

IBM System x3550 M4

I have added a page about the IBM Servers I am using as VMware hosts this information can be found at the link bellow.

IBM System x3550 M4

IBM RackSwitch G8124E (Blade Network Technology)

I have added in some information about the IBM G8124E switches, most of the information comes from the IBM website, I am trying to keep it together so it easily accessible. more information about the switches can be found at the link bellow.

IBM RackSwitch G8124E

Emulex VFA III 10Gbs NIC's

In my other post I  talk about the Emulex VFA III 10Gbs Network Cards, so I thought I would put up more information about them and there capabilities. To see more please go to the following page

Emulex VFA III 10Gbs NIC's

VMWare and IBM Storwize V7000

I am currently working with IBM Storwize V7000, IBM x3550M4 VMware Hosts, IBM G8124E Blade Network Technology (BNT) Switches, Emulex VFA III 10Gbs Network Cards.

The storage is using 10Gbs ethernet and iSCSI Software initiator, VMware and IBM V7000 don't support the Hardware initiator.

If you would like to see more details of the set up and the hurdles I have come across please go to the following page.

VMware and IBM Storwize V7000

This is a work in progress at the moment and will get updated when I can.