Overview
The extension of a SAN Fabric over a Ethernet network can be done using FCIP Gateways (Fibre Channel to IP Router) such as the IBM SAN06 B-R that I am using, I have 2 of the FCIP Gateways at each location for redundancy purposes.
This part is not my strong point I had a 3rd party set this up, but I have had to troubleshoot this set up and can share what I have come across.
 |
| Physical connections |
Configuration
- I have the base licenses no advanced features
- I have 2 zones configured which are extended across the sites.
- Each SAN06B-R has 2 Gigabit Ethernet ports connected to the Cisco 2960G
- The ports have the MTU set to 1400 to compensate for the FCIP header and the link can only accept 1500 MTU frames
- Compression has been enabled to level 1
- The Gigabit Ethernet ports FCIP Tunnels have a max and a min communication rate set to 220,000 Kbs bits not bytes this leaves head room for other data on the point to point link as it is only 1Gbs
The fabric extension needs to be in place before the relationship between the 2 IBM Storewize V7000's can be established.
Problems Encountered
I was experiencing frequent FCIP Tunnel drops which was causing replication to stop, after trying numerous things with IBM support and IBM storage consultants it turned out I had a faulty SAN06B-R which was swapped out and resolved those issues.
I have some database backups which are running between 11pm and midnight which produces so much change that the point to point link can't keep up with those changes. So I have written 2 scripts which runs from a Linux machine on the network and is initiated from crontab to stop replication while the backup runs then restart the replication again at midnight. This is not ideal but its what we are doing at the moment. Once all the replication jobs are configured I am going to revisit this.
There are some IBM Storewize V7000 parameters that can be tweaked as well I will cover them on the IBM Storewize V7000 Replication section which can be reached here
Hi, I'm setting up a similar configuration between 2 DS8100 using SAN06B too, but my tunnel keeps going down, and port 16 too. What licenses do you needed to keep the tunnel alive and working?.
ReplyDeleteI set up the IP on ge0, routes (also portcmd --ping works between the link) and fciptunnel. do I need to setup something else?
Thanks!
Javier, I am using the base licence the problems I encountered was the compression level, bandwidth settings and MTU size. You need to work out what is reporting the drop is it the SANs or the FCIP (SAN06B-R) gateways. If it is the SANs and you are using global mirror it might be the settings of that, I dont know the DS8100 but in the v7000 world there is settings like gmlinktolerance, if it is the FCIP gateways it might be too much data is trying to get down the link or the MTU setting is too big, what sort of latency are you getting across the link and what is you bandwidth, does you link support Jumbo frames or not. I used a tool from brocade called SAN Health check which basically tests the link for you and produces some results, once the test is complete you send it to brocade and they give you the results. You can download that from here http://www.brocade.com/services-support/drivers-downloads/san-health-diagnostics/index.page I sent logs to IBM support and after a few weeks and loads of SAN health checks I had a faulth SAN06B-R. Hope this helps.
Delete