blog.virtualtacit.com

In case you guys missed it, NetApp offered a shakeup back in July about the ability of their V-series line to de-dupe their competitors primary storage, noted here. I would have to agree with the general consensus that deduplication has its place and its place aint’ on primary storage. Applying de-dupe to secondary, tertiary storage and backup operations is really the meat behind this features punch.

The goal here is to provide your production data with the means to achieve high throughput, low latency execution, right? What you are doing is permeating your critical business infrastructure with an operation that is known to degrade performance. Not to mention this doesn’t in any way, shape or form provide the customer with an end to end, low maintenance, SUPPORTED solution. As a customer you must decide whether or not your investment in your array (whether EMC, HP, HDS, IBM, etc)  is all for not. 

By fronting your array with the V-series essentially strips all management capabilities of your array and reduces it to JBOD. Your investment is no longer an investment. E-labs, EMC’s own interoperability entity, works with associated vendors to resolve issues for qualified configurations. These support agreements run deep with various vendors but NetApp, particularly the V-series, is not one of them.

So here is what NetApp suggests:

· Support for de-duplication of primary data on third party storage arrays from EMC, HDS, HP, and others when connected to NetApp V-Series Virtualization systems.

· NetApp de-duplication, a feature of the Data ONTAP operating system provided at no cost on FAS systems, is now also offered free with the V-Series.

· End-to-end de-duplication including primary data, as opposed to other vendors’ de-duplication of only backup or archive environments.

· Improved business efficiency and reduced data management complexity using V-Series with non-NetApp storage arrays.

· De-duplication helps improve space efficiency and reduce raw storage requirements.

· By using V-Series with de-duplication, customers are able to better control their heterogeneous data growth while reducing costs and simplifying data management.

· More than 10,000 NetApp systems and 2,500 customers running NetApp de-duplication technology.

· All NetApp storage technologies will include de-duplication by the end of 2008.

Ok so here’s the rub when it comes to data de-dupe on NetApp Filers:

· Active snapshots? sorry you can’t de-dupe 

·Severe volume limitations are imposed as part of the 3D process

· No de-dupe over FlexVols, Aggregates, or Filers.

· Backup to tape inflates data to pre-dedupe size

· Since de-duplication is a post-process operation, NetApp offers no reduction of capacity requirements for initial purchase of new systems.

· No reclamation of space in block based storage (FC and iSCSI)

· Scheduling complications are now a reality. Avoiding periods of snapshotting, replication, archiving and general heavy work loads can be difficult.

· NetApp says “If there is very little new data, run de-duplication infrequently, because it doesn’t make sense to unnecessarily consume CPU resources.” http://www.netapp.com/us/library/technical-reports/tr-3505.html

·  De-duplication itself is free, but are SnapVault and SnapMirror?  Should I remind you that nothing in life is free.

De-dupe, like every other storage feature, whether its EMC, NetApp, DataDomain,etc, has its positives and negatives. Just make sure you as a customer look beyond the marketing wooglie booglie and understand the technology you are depending on.

One last thought, if you do decide to turn on NetApp 3D please take 10 minutes to fill out a “I told you so” form that releases any wrong doing from NetApp  when your performance dips below the equator, http://www.crn.com/storage/209901632, I kid you not….

The Clariion line has long been my bread and butter when it comes to midrange storage arrays. In this market, there is no better in my opinion, so its not surprising that EMC has continued the line into its most innovative generative release.

To begin, lets rundown the marketing dribble as to what each model has to offer from a physical and software perspective. Note–Glad there is special attention this time around to designate the last three numbers of the model name as the max supported drives, long over due.

CX4-120

• Scalable up to 120 drives
• Maximum 6G of R/W cache
• 4 Fiber/4 iSCSI
• Maximum 16 FE FC and iSCSI

CX4-240

• Scalable up to, you guessed it, 240 drives
• Maximum 8G of R/W cache
• 4 Fiber/4 iSCSI
• Maximum 20 FE FC and/or iSCSI

CX4-480

• Scalable up to 480 drives (This was the max drive account in the CX3-80)
• Maximum 16G of R/W cache
• 8 Fiber/4 iSCSI
• Maximum 24 FE FC and/or iSCSI
• Flash drives..giddy up

CX4-960 (This is where the lines begin to blur between midrange and high-end arrays)

• Scalable up to 960 drives
• Maximum 32G of R/W cache
• 8 Fiber/4 iSCSI
• Maximum 32 FE FC and/or iSCSI
• Flash drives

Here are some the points that are new to this release…

• Virtual Provisioning, pooled storage
• Flash Drives - Up to 30 times IOPS than traditional FC drives, less 1 ms response time–sick I tell you sick
• Drive Spin Down-Sleep mode for inactive drives, sweet spot for B2D disks, test and development, etc.
• Integrated Recoverpoint Splitter-This in itself is game changing, moving the I/O splitter from the host or FC-SW is huge for the adoption of this technology and more explicitly VMware’s Site Recovery Manager.
• UltraFlex Technology-Online hot pluggable I/O modules
• 64-Bit Flare, Multi-Core Processers and up to 32G of Cache

EMC has done alot to bring “Green Storage” to light, with the introduction of Drive Spin Down, Adaptive Cooling and LP SATA drives. There has also been a merging of sorts of functionality used in the Celerra line to the CX line, such as Virtual Provisioning. The ability to pool storage and expand capacity nondisruptively.

As you can see the CX4 has brought significant changes to the Clariion product line, more than I can ever cover in a single post. So expect more on this topic in the future.

As a follow up to the post, Unified I/O-FCoEasy does it….Successor to FC?, I thought it would be fair to drop some info on Infiniband and its place in the storage ecosystem.

IB from a raw throughput perspective is no doubt head of the class. Low latency, high bandwidth, and the ability to collapse both FC and Ethernet on a single transport are all contributing factors to IB’s push toward the de facto standard for high speed I/O interconnects. Although most notably a main staple in HPC (High Performance Computing) environments it quickly is targeting its efforts toward virtualization. And why shouldn’t it, the inevitable build up or build out of pure physical cabling to support even the most simplest designs makes this consolidated technology a perfect fit. VMware has already announced support for Mellanox HCA’s in its ESX 3.5 release, signaling that the tides are now split between IB and 10GigE as viable Unified options–but wait a minute, lets look under the covers shall we..

Comparison of both competing technologies above– (notice FCoE and IBoE are not listed but both are defining technologies in development)

So where do we stand from both IB’s perspective and Ethernet as a transport; the good, the bad, the ugly–noted here–http://www.nowmicro.com/NM_PDF/wp-cisco-interconnects.pdf

Advantages of IB

• Reduced CPU and memory overheads by utilizing specialized HCA hardware
• Remote Direct Memory Access (RDMA) allows for the placement of information from one computer system into the memory of another computer
  system, reducing latency and minimizing the processing overhead demands at the host system
• Low end-to-end system latency (from 5 to 10 microseconds, depending on the application)
• Multiple vendors supporting and shipping products
• Consolidated I/O with network, management, and storage all on one interface
• Implements various protocols, such as general-purpose remote-procedure call (RPC), direct-access transports (as part of RDMA), sockets direct protocol for TCP/IP, and more

Disadvantages of IB

• There is only one provider of an InfiniBand core chipset (Mellanox), constituting a risk of supply disruption. (and this is huge in my book, free market in effect-not good)
• This lack of diversity also generally reduces innovation and over time drives prices higher.
• There is limited expertise in the field with the protocol.
• It is a new networking protocol and relies heavily on gateway functionality (storage area network and IP) to get the full benefits of a unified fabric.
• Limited diagnostic and troubleshooting tools are available in shipping products.
• It has had limited use in enterprise environments, which is the true test for sustaining long-term viability of the protocol. (here we are talking about significant up front cost, you are essentially up fitting you entire environment with gear to support this technology)

Ok, not nearly as sexy as first discussed…how about Ethernet..

Advantages of Ethernet

• Long networking history
• Knowledge base of the protocol and tools available–come on who doesnt know how to configure a NIC..what about an HCA? —that’s what I thought
• Based on standards and available from multiple vendors

Downside of Ethernet

• RDMA offload network interface cards are new and relatively expensive
• 10 Gigabit Ethernet is still relatively expensive….and dropping as we speak
• End-to-end MPI latency may be longer in Ethernet networks (testing is still in progress)
• Other interconnects are more competitive in price performance at this time….give it time

Are we looking at both hardware and soft costs with IB when comparing price/performance per port to Ethernet? I am not so sure…this is definitely a topic worth revisting…thanks to JN for kicking up this topic

There has been a lot of news over the last several months about FCoE and its applicability in the data center. With the advent of 10GbE and the premise of data center unification I can understand why this technology has gotten legs. Joining the ranks of iSCSI, iFCP, and FCIP, FCoE offers the consumer yet another means to shove block based data traffic over an Ethernet network.  But unlike the former protocols,  FCoE is purely touted as a “Data Center” protocol, non-routable and connection oriented much like FC.

The practicality of this protocol comes into play with the use of a CNA or Converged Network Adapter, ultimately replacing the need for Ethernet NICs and FC HBAs. Less adapters equates to less cabling, less power and less complexity. Paradisiacal, huh???

Well hold on…undeniably the landscape of Ethernet will need to change to support this convergence of data and storage traffic. To further this unification, Converged Enhanced Ethernet  (or DCE, Data Center Ethernet) is under development to allow for FC to pipe over Ethernet networks.

What follows are the modifications that need to take place to Ethernet to support Fiber Channel.

• Encapsulation of native fiber channel frames within an Ethernet frame.
• Extensions to the Ethernet protocol itself to enable a lossless Ethernet fabric
• Replacing the fiber channel link with MAC addresses in a lossless Ethernet

Fiber Channel, it seems, is architecturally  dammed. With Ethernet extending within the 10G range and beyond (40G and 100G under development) how can FC ever compete? By switching transports to Ethernet is how. Protecting FC investments is the key to FCoE’s adoption and all vendors with business interests in FC know that.  Cisco, of particular interest, is leading this march. With its buy-in to Nuova Systems in 2006 and most recent buy-out in early 2008, it is well positioned to capitalize on future FC ecosystem upheavals. The Nexus 5000 series of DC class switches, released in April ‘08, is the first real step toward Unified I/O and you better believe it will support FCoE, with a few license buy-in’s of course.

With that said, the adoption of FCoE will be slow going despite its backing. Companies in general will not be quick to uproot there existing native FC environments for the “new kid” technology on the block until it is tested and proven from soup to nuts.

What’s exciting about the introduction of FCoE to the market is that it will force enhancements to Ethernet that will ultimately benefit iSCSI and network storage protocols in general, further driving the server consolidation and virtualization cause. Its proliferation will be built on the back of DCE and its adoption will be in the hands of you and me. So educate yourself, its coming…

For more info on FCoE and the developments that are happening in this space, please visit http://www.fcoe.com/

Also be sure to check out the compelling video describing the hurdles of explosive data growth at CNBC and how they will tackle their challenges with FCoE, http://video.computerworld.com/services/link/bcpid1351827287/bctid1410385428.

****Updated****

Thanks to Omar for commenting and correcting my inaccuracies, as FCoE does not require an upheaval but is complementary and non-disruptive to existing FC environments. He also noted, the Nexus 5000 is currently being certified in EMC E-Labs and should be customer implemented by year’s end. I may add that the current MDS product line does support FCoE in their third generation modules, further protecting your current investment.

Comments are always welcome.

It was always time consuming for me to decipher Cisco documentation that related to port group bandwidth and module overscription rates for their Gen-2 FC Modules. So what follows is a brief interpretation of how those ratios are determined and why they matter. Currently the modules available are 12 port, 24 port, 48 port and the 4 port 10G cards. The 12 port card offers a full 4G line rate per port, the 24 and 48 port cards share their bandwidth and operate according to the following subscription ratio’s:

So what this means is that if all ports were active on a 24 port module they would operate at the full line rate of 2Gbps. Broken down this equates to four 6 port port groups, each group capable of 12.8 Gbps. The 48 port modules consist of four 12 port port groups, each group again capable of 12.8 Gbps. That being said the full line rate of each port on a 48 port module if all ports were active would be 1Gbps. The maximum bandwidth per port per module, minus the 10G mod, is 4Gbps. So why does all this matter? Well its extremely important for host placement on those fiber switches. Ideally if you had a pleothra of modules in your director, you could seperate your traffic as follows:

12-port 1/2/4 FC Module-high perfomance servers, front end array ports, ISL’s, and tape drives/libraries

24-port 1/2/4 FC Module-server connections, front end array ports

48-port 1/2/4 FC Module-tier storage solutions (low,medium and high), server connections

4-port 10Gbps FC Module-ISL consolidation, high performance metro connections

These recommendations are highly dependent on your environment as there is nothing preventing you from putting a front end array port on a 48 port card. But being aware of what other devices exist on that module can save you trouble in the long run, not to mention it will ease you down the path of scalability. In addition, you should not only be cognizant of per module placement but also per port group placement. Remember you only have 12.8Gbps of bandwidth per port group so loading up all you front end ports, high performance servers, etc. into one port group is a recipe for a poor design.