Celerra, NFS and ESX- Purpose Built for You and Me
For those of you that have not gone the route of NFS, specifically NFS on Celerra, for your VI, then listen up here a few benefits of such a design..
Note: Keep in mind that each environment is different and that no one solution is ideal for everyone. I am by no means promoting NFS for all performance situations but rather informing you of the capabilities of the Celerra and how NFS fits into the equation. NFS has its challenges in itself just like every other storage protocol so be diligent and work through the numbers as to what works best for you.
Celerra Virtual Provisioning/Automatic File Extension
Allocating based on actual disk consumption rather than projected pre-allocated capacities. The days of fat or over provisioning are over, additional storage is allocated on a per file system basis only when needed. When the amount of used storage hits a predefined threshold or watermark the file system is automatically extended to compensate. NFS datastores can quickly and dynamically be expanded without jumping through hoops.
Advanced Networking Options for High Availability
- Failsafe-Active-Passive configuration across CGE ports, across separate switches. All connections in this pairing share the same MAC address, if the active link fails the passive link becomes active and therefore assumes the identify of what was the active link. A couple of things with FSN's:
- Here is what is cool, FSN can include single GigE ports, channel devices or LACP devices, although no more than 4 per FSN. So in certain situations, you could configure two separate virtual devices, one configured as a Etherchannel device and one configured as LACP device and then FSN between the two, remember only one device will be active at one time.
- On Cisco switches, enable portfast for each DM port
- FSN's do not rely on any specific switch functionality so it truly is the most neutral solution across switch vendors.
-
- Etherchannel- I won't go into the specifics on Etherchannel as that is common knowledge, but I will say this will not increase bandwidth, it exists to provide HA across a single switch, so a port failure. Single MAC address for the channel applies, static configured channels only no PAgP.
- LACP- Much like Etherchannel but the number of links in the aggregation is not required to be in multiples of 2 like EC. LACP is certainly more robust as it relates to failure detection than the latter.
What else with AN? Statistical LB..
Well I did allude to the fact that all of these mechanisms (EtherC and LACP) are purely for HA, but technically you could use Statistical Load Balancing on a per DM basis. This allows for distribution of frames across links in the channel by using a portion of the source and destination MAC address to determine the link to transmit down. You can configure SLB by IP, TCP ports and IP address. Here is the command:
$ server_param server_2 -facility trunk -modify LoadBalance -value TCP (change the value according to what LB method you want to use, available options are TCP, IP (default) and mac)
SnapSure checkpoints
The ability to recover a VM in its entirety from a CIFS share holds some weight with me. Celerra SnapSure gives you just that, Point in Time accessibility of NFS filesystems via a CIFS share divulged via the .ckpt shared hidden directory, ie. \\cifs_server_name\share_name\.ckpt. This my friends is the essence of ultra near-line recovery, and has saved me hours in full VM recovery time. I haven't seen this process video documented anywhere so I may take a stab at it in the future.
Additional benefits and tips to assist the cause...
NFS Performance Tuning
- Leverage the uncached and no prefetch mount options on the Celerra DM to enhance write performance:
- server_mount server_2 -o uncached fs_name mount_point
- server_mount server_2 -o noprefetch fs_name mount_point
Furthermore, the uncached write option enhances performance of applications with many connections to large files. Could produce an increase in performance up to 30%. Turn on for all NFS specific filesystem's, if replicating make sure you turn on this option on both the primary and replicated file system.
- Avoid virtual machine swapping to NFS volumes. Edit virtual machine config file to add sched.swap.dir place swap on local storage (supported inline with VMotion in ESX 3.5)
- 8 NFS mounts per ESX Server allowed by default adjust “NFS.MaxVolumes” , note:raising it to 32 increases the amount of physical memory needed, so size appropriately.
- NFS client read/write size, set NFS.ReceiveBufferSize and NFS.SendBufferSize to a multiple of 32k ensure both are identical
NFS Security
- Separation of traffic through the use of VLAN's
- Limiting access to certain host IP addresses
- Limiting access to certain user accounts
- ESX provides the use of a non-root account called a delegate account to mount the NFS files system. This will avoid exporting the file system with root access.
Creating an NFS Datastore via Celerra Manager
Why mess with a good thing, video on said title, http://www.youtube.com/watch?v=B5Qn4EMgbDc. Remember, considering you are using the Celerra file system for your datastore most Celerra features apply such as SnapSure, Celerra Replicator, SRDF and Quotas.
And finally, Feature Set Comparison to Storage Protocols
And if you in doubt here is how each protocol lines up to applicable feature sets.
| Feature | FC SAN | iSCSI | NFS |
| ESX Boot | Yes | Yes (hardware initiator only) | No |
| Virtual Machine Boot | Yes | Yes | Yes |
| Raw Device Mapping | Yes | Yes | NA |
| LUN Extension | Yes | Yes | Yes |
| Replication | Yes | Yes | Yes |
| Replication Type | Application Consistent | Application Consistent | Crash Consistent |
| Virtual Machine as Initiator | No | Yes | No |
| Security | NA | CHAP | Unix_Auth |







Wednesday, September 10, 2008 at 7:08PM
Reader Comments (1)
[...] woman on the other side of the room with the really annoying laugh. Yesterday Joe put up another great post about VMware and NFS. Articles like these always start some really good discussion amongst the engineers. The topic [...]