blog.virtualtacit.com

For those Celerra based CIFS servers that receive “Unable to join CIFS Server to Active Directory / Domain Controller” during domain additions, or any of the following, a simple fix exists..

• server_x:201:2119:E: server_cifs server_x -Join compname=mycifsserver domain=mydomain.com admin=administrator: failed to complete command
• LDAP: 3:  Ldap bind: LDAP server is down
  LDAP: 3:  LdapClient::connect: error message: Server down, (error code 81)
  SMB: 3:  DomainJoin::findServer: Unable to contact any domain controller in domain mydomain.com
  ADMIN: 3:  Command failed:  domjoin compname=mycifsserver domain=mydomain.com admin=administrator password=********************** init

Beyond the normal fat fingering of a password and incorrect domain or DNS settings the above commonly is revealed when the DC you are pointing to has NIC teaming or Load Balancing enabled. What follows is a brief example noted in EMC Primus #emc150780….

“In the example below, there are two NIC’s in the team on the DC: the team “leader”, which has a MAC address ending in 5c, and another team member which has a MAC address ending in 5b.

1. The Data Mover gets the IP of the DC from the DNS server. That IP resolves to the MAC address of the primary, or leader in the NIC team, which is 5c.
2. The Data Mover sends the SYN to the DC.
3. The NIC team “leader” and the teaming software will then assign a different NIC, 5b, in the team to respond with a SYN,ACK to the Data Mover.
4. The Data Mover then replies to the SYN,ACK with an ACK.
5. The Data Mover replies with the ACK back to the same MAC address, 5b, that it received the SYN,ACK from. See the example below.

Data Mover (SYN)                                     –>               Domain Controller (5c MAC address obtained from DNS)
Domain Controller (’5b’) (SYN,ACK)        –>               Data Mover
Data Mover (ACK)                                    –>               Domain Controller (5b)

Here are the retransmits in the tcpdump. The Data Mover doesn’t get a response from this point onwards from the Domain Controller:

Data Mover (ACK)                                    –>               Domain Controller (5b) Retransmit
Data Mover (ACK)                                    –>               Domain Controller (5b) Retransmit
Data Mover (ACK)                                    –>               Domain Controller (5b) Retransmit
Data Mover (FIN,ACK)                             –>               Domain Controller (5b)

The Data Mover finally gives up waiting for a response and sends the FIN,ACK to end the conversation to the Domain Controller.

The reason this has happened is because MAC address 5b is receiving packets from the Data Mover and does not know what to do with them. As far as the Domain Controller is concerned, the Data Mover should be talking to MAC address 5c. MAC address 5b ignores the packets received from the Data Mover.”

To rectify such a problem, simply run the following command on the data mover where the CIFS server exists..no reboot required.

server_param server_2 -facility ip -modify reflect -value 0

You be the judge. Great video supporting the casualness of celerra installs…

We have all been there…unexplained situations where production is down and your head is on the chopping block to get it resolved as quick as possible. Log hunting is not fun in situations like this and in all honesty who can remember every pertinent log file to aid you in your quest to recoverability. So to assist with gathering the following script can be run from the control station to collect all supporting logs, this likens the vm-support scripts for ESX support logs. To initiate run the following:

[root@NS20 nasadmin]# /nas/tools/collect_support_materials

The following is the output you should receive, blue text represents dump location:

collect_support_materials[21922]: The collection script revision 2.9.1 has started.
Collecting output from server_log
Collecting output from internal commands
Collecting event log configuration files
Collecting files from .etc dir of each DM
Collecting Mirrorview DR logs
Collecting Backend Monitor logs
Collecting /var logs
Collecting upgrade logs
Collecting files from /etc
Collecting files from /proc
Collecting /http/logs and /tomcat/logs
Collecting Celerra Manager tasks
Collecting cron files
Collecting Control Station process information and versions
Collecting /nas/jserver/debug_of_core* files
Collecting /nas/bdb files
Now running material collection script for longer running commands.
Collecting complete nas dir listing
Collecting output from nas commands
Collecting Replicator information
Collecting RDF information
Collecting DHSM information
Collecting Symmetrix information
Collecting Clariion information
Collecting output from other CS commands
Collecting other files from /nas, /nas/site, /nas/sys,
/nas/rdf, and /nas/dos

Collecting /nas/log/*, /nas/log/webui/*, /nas/ConnectHome/*,
/nas/jserver/logs and /nas/log/connectemc/*
Material Collection File:
/nas/var/emcsupport/support_materials_APM00081702890.080603_0939.zip has been generated.

**********************************************************************
Please include file /nas/var/emcsupport/support_materials_APM00081702890.080603_0939.zip (Path to compressed logs)
with materials submitted to EMC for problem investigation.
**********************************************************************

collect_support_materials[21922]: The collection script has finished successfully.

Whether its common knowledge or not I thought it would be worth sharing tips to controlling sporadic SavVol growth. For those that arent privy, SavVol is an area of a celerra volume where point-in-time blocks of data of a filesystem are kept. Snapsure, which is the enabler for checkpoints on the celerra, maintains a copy on first modify canon. Now without getting too in depth, as there are 70 page documents written on this process alone, lets assume you have previous experience with the Celerra and an understanding of Snapsure more importantly.

So administratively, space conservation as it relates to checkpoints doesnt have to be a manual task. If you primary concern is to maintain rolling checkpoints operating within a set amount of disk space then this process may be worth entertaining. To achieve this, we will specify a high water mark of zero percent during the creation of the checkpoint. This instructs SS not to extend the SavVol if the checkpoint reaches beyond the capacity of the filesystem. Rather, the oldest checkpoint is deleted and the freed space is used to keep the most recent checkpoint engaged. Each time a checkpoint needs space to complete, this process is repeated.

As stated, this method deletes the oldest checkpoint to keep the most recent active. So if any of that latter checkpoint data is needed then the following must be done:

• Backup the checkpoint to tape
• Extend SavVol to keep the checkpoint from being deleted

Remember at any time you can verify the SavVol space by running fs_ckpt “fs_name” -list command.