0

Dude, Where’s My File Share Witness

The question of: I have just ran New-DatabaseAvailabilityGroup and I do not see a File Share Witness (FSW) folder or cluster resources. Where are all my Database Availability Group (DAG) and cluster resources…? What went wrong?

This is a common scenario when an admin is building a DAG, and during the process they check to see that what has changed on the servers. One thing that that causes a false alarm is the lack of a FSW immediately after the DAG has been created, and also a “missing” cluster.  Is this something to be concerned about?  When are the cluster and FSW resources actually created?  Let’s  go hit the lab and find out!

In this example, we will be using the default DAG type in Exchange 2013 which is to create the underlying cluster with the Administrative Access Point.  Exchange 2013 SP1 and newer introduced the option to create a cluster without the Administrative Access Point when installed on Windows 2012 R2.   This has sometimes been called the IP-less DAG, since there are no longer any cluster resources which require the underlying cluster IP.

 

Lab Starting Configuration

The below is an Exchange 2013 CU12 based lab, where all Exchange servers are running the same build.  Domain controllers and the OS on the Exchange servers are all Windows Server 2012 R2.  First up, note that there are no DAGs present as we begin this exercise.

 

 Get-DatabaseAvailabilityGroup

 

No DAGs Present

On CONSEA-MB1 the Windows Failover Cluster RSAT tool was installed so we can search for any existing clusters.  Opening up Failover Cluster Manager, we cannot see any existing clusters.

No Clusters Present

 

 

Create Database Availability Group

Next, let’s create a DAG in PowerShell by running:

 

 New-DatabaseAvailabilityGroup –Name DAG-01 -DatabaseAvailabilityGroupIPAddresses 192.168.2.50

 

 

DAG AD Object Created

 

Since there are separate CAS/Mailbox servers in the AD site, Exchange automatically chose to place the FSW onto the CONSEA-CAS2 server.  You will note that this was not specified in the above command.  The IP address was specified for completeness.  The details of the DAG can be seen below.

 

 Get-DatabaseAvailabilityGroup –Identity DAG-01 | Format-List

 

Newly Created DAG - No Servers Added Yet

At this point the DAG exists only in AD as a configuration object. There is no cluster, and now FSW content has been created.  For reference the DAG object can be viewed with  Get-ADObject

Get-AdObject “CN=DAG-01,CN=Database Availability Groups,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=Contoso,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=contoso,DC=com”

DAG Object in AD

 

Note that there are no servers listed in the started servers list.  We can see this via Exchange cmdlets, or by directly querying the underlying AD attributes. The below shows the list of started servers in the DAG.

ADSIEdit Showing List of DAG StartedMailboxServers

 

We saw that the FSW is to be held on the CONSEA-CAS2 server.  On that machine the root folder for the witness directories was created, but is empty.  This is shown below.

DAGFileShareWitnessFolder Created - No FSW Folder Yet Created

The NTFS ACLs for the directory are below.  Note that the Exchange Trusted Subsystem group is present in the local Administrators group as Exchange 2013 was previously installed onto CONSEA-CAS2.

DAGFileShareWitnessFolder Default NTFS ACLs

 

Add First Mailbox Server to the DAG

Since the underlying OS is Windows Server 2012 or newer we must pre-stage the DAG CNO.  If you fail to do this, then you will be the lucky winner of this error message: “You Must Provide A Value For This Property”.    The pre-stage process and instructions are contained in this post.  Once you have done the necessary, return back here to continue.

 

At this point we have not yet created the underlying cluster, as we have still not added first mailbox server to the DAG.  It is the act of adding the first mailbox server to the DAG which will initiate the call to create and build out the cluster.  Once AD has replicated with the newly created DAG, we can add the first mailbox server into the DAG.  The below process uses PowerShell, though you can use the EAC if you want.

 Add-DatabaseAvailabilityGroupServer -Identity DAG-01 -MailboxServer CONSEA-MB1

Adding First Mailbox Server to The DAG

 

Adding First Mailbox Server to The DAG - Forming Cluster

The comment that it will take <45 seconds is a little optimistic.  But once it has done it’s thing all is good.

Adding First Mailbox Server to The DAG - Adding Server to Cluster

If there are any errors to check and troubleshoot start with the DAG log file which is located in:

C:\ExchangeSetupLogs\DagTasks

A full path example would be:

C:\ExchangeSetupLogs\DagTasksdagtask_2016-09-15_04-56-51.831_add-databaseavailabiltygroupserver.log

The cluster has now been formed.  However note that there is no FSW currently is use as there is only a single server.  We can see that in the highlighted section below, where the witness type is set as None.

Failover Cluster Manager - No FSW

And the same is noted in Get-DatabaseAvailabilityGroup – the WitnessShareInUse is currently NONE.

Get-DatabaseAvailabilityGroup - No FSW

 

The FSW folder is still empty.  This is because the FSW is not in use as the cluster is currently in node majority.

 

Note On IP Address

Remember in the command above to create the DAG, the IP address was specified.  If the IP address is not specified, and there is no DHCP available then Failover Cluster Manger will look a little bit angry as the Cluster’s IP address will be in a failed state.  Just add the IP address using the Exchange Admin Centre or PowerShell and you will be in business.  This assumes that you are not creating the DAG without the Administrative Access Point, or in the vernacular term IP-less DAG.

No DAG IP Specified - Failover Cluster Is a Tad Angry

The below could be used to add one IP4 address to the DAG.

 

Set-DatabaseAvailabilityGroup –Identity DAG-01 –DatabaseAvailabilityGroupIPAddresses 192.168.2.50

 

Note that an additional IP will have to be added for the MAPI network if there is also a second secondary site.  Simply re-running the above command with the additional IP will overwrite the original IP leaving only the second one present.  Both IPs are to be specified.  This is shown in the example below”

Set-DatabaseAvailabilityGroup -Identity DAG-01 -DatabaseAvailabilityGroupIpAddresses 192.168.2.50,10.0.1.8

 

 

 

Adding Second Mailbox Server To DAG

The second mailbox server will be added to the DAG.  On this server, the Windows Failover Cluster bits were not present so were automatically installed.

Adding Second Exchange Server to DAG

After the command completes, the second server is visible using Get-DatabaseAvailabilityGroup.  This is highlighted below.

Second Exchange Server Added to DAG

 

We can also use Cluster.exe /Quorum to have a look at the cluster details.

Checking Cluster Quorum Type Using Cluster.exe

 

When the second node is added, Exchange automatically changes the quorum type for us, note that after adding the second node it is now Node and File Share Majority.  Since the FSW is in use it’s contents can be viewed.

Contents of Exchange FSW Folder

 

 

As a side bar you will note that the FSW folder contains a bit of a random number.

C:\DAGFileShareWitnesses\DAG-01.contoso.com49bf4e3d-caa5-4df6-8e2d-38293f9533d7

Well not much in computers is really random.  This does look like a GUID.  Is it the GUID of the DAG object?

Retrieving Exchange DAG GUID

Nope – the folder name is based off the FSW GUID.  This can be seen in the cluster log when the resource was created.  First we need to generate the cluster log in Windows 2012 and newer.  The Get-ClusterLog cmdlet is used for this task.

Get-ClusterLog -Destination C:\Temp\Clusterlog

Generating Windows 2012 R2 Cluster Log

Looking through the log we see the FSW GUID listed when the it’s cluster resource is created.

Windows Failover Cluster Log Showing FSW Resource GUID

The relevant excerpt from the cluster log file is below:

0000312c.00002ca0::2016/09/15-21:58:23.417 INFO  [RCM] rcm::RcmApi::CreateResource: (Cluster Group, File Share Witness (\consea-cas2.contoso.comDAG-01.contoso.com), 49bf4e3d-caa5-4df6-8e2d-38293f9533d7, File Share Witness)

Alternatively we can look at the resource’s properties.  In this case this we are using PowerShell to retrieve the ID of the FSW:

Get-ClusterResource “File Share Witness (\consea-cas2.contoso.comDAG-01.contoso.com)”

(Get-ClusterResource “File Share Witness (\consea-cas2.contoso.comDAG-01.contoso.com)”).ID

 

Retrieving File Share Witness GUID

 

Cheers,

Rhoderick

prod-wp-nimda

Leave a Reply

Your email address will not be published. Required fields are marked *