How To Set NSLookup to Use TCP

One of the great things at Microsoft is the interaction with very smart colleagues.  A question was posed regarding DNS.  Most admins will know that DNS uses port 53, but is that TCP or UDP?

The answer is Consultant’s answer #1 "It depends"…

From a client resolver perspective this old TechNet documents:

During DNS resolution, DNS messages are sent from DNS clients to DNS servers or between DNS servers. Messages are sent over UDP and DNS servers bind to UDP port 53. When the message length exceeds the default message size for a User Datagram Protocol (UDP) datagram (512 octets), the first response to the message is sent with as much data as the UDP datagram will allow, and then the DNS server sets a flag indicating a truncated response. The message sender can then choose to reissue the request to the DNS server using TCP (over TCP port 53). The benefit of this approach is that it takes advantage of the performance of UDP but also has a backup failover solution for longer queries.

The DNS Server service in Windows Server 2008 supports Extension Mechanisms for DNS (EDNS0, as defined in RFC 2671), which allow DNS requestors to advertise the size of their UDP packets and facilitate the transfer of packets larger than 512 bytes. When a DNS server receives a request over UDP, it identifies the requestor’s UDP packet size from the option (OPT) resource record and scales its response to contain as many resource records as are allowed in the maximum UDP packet size specified by the requestor

UDP packets are smaller in size. UDP packets can not be greater then 512 bytes. So any application needs data to be transferred greater than 512 bytes require TCP in place. For example, DNS uses both TCP and UDP for valid reasons described below. Note that UDP messages are not larger than 512 Bytes and are truncated when greater than this size.  DNS uses TCP for Zone transfer and UDP for name queries either regular (primary) or reverse. UDP can be used to exchange small information whereas TCP must be used to exchange information larger than 512 bytes. If a client doesn't get response from DNS it must re-transmit the data using TCP after 3-5 seconds of interval.

The reason it is worthwhile to point out the age of the above is the comment on the maximum size of the UDP packet.  This has been an issue with some firewalls which block this traffic.  For details please see:

Some DNS name queries are unsuccessful after you deploy a Windows-based DNS server  EDNS0 allows larger User Datagram Protocol (UDP) packet sizes. However, some firewall programs may not allow UDP packets that are larger than 512 bytes. Therefore, these DNS packets may be blocked by the firewall.

Let's contrast the TCP and UDP lookup process.  We will resolve the A record in DNS called mail.TailspinToys.ca from a separate environment so the queries are over the Internet.  Note that we will query from WingtipToys.ca so pay attention to the DNS name.

UDP sockets are "message-oriented sockets" (as opposed to "stream-oriented sockets"; TCP sockets are stream-oriented). This is why the option we enable to use TCP in Nslookup is called the Virtual Circuit (VC) .

Reviewing Standard UDP Nslookup Query

Using standard Nslookup, we query for mail.TailspinToys.ca

This correctly resolves the A record to the external IP of 13.92.177.139 (this is in Azure in case you were curious).

From the network perspective, the queries issued from the resolver to its DNS server are UDP based.  You can see this in Wireshark below.

Reviewing TCP Nslookup Query

We will query for the same endpoint, but first the Virtual Circuit option will be enabled with the command:

 set vc

Note that the option is case sensitive.

Again the A record is correctly resolved.  Note that the Wireshark trace is totally different.  Immediately we see the TCP three way handshake, that s not needed for UDP.  This is some of the TCP overhead that UDP was designed to avoid.  The TCP handshake is highlighted in the red box, and the DNS query process starts in frame 6.

Looking at the details for frame 6, we see that TCP is now used for the query's transport mechanism:

Bootnote

You may have noticed text along the top of the Wireshark window.  This indicates that we are capture on an interface called NIC1 and the capture filter is what is shown inside the brackets.

Cheers,

Rhoderick