TCP Port Scanning

How can hackers assess a computer on a network to see which services it is running(and possibly find exploitable ones)? The short answer: port scanning.

Some types of port scans include:

1.     SYN scan
2.     XMAS scan
3.     NULL scan
4.     FIN scan

SYN Scan - As stated in my post about the 3 way handshake, when a machine gets a TCP packet with the SYN flag set, it should respond with a packet with the SYN/ACK flags set. This sequence of events only happens if there is a service listening on the port to which the packet was sent; if there is no service listening on the given port (ie the port is in the CLOSED state), then the machine will not respond with SYN/ACK, and we know the port is not open. So the basic idea is to just send a packet with the SYN flag set the desired host on the port we want to query. If we get a response with the SYN/ACK flags set, then we know that the port is  open.

XMAS Scan- The idea behind the XMAS scan is that we set the FIN, PSH and URG flags. The reason that we do this is because we want to set an invalid combination of TCP header flags. Different TCP stacks will respond to these boundary conditions in different ways, betraying the true state of the port.

NULL scan- Similar to XMAS scan, except that we don't set ANY TCP flags. The reasoning here is that different TCP stacks respond to NULL packets in different ways.

FIN scan- Similar to the above two scan types, but only the FIN flag is set in the packet. Depending on the way the TCP stack responds to this boundary condition, we can tell the state of the desired port.

In all except the SYN scan above, the responses to each type of scan is completely implementation dependent. The reason for this is because the TCP standards were published in the form of a many of RFCs (Requests For Comments). These RFC specifications have some ambiguities and some places where they are open to interpretation of the reader (in this case, the implementer of the TCP stack). For example, the developers that wrote the Windows TCP stack might have understood what it was saying differently than the developers that wrote the Linux TCP stack. These differences in interpretation can be exploited to figure out "What Operating System X does when packet Y is sent to a port in state Z". After experimentally understanding how a certain Operating System reacts to these boundary conditions, a set of rules can be made, giving rise to the technique of "Operating System Fingerprinting" via the network. There is a very good open source project called "nmap" that uses a lot of these techniques. Find it here:

http://nmap.org/

References:

http://nmap.org/book/man-port-scanning-techniques.html