Comments on: Routers treat HTTPS and HTTP traffic differently

By: Ryu

Ryu — Wed, 21 Oct 2009 18:27:25 +0000

I’d add that I _do_ see the same behaviour as you when visiting paypal.com. But this is definitely not a universal web server / https truth.

——-
# tcpdump -v src http://www.paypal.com and tcp port 443
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
13:25:15.536007 IP (tos 0×0, ttl 246, id 52428, offset 0, flags [DF], proto TCP (6), length 44) 66.211.169.65.https > my.host.here.59435: S, cksum 0×26b7 (correct), 1966361028:1966361028(0) ack 2475762734 win 8190
13:25:15.585680 IP (tos 0×0, ttl 246, id 19924, offset 0, flags [DF], proto TCP (6), length 1420) 66.211.169.65.https > my.host.here.59435: P 1:1381(1380) ack 114 win 40845
13:25:15.585794 IP (tos 0×0, ttl 246, id 20436, offset 0, flags [DF], proto TCP (6), length 1420) 66.211.169.65.https > my.host.here.59435: P 1381:2761(1380) ack 114 win 40845
13:25:15.585903 IP (tos 0×0, ttl 246, id 20948, offset 0, flags [DF], proto TCP (6), length 1420) 66.211.169.65.https > my.host.here.59435: P 2761:4141(1380) ack 114 win 40845
13:25:15.585932 IP (tos 0×0, ttl 246, id 21460, offset 0, flags [DF], proto TCP (6), length 243) 66.211.169.65.https > my.host.here.59435: P 4141:4344(203) ack 114 win 40845
13:25:15.837024 IP (tos 0×0, ttl 246, id 54784, offset 0, flags [DF], proto TCP (6), length 40) 66.211.169.65.https > my.host.here.59435: ., cksum 0xad2c (correct), ack 304 win 40655
13:25:15.837831 IP (tos 0×0, ttl 246, id 62464, offset 0, flags [DF], proto TCP (6), length 91) 66.211.169.65.https > my.host.here.59435: P 4344:4395(51) ack 304 win 40655
13:25:15.887928 IP (tos 0×0, ttl 246, id 59400, offset 0, flags [DF], proto TCP (6), length 40) 66.211.169.65.https > my.host.here.59435: ., cksum 0xacf9 (correct), ack 1005 win 39954
13:25:16.461813 IP (tos 0×0, ttl 246, id 22901, offset 0, flags [DF], proto TCP (6), length 1005) 66.211.169.65.https > my.host.here.59435: P 4395:5360(965) ack 1005 win 39954
13:25:16.461832 IP (tos 0×0, ttl 246, id 25973, offset 0, flags [DF], proto TCP (6), length 77) 66.211.169.65.https > my.host.here.59435: P, cksum 0xe9cc (correct), 5360:5397(37) ack 1005 win 39954
——-

By: Ryu

Ryu — Wed, 21 Oct 2009 18:20:03 +0000

@Mark:

I am not seeing the behaviour you’re describing. Take https://gmail.com for instance:

——-
# tcpdump -v src mail.google.com and tcp port 443
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
13:17:40.901281 IP (tos 0×0, ttl 51, id 21756, offset 0, flags [none], proto TCP (6), length 60) gy-in-f17.1e100.net.https > my.host.here.53272: S, cksum 0xf329 (correct), 616756817:616756817(0) ack 3946542441 win 5672
13:17:40.928660 IP (tos 0×0, ttl 51, id 21757, offset 0, flags [none], proto TCP (6), length 52) gy-in-f17.1e100.net.https > my.host.here.53272: ., cksum 0×36fc (correct), ack 115 win 89
13:17:40.929612 IP (tos 0×0, ttl 51, id 21758, offset 0, flags [none], proto TCP (6), length 1470) gy-in-f17.1e100.net.https > my.host.here.53272: . 1:1419(1418) ack 115 win 89
13:17:40.929671 IP (tos 0×0, ttl 51, id 21759, offset 0, flags [none], proto TCP (6), length 331) gy-in-f17.1e100.net.https > my.host.here.53272: P 1419:1698(279) ack 115 win 89
13:17:40.960292 IP (tos 0×0, ttl 51, id 21760, offset 0, flags [none], proto TCP (6), length 278) gy-in-f17.1e100.net.https > my.host.here.53272: P 1698:1924(226) ack 301 win 106
13:17:41.027614 IP (tos 0×0, ttl 51, id 21761, offset 0, flags [none], proto TCP (6), length 52) gy-in-f17.1e100.net.https > my.host.here.53272: ., cksum 0×2ae5 (correct), ack 1088 win 130
13:17:41.157382 IP (tos 0×0, ttl 51, id 21762, offset 0, flags [none], proto TCP (6), length 828) gy-in-f17.1e100.net.https > my.host.here.53272: P 1924:2700(776) ack 1088 win 130
13:17:41.158629 IP (tos 0×0, ttl 51, id 21763, offset 0, flags [none], proto TCP (6), length 397) gy-in-f17.1e100.net.https > my.host.here.53272: P 2700:3045(345) ack 1088 win 130
——-

By: mark

mark — Tue, 20 Oct 2009 21:45:03 +0000

Dmitriy I tried to email this to you but it bounced:

Hi Dmitriy,

Any chance you can send me a packet capture of an SSL session where you’re seeing that the DF flag isn’t set? I’ve only ever seen the DF flag set for HTTPS and I’m very interested in what you’re seeing. Just add -w filename.dump to your tcpdump options. My email is mmaunder at gmail dot com.

Thanks!!

Mark.

By: Dmitriy

Dmitriy — Tue, 20 Oct 2009 21:37:16 +0000

@Mark

You can open dump files with tcpdump on Linux with “tcpdump -r filename” and you can see whether packets have DF set or not in tcpdump by calling it with a couple of extra -v flags (tcpdump -vv). Even though yes, Wireshark rulez

I used the same methodology for my tests – and I am still not able to recreate it under Linux or Mac OS X, with wget or curl.

I am not trying to say that your findings are wrong, esp. since they are based on a real-life scenario that you were troubleshooting.

I just think there might be more to it, some factor that hasn’t been uncovered yet.

By: mark

mark — Tue, 20 Oct 2009 20:45:33 +0000

I’m not sure why DF seems to be set on all SSL traffic. You can find out how to set up PMTUD on a socket which causes the DF flag to be set here:

http://manpages.ubuntu.com/manpages/jaunty/man7/ip.7.html

Also Google the socket option IP_MTU_DISCOVER. I’ve seen a few results that bring up openssl source code.

I also saw a mention of the bonk attack being a DoS for fragmented packets and it may be related to why SSL authors set IP_MTU_DISCOVER on their servers.

Please do share what you find.

By: Swoög

Swoög — Tue, 20 Oct 2009 20:01:09 +0000

That’s interesting… It does not seem to be any restriction (neither in the protocol, nor in technical implementation) justifying the DF option to be on for encrypted protocols…

Why implementers did so ? is there a reasonable explanation of that or is it just “feature-not-a-bug” they thought useful ?

Swoög

By: mark

mark — Tue, 20 Oct 2009 19:52:56 +0000

Hi Dmitri,

Sure. On another user’s suggestion I’ve included a reference to PMTUD which is what I’m describing in the ICMP example.

The DF/non-DF data is from my own research. I was recently diagnosing a network problem that existed via HTTPS and not HTTP and discovered the difference. On Linux, most secure protocols like SCP, SSH and HTTPS set the DF flag on IP packets.

You can duplicate my research as follows:

Run the following command on your linux box to capture all SSL packets inbound and outbound. I’m assuming your ethernet card that will see the traffic is eth0:

tcpdump -w dump1.dump -s 1600 -i eth0 port 443

Then fetch a secure web document from your linux box using curl or your favorite command line web client:

curl -k –connect-timeout 20 –max-time 60 -v “https://www.paypal.com/”

Then hit CTRL-C to stop the tcpdump capture. Copy dump1.dump to your local machine. Install Wireshark (google it).

Then open the captured file with Wireshark. Click one of the data packets. Look at the IP part of the packet and you’ll see the DF flag is set.

Now repeat the process above, but modify the tcpdump port to be 80 and fetch a regular web document from a non-secure site. You’ll notice the packets do not have the DF flag set.

You can test this on a variety of protocols. Try it on SSH and SCP. Also note the size of the packets that are being captured in each case. With things like SSH the packets are small – usually single keystrokes. With HTTPS they are very large which is why MTU problems manifest themselves with HTTPS but not SSH which also has the DF flag set.

Mark.

By: Dmitriy

Dmitriy — Tue, 20 Oct 2009 19:44:43 +0000

Interesting. I just did a quick test and I saw both client and server set DF when talking either HTTP or HTTPS.

A quick grep on RFC2818 did not reveal any mentions of DF either.

Could you please describe sources based on which you reached this conclusion? Will greatly appreciate it!

Thanks.