net/http 库的一个让人抓狂的 bug

lubyruffy — Wed, 03 Sep 2014 21:10:26 +0800

我用 net http 去请求一个 https 的页面，如下两种方法在某些网站返回居然不同： 1、先用 ip 连接，然后手动设置 Host 头 2、直接用 host 连接绝大部分情况下都是正常的，只有在少部分网站返回的结果完全不一样（一个返回 200，一个返回 404），比如下面的演示：

#!/usr/bin/env ruby require 'net/http' require 'uri' require 'open-uri' require 'openssl'

@uri = URI('https://ebs.shasteel.cn')

def get_ip_of_host(host) require 'socket' ip = Socket.getaddrinfo(host, nil) return nil if !ip || !ip[0] || !ip[0][2] ip[0][2] rescue => e nil end

def test_connect_ip ip = get_ip_of_host(@uri.host) http = Net::HTTP.new(ip, @uri.port) http.use_ssl = true if @uri.scheme == 'https' http.verify_mode = OpenSSL::SSL::VERIFY_NONE http.set_debug_output($stdout) http.start { |h| request = Net::HTTP::Get.new @uri.request_uri request['Host'] = @uri.host response = h.request request puts response.code } end

def test_connect_host http = Net::HTTP.new(@uri.host, @uri.port) http.use_ssl = true if @uri.scheme == 'https' http.verify_mode = OpenSSL::SSL::VERIFY_NONE http.set_debug_output($stdout) http.start { |h| request = Net::HTTP::Get.new @uri.request_uri response = h.request request puts response.code } end

test_connect_ip #will be 400 test_connect_host #will be 200

============我打开了发包调试，可以看到发送和接收的包内容============== opening connection to 61.177.60.85:443... opened starting SSL for 61.177.60.85:443... SSL established <- "GET / HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: /\r\nUser-Agent: Ruby\r\nHost: ebs.shasteel.cn\r\n\r\n" -> "HTTP/1.1 400 Bad Request\r\n" -> "Date: Wed, 03 Sep 2014 11:43:55 GMT\r\n" -> "Server: Apache/2.2.17 (Win32) mod_ssl/2.2.17 OpenSSL/0.9.8o\r\n" -> "Content-Length: 226\r\n" -> "Connection: close\r\n" -> "Content-Type: text/html; charset=iso-8859-1\r\n" -> "\r\n" reading 226 bytes... -> "" -> "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n

\n400 Bad Request\n\n

Bad Request

Your browser sent a request that this server could not understand.
\n

\n\n" read 226 bytes Conn close 400

opening connection to ebs.shasteel.cn:443... opened starting SSL for ebs.shasteel.cn:443... SSL established <- "GET / HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: /\r\nUser-Agent: Ruby\r\nHost: ebs.shasteel.cn\r\n\r\n" -> "HTTP/1.1 200 OK\r\n" -> "Date: Wed, 03 Sep 2014 11:43:55 GMT\r\n" -> "Server: Apache/2.2.17 (Win32) mod_ssl/2.2.17 OpenSSL/0.9.8o\r\n" -> "Accept-Ranges: bytes\r\n" -> "Content-Length: 341\r\n" -> "Last-Modified: Fri, 16 Dec 2011 01:35:36 GMT\r\n" -> "X-Powered-By: Servlet/2.4 JSP/2.0\r\n" -> "Content-Type: text/html\r\n" -> "\r\n" reading 341 bytes... -> "" -> "<%@ page language=\"java\" contentType=\"text/html; charset=UTF-8\"\n pageEncoding=\"UTF-8\"%>\n<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3.org/TR/html4/loose.dtdn\">\\n... \n\t\t\n\t\t\n\n

\n\n

lubyruffy (LubyRuffy)

net/http 库的一个让人抓狂的 bug

Bad Request