Rails 简单处理掉所有 Rails ActionController::RoutingError 的错误
项目上线以后，难免都会遇到爬虫，或某些奇怪的东西扫描网站，然后我们的 production.log 就会出现很多这样的错误信息：
Started GET "/register.php" for 110.75.186.226 at 2013-05-23 05:47:32 +0800
ActionController::RoutingError (No route matches [GET] "/register.php"):
actionpack (3.2.13) lib/action_dispatch/middleware/debug_exceptions.rb:21:in `call'
actionpack (3.2.13) lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
railties (3.2.13) lib/rails/rack/logger.rb:32:in `call_app'
railties (3.2.13) lib/rails/rack/logger.rb:16:in `block in call'
activesupport (3.2.13) lib/active_support/tagged_logging.rb:22:in `tagged'
railties (3.2.13) lib/rails/rack/logger.rb:16:in `call'
actionpack (3.2.13) lib/action_dispatch/middleware/request_id.rb:22:in `call'
rack (1.4.5) lib/rack/methodoverride.rb:21:in `call'
rack (1.4.5) lib/rack/runtime.rb:17:in `call'
activesupport (3.2.13) lib/active_support/cache/strategy/local_cache.rb:72:in `call'
rack (1.4.5) lib/rack/lock.rb:15:in `call'
rack-cache (1.2) lib/rack/cache/context.rb:136:in `forward'
rack-cache (1.2) lib/rack/cache/context.rb:245:in `fetch'
rack-cache (1.2) lib/rack/cache/context.rb:185:in `lookup'
rack-cache (1.2) lib/rack/cache/context.rb:66:in `call!'
rack-cache (1.2) lib/rack/cache/context.rb:51:in `call'
railties (3.2.13) lib/rails/engine.rb:479:in `call'
railties (3.2.13) lib/rails/application.rb:223:in `call'
railties (3.2.13) lib/rails/railtie/configurable.rb:30:in `method_missing'
虽然表面上没有什么大问题，但是这种错误的请求打乱了我们的 Log，尤其是当你用了 exception_notification 这类东西做提醒的时候，你的日常工作将会被这些打乱（当然可以在提交前屏蔽 ActionController::RoutingError 的错误，那是另外回事)。
只需要简单几步就能处理好
routes.rb 的最后加上
match '*path', via: :all, to: 'home#error_404'
home_controller.rb 加上
def error_404
render file: "#{Rails.root}/public/404.html", status: 404, layout: false
end
然后 Log 里面干净了：
Started HEAD "/register.php" for 127.0.0.1 at 2014-09-10 19:05:41 +0800
Processing by HomeController#error_404 as */*
Rendered public/404.html (0.6ms)
Completed 404 Not Found in 10ms (Views: 10ms | ActiveRecord: 0.0ms)
最麻烦的是 ArgumentError::InvalidByteSequence ... 不知有没有办法处理 原因是 Rack 层解析请求时候遇到非法的字符抛出，还没有到 Rails 的路由层呢
#5 楼 @small_fish__ 这个 Rack 和 Rails 两边相互推诿，Rails 那边认为是 Rack 应该处理这种情况，Rack 说这种事情跟他们无关
#5 楼 @small_fish__ #7 楼 @jasl rack-utf8_sanitizer 可以干掉这些非法字符。你说的 Rack::Utils，应该类似 escape_utils ... 它们不是一个事。
来一个测试，这里包含了非法字符
curl http://rails-bestpractices.com/search\?utf8\=%E2%9C%93\&search\=foo%E0\&commit\=Search
不要一直玩...!
昨天我还处理了一个 Rails 4.1 以前升级 4.1 后 Cookie 异常的问题，之前 Ruby China 后台捕捉到很多 JSON::ParserError
A JSON::ParserError occurred in articles#show:
795: unexpected token at '{I"session_id:ETI"%4caed1ded93646981589c00f1c56b926;'
app/controllers/articles_controller.rb:17:in `show'
https://github.com/ruby-china/ruby-china/commit/492fa7f94e1f2f6d445a65fbe18ddbd9da29d3ca
有些“爬虫”很不“规范”，对这种的我们用黑名单 + 简单行为识别，简单粗暴但有效：
# set in application.rb
# config.middleware.insert_after 'Rack::MethodOverride', 'BlockBadSpider'
class BlockBadSpider
def initialize(app)
@app = app
end
def call(env)
bad_spider_keywords = ["EasouSpider", "EMail Exractor"] # put more here is found other bad spider
# how to test: `curl -A 'EasouSpider' "http://localhost:3000"`
is_bad_spider = bad_spider_keywords.any? { |keyword| env["HTTP_USER_AGENT"].include?(keyword) }
if env["HTTP_USER_AGENT"].downcase.include?("spider") &&
env['CONTENT_TYPE'] == 'application/x-www-form-urlencoded' &&
env['REQUEST_METHOD'] == 'GET' &&
env['rack.input'] != nil &&
env["rack.request.form_input"] == env["rack.input"] &&
env["rack.request.form_vars"] != nil &&
(env["rack.request.form_hash"] == nil) || (env["rack.request.form_vars"] == 'NULL') # use 'NULL' make it easy to test with `curl`
acts_as_bad_spider = true
else
acts_as_bad_spider = false
end
if is_bad_spider or acts_as_bad_spider
# return 406 error when some request header is invalid
# https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#4xx_Client_Error
# 406 Not Acceptable
[406, {}, ["The requested resource is only capable of generating content not acceptable according to the Accept headers sent in the request (406)"]]
else
@app.call(env)
end
end
end
if ::Rails.env.production?
match '*path', via: :all, to: 'home#error_404'
end
需要加上环境，否则 rails 4.2 的 /rails/mailers 没法用
@linjunzhugg Please see 3.11 Route Globbing and Wildcard Segments
get 'photos/*other', to: 'photos#unknown' This route would match photos/12 or /photos/long/path/to/12, setting params[:other] to "12" or "long/path/to/12". The fragments prefixed with a star are called "wildcard segments"
#11 楼 @jasl 我的 https://github.com/lulalala/utf8-cleaner (fork) 會過濾掉 invalid characters
