很多地方借鉴 scrapy，但比 scrapy 好的地方：
Scrapy 用得都是泪, 几个痛点:
基于这些, 我觉得 kimurai 相当不错了 (即使用的人不多)
lol...被迫删掉了，项目里借鉴了一些作者半年前给一家公司干 freelancer 活时候写的代码 具体看这吧：https://github.com/vfreefly/kimurai#repository-was-removed
however... 这里很多 forks: https://github.com/vfreefly/kimurai/network/members
Kimurai will be reopen soon
and will keep the same design and features. I have plans to rewrite a few parts of a framework so it will be 100% open source without any doubts. Also, there is work to do to allow run multiple crawlers from inside a single Ruby process (run crawlers using background jobs). And, there will be added "mechanize-only" mode (mechanize engine without Capybara dependency).