Ruby 的 GC 不释放内存给回系统的？

aikko — Fri, 30 Aug 2013 11:35:08 +0800

最近我的一个 ruby on rails 的项目，部署采用的是（nginx+passenger），由于访问量大概增了 6 倍左右，出现了一个之前没有注意到的问题，就是每个 ruby 进程的内存一直不断地增长，有一次跑了 3 天没管，多个 ruby 进程从刚开始的 100 多 M 到达 500M 左右，不得已每天晚上 5 点左右重启一下服务以便释放内存。换了 ruby 2.0 有所改善，但是还是不明显。

跑了 11 几个小时左右

跑了 80 几个小时左右

在网上看到了Robbin 的这篇文章后，在生产环境选了一台机器观察下，我发现从来没有内存使用有降低的情况。于是开始猜测是不是没有触发 GC，我的 ruby gc 参数参考 37singals 和 twitter 的配置如下：

export RUBY_HEAP_MIN_SLOTS=1250000
export RUBY_HEAP_SLOTS_INCREMENT=100000
export RUBY_HEAP_SLOTS_GROWTH_FACTOR=1
export RUBY_GC_MALLOC_LIMIT=30000000
export RUBY_HEAP_FREE_MIN=12500

用 GC::Profiler 去观察 ruby 的 GC 情况，得到结果如下：

明显很多次自动触发了 GC，后面我自己强制调用 GC.start，Use Size 和 Total Size 都降低了很多，但是该 ruby 进程的内存占用量始终不降。（中间我观察有几次次非常小幅的下降----每次大概 1% 约 40m，但总体在增长）

当执行完 GC 后，如果在改 ruby 进程里面再次大量的分配对象时，整体内存增长不大，于是我猜测 ruby 所谓的 GC 是不是只是进程内部把一使用的内存回收以便下次再次分配时使用，但是没有释放给系统？求大神指点。

如何降低页面 render 时的耗时以及 CPU 资源

aikko — Wed, 26 Jun 2013 17:01:34 +0800

最近在做一个项目，最后瓶颈在 cpu 上面（机器配置：16 核，32G），基本当访问页面在 200req/s 的时候，16 个 cpu 基本都被占满，无法再提高响应能力。服务器部署的方式（nginx+passenger+redis+mysql）

passenger_max_pool_size 32;
passenger_min_instances 32;

渲染 view 的时间居然用到了 900 多毫秒

Completed 200 OK in 1443ms (Views: 937.2ms | ActiveRecord: 389.0ms)

小弟初次使用 rails，求大神帮忙分析下。

Started GET "/shows/19826" at 2013-06-26 16:50:06 +0800
Processing by ShowsController#show as HTML
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (1.5ms)
  Rendered shows/_user.html.erb (46.8ms)
  Rendered shows/show/_operate.html.erb (1.7ms)
  Rendered posts/_zhan.html.erb (3.9ms)
Read fragment views/post_82397_comments (0.7ms)
   (9.9ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82397
  Rendered shows/_comments.html.erb (377.2ms)
  Rendered shows/_first_floor.html.erb (453.8ms)
Read fragment views/user_28274_medals (0.4ms)
  Rendered shared/_medals.html.erb (0.6ms)
  Rendered shows/_user.html.erb (2.9ms)
  Rendered shows/show/_operate.html.erb (0.5ms)
   (7.6ms)  SELECT COUNT(*) FROM `wooers` WHERE `wooers`.`show_id` = 19826
  CACHE (0.0ms)  SELECT COUNT(*) FROM `wooers` WHERE `wooers`.`show_id` = 19826
  CACHE (0.0ms)  SELECT COUNT(*) FROM `wooers` WHERE `wooers`.`show_id` = 19826
   (51.5ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14411 AND `posts`.`item_type` = 'Wooer'
  Rendered shows/_choose_form.html.erb (1.5ms)
Read fragment views/index_show_19826_wooer_14411 (0.6ms)
   (54.9ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14604 AND `posts`.`item_type` = 'Wooer'
  Rendered shows/_choose_form.html.erb (0.1ms)
Read fragment views/index_show_19826_wooer_14604 (0.5ms)
   (45.8ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14762 AND `posts`.`item_type` = 'Wooer'
  Rendered shows/_choose_form.html.erb (1.4ms)
Read fragment views/index_show_19826_wooer_14762 (0.3ms)
   (47.9ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14803 AND `posts`.`item_type` = 'Wooer'
  Rendered shows/_choose_form.html.erb (1.5ms)
Read fragment views/index_show_19826_wooer_14803 (0.5ms)
   (52.4ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14970 AND `posts`.`item_type` = 'Wooer'
  Rendered shows/_choose_form.html.erb (1.5ms)
Read fragment views/index_show_19826_wooer_14970 (0.4ms)
  Rendered shows/_wooer.html.erb (364.4ms)
  Rendered shows/_wooers.html.erb (435.4ms)
Read fragment views/post_82398_comments (0.4ms)
   (10.0ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82398
  Rendered shows/_comments.html.erb (15.3ms)
  Rendered shows/_second_floor.html.erb (487.2ms)
Read fragment views/user_645_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.8ms)
  Rendered shows/_user.html.erb (5.9ms)
  Rendered shows/show/_operate.html.erb (0.6ms)
  Rendered posts/_zhan.html.erb (2.7ms)
Read fragment views/post_82401_comments (0.5ms)
   (9.8ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82401
  Rendered shows/_comments.html.erb (15.1ms)
  Rendered shows/_common_floor.html.erb (34.1ms)
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.7ms)
  Rendered shows/_user.html.erb (5.4ms)
  Rendered shows/show/_operate.html.erb (0.6ms)
  Rendered posts/_zhan.html.erb (2.2ms)
Read fragment views/post_82402_comments (0.4ms)
   (9.9ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82402
  Rendered shows/_comments.html.erb (14.1ms)
  Rendered shows/_common_floor.html.erb (25.6ms)
Read fragment views/user_28274_medals (0.4ms)
  Rendered shared/_medals.html.erb (0.6ms)
  Rendered shows/_user.html.erb (4.7ms)
  Rendered shows/show/_operate.html.erb (1.0ms)
  Rendered posts/_zhan.html.erb (2.5ms)
Read fragment views/post_82403_comments (0.4ms)
   (9.9ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82403
  Rendered shows/_comments.html.erb (14.0ms)
  Rendered shows/_common_floor.html.erb (25.2ms)
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.7ms)
  Rendered shows/_user.html.erb (5.1ms)
  Rendered shows/show/_operate.html.erb (0.6ms)
  Rendered posts/_zhan.html.erb (2.2ms)
Read fragment views/post_82404_comments (0.4ms)
   (9.8ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82404
  Rendered shows/_comments.html.erb (14.7ms)
  Rendered shows/_common_floor.html.erb (25.8ms)
Read fragment views/user_28274_medals (0.4ms)
  Rendered shared/_medals.html.erb (0.6ms)
  Rendered shows/_user.html.erb (4.9ms)
  Rendered shows/show/_operate.html.erb (0.7ms)
  Rendered posts/_zhan.html.erb (97.0ms)
Read fragment views/post_82405_comments (0.7ms)
   (12.7ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82405
  Rendered shows/_comments.html.erb (19.4ms)
  Rendered shows/_common_floor.html.erb (125.3ms)
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.7ms)
  Rendered shows/_user.html.erb (5.8ms)
  Rendered shows/show/_operate.html.erb (0.7ms)
  Rendered posts/_zhan.html.erb (2.9ms)
Read fragment views/post_82406_comments (0.6ms)
   (10.1ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82406
  Rendered shows/_comments.html.erb (14.6ms)
  Rendered shows/_common_floor.html.erb (27.4ms)
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.8ms)
  Rendered shows/_user.html.erb (5.6ms)
  Rendered shows/show/_operate.html.erb (0.6ms)
  Rendered posts/_zhan.html.erb (2.5ms)
Read fragment views/post_82407_comments (0.5ms)
   (9.8ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82407
  Rendered shows/_comments.html.erb (14.1ms)
  Rendered shows/_common_floor.html.erb (26.1ms)
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.7ms)
  Rendered shows/_user.html.erb (5.0ms)
  Rendered shows/show/_operate.html.erb (0.6ms)
  Rendered posts/_zhan.html.erb (2.5ms)
Read fragment views/post_82408_comments (0.5ms)
   (10.6ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82408
  Rendered shows/_comments.html.erb (14.6ms)
  Rendered shows/_common_floor.html.erb (25.8ms)
Read fragment views/user_28274_medals (0.5ms)
  Rendered shared/_medals.html.erb (0.6ms)
  Rendered shows/_user.html.erb (5.1ms)
  Rendered shows/show/_operate.html.erb (0.6ms)
  Rendered posts/_zhan.html.erb (3.1ms)
Read fragment views/post_82409_comments (0.5ms)
   (11.0ms)  SELECT COUNT(*) FROM `comments` WHERE `comments`.`post_id` = 82409
  Rendered shows/_comments.html.erb (15.2ms)
  Rendered shows/_common_floor.html.erb (27.3ms)
  Rendered shows/_page.html.erb (5.0ms)
  Rendered shows/show.html.erb within layouts/application (1315.0ms)
  Rendered layouts/_navigation.html.erb (5.9ms)
**Completed 200 OK in 1443ms (Views: 937.2ms | ActiveRecord: 389.0ms)**

**得到各位大神的帮助，现在交优化作业，同一个页面的请求（热门帖子特例），优化效果非常明显，响应时间从 1443ms 降到 374ms，对应的日志后面有贴出。从 new relic 的统计来看，之前整站的平均响应时间为 140ms 左右，优化后降低到 80ms 左右。刚开始使用 rails，犯了很多不该犯的错误，就当一个反面教材吧，希望能帮助像我这样的菜鸟们能更好更正确的使用 rails。

一些关键请求前后响应时间数据见下图：**

优化前：

优化后：

主要从下面几个方面进行优化，有不足的地方希望大家指出 1、该加上 counter_cache 的地方全部都加上 counter_cache 2、对常用的非主键查询加索引，例如查用户的发帖，查帖子的评论等 3、使用 cache_digests，充分 cache 住 view 中的内容 4、尽量不在 view 中设计数据库查询操作，修改页面结构减少 render 5、对于一些实时性要求不高的页面使用 action_cache（由于我的应用必须登录才能使用，action_cache 不跳过验证的）

在此感谢@nihaokid, @huacnlee, @hooopo, @kenshin54, @leonworld等各位的指点

Started GET "/shows/19826" at 2013-07-03 17:57:05 +0800
Processing by ShowsController#show as HTML
  Parameters: {"id"=>"19826"}
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (1.1ms)
  Rendered shows/show/_operate.html.erb (0.9ms)
  Rendered posts/_zhan.html.erb (2.5ms)
Read fragment views/posts/82397-20130622231908/posts/82398-20130620091124/posts/82402-20130622231900/posts/82403-20130622232014/posts/82404-20130622232021/posts/82405-20130622232023/posts/82406-20130622232025/posts/82407-20130622232027/posts/82408-20130622232030/posts/82409-20130622232032/posts/82410-20130622232034/posts/82411-20130622232036/posts/82412-20130622232038/posts/82413-20130622232040/posts/82414-20130622232043/posts/82415-20130622232045/posts/82416-20130622232048/posts/82417-20130622232050/posts/82418-20130622232052/posts/82419-20130622232054/posts/82420-20130622232056/posts/82421-20130622232058/posts/82422-20130622232100/posts/82423-20130622232102/posts/82424-20130622232104/posts/82425-20130622232106/ee5e1083a00aad667f6151cd4224c7d8 (0.5ms)
Read fragment views/post_82397_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.4ms)
  Rendered shows/_comments.html.erb (0.6ms)
  Rendered shows/_first_floor.html.erb (13.4ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.8ms)
Read fragment views/posts/82398-20130620091124/4797c0bc70520a2ac0bf90fc7a8e6395 (0.4ms)
   (0.8ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14762 AND `posts`.`item_type` = 'Wooer'
Read fragment views/index_show_19826_wooer_14762_info/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
  Rendered shows/_choose_form.html.erb (1.4ms)
Read fragment views/index_show_19826_wooer_14762/f8384c485db4fe49a341041679e8e8d9 (0.4ms)
   (0.4ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14803 AND `posts`.`item_type` = 'Wooer'
Read fragment views/index_show_19826_wooer_14803_info/f8384c485db4fe49a341041679e8e8d9 (0.4ms)
  Rendered shows/_choose_form.html.erb (1.2ms)
Read fragment views/index_show_19826_wooer_14803/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
   (0.3ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 14970 AND `posts`.`item_type` = 'Wooer'
Read fragment views/index_show_19826_wooer_14970_info/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
  Rendered shows/_choose_form.html.erb (1.5ms)
Read fragment views/index_show_19826_wooer_14970/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
   (0.3ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 17379 AND `posts`.`item_type` = 'Wooer'
Read fragment views/index_show_19826_wooer_17379_info/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
  Rendered shows/_choose_form.html.erb (1.2ms)
Read fragment views/index_show_19826_wooer_17379/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
   (0.5ms)  SELECT COUNT(*) FROM `posts` WHERE `posts`.`item_id` = 17622 AND `posts`.`item_type` = 'Wooer'
Read fragment views/index_show_19826_wooer_17622_info/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
  Rendered shows/_choose_form.html.erb (1.1ms)
Read fragment views/index_show_19826_wooer_17622/f8384c485db4fe49a341041679e8e8d9 (0.5ms)
  Rendered shows/_wooer.html.erb (24.5ms)
  Rendered shows/_wooers.html.erb (109.0ms)
Read fragment views/post_82398_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.8ms)
  Rendered shows/_second_floor.html.erb (130.3ms)
Read fragment views/users/645-20130621101340/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.9ms)
Read fragment views/posts/82401-20130620091200/df576356e2de4bcdb9c7b0559bb5ba4c (0.4ms)
  Rendered posts/_zhan.html.erb (2.3ms)
Read fragment views/post_82401_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.7ms)
  Rendered shows/_common_floor.html.erb (7.9ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.6ms)
  Rendered shows/_user.html.erb (1.1ms)
Read fragment views/posts/82402-20130622231900/df576356e2de4bcdb9c7b0559bb5ba4c (0.6ms)
  Rendered posts/_zhan.html.erb (2.1ms)
Read fragment views/post_82402_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.7ms)
  Rendered shows/_common_floor.html.erb (8.1ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.9ms)
Read fragment views/posts/82403-20130622232014/df576356e2de4bcdb9c7b0559bb5ba4c (0.5ms)
  Rendered posts/_zhan.html.erb (2.3ms)
Read fragment views/post_82403_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.7ms)
  Rendered shows/_common_floor.html.erb (7.6ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.8ms)
Read fragment views/posts/82404-20130622232021/df576356e2de4bcdb9c7b0559bb5ba4c (0.4ms)
  Rendered posts/_zhan.html.erb (2.4ms)
Read fragment views/post_82404_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.8ms)
  Rendered shows/_common_floor.html.erb (7.6ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.8ms)
Read fragment views/posts/82405-20130622232023/df576356e2de4bcdb9c7b0559bb5ba4c (0.4ms)
  Rendered posts/_zhan.html.erb (2.1ms)
Read fragment views/post_82405_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.6ms)
  Rendered shows/_comments.html.erb (0.9ms)
  Rendered shows/_common_floor.html.erb (7.8ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.8ms)
Read fragment views/posts/82406-20130622232025/df576356e2de4bcdb9c7b0559bb5ba4c (0.4ms)
  Rendered posts/_zhan.html.erb (2.2ms)
Read fragment views/post_82406_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.4ms)
  Rendered shows/_comments.html.erb (0.6ms)
  Rendered shows/_common_floor.html.erb (7.4ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.6ms)
  Rendered shows/_user.html.erb (0.9ms)
Read fragment views/posts/82407-20130622232027/df576356e2de4bcdb9c7b0559bb5ba4c (0.5ms)
  Rendered posts/_zhan.html.erb (2.2ms)
Read fragment views/post_82407_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.7ms)
  Rendered shows/_common_floor.html.erb (7.6ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.5ms)
  Rendered shows/_user.html.erb (0.8ms)
Read fragment views/posts/82408-20130622232030/df576356e2de4bcdb9c7b0559bb5ba4c (0.5ms)
  Rendered posts/_zhan.html.erb (2.4ms)
Read fragment views/post_82408_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.5ms)
  Rendered shows/_comments.html.erb (0.7ms)
  Rendered shows/_common_floor.html.erb (7.7ms)
Read fragment views/users/28274-20130614075045/d23d462cb66fbbd1ff9b470e0ca7a163 (0.6ms)
  Rendered shows/_user.html.erb (1.0ms)
Read fragment views/posts/82409-20130622232032/df576356e2de4bcdb9c7b0559bb5ba4c (0.4ms)
  Rendered posts/_zhan.html.erb (2.5ms)
Read fragment views/post_82409_comments/98cc0c2a9fa04cb178257dab2f371b63 (0.4ms)
  Rendered shows/_comments.html.erb (0.6ms)
  Rendered shows/_common_floor.html.erb (8.2ms)
  Rendered shows/_page.html.erb (2.8ms)
  Rendered shows/show.html.erb within layouts/application (241.6ms)
  Rendered layouts/_navigation.html.erb (5.5ms)
Completed 200 OK in 374ms (Views: 247.9ms | ActiveRecord: 2.4ms)```

一个 utf-16 的 byte array 怎么转换成 string？

aikko — Tue, 25 Jun 2013 16:15:04 +0800

从一个 base64 编码的 string，通过.unpack("m*") 获得的是一个 utf-16 的 byte array，如何转成相对应的 string

aikko (aikko)

Ruby 的 GC 不释放内存给回系统的？

如何降低页面 render 时的耗时以及 CPU 资源

一个 utf-16 的 byte array 怎么转换成 string？