Ruby Ruby 3.2 YJIT, malloctrim, jemalloc 性能测试

kikyous · 2023年01月05日 · 最后由 xerox51 回复于 2023年01月22日 · 933 次阅读

测试脚本是个修改版的查找最大子串的代码:

f = open("./file.txt")
content = f.read()


def find_common_sub_str(s, t, threshold = 4)
    n = s.size
    m = t.size

    result = []

    # Create DP table
    dp = Array.new(2).map{|i| Array.new(m+1){0} }

    s.each_char.each_with_index do |s_char, s_index|
        i = s_index+1
        t.each_char.each_with_index do |t_char, t_index|
            j = t_index+1
            if s_char == t_char
                dp[i % 2][j] = dp[(i - 1) % 2][j - 1] + 1
                if (dp[i % 2][j] >= threshold) and ((i==n) or (j==m))
                    result.push([i,j,dp[i % 2][j]])
                end
            else
                dp[i % 2][j] = 0
                if dp[(i-1) % 2][j-1] >= threshold
                    result.push([i-1,j-1, dp[(i-1) % 2][j-1]])
                end
            end
        end
    end
    return result
end
print(find_common_sub_str(content[0,10000], content[0,10000], 25))

测试跑在 container 里,使用 earthly 运行:

VERSION 0.6

ruby32:
    FROM ruby:3.2-alpine
    ENV TIME="time result cmd: %C  real: %es  user: %Us  sys: %Ss  memory: %MKB  cpu: %P"
    COPY . .
    RUN --no-cache sh -c '/usr/bin/time ruby --yjit diff.rb'
    RUN --no-cache sh -c '/usr/bin/time ruby  diff.rb'

fullstaqruby32-jemalloc:
    FROM quay.io/evl.ms/fullstaq-ruby:3.2.0-jemalloc
    RUN apt-get update -yqq && apt-get install -yqq --no-install-recommends time
    ENV TIME="time result\ncmd:%C\nreal %es\nuser %Us \nsys  %Ss \nmemory:%MKB \ncpu %P"
    COPY . .
    RUN --no-cache bash -c '/usr/bin/time ruby --yjit diff.rb'
    RUN --no-cache bash -c '/usr/bin/time ruby diff.rb'

fullstaqruby32-malloctrim:
    FROM quay.io/evl.ms/fullstaq-ruby:3.2.0-malloctrim-stretch
    RUN apt-get update -yqq && apt-get install -yqq --no-install-recommends time
    ENV TIME="time result\ncmd:%C\nreal %es\nuser %Us \nsys  %Ss \nmemory:%MKB \ncpu %P"
    COPY . .
    RUN --no-cache bash -c '/usr/bin/time ruby --yjit diff.rb'
    RUN --no-cache bash -c '/usr/bin/time ruby diff.rb'

输出:

 1. Init 🚀
————————————————————————————————————————————————————————————————————————————————

           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)

 2. Build 🔧
————————————————————————————————————————————————————————————————————————————————

     ruby:3.2-alpine | --> Load metadata linux/amd64
q/e/fullstaq-ruby:3.2.0-jemalloc | --> Load metadata linux/amd64
q/e/fullstaq-ruby:3.2.0-malloctrim-stretch | --> Load metadata linux/amd64
+fullstaqruby32-jemalloc | --> FROM quay.io/evl.ms/fullstaq-ruby:3.2.0-jemalloc
             context | --> local context .
+fullstaqruby32-jemalloc | [          ]   0% resolve quay.io/evl.ms/fullstaq-ruby:3.2.0-jemalloc@sha256:d34dccb3cd5109ba13a4de9bc4be8d3ffc6b458d3b9[██████████] 100% resolve quay.io/evl.ms/fullstaq-ruby:3.2.0-jemalloc@sha256:d34dccb3cd5109ba13a4de9bc4be8d3ffc6b458d3b9c12abde6e4d39de7363c8
+fullstaqruby32-malloctrim | --> FROM quay.io/evl.ms/fullstaq-ruby:3.2.0-malloctrim-stretch
+fullstaqruby32-malloctrim | [          ]   0% resolve quay.io/evl.ms/fullstaq-ruby:3.2.0-malloctrim-stretch@sha256:0280d75ef0a524d180a3973224302cf5a[██████████] 100% resolve quay.io/evl.ms/fullstaq-ruby:3.2.0-malloctrim-stretch@sha256:0280d75ef0a524d180a3973224302cf5a1a9ae3b49032e7f4da4bb90bf11ad0f
             +ruby32 | --> FROM ruby:3.2-alpine
             +ruby32 | [          ]   0% resolve docker.io/library/ruby:3.2-alpine@sha256:a3130ddd747ae03e2a5ccc8b200840b5dc2d6804a021fdf6a0127[██████████] 100% resolve docker.io/library/ruby:3.2-alpine@sha256:a3130ddd747ae03e2a5ccc8b200840b5dc2d6804a021fdf6a012777c0d3f7a57
             context | [          ]   0% transferring .:
             context | [██████████] 100% transferring .:
             +ruby32 | *cached* --> COPY . .
+fullstaqruby32-jemalloc | *cached* --> RUN apt-get update -yqq && apt-get install -yqq --no-install-recommends time
+fullstaqruby32-jemalloc | *cached* --> COPY . .
             +ruby32 | --> RUN --no-cache sh -c '/usr/bin/time ruby --yjit diff.rb'
+fullstaqruby32-malloctrim | *cached* --> RUN apt-get update -yqq && apt-get install -yqq --no-install-recommends time
+fullstaqruby32-malloctrim | *cached* --> COPY . .
             ongoing | +ruby32 (17 seconds ago)
             +ruby32 | [[10000, 10000, 10000]]
             +ruby32 | time result cmd: ruby --yjit diff.rb  real: 18.48s  user: 18.40s  sys: 0.06s  memory: 92528KB  cpu: 99%
+fullstaqruby32-jemalloc | --> RUN --no-cache bash -c '/usr/bin/time ruby --yjit diff.rb'
             ongoing | +fullstaqruby32-jemalloc (17 seconds ago)
+fullstaqruby32-jemalloc | [[10000, 10000, 10000]]
+fullstaqruby32-jemalloc | time result
+fullstaqruby32-jemalloc | cmd:ruby --yjit diff.rb
+fullstaqruby32-jemalloc | real 17.25s
+fullstaqruby32-jemalloc | user 17.13s 
+fullstaqruby32-jemalloc | sys  0.10s 
+fullstaqruby32-jemalloc | memory:32456KB 
+fullstaqruby32-jemalloc | cpu 99%
+fullstaqruby32-malloctrim | --> RUN --no-cache bash -c '/usr/bin/time ruby --yjit diff.rb'
             ongoing | +fullstaqruby32-malloctrim (17 seconds ago)
+fullstaqruby32-malloctrim | [[10000, 10000, 10000]]
+fullstaqruby32-malloctrim | time result
+fullstaqruby32-malloctrim | cmd:ruby --yjit diff.rb
+fullstaqruby32-malloctrim | real 18.31s
+fullstaqruby32-malloctrim | user 18.24s 

+fullstaqruby32-malloctrim | sys  0.04s 
+fullstaqruby32-malloctrim | memory:23692KB 
+fullstaqruby32-malloctrim | cpu 99%
             +ruby32 | --> RUN --no-cache sh -c '/usr/bin/time ruby diff.rb'
             ongoing | +ruby32 (32 seconds ago)
             +ruby32 | [[10000, 10000, 10000]]
             +ruby32 | time result cmd: ruby diff.rb  real: 34.22s  user: 34.07s  sys: 0.14s  memory: 90000KB  cpu: 99%
+fullstaqruby32-jemalloc | --> RUN --no-cache bash -c '/usr/bin/time ruby diff.rb'
             ongoing | +fullstaqruby32-jemalloc (32 seconds ago)
+fullstaqruby32-jemalloc | [[10000, 10000, 10000]]
+fullstaqruby32-jemalloc | time result
+fullstaqruby32-jemalloc | cmd:ruby diff.rb
+fullstaqruby32-jemalloc | real 32.69s
+fullstaqruby32-jemalloc | user 32.61s 
+fullstaqruby32-jemalloc | sys  0.06s 
+fullstaqruby32-jemalloc | memory:32068KB 
+fullstaqruby32-jemalloc | cpu 99%
+fullstaqruby32-malloctrim | --> RUN --no-cache bash -c '/usr/bin/time ruby diff.rb'
             ongoing | +fullstaqruby32-malloctrim (32 seconds ago)
+fullstaqruby32-malloctrim | [[10000, 10000, 10000]]
+fullstaqruby32-malloctrim | time result
+fullstaqruby32-malloctrim | cmd:ruby diff.rb
+fullstaqruby32-malloctrim | real 33.97s
+fullstaqruby32-malloctrim | user 33.88s 
+fullstaqruby32-malloctrim | sys  0.09s 
+fullstaqruby32-malloctrim | memory:23080KB 
+fullstaqruby32-malloctrim | cpu 99%
              output | --> exporting outputs

结果

跑了三次

version time memory
ruby3.2 34.98s,34.10s,34.22s 89568KB,89584KB,90000KB
ruby3.2 --yjit 19.83s,17.96s,18.48s 92720KB,93280KB,92528KB
ruby3.2-malloctrim 31.91s,33.60s,33.97s 23240KB,23228KB,23080KB
ruby3.2-malloctrim --yjit 18.89s,18.27s,18.31s 23840KB,23736KB,23692KB
ruby3.2-jemalloc 32.17s,32.71s,32.69s 31824KB,32036KB,32068KB
ruby3.2-jemalloc --yjit 16.66s,17.10s,17.25s 32524KB,32424KB,32456KB

看的出来在此场景下 yjit 有超过 40% 的性能提升,jemalloc 和 malloctrim 对内存占用的优化很明显,甚至 malloctrim 的内存是最低的

把我的网站升级到 ruby 3.2 + yjit + jemalloc 了

你的代码缩进风格怎么像是在写 Python

需要 登录 后方可回复, 如果你还没有账号请 注册新账号