翻译 Redis-objects 介绍

xiajian · 2015年05月25日 · 最后由 zcy4coding 回复于 2018年04月23日 · 5977 次阅读

注: 官方 Readme 的翻译,译的不好,还请各位指正

Redis::Objects - Map Redis types directly to Ruby objects , 将 Redis 类型直接映射为 Ruby 对象。

This is not an ORM. People that are wrapping ORM’s around Redis are missing the point.

这并不是 ORM,想要在 Redis 上包装 ORM 的人未得要领。

The killer feature of Redis is that it allows you to perform atomic operations on individual data structures, like counters, lists, and sets. The atomic part is HUGE. Using an ORM wrapper that retrieves a "record", updates values, then sends those values back, removes the atomicity, cutting the nuts off the major advantage of Redis. Just use MySQL, k?

Redis 的杀手级特性是:可以在单独的数据结构 (比如计数器,列表,集合) 上执行原子性的操作,其中原子性部分作用是巨大的。 使用 ORM 层来检索记录,更新值,回送值,删除值将会抵消原子性的作用,从而不能发挥 Redis 的巨大优势。要 ORM,直接用 MySQL 就好了。

This gem provides a Rubyish interface to Redis, by mapping Redis data types to Ruby objects, via a thin layer over the redis gem. It offers several advantages over the lower-level redis-rb API:

该 gem 提供了 Redis 的 Rubyish 接口,将Redis data types映射为 Ruby 对象。通过对redisgem 的薄封装,相对 redis-rb API, redis-objects 提供如下的一些优势:

  1. Easy to integrate directly with existing ORMs - ActiveRecord, DataMapper, etc. Add counters to your model!
  2. 很容易集成到现有的 ORM 中,诸如 ActiveRecord, DataMapper 之类。并向模型中添加计数器。
  3. Complex data structures are automatically Marshaled (if you set :marshal => true)
  4. 复杂的数据结构将自动序列化 (设置:marshal => true )
  5. Integers are returned as integers, rather than '17'
  6. 整数就按整数返回,而不是'17'
  7. Higher-level types are provided, such as Locks, that wrap multiple calls
  8. 提供高层次的类型,比如 Locks(锁) - 包装了多个调用

This gem originally arose out of a need for high-concurrency atomic operations; for a fun rant on the topic, see An Atomic Rant, or scroll down to Atomic Counters and Locks in this README.

gem 起源于高并发的原子操作的需求; 关于该话题的有趣咆哮 - An Atomic Rant, 或者滚动到 Atomic Counters and Locks的章节处。

There are two ways to use Redis::Objects, either as an include in a model class (to tightly integrate with ORMs or other classes), or standalone by using classes such as Redis::List and Redis::SortedSet.

存在两种使用Redis::Objects的方式:

  • 在 model 类中使用,即与 ORM 或其他类紧密集合
  • 单独使用诸如Redis::ListRedis::SortedSet这样的类

安装和设置 (Installation and Setup)

在 Gemfile 中添加如下内容:

gem 'redis-objects'

Redis::Objects needs a handle created by Redis.new or a ConnectionPool:

Redis::Objects 需要由Redis.new创建的句柄,或者一个连接池 (ConnectionPool):

The recommended approach is to use a ConnectionPool since this guarantees that most timeouts in the redis client do not pollute your existing connection. However, you need to make sure that both :timeout and :size are set appropriately in a multithreaded environment.

推荐使用ConnectionPool,这样能够保证redis客户端不会污染现有的连接。但是,在多线程的环境中,需要确保正确的设置:timeout:size

require 'connection_pool'
Redis::Objects.redis = ConnectionPool.new(size: 5, timeout: 5) { Redis.new(:host => '127.0.0.1', :port => 6379) }

Redis::Objects can also default to Redis.current if Redis::Objects.redis is not set.

如果Redis::Objects.redis没有设置,Redis::Objects 将会默认使用Redis.current

Redis.current = Redis.new(:host => '127.0.0.1', :port => 6379)

如果在 Rails 中,这些配置可以放在config/initializers/redis.rb

Remember you can use Redis::Objects in any Ruby code. There are no dependencies on Rails. Standalone, Sinatra, Resque - no problem.

记住,可以在任意的 Ruby 代码中使用 Redis::Objects,其并不依赖 Rails。单独使用,在 Sinatra 或 Resque 中,都没有问题。

Alternatively, you can set the redis handle directly:

当然,也可直接是设置redis句柄:

Redis::Objects.redis = Redis.new(...)

Finally, you can even set different handles for different classes:

最后,可以为不同的类设置不同的句柄:

class User
  include Redis::Objects
end
class Post
  include Redis::Objects
end

# you can also use a ConnectionPool here as well
User.redis = Redis.new(:host => '1.2.3.4')
Post.redis = Redis.new(:host => '5.6.7.8')

As of 0.7.0, redis-objects now autoloads the appropriate Redis::Whatever classes on demand. Previous strategies of individually requiring redis/list or redis/set are no longer required.

0.7.0redis-objects可以在恰当的时候,自动加载Redis::Whatever类。先前需要单独引入redis/listredis/set的方式不再需要。

场景 1:在 Model 类中包含 Redis::Objects

Including Redis::Objects in a model class makes it trivial to integrate Redis types with an existing ActiveRecord, DataMapper, Mongoid, or similar class. Redis::Objects will work with any class that provides an id method that returns a unique value. Redis::Objects automatically creates keys that are unique to each object, in the format:

在模型类中包含 Redis::Objects,从而将 Redis 类型整合进现存的 ActiveRecord,DataMapper,Mongoid,或单个类中。 Redis::Objects 可以在任何提供了 id 方法 (返回独一无二的值) 的类中工作. Redis::Objects 将自动创建针对每个对象创建独特的键, 以如下的格式:

model_name🆔field_name

For illustration purposes, consider this stub class:

为了演示处理,考虑如下的桩类:

class User
  include Redis::Objects
  counter :my_posts
  def id
    1
  end
end

user = User.new
user.id  # 1
user.my_posts.increment
user.my_posts.increment
user.my_posts.increment
puts user.my_posts.value # 3
user.my_posts.reset
puts user.my_posts.value # 0
user.my_posts.reset 5
puts user.my_posts.value # 5

Here's an example that integrates several data types with an ActiveRecord model:

如下,是将一些 Redis 数据类型集成到 ActiveRecord 模型中的例子:

class Team < ActiveRecord::Base
  include Redis::Objects

  lock :trade_players, :expiration => 15  # sec
  value :at_bat
  counter :hits
  counter :runs
  counter :outs
  counter :inning, :start => 1
  list :on_base
  list :coaches, :marshal => true
  set  :outfielders
  hash_key :pitchers_faced  # "hash" is taken by Ruby
  sorted_set :rank, :global => true
end

列表类型与 Ruby 的数组操作类似:

@team = Team.find_by_name('New York Yankees')
@team.on_base << 'player1'
@team.on_base << 'player2'
@team.on_base << 'player3'
@team.on_base    # ['player1', 'player2', 'player3']
@team.on_base.pop
@team.on_base.shift
@team.on_base.length  # 1
@team.on_base.delete('player2')

集合的操作也类似:

@team.outfielders << 'outfielder1'
@team.outfielders << 'outfielder2'
@team.outfielders << 'outfielder1'   # dup ignored
@team.outfielders  # ['outfielder1', 'outfielder2']
@team.outfielders.each do |player|
  puts player
end
player = @team.outfielders.detect{|of| of == 'outfielder2'}

可以在对象之间取交集和并集操作,非常酷:

@team1.outfielders | @team2.outfielders   # outfielders on both teams
@team1.outfielders & @team2.outfielders   # in baseball, should be empty :-)

Counters can be atomically incremented/decremented (but not assigned):

计数器可以原子的增加或减少,但不能赋值:

@team.hits.increment  # or incr
@team.hits.decrement  # or decr
@team.hits.incr(3)    # add 3
@team.runs = 4        # exception

定义一个像id变量那样的方法非常的容易:

class User
  include Redis::Objects
  redis_id_field :uid
  counter :my_posts
end

user.uid                # 195137a1bdea4473
user.my_posts.increment # 1

Finally, for free, you get a redis method that points directly to a Redis connection:

最后,将免费的得到一个指向 Redis 连接的redis方法:

Team.redis.get('somekey')
@team = Team.new
@team.redis.get('somekey')
@team.redis.smembers('someset')

可以使用redis直接调用任何Redis API command

Option 2: Standalone Usage

There is a Ruby class that maps to each Redis type, with methods for each Redis API command.

有一些将 Ruby 类直接映射成 Redis 类型,并提供了针对Redis API command的相应方法。

Note that calling new does not imply it's actually a "new" value - it just creates a mapping between that Ruby object and the corresponding Redis data structure, which may already exist on the redis-server.

注意,这里调用new方法,并不意味创建一个"新"的值 - 它只是创建了一个从 Redis 数据结构到 Ruby 对象之间的映射, 而前者已经在redis-server中存在。

Counters

counter_name是 Redis 中存储的键。

@counter = Redis::Counter.new('counter_name')
@counter.increment  # or incr
@counter.decrement  # or decr
@counter.increment(3)
puts @counter.value

gem 也提供了干净的方法去处理原子性的块:

@counter.increment do |val|
  raise "Full" if val > MAX_VAL  # rewind counter
end

See the section on Atomic Counters and Locks for cool uses of atomic counter blocks.

查看Atomic Counters and Locks章节,找到更多很酷的原子计数块。

Locks

A convenience class that wraps the pattern of using setnx to perform locking.

Lock 是包装了using setnx to perform locking模式的极其方便的类:

@lock = Redis::Lock.new('serialize_stuff', :expiration => 15, :timeout => 0.1)
@lock.lock do
  # do work
end

This can be especially useful if you're running batch jobs spread across multiple hosts.

这在运行分散在多台主机上的批量 job 时,非常有用。

Values

Value 对象中,简单对象很好处理:

@value = Redis::Value.new('value_name')
@value.value = 'a'
@value.delete

复杂的对象使用:marshal => true也没多大问题:

@account = Account.create!(params[:account])
@newest  = Redis::Value.new('newest_account', :marshal => true)
@newest.value = @account.attributes
puts @newest.value['username']

Lists

列表的操作与 Ruby 数组类似:

@list = Redis::List.new('list_name')
@list << 'a'
@list << 'b'
@list.include? 'c'   # false
@list.values  # ['a','b']
@list << 'c'
@list.delete('c')
@list[0]
@list[0,1]
@list[0..1]
@list.shift
@list.pop
@list.clear
# etc

可以限定列表的大小,使其仅存放 N 个元素:

# Only holds 10 elements, throws out old ones when you reach :maxlength.
@list = Redis::List.new('list_name', :maxlength => 10)

Complex data types are now handled with :marshal => true:

复杂的数据可以通过设置:marshal => true:

@list = Redis::List.new('list_name', :marshal => true)
@list << {:name => "Nate", :city => "San Diego"}
@list << {:name => "Peter", :city => "Oceanside"}
@list.each do |el|
  puts "#{el[:name]} lives in #{el[:city]}"
end

Hashes

Hashes work like a Ruby Hash, with a few Redis-specific additions. (The class name is "HashKey" not just "Hash", due to conflicts with the Ruby core Hash class in other gems.)

Hash 与 Ruby 的 Ruby Hash作用类似,并带有一些 Redis 特定的附加特性。类名为 HashKey,而不是 Hash,是为了避免与 Ruby core 的冲突。

@hash = Redis::HashKey.new('hash_name')
@hash['a'] = 1
@hash['b'] = 2
@hash.each do |k,v|
  puts "#{k} = #{v}"
end
@hash['c'] = 3
puts @hash.all  # {"a"=>"1","b"=>"2","c"=>"3"}
@hash.clear

Redis 也添加了自增以及 bulk 的操作 (??):

@hash.incr('c', 6)  # 9
@hash.bulk_set('d' => 5, 'e' => 6)
@hash.bulk_get('d','e')  # "5", "6"

Remember that numbers become strings in Redis. Unlike with other Redis data types, redis-objects can't guess at your data type in this situation, since you may actually mean to store "1.5".

注意: Redis 中将字符存作数字。与其他 Redis 数据类型不同,redis-objects并在这种情况下,猜测数据类型,因为你有可能想 存的是"1.5"。

Sets

Sets work like the Ruby Set class. They are unordered, but guarantee uniqueness of members.

Set 类型与 Ruby 的Set类型相似,其是无序且不重复的。

@set = Redis::Set.new('set_name')
@set << 'a'
@set << 'b'
@set << 'a'  # dup ignored
@set.member? 'c'      # false
@set.members          # ['a','b']
@set.members.reverse  # ['b','a']
@set.each do |member|
  puts member
end
@set.clear
# etc

可以很方便的执行交并补差等运算:

@set1 = Redis::Set.new('set1')
@set2 = Redis::Set.new('set2')
@set3 = Redis::Set.new('set3')
members = @set1 & @set2   # intersection
members = @set1 | @set2   # union
members = @set1 + @set2   # union
members = @set1 ^ @set2   # difference
members = @set1 - @set2   # difference
members = @set1.intersection(@set2, @set3)  # multiple
members = @set1.union(@set2, @set3)         # multiple
members = @set1.difference(@set2, @set3)    # multiple

或者,将其存到 Redis 中:

@set1.interstore('intername', @set2, @set3)
members = @set1.redis.get('intername')
@set1.unionstore('unionname', @set2, @set3)
members = @set1.redis.get('unionname')
@set1.diffstore('diffname', @set2, @set3)
members = @set1.redis.get('diffname')

同上,复杂的数据类型可以使用:marshal => true :

@set1 = Redis::Set.new('set1', :marshal => true)
@set2 = Redis::Set.new('set2', :marshal => true)
@set1 << {:name => "Nate",  :city => "San Diego"}
@set1 << {:name => "Peter", :city => "Oceanside"}
@set2 << {:name => "Nate",  :city => "San Diego"}
@set2 << {:name => "Jeff",  :city => "Del Mar"}

@set1 & @set2  # Nate
@set1 - @set2  # Peter
@set1 | @set2  # all 3 people

Sorted Sets

Due to their unique properties, Sorted Sets work like a hybrid between a Hash and an Array. You assign like a Hash, but retrieve like an Array:

由于有序集合独特的特性,其更像哈希 (Hash) 和数组的混合体 - 即像 Hash 那样赋值,像数组那样检索:

@sorted_set = Redis::SortedSet.new('number_of_posts') # 建立键名为`number_of_posts`的有序集合
@sorted_set['Nate']  = 15
@sorted_set['Peter'] = 75
@sorted_set['Jeff']  = 24

# Array access to get sorted order
@sorted_set[0..2]           # => ["Nate", "Jeff", "Peter"]
@sorted_set[0,2]            # => ["Nate", "Jeff"]

@sorted_set['Peter']        # => 75
@sorted_set['Jeff']         # => 24
@sorted_set.score('Jeff')   # same thing (24),这里说,score与[]方法作用相同

@sorted_set.rank('Peter')   # => 2
@sorted_set.rank('Jeff')    # => 1

@sorted_set.first           # => "Nate"
@sorted_set.last            # => "Peter"
@sorted_set.revrange(0,2)   # => ["Peter", "Jeff", "Nate"]

@sorted_set['Newbie'] = 1
@sorted_set.members         # => ["Newbie", "Nate", "Jeff", "Peter"]
@sorted_set.members.reverse # => ["Peter", "Jeff", "Nate", "Newbie"]

@sorted_set.rangebyscore(10, 100, :limit => 2)   # => ["Nate", "Jeff"]
@sorted_set.members(:with_scores => true)        # => [["Newbie", 1], ["Nate", 16], ["Jeff", 28], ["Peter", 76]]

# atomic increment
@sorted_set.increment('Nate')
@sorted_set.incr('Peter')   # shorthand
@sorted_set.incr('Jeff', 4)

The other Redis Sorted Set commands are supported as well; see Sorted Sets API.

其他的 Redis 有序集合的命令也支持,具体参考Sorted Sets API

Atomic Counters and Locks

You are probably not handling atomicity correctly in your app. For a fun rant on the topic, see An Atomic Rant.

你可能不能在应用中正确的处理原子性。正如那个有趣的咆哮所言: An Atomic Rant

Atomic counters are a good way to handle concurrency:

原子计数是处理并发的好方法,例如,下面代码中增加活跃玩家的代码:

@team = Team.find(1)
if @team.drafted_players.increment <= @team.max_players
  # do stuff
  @team.team_players.create!(:player_id => 221)
  @team.active_players.increment
else
  # reset counter state
  @team.drafted_players.decrement
end

An atomic block gives you a cleaner way to do the above. Exceptions or returning nil will rewind the counter back to its previous state:

使用 原子块 可以更加简洁的处理上述情况。异常或返回为 nil 会将计数器回滚到先前的状态:

@team.drafted_players.increment do |val|
  raise Team::TeamFullError if val > @team.max_players  # rewind
  @team.team_players.create!(:player_id => 221)
  @team.active_players.increment
end

Here's a similar approach, using an if block (failure rewinds counter):

下面有个类似的方法,使用了 if 块来在失效时回滚计数器:

@team.drafted_players.increment do |val|
  if val <= @team.max_players
    @team.team_players.create!(:player_id => 221)
    @team.active_players.increment
  end
end

Class methods work too, using the familiar ActiveRecord counter syntax:

类方法使用了类似 ActiveRecord 的计数语法,来保证原子递增:

Team.increment_counter :drafted_players, team_id
Team.decrement_counter :drafted_players, team_id, 2
Team.increment_counter :total_online_players  # no ID on global counter

Class-level atomic blocks can also be used. This may save a DB fetch, if you have a record ID and don't need any other attributes from the DB table:

类层次的原子块也能很有用。如果有一条记录 ID,并且不需要从数据获取其他的属性,这将节省从 DB 中的读取数据:

Team.increment_counter(:drafted_players, team_id) do |val|
  TeamPitcher.create!(:team_id => team_id, :pitcher_id => 181)
  Team.increment_counter(:active_players, team_id)
end

Locks

Locks work similarly. On completion or exception the lock is released:

锁的工作机制很相似,在完成或异常发生时,锁将会被释放:

class Team < ActiveRecord::Base
  lock :reorder # 申明一个锁
end

@team.reorder_lock.lock do
  @team.reorder_all_players
end

类层次的锁,概念相似:

Team.obtain_lock(:reorder, team_id) do
  Team.reorder_all_players(team_id)
end

Lock expiration. Sometimes you want to make sure your locks are cleaned up should the unthinkable happen (server failure). You can set lock expirations to handle this. Expired locks are released by the next process to attempt lock. Just make sure you expiration value is sufficiently large compared to your expected lock time.

锁过期失效。 有时,需要确保在发生不可预测的事情 (服务器故障) 时,锁会自动的清除。这可以通过设置锁的过期。失效的锁将会在下一个进程尝试锁时,自动释放。 确保失效值大于你所期待的锁的时间。

class Team < ActiveRecord::Base
  lock :reorder, :expiration => 15.minutes
end

Keep in mind that true locks serialize your entire application at that point. As such, atomic counters are strongly preferred.

谨记,锁会在某个点序列化整个应用程序。所以,更加推荐原子操作。

过期 (Expiration)

Use :expiration and :expireat options to set default expiration.

可以使用:expiration:expireat来设置默认的过期时间:

value :value_with_expiration, :expiration => 1.hour
value :value_with_expireat, :expireat => Time.now + 1.hour

作者 Author

Copyright (c) 2009-2013 Nate Wiger. All Rights Reserved. Released under the Artistic License.

借道问下,这个 gem 包是否线程安全?因为一个项目里面有用,puma 做 server,有并发量大的场景。

#1 楼 @as181920 如果自己写连接语句的话,那可能就要自己负责处理线程安全之类的了。gem 包推荐使用连接池,这样,会安全一点吧。具体的,我也没试过。不过 Rails 4 不是内置了线程安全的吗?

如上文:

推荐使用[ConnectionPool](https://github.com/mperham/connection_pool),这样能够保证redis客户端不会污染现有的连接。但是,在多线程的环境中,需要确保正确的设置:timeout和:size。

#2 楼 @xiajian Rails 框架线程安全不代表自己的应用是线程安全的。

How Do I Know Whether My Rails App Is Thread-safe or Not? https://bearmetal.eu/theden/how-do-i-know-whether-my-rails-app-is-thread-safe-or-not/

#3 楼 @rei 可能是我经验不足吧,不是特别明白线程安全相关的问题,也没遇到过这类问题。多线程这么麻烦,干脆直接用多进程得了。

xiajian 回复

redis-object 的 lock 和 rails 的 lock 命名冲突,比如 User.lock.find(1),本意是使用数据库的行级锁,这时调用的确是 redis-object 的 lock

最近看到这个 gem,进来讨论一下。 rails 的 orm 基于数据库已经有事物,锁和隔离级别了,是提供了解决并发问题的方法的。 这个 gem 的作者说了killer feature 是可以使你在一个 redis 内的一个单独数据结构上实现原子性操作解决并发问题。而不用像在 rails orm 事务中操作多个 model。 引入这个 gem 实际上是把并发问题从项目里的关系数据库转移到了 redis,这会增加项目里成员额外的学习成本,好处也不明显。

需要 登录 后方可回复, 如果你还没有账号请点击这里 注册