服务器不上个监控总是没有安全感,偶尔出点问题,追踪起原因也不是很方便。经过在monit、god、eye之间选择,最终选择了 monit,最主要的优点就是占用内存很小,而且和 ruby 无关,所以也不用考虑 root 用户下 ruby 环境的问题。
monit的安装和简单使用就不做简单的介绍,大家可以在官网上查看。需要注意的就是建议使用 root 用户来安装 monit,这样就比较方便的使用 monit 启动其他用户下的服务,不用考虑权限问题。
mina,部署利器,具体使用方法可以到官网查看,不再介绍。
这里主要分享,使用 monit+mina 对服务器进行监控的一些监控和部署脚本,以及踩过的坑。
部分代码如下:
config/deploy.rb
Dir[File.dirname(__FILE__) + '/mina/*.rb'].each {|file| require file }
environments = {
'production1' => {
domain: '115.115.115.115',
branch: 'master',
env: 'production1',
path: "/var/app/wedding",
user: "root",
user_home: "/root",
rvm_home: "/usr/local/rvm",
thin_ports: 3000..3003
},
'production2' => {
domain: '222.222.222.222',
branch: 'master',
env: 'production2',
path: "/var/app/wedding",
user: "cloud",
user_home: "/home/cloud",
rvm_home: "/usr/local/rvm",
thin_ports: 2000..2007
}
}
rails_env = environments.keys.include?(ENV['RAILS_ENV']) ? ENV['RAILS_ENV'] : 'develop'
env_attrs = environments[rails_env]
branch = env_attrs[:branch]
domain = env_attrs[:domain]
rvm_home = env_attrs[:rvm_home]
rvm_path = rvm_home + "/bin/rvm"
env = env_attrs[:env]
path = env_attrs[:path]
user = env_attrs[:user]
rvm = "ruby-2.1.2"
thin_ports = env_attrs[:thin_ports]
user_home = env_attrs[:user_home]
set :rails_env, env
set :domain, domain
set :deploy_to, path
set :repository, '[email protected]:huoshaoyun/wedding-memo-rails.git'
set :branch, branch
set :ssh_options, '-A'
set :port, '29168' if rails_env == 'production2'
set :user, user
set :term_mode, :nil
set :rvm_path, rvm_path
## script templates
## 根据template文件生成脚本,需要设置必要的变量
set :rvm_home, rvm_home
set :user_home, user_home
set :rvm, rvm
set :path, env_attrs[:path]
set :thin_ports, thin_ports
config/templates/monit/monitrc.erb
#这里是monitrc的配置,省略1000行代码。
include /etc/monit/conf.d/*
config/templates/monit/mysql.erb
记得加上as uid user and gid usergroup
,这样就可以以非 root 用户启动服务。
check process mysqld with pidfile /var/run/mysqld/mysqld.pid
group mysql
start program = "/etc/init.d/mysql start" as uid <%= user %> and gid <%= user %>
stop program = "/etc/init.d/mysql stop" as uid <%= user %> and gid <%= user %>
if failed host 127.0.0.1 port 3306 then restart
if 5 restarts within 5 cycles then timeout
config/templates/monit/nginx.erb
check process nginx with pidfile /run/nginx.pid
start program = "/etc/init.d/nginx start" as uid <%= user %> and gid <%= user %>
stop program = "/etc/init.d/nginx stop" as uid <%= user %> and gid <%= user %>
config/templates/monit/redis.erb
# redis
check process redis with pidfile <%= user_home %>/pids/redis.pid
start = "/bin/bash -c '/bin/sh <%= user_home %>/mysh/startRedis.sh'" as uid <%= user %> and gid <%= user %>
stop program = "/etc/init.d/redis-server stop" as uid <%= user %> and gid <%= user %>
group redis
config/templates/monit/sidekiq.erb
check process sidekiq
with pidfile /var/app/wedding/shared/pids/sidekiq.pid
start program = "/etc/init.d/sidekiq start" with timeout 90 seconds
stop program = "/etc/init.d/sidekiq stop" with timeout 90 seconds
group sidekiq
config/templates/monit/thin.erb
<% thin_ports.each do |thin_port| %>
check process thin-<%= thin_port %> with pidfile /var/app/wedding/tmp/pids/thin.<%= thin_port %>.pid
start = "/bin/bash -c '/bin/sh <%= user_home %>/mysh/thin.<%= thin_port %>.sh start'" as uid <%= user %> and gid <%= user %>
stop = "/bin/bash -c '/bin/sh <%= user_home %>/mysh/thin.<%= thin_port %>.sh stop'" as uid <%= user %> and gid <%= user %>
if 3 restarts within 5 cycles then timeout
if failed port <%= thin_port %> protocol http with timeout 30 seconds for 2 cycles then restart
group thin
<% end %>
config/templates/mysh/redis-server.sh.erb
#!/bin/sh
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
# SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
# BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
# DAMAGE.
# 存在bug不能正常start,只能stop
### BEGIN INIT INFO
# Provides: redis-server
# Required-Start: $syslog
# Required-Stop: $syslog
# Should-Start: $local_fs
# Should-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: redis-server - Persistent key-value db
# Description: redis-server - Persistent key-value db
### END INIT INFO
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/bin/redis-server
DAEMON_ARGS=/etc/redis.conf
NAME=redis-server
DESC=redis-server
PIDFILE=<%= user_home %>/pids/redis.pid
test -x $DAEMON || exit 0
test -x $DAEMONBOOTSTRAP || exit 0
set -e
case "$1" in
start)
echo -n "Starting $DESC: "
touch $PIDFILE
chown <%= user %>:<%= user %> $PIDFILE
if start-stop-daemon --start --quiet --umask 007 --pidfile $PIDFILE --chuid <%= user %>:<%= user %> --exec $DAEMON -- $DAEMON_ARGS
then
echo "$NAME."
else
echo "failed"
fi
;;
stop)
echo -n "Stopping $DESC: "
if start-stop-daemon --stop --retry 10 --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
then
echo "$NAME."
else
echo "failed"
fi
rm -f $PIDFILE
;;
restart|force-reload)
${0} stop
${0} start
;;
*)
echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload}" >&2
exit 1
;;
esac
exit 0
这里存在一个问题不能通过这个脚本来启动 redis,只能停止 redis,所以又单独写了一个 startRedis。
config/templates/mysh/startRedis.sh.erb
#!/bin/sh
cd $HOME && /usr/local/bin/redis-server $HOME/redis-2.8.16/redis.conf
config/templates/mysh/thin.sh.erb
需要 export ruby 环境,否则 root 用户下没有办法来启动 thin,而且export PATH="${PATH}:<%= rvm_home %>/gems/<%= rvm %>/bin:/usr/bin:<%= rvm_home %>/bin:<%= rvm_home %>/gems/<%= rvm %>@global/bin:<%= rvm_home %>/rubies/<%= rvm %>/bin/"
要将${PATH}
放到前面,这样就不会被系统默认的 ruby 环境覆盖。
#!/bin/bash
# Set the environment, as required by Monit
export PATH="${PATH}:<%= rvm_home %>/gems/<%= rvm %>/bin:/usr/bin:<%= rvm_home %>/bin:<%= rvm_home %>/gems/<%= rvm %>@global/bin:<%= rvm_home %>/rubies/<%= rvm %>/bin/"
export GEM_PATH="<%= rvm_home %>/gems/<%= rvm %>:<%= rvm_home %>/gems/<%= rvm %>@global"
export GEM_HOME="<%= rvm_home %>/gems/<%= rvm %>"
start () {
cd <%= path %>/current
BUNDLE_GEMFILE=<%= path %>/current/Gemfile bundle exec thin -C /etc/thin/wedding.<%= thin_port %>.yml -d start
}
stop () {
kill -s QUIT $(cat <%= deploy_to %>/tmp/pids/thin.<%= thin_port %>.pid)
}
case $1 in
start)
start
;;
stop)
stop
;;
*)
echo $"Usage: $0 {start|stop}"
exit 1
;;
esac
exit 0
config/templates/thin.yml.erb
---
chdir: /var/app/wedding/current
environment: <%= rails_env %>
address: 0.0.0.0
port: <%= thin_port %>
timeout: 30
log: /var/app/wedding/shared/log/thin.log
pid: /var/app/wedding/tmp/pids/thin.pid
max_conns: 1024
max_persistent_conns: 100
require: []
wait: 30
servers: <%= servers_number %>
daemonize: true
config/mina/base.rb
require 'erb'
def template(from, to)
queue %{echo "-----> Put #{from} file to #{to}"}
erb = File.read(File.expand_path("../../templates/#{from}", __FILE__))
put ERB.new(erb).result(binding), to
end
def put(content, file)
queue %[echo #{escape content} > "#{file}"]
end
def escape(str)
Shellwords.escape(str)
end
config/mina/mysh.rb
namespace :mysh do
task :setup do
invoke :'mysh:thin'
invoke :'mysh:sidekiq'
invoke :'mysh:redis-server' if rails_env == 'production2'
%w[startRedis.sh].each do |name|
destination = "$HOME/mysh/#{name}"
template "mysh/#{name}.erb", destination
queue "chmod a+x #{destination}"
end
end
task :thin do
thin_ports.each do |thin_port|
set :thin_port, thin_port
destination = "$HOME/mysh/wedding.#{thin_port}.sh"
template "mysh/thin.sh.erb", destination
queue "chmod a+x #{destination}"
end
end
task :sidekiq do
destination = "$HOME/mysh/sidekiq.sh"
template "mysh/sidekiq.sh.erb", destination
queue "chmod a+x #{destination}"
queue "sudo cp #{destination} /etc/init.d/sidekiq"
end
task "redis-server" do
destination = "$HOME/mysh/redis-server.sh"
template "mysh/redis-server.sh.erb", destination
queue "chmod a+x #{destination}"
queue "sudo cp #{destination} /etc/init.d/redis-server"
end
end
通过mina mysh:setup
来生成服务器相关的启动脚本。
config/mina/thin.rb
namespace :thin do
task :setup do
thin_ports.each do |thin_port|
destination = "/etc/thin/wedding.#{thin_port}.yml"
set :thin_port, thin_port
set :servers_number, 1
template "thin.yml.erb", destination
end
invoke :'thin:railsapp'
end
task "railsapp" do
destination = "/etc/thin/railsapp.yml"
set :thin_port, thin_ports.first
set :servers_number, thin_ports.size
template "thin.yml.erb", destination
end
end
通过mina thin:setup
来生成 thin 的配置文件。
config/mina/monit.rb
namespace :monit do
desc "Setup all Monit configuration"
task :setup do
queue %{echo "-----> Setting up Monit..."}
monit_config "monitrc", "/etc/monit/monitrc"
monit_config "monitrc", "/etc/monitrc"
invoke :'monit:mysql'
invoke :'monit:redis' if rails_env == 'production2'
invoke :'monit:nginx'
invoke :'monit:thin'
invoke :'monit:sidekiq'
end
after "deploy:setup", "monit:setup"
task(:nginx) { monit_config "nginx" }
task(:mysql) { monit_config "mysql" }
task(:redis) { monit_config "redis" }
task(:thin) { monit_config "thin" }
task(:sidekiq) { monit_config "sidekiq" }
%w[start stop syntax reload].each do |command|
desc "Run Monit #{command} script"
task command do
queue %{echo "-----> Monit #{command}"}
queue "sudo service monit #{command}"
end
end
task :status do
queue "sudo monit status"
end
end
def monit_config(name, destination = nil)
destination ||= "/etc/monit/conf.d/#{name}"
template "monit/#{name}.erb", "$HOME/monit/#{name}"
queue "sudo cp $HOME/monit/#{name} #{destination}"
queue "sudo chown root #{destination}"
queue "sudo chmod 600 #{destination}"
end
通过mina monit:setup
来生成 monit 的配置文件。这里需要注意,一定要将 monit 的一些配置文件的 owner 修改为 root,即和 monit 安装用户是同一用户,否则是无法启动的。另外需要注意的是,需要考虑各项服务的 pid 文件,某些 service 是默认关闭 pid 文件的,需要修改相应的配置文件打开。
将配置文件和脚本模版化,可以在配置完一台服务器后,非常迅速的配置完成其他服务器。
一切搞定,这样就可以随时查看服务器的状态,来点效果图。 试着 kill 掉一个 thin