EzDevInfo.com

Webrick as production server vs. Thin or Unicorn?

It seems like it's taken for granted that you must not use Webrick as production server, but I can't really find anywhere mentioning why. The consensus seems to be: "Webrick is ok for development, but Thin or Unicorn is the choice for production, period."

I did look up Thin server's homepage and it talks about requests/second but I don't really understand the graph since there's no annotation.

Can anyone let me know why I should use Thin or Unicorn compared to Webrick? Also is there any benefit to using Webrick for development? I've been using Webrick since it comes with rails, and I think there should be a reason why it's default.

I'm using Heroku by the way.

Unicorn exit timeout on Heroku after trapping TERM and sending QUIT

I am receiving R12 Exit Timeout errors for a Heroku app running unicorn and sidekiq. These errors occur 1-2 times a day and whenever I deploy. I understand that I need to convert the shutdown signals from Heroku for unicorn to respond correctly, but thought that I had done so in the below unicorn config:

worker_processes 3
timeout 30
preload_app true

before_fork do |server, worker|
  Signal.trap 'TERM' do
    puts "Unicorn master intercepting TERM and sending myself QUIT instead. My PID is #{Process.pid}"
    Process.kill 'QUIT', Process.pid
  end

  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.connection.disconnect!
    Rails.logger.info('Disconnected from ActiveRecord')
  end
end

after_fork do |server, worker|
  Signal.trap 'TERM' do
    puts "Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is #{Process.pid}"
  end

  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    Rails.logger.info('Connected to ActiveRecord')
  end

  Sidekiq.configure_client do |config|
    config.redis = { :size => 1 }
  end
end

My logs surrounding the error look like this:

Stopping all processes with SIGTERM
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 7
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 11
Unicorn worker intercepting TERM and doing nothing. Wait for master to sent QUIT. My PID is 15
Unicorn master intercepting TERM and sending myself QUIT instead. My PID is 2
Started GET "/manage"
reaped #<Process::Status: pid 11 exit 0> worker=1
reaped #<Process::Status: pid 7 exit 0> worker=0
reaped #<Process::Status: pid 15 exit 0> worker=2
master complete
Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
Stopping remaining processes with SIGKILL
Process exited with status 137

It appears that all of the child processes were successfully reaped before the timeout. Is it possible master is still alive? Also, should the router still be sending web requests to the dyno during shut down, as shown in the logs?

FWIW, I'm using Heroku's zero downtime deployment plugin (https://devcenter.heroku.com/articles/labs-preboot/).

Should I use thin or unicorn on Heroku Cedar

I recently 'upgraded' my app to the cedar platform on heroku. By default I am using thin as a web server. But I have always been tempted to use unicorn for concurrency and having my dyno dollar go father. But I worry there are some gotchas in using something other than Thin.

Does anyone have real life experience with this decision?

Thanks!

Jonathan

Notes:

• This was the article that got me excited about the idea: http://michaelvanrooijen.com/articles/2011/06/01-more-concurrency-on-a-single-heroku-dyno-with-the-new-celadon-cedar-stack/
• I know every app is different, and that you should build a staging env and try it for yourself. But if it looks great in your staging env, are they any pitfalls that we should know about?

I want to know reasons why everyone shouldn't do this

How can I use unicorn as "rails s"?

A new rails project's gemfile shows:

# Use unicorn as the app server
gem 'unicorn'

rails s --help shows:

Usage: rails server [mongrel, thin, etc] [options]

Yet, doing:

rails s unicorn

I get:

/Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/rack-1.4.5/lib/rack/handler.rb:63:in `require': cannot load such file -- rack/handler/unicorn (LoadError)
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/rack-1.4.5/lib/rack/handler.rb:63:in `try_require'
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/rack-1.4.5/lib/rack/handler.rb:16:in `get'
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/rack-1.4.5/lib/rack/server.rb:272:in `server'
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/railties-3.2.13/lib/rails/commands/server.rb:59:in `start'
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/railties-3.2.13/lib/rails/commands.rb:55:in `block in <top (required)>'
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/railties-3.2.13/lib/rails/commands.rb:50:in `tap'
from /Users/patrick/.rvm/gems/ruby-1.9.3-head@keynote/gems/railties-3.2.13/lib/rails/commands.rb:50:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'

I've used unicorn in the past on other projects, but always had to run the unicorn command and specify a config file which is a bit of a pain. I am wondering how I can just simply make it work by using "rails s"... Is this possible?

Why would I want to use unicorn or thin instead of WEBrick for development purposes?

I've recently found that some people prefer using unicorn_rails instead of the default WEBrick as a web server for developing Rails applications.

I understand that if I wanted to use unicorn in production, it could make kind of sense to try it out in development, but since the configuration is different in production, is it even relevant?

Is there any real, tangible advantage that I would get from using thin or unicorn instead of WEBrick for developing a Rails application, such as speed or some additional features? Or is this just a matter of personal preference?

Using God to monitor Unicorn - Start exited with non-zero code = 1

I am working on a God script to monitor my Unicorns. I started with GitHub's examples script and have been modifying it to match my server configuration. Once God is running, commands such as god stop unicorn and god restart unicorn work just fine.

However, god start unicorn results in WARN: unicorn start command exited with non-zero code = 1. The weird part is that if I copy the start script directly from the config file, it starts right up like a brand new mustang.

This is my start command:

/usr/local/bin/unicorn_rails -c /home/my-linux-user/my-rails-app/config/unicorn.rb -E production -D

I have declared all paths as absolute in the config file. Any ideas what might be preventing this script from working?

How to preserve request url with nginx proxy_pass

I was trying to use Thin app server and had one issue.

When nginx proxies the request to Thin (or Unicorn) using proxy_pass http://my_app_upstream; the application receives the modified URL sent by nginx (http://my_app_upstream).

What I want is to pass the original URL and the original request from client with no modification as the app relies heavily on it.

The nginx' doc says:

If it is necessary to transmit URI in the unprocessed form then directive proxy_pass should be used without URI part.

But I don't understand how exactly to configure that as the related sample is actually using URI:

location  /some/path/ {
  proxy_pass   http://127.0.0.1;
}

So could you please help me figuring out how to preserve the original request URL from the client?

Thanks,
Dima.

How to improve memory sharing between unicorn processes with Ruby 2.0 on Linux

Ruby 2.0 introduces a copy-on-write friendly garbage collector. My processes don't seem to keep memory shared for more than a few minutes - it seems to move from shared_dirty to private_dirty quite quickly.

Some others have had success getting this to work:

• https://gist.github.com/kenn/5105175
• http://marianposaceanu.com/articles/on-ruby-2-0-memory-usage-unicorn-and-heroku

This program can be used to check memory stats on Linux: https://gist.github.com/kenn/5105061

My unicorn configuration: https://gist.github.com/inspire22/f82c77c0a465f1945305

For some reason my unicorn apps, also with preload_app=true, have much less shared memory. Ruby 2.0-p195, rails 3.2, linux 2.6.18 (centos)

[root@thorn script]# ruby memstats.rb 4946
Process:             4946
Command Line:        unicorn_rails worker[4] -c /u/apps/newap/current/lib/unicorn.rb -E production -D
Memory Summary:
  private_clean                   0 kB
  private_dirty              56,324 kB
  pss                        60,256 kB
  rss                        83,628 kB
  shared_clean                4,204 kB
  shared_dirty               23,100 kB
  size                      108,156 kB
  swap                           68 kB 

If I shutdown the master process entirely (not just a HUP) then restart it and immediately check a worker before any requests have queued, I get a better story:

[root@thorn script]# ruby memstats.rb 5743
Process:             5743
Command Line:        unicorn_rails worker[4] -c /u/apps/newap/current/lib/unicorn.rb -E production -D
Memory Summary:
  private_clean                   0 kB
  private_dirty              21,572 kB
  pss                        27,735 kB
  rss                        66,296 kB
  shared_clean                2,484 kB
  shared_dirty               42,240 kB
  size                       91,768 kB
  swap                            0 kB

But within 5 seconds of being started up, they're back to ~20MB of shared_clean+shared_dirty.

I suspected that swapping might be causing the problem, but after lowering swappiness and making sure that neither the parent nor child processes are being swapped out (using swapstats.rb), the problem persists.

I don't understand exactly what shared_dirty memory is, and how it gets turned into private memory. I'd also love suggestions for improving the longevity and amount of my shared memory. Thanks!

bundler incorrectly trying to install "development" and "test" group gems in production

I have a small web app, which uses a bunch of gems. Some of them are only used for test and development environments. Now, when I try to start unicorn on the production server using the following command, it fails.

unicorn_rails -E production -D -c config/unicorn.rb

The error I see in the log files is:

Refreshing Gem list
Could not find gem 'spork (>= 0.9.0.rc2, runtime)' in any of the gem sources listed in your Gemfile.
Try running `bundle install`.

I've pasted my gemfile below:

source 'http://rubygems.org'

gem 'rails', '3.0.1'
gem 'unicorn'
gem 'mongoid', '>= 2.0.0.beta.19'
gem 'devise'
gem 'cancan'
gem 'haml', '>= 3.0.0'
gem 'bson'
gem 'bson_ext'
gem 'formtastic'
gem 'bluecloth'

group :production do
  gem 'capistrano'
end

group :development do
  gem 'haml-rails'
  gem 'hpricot', '0.8.2'
  gem 'ruby_parser', '2.0.5'
  gem 'less'
  gem 'rspec-rails', '>= 2.0.1'
end

group :development,:test do
  gem 'spork', '>=0.9.0.rc2'
  gem 'mongoid-rspec'
end

group :test do
  gem 'factory_girl_rails'
  gem 'autotest'
  gem 'cucumber-rails'
  gem 'cucumber'
  gem 'capybara'
  gem 'shoulda'
  gem 'database_cleaner'
  gem 'test_notifier'
  gem 'rspec', '2.0.1'
  gem 'launchy' 
end

Bundler is supposed to detect the right environment and ignore the other gems, right? Right now, I am deleting all the lines which are not in the default group on the server to get this working, but that's an ugly hack.

Rails Performance Tuning for Production?

I'm getting close to deploying an application built on Rails 3.1.x and started running some performance tests. After fiddling with ab for a bit, I'm seeing some very discouraging results yielding around 15 requests / second on Heroku.

When testing locally I see similar results that really demonstrates that it's an app issue more than anything.

I'm running Unicorn, which is about 40% faster than Thin on Celadon Cedar. Further, I'm using the PGSQL shared db.

I'm hopeful that someone could share a laundry list or essentially a launch checklist that I should move through when prepping an app for production and moving into the need for speed tuning. So far I've not found a real concise list of actionable items to move through that seems to make sense given my situation.

Or if you have solid practical experience moving through issues like this, any input would be appreciated!

Why does Unicorn needs to be deployed together with Nginx?

I would like to know the difference between Nginx and Unicorn. As far as I understand, Nginx is a web server while Unicorn is a Ruby HTTP server.

Since both Nginx and Unicorn can handle HTTP requests, what is the need to use the combination of Nginx and Unicorn for RoR applications?

Unicorn completely ignores USR2 signal

I'm experiencing a rather strange problem with unicorn on my production server. Although the config file states preload_app true, sending USR2 to the master process does not generate any response, and it seems like unicorn is ignoring the signal altogether. On another server sending USR2 changes the master process to and (old) state and starts a new master process successfully. The problematic server is using RVM & bundler, so I'm assuming it's somehow related (the other one is vanilla ruby). Sending signals other than USR2 (QUIT, HUP) works just fine. Is there a way to trace what's going on behind the scenes here? Unicorn's log file is completely empty.

Is there a way to tell RubyMine to not use webrick?

When I start my app up in RubyMine I want to be able to use unicorn and my unicorn configs. Is there any way to tell it not to use webrick but use something else like unicorn or thin?

Unicorn unable to write pid file

I am using deploying a Ruby on Rails app to a Linode VPS using Capistrano. I am using Unicorn as the application server and Nginx as the proxy. My problem is that I am not able to start Unicorn because of an apparent permissions issue, but I'm having a hard time tracking it down.

Unicorn is started using this Capistrano task:

task :start, :roles => :app, :except => { :no_release => true } do
    run <<-CMD
      cd #{current_path} && #{unicorn_bin} -c #{unicorn_config} -E #{rails_env} -D
    CMD
end

I get back and ArgumentError indicating that the path to the pid file is not writeable.

cap unicorn:start                                                                           master [d4447d3] modified
  * executing `unicorn:start'
  * executing "cd /home/deploy/apps/gogy/current && /home/deploy/apps/gogy/current/bin/unicorn -c /home/deploy/apps/gogy/shared/config/unicorn.rb -E production -D"
    servers: ["66.228.52.4"]
    [66.228.52.4] executing command
 ** [out :: 66.228.52.4] /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/lib/unicorn/configurator.rb:88:in `reload':
 ** [out :: 66.228.52.4] directory for pid=/home/deploy/apps/shared/pids/unicorn.pid not writable (ArgumentError)
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/lib/unicorn/configurator.rb:84:in `each'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/lib/unicorn/configurator.rb:84:in `reload'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/lib/unicorn/configurator.rb:65:in `initialize'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:102:in `new'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/lib/unicorn/http_server.rb:102:in `initialize'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/bin/unicorn:121:in `new'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/shared/bundle/ruby/1.8/gems/unicorn-4.1.1/bin/unicorn:121
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/current/bin/unicorn:16:in `load'
 ** [out :: 66.228.52.4] from /home/deploy/apps/gogy/current/bin/unicorn:16
 ** [out :: 66.228.52.4] master failed to start, check stderr log for details
    command finished in 1032ms
failed: "rvm_path=/usr/local/rvm /usr/local/rvm/bin/rvm-shell 'default' -c 'cd /home/deploy/apps/gogy/current && /home/deploy/apps/gogy/current/bin/unicorn -c /home/deploy/apps/gogy/shared/config/unicorn.rb -E production -D'" on 66.228.52.4

Finally, here is the relevant sections of my Unicorn configuration file (unicorn.rb)

# Ensure that we're running in the production environment
rails_env = ENV['RAILS_ENV'] || 'production'

# User to run under
user 'deploy', 'deploy'

# We will spawn off two worker processes and one master process
worker_processes 2

# set the default working directory
working_directory "/home/deploy/apps/gogy/current"

# This loads the application in the master process before forking
# worker processes
# Read more about it here:
# http://unicorn.bogomips.org/Unicorn/Configurator.html
preload_app true

timeout 30

# This is where we specify the socket.
# We will point the upstream Nginx module to this socket later on
listen "/home/deploy/apps/shared/sockets/unicorn.sock", :backlog => 64

pid "/home/deploy/apps/shared/pids/unicorn.pid"

# Set the path of the log files 
stderr_path "/home/deploy/apps/gogy/current/log/unicorn.stderr.log"
stdout_path "/home/deploy/apps/gogy/current/log/unicorn.stdout.log"

I'm deploying with Capistrano under the 'deploy' user and group and that's what Unicorn should be run under also.

Does anyone have any ideas why Unicorn can't write out the pid file? Any help would be greatly appreciated!!!

• Mike

Unicorn restart issue with capistrano

We're deploying with cap and using a script that send USR2 to the unicorn process to reload and it usually works but every once in a while it will fail. When that happens looking in the unicorn log reveals that it's looking for a Gemfile in an old release directory that no longer exists.

Exception :

/usr/local/lib/ruby/gems/1.9.1/gems/bundler-1.0.21/lib/bundler/definition.rb:14:in `build': /var/www/railsapps/inventory/releases/20111128233407/Gemfile not found (Bundler::GemfileNotFound)

To clarify that's not the current release but an older one that's since been removed.

When it works it does seem to work correctly - ie it does pickup the new code - so I don't think it's somehow stuck referring to the old release.

Any ideas?