dalli
High performance memcached client for Ruby
Is it possible to somehow run Rails.cache.clear and only clear keys with a certain name/string?
I don't want to clear the entire cache...just keys with the string blog/post in the name (ie. blog/post/1, blog/post/2).
I'm using dalli with memcached for my cache and running Rails 3.0.6.
Source: (StackOverflow)
I am not able to use memcached as session store with rails 4 using dalli gem.
Here's what I have done.
I added dalli gem to Gemfile
gem 'dalli'
I added the following line in config/initializers/session_store.rb
Rails.application.config.session_store ActionDispatch::Session::CacheStore, :expire_after => 20.minutes
And I added the following line in development.rb
config.cache_store = :dalli_store
Now when I start my development server with thin server without starting memcached server, I still can login as usual. Should I get some error like no memcached server running or something like that.
I am not sure if rails is using memcached as session store or not.
Can someone tell me what have I missed in using memcached as session store in development environment?
For your information, I have been using devise as authentication gem.
Thanks
Source: (StackOverflow)
I have been struggling for a while to find out if there is some default expiration time set by Rails, in case we don't provide any while storing a key-value pair to memcache?
e.g. Rails.cache.write('some-key', 'some-value')
Would rails set some expiration time by default if we haven't specified?
Source: (StackOverflow)
Hi I'm having trouble setting up my Rails project on my server because apache keeps complaining
DalliError: No server available.
I installed memcached on my ubuntu machine, but it still doesn't work. My rails project also has config.cache_store = :dalli_store, 'localhost:11211', { :namespace => "production" } in environments/production.rb. How would I debug this?
My log shows before each request:
localhost:11211 failed (count: 6)
DalliError: No server available
telnet to 11211:
root@s2:/usr/local/www/production/current/log# telnet localhost 11211
Trying 127.0.1.1...
telnet: Unable to connect to remote host: Connection refused
Source: (StackOverflow)
I am attempting to generate a fragment cache (using a Dalli/Memcached store) however the key is being generated with "#" as part of the key, so Rails doesn't seem to be recognizing that there is a cache value and is hitting the database.
My cache key in the view looks like this:
cache([@jobs, "index"]) do
The controller has:
@jobs = @current_tenant.active_jobs
With the actual Active Record query like this:
def active_jobs
self.jobs.where("published = ? and expiration_date >= ?", true, Date.today).order("(featured and created_at > now() - interval '" + self.pinned_time_limit.to_s + " days') desc nulls last, created_at desc")
end
Looking at the rails server, I see the cache read, but the SQL Query still runs:
Cache read: views/#<ActiveRecord::Relation:0x007fbabef9cd58>/1-index
Read fragment views/#<ActiveRecord::Relation:0x007fbabef9cd58>/1-index (1.0ms)
(0.6ms) SELECT COUNT(*) FROM "jobs" WHERE "jobs"."tenant_id" = 1 AND (published = 't' and expiration_date >= '2013-03-03')
Job Load (1.2ms) SELECT "jobs".* FROM "jobs" WHERE "jobs"."tenant_id" = 1 AND (published = 't' and expiration_date >= '2013-03-03') ORDER BY (featured and created_at > now() - interval '7 days') desc nulls last, created_at desc
Any ideas as to what I might be doing wrong? I'm sure it has to do w/ the key generation and ActiveRecord::Relation, but i'm not sure how.
Source: (StackOverflow)
We have a page which shows the top groups for our application. The calculation of the leader board is expensive so we cache the results for an hour as follows:
@groupboard = Rails.cache.fetch("top_groups", :expires_in => 1.hour) do
Group.top_groups
end
This is giving an error after the cache has been written the first time. Poking around in the console I see that Group.top_groups returns an array of items that look as follows:
[#<Group id: 4, name: "IBP", rank: 6, users_count: 13, leader_id: 4662>, 3887]
When I look at the result returned from the cache it looks as follows:
[#<Group id: 4, name: "IBP", rank: 6, users_count: 13, leader_id: 4662>, :@new_record_before_save], false, 3887]
Does anyone know what would be causing @new_record_before_save and the 'false' value to be inserted into all the entries for this object in the cache?
We are using Dalli, Memcached 1.4.9, Rails 3.2.4, Ruby 1.9.2
Source: (StackOverflow)
I'm trying to develop application with caching in development mode, but development.log spammed is heavily by cache logs. I'm use dalli, and I know, that dalli has silence! method (https://github.com/mperham/dalli/commit/892020fbc73613ccc84412ce04b85b7fda645e63), but how to use this method?
I found some old instructions, where it is suggested to call in on config.cache_store, but it is a symbol, and don't has this method:
config.cache_store = :dalli_store
config.cache_store.silence!
Throws exception.
Source: (StackOverflow)
I'm using dalli gem as a memcached client in my Rails 4 app. Having multiple cached fragments in my view I couldn't but wonder how efficiently does Rails handles retrieval of data from memcached server. Does it make a new connection per each fragment or does it retrieve all the necessary fragments in a single connection?
Source: (StackOverflow)
I'm going to use Dalli cache as key-value store.
Usually in production and development environments we have line
config.cache_store = :dalli_store
so then we can use Rails.cache construction to read from and write to cache.
But in test environment usually we don't have this config line.
What is the write way to setup cache in test environment in purpose to test my storing logic?
P.S. I'm using Linux(Ubuntu)
Source: (StackOverflow)
I am running Ruby on Rails and am using the Dalli gem to access memcached.
Question: how do I delete a range of keys (not multiple, but a range) for something like: delete all memcached entries with a key that begins with "USERINFO", in other words, how can I use wildcards to delete a range of keys?
Source: (StackOverflow)
I want to use memcached in Rails.
So I set in my local. Good. and deploy to my server.. a problem comes up
The problem is that dalli cannot be loaded in active_support/cache/mem_cache_store.rb..
My server environment :
ubuntu / nginx / unicorn / Rails 4 / capistrano 2
unicorn.log (Edit (add log more) )
E, [2014-08-09T10:46:52.306478 #3576] ERROR -- : reaped #<Process::Status: pid 25225 exit 1> worker=1
I, [2014-08-09T10:46:52.306772 #3576] INFO -- : worker=1 spawning...
I, [2014-08-09T10:46:52.315784 #25228] INFO -- : worker=1 spawned pid=25228
I, [2014-08-09T10:46:52.316264 #25228] INFO -- : Refreshing Gem list
You don't have dalli installed in your application. Please add it to your Gemfile and run bundle install
E, [2014-08-09T10:46:52.377766 #25221] ERROR -- : Could not find cache store adapter for mem_cache_store (cannot load such file -- dalli) (RuntimeError)
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/activesupport-4.1.4/lib/active_support/cache.rb:105:in `rescue in retrieve_store_class'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/activesupport-4.1.4/lib/active_support/cache.rb:103:in `retrieve_store_class'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/activesupport-4.1.4/lib/active_support/cache.rb:59:in `lookup_store'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/railties-4.1.4/lib/rails/application/bootstrap.rb:62:in `block in <module:Bootstrap>'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/railties-4.1.4/lib/rails/initializable.rb:30:in `instance_exec'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/railties-4.1.4/lib/rails/initializable.rb:30:in `run'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/railties-4.1.4/lib/rails/initializable.rb:55:in `block in run_initializers'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:150:in `block in tsort_each'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:183:in `block (2 levels) in each_strongly_connected_component'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:219:in `each_strongly_connected_component_from'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:182:in `block in each_strongly_connected_component'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:180:in `each'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:180:in `each_strongly_connected_component'
/usr/local/rbenv/versions/2.0.0-p451/lib/ruby/2.0.0/tsort.rb:148:in `tsort_each'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/railties-4.1.4/lib/rails/initializable.rb:54:in `run_initializers'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/railties-4.1.4/lib/rails/application.rb:300:in `initialize!'
/bps_data/apps/chimiseng/releases/20140809040739/config/environment.rb:5:in `<top (required)>'
config.ru:4:in `require'
config.ru:4:in `block in <main>'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/builder.rb:55:in `instance_eval'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/rack-1.5.2/lib/rack/builder.rb:55:in `initialize'
config.ru:1:in `new'
config.ru:1:in `<main>'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn.rb:48:in `eval'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn.rb:48:in `block in builder'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:764:in `call'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:764:in `build_app!'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:628:in `init_worker_process'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:651:in `worker_loop'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:525:in `spawn_missing_workers'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:536:in `maintain_worker_count'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:294:in `join'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/gems/unicorn-4.8.3/bin/unicorn:126:in `<top (required)>'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/bin/unicorn:23:in `load'
/bps_data/apps/chimiseng/shared/bundle/ruby/2.0.0/bin/unicorn:23:in `<main>'
@@@@Edit START@@@@
add info about this..
in /path/to/shared/bundle/gems/activesupport-4.1.4/lib/activesupport/cache.rb:103-105
102 def retrieve_store_class(store)
103 require "active_support/cache/#{store}"
104 rescue LoadError => e
105 raise "Could not find cache store adapter for #{store} (#{e})"
106 else
107 ActiveSupport::Cache.const_get(store.to_s.camelize)
108 end
in active_support/cache/mem_cache_store.rb required
1 begin
2 require 'dalli' # <- error !!
3 rescue LoadError => e
4 $stderr.puts "You don't have dalli installed in your application. Please add it to your Gemfile and run bundle install"
5 raise e
6 end
@@@@@Edit END@@@@@
and Here is config/environment/production.rb and Gemfile
Gemfile
source 'https://rubygems.org'
...
# Use unicorn as the app server
gem 'unicorn'
gem 'dalli'
gem 'capistrano', "~> 2.15.0"
gem 'net-ssh', '~> 2.7.0'
gem 'rb-readline'
...
config/environments/production.rb
...
config.cache_classes = true
config.action_controller.perform_caching = true
config.cache_store = :mem_cache_store
...
command 'bundle show' at APP_DIR (exists .bundle folder)
...
* dalli (2.7.2)
...
and ls /path/to/shared/bundle/ruby/2.0.0/gems
...
dalli-2.7.2
...
of course,, APP_DIR/.bundle/config is defaults.
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: /path/to/shared/bundle
BUNDLE_WITHOUT: development:test
BUNDLE_DISABLE_SHARED_GEMS: '1'
and /etc/memcached.conf
-d
logfile /path/to/shared/log/memcached.log
-m 64
-p 11211
-u memcache
-l 127.0.0.1
at my server, in APP_DIR command 'rails c -e production'
Loading production environment (Rails 4.1.4)
irb(main):001:0> Rails.cache.stats
Dalli::Server#connect localhost:11211
localhost:11211 failed (count: 0) Errno::ECONNREFUSED: Connection refused - send
=> {"localhost:11211"=>nil}
irb(main):002:0>
irb(main):003:0* dc = Dalli::Client.new
=> #<Dalli::Client:0x007fa410d27d58 @servers=["127.0.0.1:11211"], @options={}, @ring=nil>
irb(main):004:0>
irb(main):005:0* dc.stats
.......
irb(main):006:0> y _
---
127.0.0.1:11211:
pid: '6657'
uptime: '476'
time: '1407552100'
version: 1.4.13
libevent: 2.0.16-stable
pointer_size: '64'
rusage_user: '0.000000'
rusage_system: '0.032002'
curr_connections: '5'
total_connections: '7'
connection_structures: '6'
reserved_fds: '20'
cmd_get: '0'
cmd_set: '1'
cmd_flush: '0'
cmd_touch: '0'
get_hits: '0'
get_misses: '0'
delete_misses: '0'
delete_hits: '0'
incr_misses: '0'
incr_hits: '0'
decr_misses: '0'
decr_hits: '0'
cas_misses: '0'
cas_hits: '0'
cas_badval: '0'
touch_hits: '0'
touch_misses: '0'
auth_cmds: '0'
auth_errors: '0'
bytes_read: '160'
bytes_written: '3709'
limit_maxbytes: '67108864'
accepting_conns: '1'
listen_disabled_num: '0'
threads: '4'
conn_yields: '0'
hash_power_level: '16'
hash_bytes: '524288'
hash_is_expanding: '0'
expired_unfetched: '0'
evicted_unfetched: '0'
bytes: '73'
curr_items: '1'
total_items: '1'
evictions: '0'
reclaimed: '0'
=> nil
irb(main):007:0>
and ps aux | grep memcached in my server
memcache 6657 0.0 0.2 324248 2228 ? Sl 11:33 0:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1
and /etc/init.d/unicorn_myapp restart
reloaded OK
but.. server rails application not work..
@@@Edit start (add info)@@@
in my server , rails c -e production..
Loading production environment (Rails 4.1.4)
irb(main):001:0>require 'dalli'
=> false
irb(main):002:0>
this means.. dalli is already required. isn't it?
hmm.. why cannot be required in active_support/cache/mem_cache_store.rb
@@@Edit end @@@
But it's work very well at my local ubuntu!!
local bundle installed list is same server (except for gem spring)..
Loading production environment (Rails 4.1.4)
irb(main):001:0> Rails.cache.stats
Dalli::Server#connect localhost:11211
=> {"localhost:11211"=>{"pid"=>"1976", "uptime"=>"13302", "time"=>"1407552926", "version"=>"1.4.13", "libevent"=>"2.0.16-stable", "pointer_size"=>"64", "rusage_user"=>"0.724000", "rusage_system"=>"0.296000", "curr_connections"=>"5", "total_connections"=>"10", "connection_structures"=>"7", "reserved_fds"=>"20", "cmd_get"=>"0", "cmd_set"=>"0", "cmd_flush"=>"1", "cmd_touch"=>"0", "get_hits"=>"0", "get_misses"=>"0", "delete_misses"=>"0", "delete_hits"=>"0", "incr_misses"=>"0", "incr_hits"=>"0", "decr_misses"=>"0", "decr_hits"=>"0", "cas_misses"=>"0", "cas_hits"=>"0", "cas_badval"=>"0", "touch_hits"=>"0", "touch_misses"=>"0", "auth_cmds"=>"0", "auth_errors"=>"0", "bytes_read"=>"244", "bytes_written"=>"5617", "limit_maxbytes"=>"67108864", "accepting_conns"=>"1", "listen_disabled_num"=>"0", "threads"=>"4", "conn_yields"=>"0", "hash_power_level"=>"16", "hash_bytes"=>"524288", "hash_is_expanding"=>"0", "expired_unfetched"=>"0", "evicted_unfetched"=>"0", "bytes"=>"0", "curr_items"=>"0", "total_items"=>"0", "evictions"=>"0", "reclaimed"=>"0"}}
What's the problem??
Please Help me T^T
i am being crazy because of this..
Anyway.. Thank you for reading !
Source: (StackOverflow)
We are facing a weird problem where-in few keys/cookies are getting deleted/missing from the session in our rails 3.2 app (staging environment) which has the following cache settings/configuration.
The session store is cookie store and has been configured as follows:
Appsrv::Application.config.session_store :cookie_store, {
:key => 'SSID',
:path => '/',
:domain => APP_CONFIG['site_url'].sub(/^https?:\/\//, ""),
:expire_after => 30.minutes,
:secret => 's23asdfe443534afdgstreggv234324we434',
:secure => false # cookie not for just https
}
The rails cache store is a memcache store and we use dalli gem to integrate rails and memcache.
The following configuration has been added to different .rb files:
config.cache_store = :dalli_store, 'staging01:11211', 'staging02:11211', 'staging03:11211', 'staging04:11211'
{:namespace => "appsrv", :expires_in => 86400, :compression => true}
config.action_controller.perform_caching = true
The flow where this scenario of missing cookies happens is a bit complicated though. The scenario where this happens is as follows:
- User starts a session by visiting a page on Appsrv(rails 3.2 app, some variables identifying user transaction are set in session at this time)
- Goto external website for auth.
- External website redirects to a Java App server after successful auth.
- Java App server records the auth and redirects back to Appsrv.
- But after this redirection the session variables set in step 1 are gone.
Some weird findings:
Things work perfectly in the development environment where the only difference is:
config.action_controller.perform_caching = false
If we have the same setting in staging "config.action_controller.perform_caching=false" then staging also works fine. Even with "config.action_controller.perform_caching" set to false, caching actually happens properly in controllers and models.
So the questions are:
1. Why do the session cookies get deleted when the config.action_controller.perform_caching is set to true?
2. What is the significance of config.action_controller.perform_caching configuration if setting it to false also allows cache to happen properly?
Source: (StackOverflow)
I'm trying to use fragment caching with Jbuilder as one of the view is taking quite a bit to render (only view rendering can take 2500ms+
Note that this does work in other calls, but this one can't seem to work and I can't figure why. Second : this works on my local machine but fails in heroku.
Here is the error in heroku :
2013-09-18T21:05:46.425034+00:00 app[web.1]: Rendered api/shop/products/_product.json.jbuilder (3.2ms)
2013-09-18T21:05:46.606141+00:00 app[web.1]: Marshalling error for key 'shop/products/344-20130914175034924266000/shop/products/346-20130914175035358419000/shop/products/345-20130914175035153905000/en/b5262bbbd44fb696ffdece67a464e218': no _dump_data is defined for class Proc
2013-09-18T21:05:46.606141+00:00 app[web.1]: You are trying to cache a Ruby object which cannot be serialized to memcached.
2013-09-18T21:05:46.606141+00:00 app[web.1]: /app/vendor/bundle/ruby/2.0.0/gems/dalli-2.6.4/lib/dalli/server.rb:397:in `dump'
2013-09-18T21:05:46.606141+00:00 app[web.1]: /app/vendor/bundle/ruby/2.0.0/gems/dalli-2.6.4/lib/dalli/server.rb:397:in `serialize'
2013-09-18T21:05:46.606141+00:00 app[web.1]: /app/vendor/bundle/ruby/2.0.0/gems/dalli-2.6.4/lib/dalli/server.rb:269:in `set'
2013-09-18T21:05:46.606141+00:00 app[web.1]: /app/vendor/bundle/ruby/2.0.0/gems/dalli-2.6.4/lib/dalli/server.rb:60:in `request'
2013-09-18T21:05:46.606141+00:00 app[web.1]: /app/vendor/bundle/ruby/2.0.0/gems/dalli-2.6.4/lib/dalli/options.rb:18:in `block in request'
Here's the simple part where I tried to use the fragment cache:
json.cache! [category[:products], I18n.locale] do
json.products category[:products] do |product|
json.partial! product
end
end
And the product partial :
json.(
product,
:id,
:name,
:picture,
:price,
:subcategory
)
json.product_options product.product_options do |option|
json.(
option,
:id,
:name,
:option_type
)
json.option_items option.product_option_items do |item|
json.(
item,
:id,
:name
)
end
end
json.partial! 'api/app_styles/app_style', app_style: product.app_style
- note that the app_style partial is used everywhere and works with other cached views
Source: (StackOverflow)
I am building a Rails backend to an iPhone app.
After profiling my application, I have found the following call to be especially expensive in terms of performance:
@messages.as_json
This call returns about 30 message objects, each including many child records. As you can see, a single message json response may make many DB calls to be composed:
def as_json(options={})
super(:only => [...],
:include => {
:user => {...},
:checkin => {...}
}},
:likes => {:only => [...],
:include => { :user => {...] }}},
:comments => {:only => [...],
:include => { :user => {:only => [...] }}}
},
:methods => :top_highlight)
end
On average the @messages.as_jsoncall (all 30 objects) takes almost 1100ms.
Wanting to optimize I've employed memcached. With the solution below, when all my message objects are in cache, average response is now 200-300ms. I'm happy with this, but the issue I have is that this has made cache miss scenarios even slower. In cases where nothing is in cache, it now takes over 2000ms to compute.
# Note: @messages has the 30 message objects in it, but none of the child records have been grabbed
@messages.each_with_index do |m, i|
@messages[i] = Rails.cache.fetch("message/#{m.id}/#{m.updated_at.to_i}") do
m.as_json
end
end
I understand that there will have to be some overhead to check the cache for each object. But I'm guessing there is a more efficient way to do it than the way I am now, which is basically serially, one-by-one. Any pointers on making this more efficient?
Source: (StackOverflow)
I am trying to implement fragment caching in a Rails 3.0.19 application and ussing dalli as cache store. Here is my cache fragment script:
- @presentations.each do |p|
- cache "presentation", p do
= render_presentation_object p
render_presetnation_object actually render out specific partial based on some conditions. I also added a sweeper into my controller.
cache_sweeper :presentation_sweeper, :only => [:create, :update, :destroy]
caches_action :update
Here is the code for sweeper:
class PresentationSweeper < ActionController::Caching::Sweeper
observe Presentation
def after_save(presentation)
expire_cache(presentation)
end
def after_update(presentation)
expire_cache(presentation)
end
def after_destroy(presentation)
expire_cache(presentation)
end
def expire_cache(presentation)
expire_fragment "presentation", presentation
end
end
When I try to update any thing from controller using this code @presentation.update_attributes(params[:presentation]) , it gave an error ActiveRecord::StatementInvalid (RuntimeError: can't modify frozen object:
Is there any thing I am missing out?
Source: (StackOverflow)