EzDevInfo.com

Mongoid 3 + Heroku (MongoHQ) cause Moped::Errors::OperationFailure

With Rails 3, after upgrading from Mongoid 2.x to Mongoid 3.x, my Heroku + MongoHQ setup stopped working. Funny thing is, that my development & test frameworks and my whole test suite passes just fine.

I suspect the problem is with my mongoid.yml file, but I've tried searching the docs, google & stackoverflow, and used all the suggested formats, including this: heroku mongohq and mongoid Mongo::ConnectionFailure or actually this: https://gist.github.com/2900804

UPDATED July 16th: This is how my mongoid.yml file looks, after trying multiple things + After what Jason from MongoHQ suggested:

development:
  sessions:
    default:
      database: development
      hosts:
        - localhost:27017

test:
  sessions:
    default:
      database: test
      hosts:
        - localhost:27017

production:
  sessions:
    default:
      uri: <%= ENV['MONGOHQ_URL'] %>
      options:
        skip_version_check: true
        safe: true

(to my understanding, it is essentially the same as the one in the links above, except that it uses the uri; I tried the other way, splitting the MONGOHQ_URL into the separate fields as well, but it didn't help)

I've tried setting mongoid as 3.0.0rc and leaving the version blank in my Gemfile. Using the github version failed due to HTTPS certificates or something, so I didn't bother trying it multiple times.

What the action controller says is this:

Moped::Errors::OperationFailure in Home#index

Showing /app/app/views/home/index.html.haml where line #2 raised:

The operation: #<Moped::Protocol::Command
  @length=68
  @request_id=4
  @response_to=0
  @op_code=2004
  @flags=[:slave_ok]
  @full_collection_name=".$cmd"
  @skip=0
  @limit=-1
  @selector={:count=>:posts, :query=>{}}
  @fields=nil>
failed with error "db assertion failure"

and when I run

heroku run console 

.. I get this:

irb(main):052:0> Location.create!
NoMethodError: undefined method `[]' for nil:NilClass
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:74:in `block in command'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:522:in `[]'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:522:in `block (3 levels) in flush'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:521:in `map'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:521:in `block (2 levels) in flush'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:113:in `ensure_connected'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:517:in `block in flush'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:532:in `logging'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:516:in `flush'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:505:in `process'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:70:in `command'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/node.rb:356:in `refresh'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/cluster.rb:101:in `block in refresh'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/cluster.rb:114:in `each'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/cluster.rb:114:in `refresh'
    from /app/vendor/bundle/ruby/1.9.1/gems/moped-1.1.1/lib/moped/cluster.rb:67:in `nodes'
... 15 levels...
    from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.6/lib/active_support/callbacks.rb:405:in `__run_callback'
    from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.6/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
    from /app/vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.6/lib/active_support/callbacks.rb:81:in `run_callbacks'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/callbacks.rb:98:in `run_callbacks'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/persistence/insertion.rb:23:in `prepare'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/persistence/operations/insert.rb:26:in `persist'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/persistence.rb:50:in `insert'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/persistence.rb:251:in `block in create!'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/threaded/lifecycle.rb:173:in `_creating'
    from /app/vendor/bundle/ruby/1.9.1/gems/mongoid-3.0.1/lib/mongoid/persistence.rb:249:in `create!'
    from (irb):52
    from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.6/lib/rails/commands/console.rb:47:in `start'
    from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.6/lib/rails/commands/console.rb:8:in `start'
    from /app/vendor/bundle/ruby/1.9.1/gems/railties-3.2.6/lib/rails/commands.rb:41:in `<top (required)>'
    from script/rails:6:in `require'
    from script/rails:6:in `<main>'irb(main):053:0> 

I think I've googled everything, run through multiple github mongoid issues, read the documentation multiple times ... and I'm running out of ideas here.

Anything come to mind I should try next?

UPDATE July 16th: this is what Heroku says when I git push heroku master (After doing what Jason from MongoHQ suggested):

git push heroku master
Counting objects: 7, done.
Delta compression using up to 16 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 372 bytes, done.
Total 4 (delta 3), reused 0 (delta 0)

-----> Heroku receiving push
-----> Ruby/Rails app detected
-----> Installing dependencies using Bundler version 1.2.0.pre
       Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
       Using rake (0.9.2.2)
       Using i18n (0.6.0)
       Using multi_json (1.3.6)
       Using activesupport (3.2.6)
       Using builder (3.0.0)
            # ... shortened this #
       Using libv8 (3.3.10.4)
       Using moped (1.1.2)
       Using origin (1.0.4)
       Using mongoid (3.0.0.rc)
       Using omniauth (1.1.0)
       Using quimby (0.4.5)
       Using bundler (1.2.0.pre)
       Using rails (3.2.6)
       Using therubyracer (0.10.1)
       Using thin (1.3.1)
       Your bundle is complete! It was installed into ./vendor/bundle
       Cleaning up the bundler cache.
-----> Writing config/database.yml to read from DATABASE_URL
-----> Preparing app for Rails asset pipeline
       Running: rake assets:precompile
       Asset precompilation completed (31.24s)
-----> Rails plugin injection
       Injecting rails_log_stdout
       Injecting rails3_serve_static_assets
-----> Discovering process types
       Procfile declares types      -> (none)
       Default types for Ruby/Rails -> console, rake, web, worker
-----> Compiled slug size is 18.7MB
-----> Launching... done, v92
       http://xxxxx.herokuapp.com deployed to Heroku

To git@heroku.com:xxxxx.git
   b2d97xy..7b0aczy  master -> master

UPDATE 2 July 16th: did as Jason & MrKurt said (first forgot to run bundle update mongoid, but that didn't help either). Now the error changed, though:

NoMethodError in Home#index

Showing /app/app/views/home/index.html.haml where line #2 raised:

undefined method `[]' for nil:NilClass

Extracted source (around line #2):

    1: %h1 Most recent posts
    2: - if @posts.length > 0

Select where not null or empty in mongoid

I modified a model so it includes a new field, such as...

field :url, :type => String

I use activeadmin, so when I create a new entry @model.url is empty, and in entries created before changing the schema it's nil. How do I select both? I have tried:

# Returns nils and strings
Model.where(:url.ne => "").count 

# Returns strings and ""
Model.where(:url.ne => nil).count

# Returns strings, nils and ""
Model.where(:url.ne => ["", nil]).count 

Or, if there's a best practice for this kind of scenario please let me know.

Query on Mongoid Hash Field

I want to query on a Hash field for a Mongoid Class. I'm not sure how I can do this with conditions?

Here is an example:

class Person
  include Mongoid::Document

  field :things, :type => Hash
end

So, let's say that I have this:

p = Person.new
p.things = {}
p.things[:tv] = "Samsung"

I want to query for the first person with a tv that is a Samsung...

People.first(:conditions => ?????

Thanks in advance.

Mongoid finders not working?

Hey all I have set up a rails3+mongoid application and when I open the rails console, none of the finders seem to work - http://d.pr/FNzC

User.all
User.find(:all, :conditions => { first_name => "John" })

both return:

#<Mongoid::Criteria
  selector: {},
  options:  {}>

Am I doing something wrong?

Mongoid / Mongodb and querying embedded documents

I have Author and Book models.

An Author has many embedded Books.

Can I query the embedded Books, or do I have to fetch Authors first to get Books?

Managing mongoid migrations

Can someone give me a short introduction to doing DB migrations in Rails using Mongoid? I'm particularly interested in lazy per document migrations. By this, I mean that whenever you read a document from the database, you migrate it to its latest version and save it again.

Has anyone done this sort of thing before? I've come across mongoid_rails_migrations, but it doesn't provide any sort of documentation, and although it looks like it does this, I'm not really sure how to use it.

I should point out I'm only conceptually familiar with ActiveRecord migrations.

Mongoid or MongoMapper? [closed]

I have tried MongoMapper and it is feature complete (offering almost all AR functionality) but i was not very happy with the performance when using large datasets. Has anyone compared with Mongoid? Any performance gains ?

Using Active Record generators after Mongoid installation?

I'm using MongoDB via Mongoid integration, as well as ActiveRecord in a project. I would like to generate migrations for active record, and Mongoid is the default when I run.

rails g migration

Any ideas how to specify AR as my default generator for migrations, models, etc?

Thanks!

How to query MongoDB directly from Ruby instead of using Mongoid?

I am writing a migration for a Rails application that uses MongoDB and Mongoid. My migration currently uses my models that use Mongoid to query and update records, but the performance is sub-par. I am essentially updating all records in a large collection and making n+20 queries. I killed the migration after taking an hour to run locally (and didn't finish). I would like to be able to run raw queries to mongo without too much effort. I'm assuming there is some way to access a mongo driver from Mongoid since Mongoid has already loaded a connection to the database. How can I access the database to run my update queries direcly?

How to implement has_many :through relationships with Mongoid and mongodb?

Using this modified example from the Rails guides, how does one model a relational "has_many :through" association using mongoid?

The challenge is that mongoid does not support has_many :through as ActiveRecord does.

# doctor checking out patient
class Physician < ActiveRecord::Base
  has_many :appointments
  has_many :patients, :through => :appointments
  has_many :meeting_notes, :through => :appointments
end

# notes taken during the appointment
class MeetingNote < ActiveRecord::Base
  has_many :appointments
  has_many :patients, :through => :appointments
  has_many :physicians, :through => :appointments
end

# the patient
class Patient < ActiveRecord::Base
  has_many :appointments
  has_many :physicians, :through => :appointments
  has_many :meeting_notes, :through => :appointments
end

# the appointment
class Appointment < ActiveRecord::Base
  belongs_to :physician
  belongs_to :patient
  belongs_to :meeting_note
  # has timestamp attribute
end

Rails, Mongoid & Unicorn config for Heroku

I am using Mongoid 3, with Rails 3.2.9 and Unicorn for production. Would like to setup a before_fork & after_fork for the connection to mongodb, found the following code for active record:

before_fork do |server, worker|
  # Replace with MongoDB or whatever
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.connection.disconnect!
    Rails.logger.info('Disconnected from ActiveRecord')
  end
end

after_fork do |server, worker|
  # Replace with MongoDB or whatever
  if defined?(ActiveRecord::Base)
    ActiveRecord::Base.establish_connection
    Rails.logger.info('Connected to ActiveRecord')
  end
end

What is the relevant code for Mongoid (to connect and disconnect)?

Update:

You dont actually need to do this, so for people coming to view this question see:

http://mongoid.org/en/mongoid/docs/rails.html

"Unicorn and Passenger

When using Unicorn or Passenger, each time a child process is forked when using app preloading or smart spawning, Mongoid will automatically reconnect to the master database. If you are doing this in your application manually you may remove your code."

Though it would still be interesting to know what would be the equivalent Mongoid code.

How to reference an embedded document in Mongoid?

Using Mongoid, let's say I have the following classes:

class Map
  include Mongoid::Document

  embeds_many :locations
end

class Location
  include Mongoid::Document

  field :x_coord, :type => Integer
  field :y_coord, :type => Integer

  embedded_in      :map, :inverse_of => :locations
end


class Player
  include Mongoid::Document

  references_one   :location
end

As you can see, I'm trying to model a simple game world environment where a map embeds locations, and a player references a single location as their current spot.

Using this approach, I'm getting the following error when I try to reference the "location" attribute of the Player class:

Mongoid::Errors::DocumentNotFound: Document not found for class Location with id(s) xxxxxxxxxxxxxxxxxxx.

My understanding is that this is because the Location document is embedded making it difficult to reference outside the scope of its embedding document (the Map). This makes sense, but how do I model a direct reference to an embedded document?

Mongo DB Design, embedding vs relationships

I'm building a simple accounting system where a user has many bills. Now I'm trying to decide if bills should be its own collection, or nested within the user. I'm leaning towards the former but I've NEVER done any noSQL stuff so I'm just going by trial and error and what I think makes sense to me.

I understand that Mongo has a 4mb document size limit which is what's making me think that I should have a separate collection for bills, as these will accumulate daily and could eventually take up a large amount of space.

I'm just looking for opinions on the matter. Basically I'll be querying for bills of a user between different date periods (as you can imagine an accounting system would do).

Not that it really matters but I'm using Mongoid in a Rails3 project. I figured I'd do something like:

class User
  references_many :bills
end

class Bill
  referenced_in :user
end

Any comments or design suggestions are greatly appreciated.

Heroku & MongoHQ: ActionView::Template::Error (The operation: #

app[web.1]: >> Thin web server (v1.4.1 codename Chromeo)
app[web.1]: >> Maximum connections set to 1024
app[web.1]: >> Listening on 0.0.0.0:58731, CTRL+C to stop
heroku[web.1]: State changed from starting to up
app[web.1]: 
app[web.1]: 
app[web.1]: Started GET "/" for 212.86.9.90 at 2012-07-31 08:08:07 +0000
heroku[router]: GET cool-name-123.herokuapp.com/ dyno=web.1 queue=0 wait=0ms service=171ms status=200 bytes=1286
heroku[router]: GET cool-name-123.herokuapp.com/assets/application-8e7bfeeffc9291864e5b42d908c2fdda.css dyno=web.1 queue=0 wait=0ms service=11ms status=200 bytes=92524
heroku[router]: GET cool-name-123.herokuapp.com/assets/application-aa557bde70f1236cdf90c913043c4382.js dyno=web.1 queue=0 wait=0ms service=16ms status=200 bytes=122836
heroku[router]: GET cool-name-123.herokuapp.com/favicon.ico dyno=web.1 queue=0 wait=0ms service=3ms status=200 bytes=0
app[web.1]: 
app[web.1]: 
app[web.1]: Started GET "/originals/new" for 212.86.9.90 at 2012-07-31 08:08:11 +0000
heroku[router]: GET cool-name-123.herokuapp.com/originals/new dyno=web.1 queue=0 wait=0ms service=31ms status=200 bytes=1808
app[web.1]: 
app[web.1]: 
app[web.1]: Started POST "/originals" for 212.86.9.90 at 2012-07-31 08:08:34 +0000
app[web.1]: 
app[web.1]: Moped::Errors::AuthenticationFailure (The operation: #<Moped::Protocol::Commands::Authenticate
app[web.1]:   @length=154
app[web.1]:   @request_id=3
app[web.1]:   @response_to=0
app[web.1]:   @op_code=2004
app[web.1]:   @flags=[]
app[web.1]:   @full_collection_name="app123.$cmd"
app[web.1]:   @limit=-1
app[web.1]:   @skip=0
app[web.1]:   @selector={:authenticate=>1, :user=>"heroku", :nonce=>"xyz", :key=>"xyz"}
heroku[router]: POST cool-name-123.herokuapp.com/originals dyno=web.1 queue=0 wait=0ms service=1486ms status=500 bytes=643
app[web.1]:   @fields=nil>
app[web.1]: failed with error "auth fails"):
app[web.1]:   app/controllers/originals_controller.rb:19:in `block in create'
app[web.1]:   app/controllers/originals_controller.rb:18:in `create'
app[web.1]: 
app[web.1]: 
heroku[router]: GET cool-name-123.herokuapp.com/favicon.ico dyno=web.1 queue=0 wait=0ms service=3ms status=304 bytes=0
app[web.1]: 
app[web.1]: 
app[web.1]: Started GET "/" for 212.86.9.90 at 2012-07-31 08:09:40 +0000
heroku[router]: GET cool-name-123.herokuapp.com/ dyno=web.1 queue=0 wait=0ms service=11ms status=304 bytes=0

DATABASE_URL        => postgres://rhalppyjrb:xyz@ec2-xyz.compute-1.amazonaws.com/rhalppyjrb
GEM_PATH            => vendor/bundle/ruby/1.9.1
LANG                => en_US.UTF-8
MONGOHQ_URL         => mongodb://heroku:xyz@flame.mongohq.com:27054/app123
PATH                => bin:vendor/bundle/ruby/1.9.1/bin:/usr/local/bin:/usr/bin:/bin
RACK_ENV            => production
RAILS_ENV           => production
SHARED_DATABASE_URL => postgres://rhalppyjrb:xyz@ec2-123.compute-1.amazonaws.com/rhalppyjrb

production:
  sessions:
    default:
          uri: <%= ENV['MONGOHQ_URL'] %>
          options:
            consistency: :strong
            skip_version_check: true
            safe: true

source 'https://rubygems.org'

gem 'bundler', '1.2.0.rc'

ruby '1.9.3'

gem 'rails', '3.2.6'
gem 'thin'
gem 'rmagick', :require => 'RMagick'
gem "carrierwave-mongoid", :git => "git://github.com/jnicklas/carrierwave-mongoid.git", :branch => "mongoid-3.0", :require => 'carrierwave/mongoid'

group :assets do
  gem 'sass-rails',   '~> 3.2.3'
  gem 'coffee-rails', '~> 3.2.1'
  gem 'uglifier', '>= 1.0.3'
  gem "therubyracer", :platform => :ruby
end

gem 'jquery-rails'
gem "haml", ">= 3.1.6"
gem "mongoid", ">= 3.0.3"
gem "devise", ">= 2.1.2"
gem "devise_invitable", ">= 1.0.2"
gem "cancan", ">= 1.6.8"
gem "rolify", ">= 3.1.0"
gem "bootstrap-sass", ">= 2.0.4.0"
gem "simple_form"

group :development do
  gem "guard", ">= 0.6.2"
  gem 'rb-fsevent'
  gem 'growl'
  gem "haml-rails", ">= 0.3.4"
  gem "guard-bundler", ">= 0.1.3"
  gem "guard-rails", ">= 0.0.3"
  gem "guard-livereload", ">= 0.3.0"
  gem "guard-rspec", ">= 0.4.3"
end

group :test do
  gem "database_cleaner", ">= 0.8.0"
  gem "mongoid-rspec", "1.4.6"
  gem "email_spec", ">= 1.2.1"
end

group :development, :test do
  gem "factory_girl_rails", ">= 3.5.0"
  gem "rspec-rails", ">= 2.11.0"
end

Mongoid OR query syntax

This must be asked alot but it is very poorly documented. There is no mention at http://mongoid.org/en/mongoid/docs/querying.html

I'm trying to check whether a user exists (below is an AND query), how can I change it to an OR type query

Username.where(:username=>@username, :email=>@email)

(Either the email or the username must match).

I have found some pretty complicated ways online including sending a direct javascript (from): http://omarqureshi.net/articles/2010-6-17-using-or-in-mongoid

Surely there must be a simple clear syntax to do this correctly?