nil? vs. ==

Just stumbled over this question on Stackoverflow.com.

What is the difference between nil? and == nil?.

There is no difference. At least when looking at the observable outcome. And I prefer nil? because of readability. But that is just a matter of taste.

However there is a slight difference in how this outcome is calculated.

nil?

nil? is a method defined on Object and NilClass.

# NilClass
rb_true(VALUE obj)
{
  return Qtrue;
}

# Object
static VALUE
rb_false(VALUE obj)
{
return Qfalse;
}

Unless you mess with this implementation through monkeypatching a nil? check is a simple method call. For your own objects (unless they inherit from BasicObject which does not implement nil?) the implementation stems from Object and will always return false.

== nil

a == b is just syntactic sugar for sending the == message to the left hand side, passing the right hand side as sole argument. This translates to a.==(b) in the general case. And to a.==(nil) in our specific case.

== is a method defined on BasicObject:

rb_obj_equal(VALUE obj1, VALUE obj2)
{
    if (obj1 == obj2) return Qtrue;
    return Qfalse;
}

From the docs:

At the Object level, == returns true only if obj and other are the same object.

So this returns true if two variables are pointing to the same object or if the receiving class has overridden the == method in another way. And subclasses are meant to override == to implement class specific behavior.

This also means that the performance depends on the implementation of ==. Unlike nil? which should not be overridden by subclasses.

Another solution

In ruby everything besides nil and false is considered truthy. If this interpretation fits your use case, then you can avoid the check and pass the object to the if clause directly:

if some_object
  puts "some_object is neither nil nor false"
end

Performance

I’d expect nil? to be as fast as == nil. And because == might be overridden by subclasses performance should depend upon the receiver. And omitting the check should be fastest. Here is a simple benchmark to test my assumptions. As usual, I use the wonderful benchmark/ips gem by Evan Phoenix.

require 'benchmark/ips'

class ExpensiveEquals
  def ==(other)
    1000.times {}
    super(other)
  end
end

string = 'something'
number = 123
expensive = ExpensiveEquals.new
notnil = Object.new
isnil = nil

Benchmark.ips do |x|

  x.report('isnil.nil?') do
    if isnil.nil?
    end
  end

  x.report('notnil.nil?') do
    if notnil.nil?
    end
  end

  x.report('string.nil?') do
    if string.nil?
    end
  end

  x.report('number.nil?') do
    if number.nil?
    end
  end

  x.report('expensive.nil?') do
    if expensive.nil?
    end
  end

  x.report('isnil == nil') do
    if isnil == nil
    end
  end

  x.report('notnil == nil') do
    if notnil == nil
    end
  end

  x.report('string == nil') do
    if string == nil
    end
  end

  x.report('number == nil') do
    if number == nil
    end
  end

  x.report('expensive == nil') do
    if expensive == nil
    end
  end

  x.report('nil == isnil') do
    if nil == isnil
    end
  end

  x.report('nil == notnil') do
    if nil == notnil
    end
  end

  x.report('nil == string') do
    if nil == string
    end
  end

  x.report('nil == number') do
    if nil == number
    end
  end

  x.report('nil == expensive') do
    if nil == expensive
    end
  end


  x.report('isnil') do
    if isnil
    end
  end

  x.report('notnil') do
    if notnil
    end
  end

  x.report('string') do
    if string
    end
  end

  x.report('number') do
    if number
    end
  end

  x.report('expensive') do
    if expensive
    end
  end

  x.compare!
end

notnil: 11840655.4 i/s
expensive: 11801608.9 i/s - 1.00x slower
number: 11679119.8 i/s - 1.01x slower
string: 11598016.4 i/s - 1.02x slower
 isnil: 11529034.6 i/s - 1.03x slower
notnil == nil: 10397262.7 i/s - 1.14x slower
nil == expensive: 10319767.0 i/s - 1.15x slower
nil == string: 10188393.9 i/s - 1.16x slower
nil == number: 10167930.4 i/s - 1.16x slower
isnil == nil: 10120560.0 i/s - 1.17x slower
nil == notnil: 10069779.1 i/s - 1.18x slower
nil == isnil: 10055165.2 i/s - 1.18x slower
expensive.nil?:  9970905.5 i/s - 1.19x slower
string.nil?:  9967045.3 i/s - 1.19x slower
number.nil?:  9893974.5 i/s - 1.20x slower
notnil.nil?:  9581974.5 i/s - 1.24x slower
isnil.nil?:  8390963.6 i/s - 1.41x slower
string == nil:  6915330.1 i/s - 1.71x slower
number == nil:  6803969.3 i/s - 1.74x slower
expensive == nil:    25382.6 i/s - 466.49x slower

I did not expect nil? to be slower, still looking into this. But one can see that if you go for the == check, then it should be faster if you do nil == other instead of other == nil. Usual micro benchmark warnings apply.

Search across multiple models

Consider a Rails application with a search feature. Type in a name and it lists the matching Artists. Sounds simple. What if we also want to search for the city the artists were born in? Simple as well: add a join, include the city name in the where clause. Done. But what if we not only want to search for the Artists but also the Albums? And have those included in the result list as well?

Usually I try to go without additional libraries or even additional services whenever possible. Because every dependency comes at a cost. And honestly there are quite a few dependencies in Rails already. So let’s try and see how far we can get without any additional gem.

The models:

Somewhat contrived but you should get the idea.

  
class Artist < ActiveRecord::Base
  has_many :nicknames
end

class Nickname < ActiveRecord::Base
  belongs_to :artist
end

class Album < ActiveRecord::Base
end

The simple solution

Just run two queries. As simple as it can get. Perhaps wrap it in an object so your controller stays clean and the view as well:

class Search
  def initialize(query)
    @query = query
  end

  def albums
    Album.where('name like :query', query: "%#{@query}%")
  end

  def artists
    Artist.joins(:nicknames).where('nicknames.name like :query OR artists.name like :query', query: "%#{@query}%")
  end
end

It works but starts to get complicated as soon as you have pagination or you want/need to display the results in the same list. How do you merge those results?

The DB-view solution

Another solution that does not need any external dependencies: Database views. Database views can be seen as a predefined select statement that is accessible like a table. Depending on your database you can use different view types (materialized) but for this sample I want to keep it simple.

We create a database view which acts as a reverse index. It combines all the attributes we want to be part of the search and add a reference back to the model. To have multiple models included in the view we can use union which combines results from multiple tables.

This is the migration which will create the view:

class CreateSearchView < ActiveRecord::Migration
  def up
    sql = <<-SQL
      CREATE VIEW searches AS
        SELECT
          a.name || GROUP_CONCAT(n.name) AS reverse_index,
          a.id AS searchable_id, 'Artist' AS searchable_type,
          a.name AS label,
          MAX(a.updated_at, n.updated_at) AS updated_at
        FROM artists a
        JOIN nicknames n on n.artist_id = a.id
        GROUP BY a.id

        UNION ALL

        SELECT
          a.name AS reverse_index,
          a.id AS searchable_id, 'Album' AS searchable_type,
          a.name AS label,
          a.updated_at AS updated_at
        FROM albums a
    SQL

    execute(sql)
  end

  def down
    execute('DROP VIEW searches')
  end
end

Now we need to switch the schema dump format because raw SQL statements are not reflected in schema.rb

Add following line to your application.rb

config.active_record.schema_format = :sql

and then you’ll have a structure.sql instead of schema.rb after you run the migrations

Multiple things to notice:

• This is for SQLite, some functions might be different for other databases (string concatenation, max)
• Because an Artist can have multiple Nicknames we need to group the results. In order to get all the nicknames in our reverse_index column we use GROUP_CONCAT
• I added a label column to avoid N+1 selects when displaying the results
• The searchable_id and searchable_type column are named like this to make use of Rails polymorphic belongs_to association
• The different selects need to return tables of the same size/column order
• There is an updated_at column. I’ve added it to have a value I can use for ordering

With this table we can create a Search model and use it to search inside all Artist and Album records.

class Search < ActiveRecord::Base
  belongs_to :searchable, polymorphic: true
  scope :execute, -> (query) {
    where('reverse_index like :query', query: "%#{query}%")
  }
end

That’s it. You can now use it like:

2.2.0 :009 > Search.execute('slash')
  Search Load (0.4ms)  SELECT "searches".* FROM "searches" WHERE (reverse_index like '%slash%')
 => #<ActiveRecord::Relation [#<Search reverse_index: "Saul HudsonSlash", searchable_id: 2, searchable_type: "Artist", label: "Saul Hudson", updated_at: "2016-04-19 14:00:21.146916">, #<Search reverse_index: "Slash", searchable_id: 1, searchable_type: "Album", label: "Slash", updated_at: "2016-04-19 14:00:21.151739">]>

Some notes about this solution:

• The view does not have an id column, default ordering will not work
• Weighting attributes is not possible. Weighting can be used to improve the order of hits. Consider this example: when searching for “john” then a match on the name “John Doe” should be ranked higher than on a company name “Johnson & peterson”
• Performance: union can be costly, consider using union all

The takeaway

Depending on your needs there might a simple solution that does not depend on additional libraries and does not add a dependency. Don’t be afraid of SQL.

What is the method lookup path in Ruby?

A simple question but hard to answer. Why is that hard? Because Ruby has various ways of defining a method and add it to a class:

• Adding it to the singleton class
• Adding it to the class
• Include a module
• Prepend a module
• Extend a module
• Inherit from superclass

If this sounds complicated to you then that’s because it is.

So first rule: try to avoid such situations where you have a multitude of classes and modules defining the same method.

If you have more than two definitions of a method then you most likely have bigger problems than knowing about the lookup path. Also I haven’t seen many good uses of adding a method to the singleton class so far.

So how do we go about finding the lookup path? How about a small piece of code that answers this question?

module Include
  def call(level)
    puts "#{level} include"
    super(level + 1) rescue nil
  end
end

module Prepend
  def call(level)
    puts "#{level} prepend"
    super(level + 1) rescue nil
  end
end

module Extend
  def call(level)
    puts "#{level} extend"
    super(level + 1) rescue nil
  end
end

class Super
  def call(level)
    puts "#{level} super"
    super(level + 1) rescue nil
  end
end

class Klass < Super
  include Include
  prepend Prepend


  def call(level)
    puts "#{level} klass"
    super(level + 1) rescue nil
  end

end

thing = Klass.new


def thing.call(level)
  puts "#{level} singleton"
  super(level + 1) rescue nil
end

thing.extend(Extend)


thing.call(1)

What does this code do? It defines a method call for the six possibilities described above. They all print out some debugging info and then forward the call to super. Since at the end of the hierarchy the call method is not implemented I added rescue nil. Of course this would only be required for the last element in the hierarchy. But we don’t know which one this is, yet. Lets run the code and see the output:

1 singleton
2 extend
3 prepend
4 klass
5 include
6 super

What if you extend or include or prepend multiple times? The last definition comes first. That is if you have:

  
class Foo
  include Bar
  include Baz
end

then the definitions from Baz will take precedence.

And of course if you do not call super then none of the other implementations will be called.

So, now that this is solved…let’s look at another way this can be determined: ancestors. The documentation says that this “Returns a list of modules included in mod (including mod itself).”. If we extend above code to print the list of ancestors:

p thing.class.ancestors

Then we can see following:

[Prepend, Klass, Include, Super, Object, Kernel, BasicObject]

This is the order that we determined before but not complete. We are missing the methods that have been added to the singleton class. Those can be seen if we check the singleton_class instead (note that this will create the singleton class if it does not yet exist):

p thing.singleton_class.ancestors

This will print the full list of ancestors:

[#<Class:#<Klass:0x007fe34b225480>>, Extend, Prepend, Klass, Include, Super, Object, Kernel, BasicObject]

The #<Class:#<Klass:0x007fe34b225480>> is the singleton class. It exists solely for this object:

p Klass.new.singleton_class
p Klass.new.singleton_class

#<Class:#<Klass:0x007fc93913fbf0>>
#<Class:#<Klass:0x007fc93913fad8>>

This ancestry also shows how Ruby looks up methods. It does not make complicate decisions of where to look first. It just walks up the hierarchy and calls the first matching method it can find. So if the singleton class does not respond to the method, then the prepended modules will be checked until the root is reached.

Callables in Ruby

In Ruby there are several objects that respond to call. I usually refer to them as callables:

• Proc
• Lambda
• Method

There are various ways that you can invoke those callables:

• .call()
• []
• .()
• .===

I’ve talked about the case equality operator === over here. But what about the other ones? IMHO you should stick to .call() because it does not require knowledge about a special syntax.

XML Anyone?

Recently we had to deal with importing about 120Mb of XML data. Daily. It was split up into files of around 6Mb each. Those files contain information about employees, with around 60 attributes per employee and around 5000 employees per file. We need to read the files and create/update a record in the DB with the values from the XML file.

An XML file looked something like this:

<root>
  <record>
    <attr1>A name</attr1>
    <attr2>A street</attr2>
    ...
  </record>
  <record>
    ...
  </record>
</root>

Basically a CSV file in XML format…

Processing these files took ~16h. Whaaat? I don’t need to mention, that there was a bug in our code, do I (we accessed the nodes through NodeSet, see benchmark samples)? By fixing it we were able to cut down the time to about 20min. Not bad. Looked further into the code with rubyprof, I found that still a big part of time was spent in XML parsing. So I set out to build a benchmark for our use case. I was wondering if we could improve performance by replacing Nokogiri with one of the alternatives:

• OGA
• OX
• REXML

As Mike Perham described in his Kill Your Dependencies article, relying on fewer libraries (and using STDLIB instead) is better. Since Rails has a dependency I always resort to Nokogiri for XML parsing. Even though REXML works and in cases where performance does not matter could be used instead of adding another dependency to your lib.

A simplified benchmark can be found here. I was suprised that OGA (pure ruby) was that much faster than Nokogiri. OX tops that and is ~12 times faster than Nokogiri.

Nokogiri: NodeSet    280.336  (±17.1%) i/s -      1.368k
Nokogiri: Element      2.428k (± 7.4%) i/s -     12.312k
               OX     29.533k (± 3.6%) i/s -    148.188k
              OGA     18.220k (± 6.7%) i/s -     92.750k
            REXML    771.185  (± 9.2%) i/s -      3.871k

So it looks like I could improve the performance once more by switching to OGA (preferred, since pure ruby) or OX. Next step is to test the performance in real life with real input under real conditions.

Note: It looks like OX does return nil for empty bodies whereas Nokogiri returns empty string.