Talking to darner from ruby memcache gems

[jnunemaker]

I've tried kestrel-client (build on top of memcached gem), the memcached gem and the dalli gem, and all fail to perform a get from darner.

For example, I start up darner:

$ darner -d /var/spool/darner/
[INFO] 2012-Sep-07 19:41:46.008201: darner: queue server
[INFO] 2012-Sep-07 19:41:46.008847: build: Sep  7 2012 (18:54:17) v0.1.4 (c) Wavii, Inc.
[INFO] 2012-Sep-07 19:41:46.008932: listening on port: 22133
[INFO] 2012-Sep-07 19:41:46.008978: data dir: /var/spool/darner/
[INFO] 2012-Sep-07 19:41:46.009047: starting up

Then start up irb, with dalli:

$ irb -r rubygems -r dalli
1.9.3p0 :001 > client = Dalli::Client.new 'localhost:22133'
 => #<Dalli::Client:0x007fae532e3f20 @servers="localhost:22133", @options={}, @ring=nil> 
1.9.3p0 :002 > client.get 'foo'
I, [2012-09-07T19:44:05.305352 #77838]  INFO -- : localhost:22133 failed (count: 0)
Dalli::RingError: No server available
    from /Users/jnunemaker/.rvm/gems/ruby-1.9.3-p0/gems/dalli-2.1.0/lib/dalli/ring.rb:45:in `server_for_key'
    from /Users/jnunemaker/.rvm/gems/ruby-1.9.3-p0/gems/dalli-2.1.0/lib/dalli/client.rb:250:in `perform'
    from /Users/jnunemaker/.rvm/gems/ruby-1.9.3-p0/gems/dalli-2.1.0/lib/dalli/client.rb:50:in `get'
    from (irb):2
    from /Users/jnunemaker/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'

Same thing from the memcached gem:

$ irb -r rubygems -r memcached
1.9.3p0 :001 > client = Memcached.new 'localhost:22133'
 => #<Memcached:0x007fe0430bc838 @struct=#<Rlibmemcached::MemcachedSt:0x007fe0430bc7e8>, @options={:hash=>:fnv1_32, :no_block=>false, :noreply=>false, :distribution=>:consistent_ketama, :ketama_weighted=>true, :buffer_requests=>false, :cache_lookups=>true, :support_cas=>false, :tcp_nodelay=>false, :show_backtraces=>false, :retry_timeout=>30, :timeout=>0.25, :rcv_timeout=>0.25, :poll_timeout=>0.25, :connect_timeout=>4, :prefix_key=>"", :prefix_delimiter=>"", :hash_with_prefix_key=>true, :default_ttl=>604800, :default_weight=>8, :sort_hosts=>false, :auto_eject_hosts=>true, :server_failure_limit=>2, :verify_key=>true, :use_udp=>false, :binary_protocol=>false, :credentials=>nil, :experimental_features=>false, :exception_retry_limit=>5, :exceptions_to_retry=>[Memcached::ServerIsMarkedDead, Memcached::ATimeoutOccurred, Memcached::ConnectionBindFailure, Memcached::ConnectionFailure, Memcached::ConnectionSocketCreateFailure, Memcached::Failure, Memcached::MemoryAllocationFailure, Memcached::ReadFailure, Memcached::ServerEnd, Memcached::ServerError, Memcached::SystemError, Memcached::UnknownReadFailure, Memcached::WriteFailure, Memcached::SomeErrorsWereReported]}, @default_ttl=604800, @not_found=#<Memcached::NotFound: Memcached::NotFound>, @not_stored=#<Memcached::NotStored: Memcached::NotStored>> 
1.9.3p0 :002 > client.get 'foo'
return else
"foo"
8
Memcached::ProtocolError: Key {"foo"=>"localhost:22133:8"}
    from /Users/jnunemaker/.rvm/gems/ruby-1.9.3-p0/gems/memcached-1.4.3/lib/memcached/memcached.rb:623:in `reraise'
    from /Users/jnunemaker/.rvm/gems/ruby-1.9.3-p0/gems/memcached-1.4.3/lib/memcached/memcached.rb:600:in `check_return_code'
    from /Users/jnunemaker/.rvm/gems/ruby-1.9.3-p0/gems/memcached-1.4.3/lib/memcached/memcached.rb:517:in `get'
    from (irb):2
    from /Users/jnunemaker/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'

set seems to work for the memcached gem, but dalli always reports darner as dead. I also tried with the raw flag true, to avoid marshaling, but it still failed.

I was able to use telnet just fine, so it seems to be something specific that the ruby gems need in the memcache protocol, but I'm not familiar enough. I can try to dig in, but I thought I would see if it is something obvious or easy that I am missing first.

I currently use kestrel with the kestrel-client gem no problem. I'd love to test out darner. Any ideas?

[henrikbjorn]

I am not a ruby developer, but tried to talk to darner anyways as it seemed wierd it wouldnt work.

# Gemfile
source 'https://rubygems.org'

gem 'memcache'

# test.rb
require 'rubygems'
require 'bundler/setup'
require 'memcache'

cache = Memcache.new :server => 'localhost:22133'

cache.set 'ruby', 'this is the payload'

puts cache.get('ruby').inspect

Running that through the cli ruby binary would output "this is the payload".

Tried with Dalli and couldnt get it to work.

[jnunemaker]

It definitely was weird to me. That gem is really old. I don't believe it has been updated in several years. memcached and dalli are probably the most widely used of late, I would guess.

[erikfrey]

@jnunemaker it looks like dalli uses the newer memcached binary protocol: https://github.com/mperham/dalli#dalli--

Neither Darner nor Kestrel currently supports the binary protocol, but it makes a lot of sense to, and it's a pretty easy one to parse.

But there's no reason why the kestrel client shouldn't work. I'll have a poke at that.

[erikfrey]

I inspected kestrel-client and found it was sending gets with whitespace after the key, probably a side effect of supporting multiple keys. Darner now handles this as of 479b5ce - thanks @jnunemaker.

To handle clients like dalli I've opened #15 - thanks for the sleuthing, guys.

[jnunemaker]

Awesome! Thanks for the fix. Can't wait to play around again.

[henrikbjorn]

Tested with kestrel-client and memcached. works :)