Something Special Making it EVEN better.

20Apr/100

AMQP OmniGraffle stencil

A simple stencil for drawing AMQP and RabbitMQ topologies in OmniGraffle. Basic, but gets the job done. Enjoy! amqp

Filed under: Uncategorized No Comments
16Apr/104

RabbitMQ HA testing with HAProxy

RabbitMQ HA deploymentLots of news and talk lately about RabbitMQ being scooped up by SpringSource (a division of VMWare itself a division of EMC) but there's still work to be done! RabbitMQ has fantastic performance and scalability characteristics, but one thing it doesn't do out of the box (yet) is easy HA. As can be seen by the many posts to the mailing list, it's also a common problem. There are some guides in the works and community projects like Beetle but I have no doubt that SpringSource investing time and money in RabbitMQ will bring some basic HA to the table.

In the meantime, we have a few options. One of the easiest is to simply setup RabbitMQ in active-passive configuration behind a TCP load balancer. Producers publish through the load balancer and consumers consume from all of the individual nodes. The big issue here is that there is no library support for this setup (at least for Java clients), but a little bit of hacking on the standard Java client library and you can come up with a client abstraction that nicely fails over for you.

Since we use big, expensive F5 BigIPs at work, we as lowly, untrusted developers have limited access to setup and configure them. In comes HAProxy. I setup two VMs, rabbit01 and rabbit02 and dropped HAProxy on rabbit01 to simulate the real deployment of producers being behind the load balancer. Here's the configuration I used.

global
 log     127.0.0.1 alert
 log     127.0.0.1 alert debug

defaults
 log        global
 mode       http
 option     dontlognull
 option     redispatch
 retries    3
 contimeout 5000
 clitimeout 50000
 srvtimeout 50000

listen rabbitmq 192.168.56.11:5000
 mode    tcp
 stats   enable
 balance roundrobin
 option  forwardfor
 option  tcpka
 server  rabbit01 192.168.56.11:5672 check inter 5000 downinter 500    
 server  rabbit02 192.168.56.12:5672 check inter 5000 backup

From here, it's pretty easy to test the failover. Fire up some of the test producers and consumers from the Java client library binaries and test away, using rabbitmqctl stop_app and start_app on the RabbitMQ nodes to simulate them coming up and down.

More posts to follow on an HA Java client library (i.e. as soon as I finish writing it)!

Filed under: Uncategorized 4 Comments
21Mar/105

RabbitMQ cluster auto-configuration debugging

So being relatively new to RabbitMQ and Erlang, I tried for the first time to auto-configure a cluster of brokers, as opposed to explicitly adding them to a cluster as I have been doing up until now. The premise is simple: 1) make sure all brokers have the same Erlang cookie so they can talk to each other, 2) create a file with a list of nodes in the cluster. The documentation is pretty clear too, "create a configuration file containing the line":

[rabbit@rabbit1, rabbit@rabbit2].

 
Makes sense, I can do that. Here's my configuration file to setup brokers on two Ubuntu VM's:

[rabbit@devins-ubuntu-vm01, rabbit@devins-ubuntu-vm02].

 
Okay, fire up RabbitMQ and everything's gonna work just fine!

=INFO REPORT==== 21-Mar-2010::18:38:40 ===
    application: rabbit
    exited: {bad_return,
                {{rabbit,start,[normal,[]]},
                 {'EXIT',
                     {{case_clause,
                          {error,
                              {cannot_read_cluster_nodes_config,
                                  "/etc/rabbitmq/rabbitmq_cluster.config",
                                  {1,erl_parse,"bad term"}}}},
                      [{rabbit,'-run_boot_step/1-lc$^1/1-1-',1},
                       {rabbit,run_boot_step,1},
                       {rabbit,'-start/2-lc$^0/1-0-',1},
                       {rabbit,'-start/2-lc$^0/1-0-',1},
                       {rabbit,start,2},
                       {application_master,start_it_old,4}]}}}}
    type: temporary

11Mar/100

QCon London 2010

It's conference season again and this year I'm lucky enough to be speaking at QCon London! Okay, I kind of cheated since Nokia is one of the sponsors, but, nevertheless. My talk is about "Scaling to billions of people and places" in the context of Nokia Maps and location based services. The crowd was small but so was the room so it felt very personal and it was nice to be able to interact and answer questions from the audience in that format.

Thanks to those that came, here's the presentation along with notes for you to follow along with.

Filed under: Uncategorized No Comments
8Mar/100

Ignite Berlin video

Sabine and I recently did a talk at Ignite Berlin on Lego Manhunt. Here's the result!

Filed under: Uncategorized No Comments