Have you been wondering what the redis_git_dirty stat is in the Redis info command is?

Me too, so I downloaded the source and started grepping. What I found was that it is a constant that is defined in the src/mkreleasehdr.sh file. The exact command run to gather the value is:

GIT_DIRTY=`git diff 2> /dev/null | wc -l`

What is it doing? It's counting the number of lines when running the git diff command against the redis source code.

The purpose? It looks like it's a development flag to indicate how different the compiled version of redis is from what the last git commit was.

Should you monitor it? Nope, it's a waste of bits. In theory, this should always be 0 in a production release.

I tend to write a fair amount of command-line applications in Python that more often than not are meant to run as daemons. I also tend to use the same patterns in doing so. At first I wrote daemonization code myself following the general pattern that can be found in many places. Then I discovered python-daemon, the reference implementation of PEP-3143.

For logging, I would often use the same code, cut-and-paste from one application to another. After digging into the logging documentation for Python 2.7, I decided that DictConfig in logging was for me, but I needed support in 2.6. I also wanted something I could install via pip instead of copying the code from 2.7's logging package and including it in my code. Thus logging-config was born. ^*Edit: I have now removed logging-config and moved to logutils thanks to a comment by Vinay Sajip below.

Today, I am releasing clihelper, a Python module that aims to make writing command-line applications and daemons in Python easier. It uses python-daemon and logging/logutils together with a YAML based configuration file to let one focus on writing the core application and not the details about how to deal with command-line option handling, configuration, logging and daemonization.

Getting started with clihelper is meant to be very straightforward; simply extend the clihelper.Controller class:

class MyApp(clihelper.Controller):
    def _process(self):
        self._logger.info('Would be processing at the specified interval now')

Next, in the main guard for the python module you are putting the MyApp class clihelper.setup method should be called, then call the clihelper.run method passing in the class that will be used as your application controller.

if __name__ == '__main__':
    clihelper.setup('MyApp', 'MyApp is just a demo', '0.0.1')
    clihelper.run(MyApp)

Next, the configuration file should be created:

Logging:
  version: 1
  formatters:
    verbose:
      format: '%(levelname) -10s %(asctime)s %(process)-6d %(processName) -15s %(name) -10s %(funcName) -20s: %(message)s'
      datefmt: '%Y-%m-%d %H:%M:%S'
  handlers:
    console:
      class: logging.StreamHandler
      formatter: verbose
      level: DEBUG
      debug_only: True
    file:
      class: logging.handlers.RotatingFileHandler
      formatter: verbose
      level: DEBUG
      filename: example.log
      maxBytes: 1024
      backupCount: 3
  loggers:
    MyApp:
      handlers: [console, file]
      level: DEBUG
      propagate: true
    clihelper:
      handlers: [console, file]
      level: DEBUG
      propagate: true

Now invoke your application via the command line. Try passing --help to see the base level options:

Usage: usage: example.py -c  [options]

MyApp is just a demo

Options:
  -h, --help            show this help message and exit
  -c CONFIGURATION, --config=CONFIGURATION
                        Path to the configuration file.
  -f, --foreground      Run interactively in console

clihelper allows you to add your own command line options and does not need to be interval based. Instead, one can use a blocking IOLoop or other similar concepts. In the class that extends clihelper.Controller, redefine the clihelper.Controller._loop method. In addtion, you'll likely want to extend the clihelper.Controller._shutdown method to tell the IOLoop to stop when the application has been signalled to stop. An example with the Tornado IO Loop may look something like:

import clihelper
from tornado import ioloop

class Test(clihelper.Controller): 
    
    def _loop(self):
        self._ioloop = ioloop.IOLoop.instance()
        self._ioloop.start()

    def _shutdown(self):
        self._ioloop.stop()

There are a few other options, so if you're inclined to try it out, I suggest reading the README and the code. I hope that someone else will find this useful. If you have any suggestions or improvements, please do not hesitate to send them my way.

In a search for a non-hacky way to disable direct mappings to iTunes for the play/pause/next/previous buttons in the keyboard in OSX, I stumbled across the "Remote control daemon" (rcd). As it turns out, disabling the iTunes launch behavior for these keys is as easy as unloading rcd with the following command:

launchctl unload -w /System/Library/LaunchAgents/com.apple.rcd.plist

I've not run across any negative impact of doing so and now when I use Enqueue, the buttons work for just it.

Do you use memcached in your operational environment for an application? I'm looking to come up with common use (not opinion) of memcached in production environments to see if my opinions are off base.

Take the survey

Since there is no plugin option in Homebrew's install of RabbitMQ one might have a hard time finding how to install RabbitMQ plugins. Assuming you used the default location of /usr/local for your homebrew setup, you only need to copy the .ez files for the plugins to the following directory:

/usr/local/Cellar/rabbitmq/2.4.1/lib/rabbitmq/erlang/lib/rabbitmq-2.4.1/plugins

In my case, I downloaded the plugins to my Downloads folder (~/Downloads) and issued the following command:

mv ~/Downloads/*ez /usr/local/Cellar/rabbitmq/2.4.1/lib/rabbitmq/erlang/lib/rabbitmq-2.4.1/plugins

Once you’ve moved the files there, you’ll need to restart the RabbitMQ server to have them loaded. If you used the launchctl instructions during the RabbitMQ brew install for starting and adding the service, you should be able to issue two commands in terminal:

launchctl stop com.rabbitmq.rabbitmq-server
launchctl start com.rabbitmq.rabbitmq-server

That’s all there is to it. If you installed the Management plugin, you can now test by going to http://localhost:55672.

On Friday, Doug Hellmann of PyMOTW fame commented on Twitter that he'd be interested in the breakdown of laptop vendors for people attending PyCon, something I had been pondering that morning as well.

In a attempt to come up with an approximation of this number, this morning I kicked off Kismet to listen on the network for approximately 10 minutes while it collected 2,371 MAC addresses. Taking the data it collected, I wrote a script that looked up the MAC addresses it found to look the manufacturer. I was surprised at inconsistency of the company names in the MAC address vendor database. I cleaned up the multiple versions of manufacturer strings and ended up with data I could then easily chart.

The results confirmed the impression one gets just walking around and looking at what people are using:

A pretty strong showing of Apple devices. It appears that the lower end numbers are either mobile devices on the network or network gear. If you're interested in the count data, here's a text file with the breakdown.

Example rejecting a message with Pika:

channel.basic_reject(delivery_tag=method_frame.delivery_tag, requeue=True)

Basic.Nack is a RabbitMQ only extension of the AMQP protocol and expands upon the Basic.Reject behavior in a few ways. The important distinction in this context is that Basic.Nack allows for multiple delivery-tags to be passed back upon receipt of the message. As a side note, Basic.Nack is also implemented in RabbitMQ as a mechanism for the server to reject a Basic.Publish message sent via from a publishing client. An example use case for server to client Basic.Nack is when the user_id property of a message sent with Basic.Publish does not match the authenticated user for the connection.

Example of sending Basic.Nack with Pika:

channel.basic_nack(delivery_tag=method_frame.delivery_tag, requeue=True)

Since RabbitMQ currently does not have a built-in dead-letter system for when requeue=False when using either Basic.Reject or Basic.Nack, I have employed using a dead-letter queue for rejected messages. When using this approach, I send a Basic.Reject message with requeue=False and then a Basic.Publish with the message to the dead-letter queue.

Example of this when using Pika:

channel.basic_recover()

Vorpal Bunny is an experimental rabbitmq-jsonrpc-channel PHP driver designed to allow expedited single call HTTP delivery for Basic.Publish calls to RabbitMQ.

I've been trying to pick up Sphinx to document my projects and have spent a few hours today trying to force it to do what I want. In essence I was looking for a way using the autodoc functionality to automatically replace module.Class references in docblock text with python domain markup.