Prevent code to execute more than once at the same time

Sometimes you don't want code to be executed more than once at the same time. E.g. for synchronizing the database with some other source.

You can do this by making a lock file at the beginning of the piece of code checking if that lockfile doesn't already exists and removing it on the end of the piece of code.

However, cross-OS there are different ways to create a lockfile. By using tendo as an installable Python package, this is made very easy.

First install it like so:

$ pip install tendo==0.2.4

Now add this to your code:

from tendo import singleton
def example_function(): # make sure this function is never executed twice
    only_once_preventer = singleton.SingleInstance(flavor_id='insert-an-unique-id')
        do_some_code()

# remove the singleton (actually redundant, because auto-removes when this function ends)
    del only_once_preventer

Note that the key-argument flavor_id is only needed when you have more than one singleton in that file.

For more details see the source code.

However, this solution doesn't work when for example you have more than 1 Celery worker, where both Celery workers are located on different machines.
A lock file will be created on one Machine but is not visible on the other machine, hence there's no lock.

So in this case it's also possible to use Memcached that sets a flag. Whereby, the location of the Memcached is shared between the two systems. In this example the Memcached is located in the database and the project is Django.

First you need to update your CACHES settings in settings.py:

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'cache_table_name',
    }
}

Now you want to create the cache table like this:

$ python manage.py createcachetable cache_table_name

It might be handy to put the creation of the cachetable into a South migration.

First, create a datamigration:

$ python manage.py datamigration project create_cache_table 

Whereby, project is the name of the application you want to put the migration in. I use literally project because that's the location where I put project wise logic in (e.g. mixins.py, base views.py, base.html etc.). Make sure that if you use Django==1.6 and your application doesn't contain a models.py, you add an empty models.py. Otherwise South is unable to create a datamigration. This will be fixed in Django==1.7.
And whereby create_cache_table is the name of your datamigration.

Now update the created datamigration like so:

# -*- coding: utf-8 -*-
from south.db import db
from south.v2 import DataMigration
from django.core.management import call_command


class Migration(DataMigration):

    def forwards(self, orm):
        call_command('createcachetable', 'cache_table_name')

    def backwards(self, orm):
        db.delete_table('cache_table_name')

    models = {

    }

    complete_apps = ['project']
    symmetrical = True

With this datamigration a cache-table will be created on migration.

What does the updated code look like?

def example_function(): 
# make sure this function is never executed twice
if cache.get('unique-cache-key-for-this-function') is not None:
# TODO: you might want to log this happend here
pass
else: cache.set('unique-cache-key-for-this-function', True, 60 * 60) # 1 hour try:
    do_some_code()
finally:
# Make sure lock is released. Whatever the circumstances
cache.delete('unique-cache-key-for-this-function')

Where unique-cache-key-for-this-function is the unique cache key.