Google App Engine memcache wrapper

According to Google's App Engine documentation, Memcache is a "high-performance, distributed memory object caching system, intended for use in speeding up dynamic web applications by alleviating load on the datastore." In this post I show a method to decrease code duplication when fetching multiple kinds of entity using Memcache.

This function forms the core of the scheme I use.

def fetch_entity(memcache_key, fetch_from_db):
    entity=memcache.get(memcache_key)
    if entity is None:
        entity=fetch_from_db()
        memcache.add(memcache_key, entity, 60)
    return entity

First Memcache is checked for a cached entity with the given key. If no entity is found in the cache, then the entity is fetched from the database using the "fetch_from_db" method passed in as a parameter. The enity is cached using the key and is returned by the function.

I use this function for fetching all the entities for my Google App Engine app at etapic.name.

Google App Engine Request Dispatcher

Google has a document that describes how to run the application. At the bottom of the page is a tip that states, "You could have all non-static URLs go to a single Python script, and have the script dispatch all dynamic URLs to handlers." In this post I will expand on this statement by showing the implementation I came up with for etapic.name. Here is the basic structure of my main method.

def main():
    url_path=r'%s(.*)'
    url_mappings=[]
    for handler in [FirstRequestHandler,
                    SecondRequestHandler,
                    …,
                    RootRequestHandler]:
        url_mappings.append((url_path%handler.app_url, handler))
    from google.appengine.ext.webapp import WSGIApplication
    application=WSGIApplication(url_mappings, debug=True)
    from wsgiref.handlers import CGIHandler
    CGIHandler().run(application)

First I setup a format string for the url_path associated with each handler. Each of my handlers has a field named app_url which is the prefix for all URLs associated with the handler. The final handler in the list in the RootRequestHandler and its app_url is ‘/’. I iterate over a list of handlers and set up the url_mappings list.

All of my handlers extend my BaseHandler. The BaseHandler has methods to create the base template dictionary, to handle get and post requests and to handle exceptions. Each of the Handlers implements the get method and imports their module and call the base classes handle method.

Test Syntax Highlighter

def real_main():
    """Every request passes through this function. It sets up a format for the
    regex pattern used as the URL path in the URL mappings."""
    url_path=r'%s(.*)'
    url_mappings=[]
    for handler in [SwapRequestHandler, 
                    BlogRequestHandler, 
                    GuestbookRequestHandler,
                    RootRequestHandler]:
        url_mappings.append((url_path%handler.app_url, handler))
    from google.appengine.ext.webapp import WSGIApplication
    application=WSGIApplication(url_mappings, debug=True)
    from wsgiref.handlers import CGIHandler
    CGIHandler().run(application)

Google App Engine Development Environment

Google App Engine is based on Python. The best IDE for Python is Eclipse with the Pydev plugin. You can run dev_appserver.py in debug mode, set break points, inspect variables, and evaluate expressions in the context of the breakpoint. To setup your appengine project in Pydev, first create a new Pydev project. Choose Python 2.5 and uncheck the option to create a source folder named 'src'. Create two source folders, one named "test" and the other the name of your project. Having a separate source folder for tests is a good idea, because that way when you run appcfg.py update you won't upload your tests to the production server. Giving your main source folder the same name as your project is a good idea, because it makes the appcfg.py update command explicit about which project you are updating.

In order to debug dev_appserver.py you need to add a source folder for the appengine source and its three libraries. In Eclipse create a new folder (not a source folder), and click the Advanced button. Check the box for linking to an existing folder and use the Browse button to choose you appengine source directory (on Windows it is typically C:\Program Files\Google\google_appengine). Now right click on the new folder and choose add as source folder. Then navigate to the following three folders and add them as source folders:

• lib/django
• lib/webbob
• lib/yaml/lib

Now right click on dev_appserver.py and select Run as Python application. You will see the help message printed in the console. Open the Run... dialog. Set the program arguments to '-c '. The -c option clears your local datastore on each startup. Set the working directory to your Eclipse project root (the parent directory of your main source folder). Now you can click debug and the server will startup. Be sure to setup Eclipse so it launches the previously launched application, because you will be working in your source code
but you will want to launch dev_appserver.py. This setting is in Preferences>Run/Debug>Launching. You will probably notice a warning message printed in the console that says Psyco is not installed. Install it, it takes 5 seconds.

This is the setup I use to develop my app engine apps, volt.appspot.com and etapic.name, and I am very happy with it.

Google App Engine Videos from Google IO

The Google App Engine team posted a list of relevant videos from Google IO. I watched all of them last night. The first two are by far the best. I’m writing a collection of applications using the app engine. My current URL is http://volt.appspot.com, but I plan to point my own domain, http://etapic.name/, to my app engine instance soon.