Serving XML-RPC over HTTPS with Django and Lighttpd

by Sam Burnett on Sept 9, 2010

xmlrpclib is a convenient way of letting Python scripts seamlessly communicate across the network. Unfortunately, by default everything is sent over unencrypted HTTP connections, making this technique unsuitable in insecure environments.

This post describes how to use xmlrpclib over HTTPS. We accomplish this using Django and Lighttpd.

Prerequisites

To begin, create a new directory for our work:

mkdir -p rpctest/serv_misc
cd rpctest

Generate an SSL Certificate

If you don’t already have one, you can generate an SSL certificate:

cd serv_misc
openssl req -new -x509 -keyout server.pem -out server.pem -days 365 -nodes
chmod 600 server.pem
cd -

Configure Lighttpd

Create a file lighttpd.conf in rpctest:

var.rootdir = "/path/to/rpctest"
var.miscdir = rootdir + "/serv_misc"

server.modules              = ( "mod_rewrite",
                                "mod_access",
                                "mod_fastcgi",
                                "mod_accesslog" )

# Just a formality: we don't actually serve any files.
server.document-root        = rootdir

server.errorlog             = miscdir + "/error.log"
accesslog.filename          = miscdir + "/access.log"

## bind to port (default: 80)
server.port                = 8000

## bind to localhost (default: all interfaces)
server.bind                = "127.0.0.1"

server.pid-file            = miscdir + "/lighttpd.pid"

# This is the part that connects lighttpd to django
fastcgi.server             = ( "/mysite.fcgi" =>
                               ( "localhost" =>
                                 (
                                   "min-procs" => 1,
                                   "socket" => miscdir + "/lighttpd.socket",
                                   "check-local" => "disable"
                                 )
                               )
                            )

# Redirect all requests to the RPC handler
url.rewrite-once = (
    "^/static(/.*)$" => "$1",
    "^(/.*)$" => "/mysite.fcgi$1",
)

# Enable SSL connections
ssl.engine                 = "enable"
ssl.pemfile                = miscdir + "/server.pem"

Configure Django

Create a new Django project:

django-admin.py startproject rpctest

Edit settings.py with your personal details. It’s best to remove all entries from INSTALLED_APPS, TEMPLATE_DIRS and MIDDLEWARE_CLASSES, because we don’t need them for our application. Also, you don’t need a database.

Put the following in urls.py:

import xmlrpclib

from django.conf.urls.defaults import *
from django.conf import settings

from djangorpc import xmlrpc, call_xmlrpc

@xmlrpc('greet')
def greet(name):
return 'Hello, %s' % name

urlpatterns = patterns('',
(r'^.*$', call_xmlrpc),
)

The last three lines are the important ones. They ensure that all requests to our Django server are routed to the XML-RPC handler. You can expose additional functions by tagging them with the @xmlrpc decorator.

Using the RPC Server

We must start both lighttpd and Django. An easy way to do this is with a tmux session (you can also use screen, if you prefer):

tmux new-session -s rpctest -d -n lighttpd '/usr/sbin/lighttpd -f lighttpd.conf -D';
tmux new-window -t rpctest -n django 'python rpctest/manage.py runfcgi method=threaded \
socket=serv_misc/lighttpd.socket pidfile=serv_misc/lighttpd.pid daemonize=false';
tmux attach -t rpctest

You can test out our new XML-RPC function using the Python shell:

Python 2.6.5 (r265:79063, Apr  1 2010, 05:28:39) 
[GCC 4.4.3 20100316 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xmlrpclib
>>> sp = xmlrpclib.ServerProxy("https://127.0.0.1:8000/server.py")
>>> sp.greet("world")
'Hello, world'
>>> 

Sending Binary Data Through xmlrpclib

To send binary data over XML-RPC, you must wrap it in an instance of xmlrpclib.Binary:

proxy = xmlrpclib.ServerProxy(rpc_url)

# Pack binary data using xmlrpclib.Binary
data = open('file.dat', 'r').read()
packed = xmlrpclib.Binary(data)
proxy.send_binary(packed)

# Unpack binary data from xmlrpclib.Binary instance
packed = proxy.receive_binary()
data = packed.data
process_binary_data(data) # Do something with the data
Send Questions and Comments to Sam Burnett.