The missing documentation for django.utils.datastructures
Thu 01 November 2012 by Thejaswi PuthrayaCross posted from my work blog.
Note
django.utils.datastructures is intentionally not documented by the django core devs because it is an internal API and is liable to change without any notice. This file is not governed by django's lenient backwards-compatible policy. You have been sufficiently warned!
With the note out of the way, let's look at the interesting datastructures in this file. You may ask why we should learn about those when we shouldn't be using them? Reading code is the best way of learning and this file has some beautiful code.
MergeDict is the first of the datastructures in the file. It provides a dictionary like interface but can look up from multiple dictionaries provided during the initialization.
Here's an example:
>>> md = MergeDict({"foo": "bar", "moo": "cow"}, {"abc": "def"})
>>> md["foo"]
'bar'
>>> md["abc"]
'def'
>>> md.get("abc")
'def'
>>> md["xyz"]
KeyError:
>>> md.items()
[('foo', 'bar'), ('moo', 'cow'), ('abc', 'def')]
>>> md.keys()
['foo', 'moo', 'abc']
>>> md.values()
['bar', 'cow', 'def']
The MergeDict is used within django in attaching values with a form widget and in request.REQUEST.
The built-in dictionary does not maintain the order of the items but the SortedDict is a subclass of the built-in dictionary that maintains the keys in exactly the same order they were inserted.
Here's an example:
>>> dd = {"foo": "bar", "moo": "cow", "abc": "def"}
{"abc": "def", "foo": "bar", "moo": "cow"}
>>> sd = SortedDict((("foo", "bar"), ("moo", "cow"), ("abc", "def")))
{"foo": "bar", "moo": "cow", "abc": "def"}
>>> dd["xyz"] = "pqr"
>>> dd
{'abc': 'def', 'foo': 'bar', 'moo': 'cow', 'xyz': 'pqr'}
>>> dd["lmn"] = "ghi"
>>> dd
{'abc': 'def', 'foo': 'bar', 'lmn': 'ghi', 'moo': 'cow', 'xyz': 'pqr'}
>>> sd["xyz"] = "pqr"
>>> sd
{'foo': 'bar', 'moo': 'cow', 'abc': 'def', 'xyz': 'pqr'}
>>> sd["lmn"] = "ghi"
{'foo': 'bar', 'moo': 'cow', 'abc': 'def', 'xyz': 'pqr', 'lmn': 'ghi'}
The SortedDict is fairly widely used inside of django generally to build a hierarchy (like models and it's parents), maintaining the order of the form fields while iterating etc.
In python 2.7, a new datastructure was introduced that mimics the SortedDict in the collections module and is called OrderedDict.
MultiValueDict is a dictionary subclass that can handle multiple values assigned to a key.
Here's an example:
>>> dd = {"abc": "def", "foo": ["bar1", "bar2"]}
>>> dd["foo"]
['bar1', 'bar2']
>>> mvd = MultiValueDict({"abc": "def", "foo": ["bar1", "bar2"]})
>>> mvd["foo"]
'bar2'
>>> mvd.getlist("foo")
['bar1', 'bar2']
>>> mvd.getlist('blah')
[]
>>> mvd.getlist('abc')
'def'
>>> mvd.setlist('xyz', ['pqr', 'ghi'])
>>> mvd
<MultiValueDict: {'xyz': ['pqr', 'ghi'], 'abc': 'def', 'foo': ['foo1', 'foo2']}>
>>> mvd.appendlist('xyz', 'ijk')
>>> mvd
<MultiValueDict: {'xyz': ['pqr', 'ghi', 'ijk'], 'abc': 'def', 'foo': ['foo1', 'foo2']}>
>>> mvd.update({'xyz': 'lmn'})
>>> mvd
<MultiValueDict: {'xyz': ['pqr', 'ghi', 'ijk', 'lmn'], 'abc': 'def', 'foo': ['foo1', 'foo2']}>
The MultiValueDict is used in binding data to request.POST, the files to request.FILES and in the get parameter parsing.
The ImmutableList is an immutable datastructure that raises errors when it is attempted to be mutated.
Here's an example:
>>> il = ImmutableList(['foo', 'bar', 'abc'])
>>> il += 'lmn'
AttributeError: ImmutableList object is immutable.
>>> il = ImmutableList(['foo', 'bar', 'abc'], warning='Custom warning')
>>> il[1] = 123
AttributeError: Custom warning
The ImmutableList is used in request.upload_handlers to prevent modification after the request.POST or request.FILES have been accessed.
The DictWrapper is a subclass of the built-in dictionary that prefixes the keys. It takes a dictionary, a function and a prefix as arguments. If a specific key lookup begins with the prefix then the value is passed through the function before it is returned.
Here's an example:
>>> dw = DictWrapper({'foo': 'bar', 'moo': 'cow'}, lambda x: x, 'abc_')
>>> dw['foo']
'bar'
>>> dw['abc_foo']
'bar'
>>> dw['xyz_foo']
KeyError: 'xyz_foo'
>>> def post_process_value(value):
... return "The value is " + value
>>> dw = DictWrapper({'foo': 'bar', 'moo': 'cow'}, post_process_value, 'abc_')
>>> dw['foo']
'bar'
>>> dw['abc_foo']
'The value is bar'
The DictWrapper is used in quoting names for SQL queries with the key prefix.
Hope you enjoyed learning about these hidden gems and how django works under the hood but take the note on the top into consideration.
Dropbox file upload handler for django
Behind the Scenes: From request to response
Cross posted from my work blog.
In the previous installment of "Behind the Scenes", we saw how the control flows from Form to File Storage. Today, we are going to see how the application reacts from request to response.
In this post, we are going to assume that we are …
read moreBehind the Scenes: From HTML Form to Storage
Cross posted from my work blog post
In this post, we are going to see what happens behind the scenes when a file is uploaded to a django powered web application.
An HTML form with a file input (atleast one) and encoding set to multipart/form-data is submitted. The MultiPartParser …
read moreReleased django-check-constraints
Ever since model validation hit the django trunk, I had been having thoughts of reviving my Google Summer of Code 2007 project. So nearly a couple of weeks back, I started resurrecting django-check-constraints. Today, I released the code into wild here.
The feature of the project is that it not …
read moreConfista: A conference management app
RFC: Comments framework usage documentation
This is my attempt at ticket #9819 which aims for better documentation to the comments framework. This is my second attempt at writing documentation and I have uploaded a patch to the ticket but wanted some feedback here without spamming the timeline.
Example of using the in-built comments app
- Follow …
A wiki with a couchdb backend and django
Recently, I had a requirement for a personal simple wiki. I checked out various options like TiddlyWiki, MoinMoin and MediaWiki but wanted something very simple and something that didn't use a database server or flat files and was portable (and hostable). So I quickly wrote up a wiki that used …
read moreReusable App: Authenticated Comments
This app had been on my todo list for a long time. Today, no matter how much I tried to procrastinate, I could not ignore it and hence decided to scratch my itch and get it done. I swore I wouldn't even get up for a glass of water until …
read more