Python Descriptors for Better Data Structures

I worked as a developer of a testing framework for a C++ server. The framework tested binary protocol implemented by the server.

Most of the work involved testers preparing test cases. The data format was primitive structures – hard to read and easy to break. Field order and all the data had to be entered manually.

At the time, I have already seen the better world – the models from Django. Have you ever wondered how those work? Step by step, I used the ideas from there to make the structures more friendly and on my way I rediscovered descriptors.

I’ll show in incremental steps, how:

• used keyword arguments to lower signal to noise ratio,
• order of definition for sorting the fields,
• realized that __call__ is used instead of assignment,
• used __setattribute__ as first step to extend primitive fields,
• discovered that I’m actually reimplementing descriptors,

and how it lead me to:

• implement printing in a way that is friendly to regression testing,
• use diff library for less code and better results,
• implement more readable validation.

I want to show how descriptors work in Python and how they enable declarative style of programming. By the end of the talk I want you to understand what is at the core of the magic behind field types used by object relational mappers like Django.