Binding Class Attributes In Python (Part 2)

Hello everyone, sorry I haven’t blogged for a while but I was busy with other matters and barely managed to allocate enough time over the weeks to write this article. My last article was about how you can use Python to bind to class attributes together so that updating one automatically updates the other. I also demonstrated how we used this method in a college project to implement a MIPS processor by defining each block, defining its inputs and outputs (in terms of inputs) and then, using the demonstrated method, tell Python that the output of this block is connected to the input of the other block and just letting everything evaluate itself automatically. This was, by far, the most viewed article on my blog, so if you haven’t read this article, I recommend you go do that now, just follow this link. The old article also contains a lot of essential background which is required for understanding this article, so if you want to make the most of this article, read the old one first or just google the topics you do not understand when they come up ^_^ .

Motivation

You might be wondering, if we already have a method for binding class attributes together, why do we need this new method. Here are my top reasons for researching this alternative method.

1. The old method requires classes to be defined in a certain way, it was not a general case solution.
2. The old method was one way only. This means that updating instance 1 updates 2, but you cannot update 2 and have it automatically update 1.
3. Once the class attributes were binded, they cannot be unbinded.
4. It is always super AWESOME and fun to think of new, and better solutions to problems :D.

Implementation

Let us start by defining a simple class, we will call it ValueContainer.

Simple enough, no? If you’re wondering what will we be using this for, then congratulations on having a brain :D but I guess you’ll have to find out shortly. Next, we will define a new class that we will be using as a descriptor. If you do not know what descriptors are, feel free to google them now. The most popular descriptor you have used is probably the property decorator but we will be defining our own descriptor now. Let’s us start simple and call this descriptor BindingDescriptor.

Okay, so this BindingDescriptor will be initialised and will be given the name of the attribute it is controlling access to. It will also initialise 2 dictionaries. One called getters and the other called setters. The intention is that getters will have instances as their keys and functions as their values. Whenever the function corresponding to a certain instance is called, the value that is “binded” will be returned. Setters has the same mapping except that the function will take an argument that will be set as the “binded” value. We will also define a method called add_instance() that we will use to insert more instances with their getter and setter functions into the getters and setters dictionaries.

Let’s put aside BindingDescriptor for now and move on to our third and final Class. Before we give it a name, You must know that I am a Communications and Electronics Engineering college student, so all the names I give are 100% biased towards my study. Therefore, I will name this class Wire, and I will add a method inside called solder(). This method, should be called with the arguments instance1, instance2, attrname1, attrname2, and to start off simple, we will assume (temporarily) that the instances that will be binded (or soldered) together have the same parent class and the attrnames that are being binded are exactly the same. We will also make the assumption that both instances have not been binded to anything previously. Here is the definition of Wire.solder() with the assumptions we mentioned before.

If you do not know what is the @staticmethod, getattr(), or setattr(), feel free to google.

Now the final piece of the puzzle (assuming our assumptions are correct), defining the __get__() and __set__() of the BindingDescriptor.

And now, let us test to see if it works (with our assumptions in effect)

It works!! (mostly). Remember the assumptions we made earlier? Lots of problems happen if they are not upheld. Moreover, tons of other problems can happen for various other reasons. Therefore, in the next section I will be discussing all these problems and their solutions. A gentle warning though, things are going to start getting messy.

Problems and their solutions

Instances will never get deleted from memory

If you’ve worked with a language like C before, you have probably dealt with the pain of cleaning up the memory manually whenever something is not useful anymore. Thankfully, pythonistas never have to deal with this; thanks to the garbage collector. The garbage collector will remove something from memory when there are no more references to it. Remember when we used the instances as keys for our getters and setters dictionaries? Using them as keys will be considered as referencing them and therefore they will NEVER get deleted from memory because there will always exist a reference to them inside the getters/setters dictionary. To solve this, we will replace the standard dictionary with a WeakKeyDictionary from the weakref module. This special dictionary will allow us to use instances as keys without increasing their reference counter and we can still deal with it as a normal dictionary (with special considerations that are not required here). Here is our updated __init__() from the BindingDescriptor class.

Using the WeakKeyDictionary will allow instances to get garbage collected as soon as all non-weak references of it get removed.

Accessing attributes on instances that are not binded

Remember the 2 final steps of the Wire.solder() method? We set the descriptor on the class of the instance and not the instance. Google will be happy to tell you why, but long story short, that’s how descriptors work. The problem is, under the same assumptions, if we try to access a third instance that is not binded (but the other 2 are) this happens.

See that KeyError exception? It’s because, after soldering a and b, the attribute name “testvalue” on all TestClass instances is now accessed via our BindingDescriptor. This descriptor’s __get__() method expects an instance to have a getter function, but since the c instance was never soldered, it does not have a getter key in this dictionary and therefore will raise a KeyError Exception. Let us fix this by using the __dict__ of the instance to return the testvalue of c. Let us also do the same on the __set__() method of the BindingDescriptor. Here are the updated __get__() and __set__()

Let us test and see if it works.

Great! It does, now let us move on to the next problem.

Soldering a third instance desolders one of the other 2 instances

In this problem, We will remove the assumption that all instances are initially soldered. Maybe you soldered b to a, and now you wish to solder c to a. Will this work with our current implementation?

As you can see, soldering c to b, desoldered a from the both of them. This is because we defined new getter and setter functions for this binding. These new getter and setter functions update a different ValueContainer instance, not the one we used in the earlier binding. What we should have done, is recycle the old getter and setter functions instead of using new ones so that all bindings reference the same ValueContainer object. Moreover, we created a new BindingDescriptor that overwrote the old BindingDescriptor; we should have also recycled the old BindingDescriptor. Solving this problem is a 3 step process. First of all, let us define 2 new methods on the BindingDescriptor

We will use retrieve_getter() and retrieve_setter() to retrieve an old getter or setter if it exists from the getters/setters dictionaries (note: accessing the dictionary using the .get() method will return None if the instance is not found in the dictionary).

Otherwise, we will just re-use the BindingDescriptor, getter, and setter. To implement this more cleanly, I moved getter and setter creation to a new method under Wire called request_new_getter_and_setter(). To be able to call this method, I changed the Wire.solder to a classmethod instead of a staticmethod. Here are the changes I made.

On the 3^rd line in the Wire.solder() method we attempt to get the old BindingDescriptor. If the descriptor does not exist, then it will raise an AttributeError exception. We handled this by saying that the potential melt is None, but what if the descriptor existed? The problem is, because attempting to access the attribute (attrname1) on the parent class is done through the descriptor itself, the descriptor must be told to return itself if the attribute was being accessed on None instance (means the access is being done on the parent class itself). Let us fix this. Here is the updated __get__() method on the BindingDescriptor

Now let us test and see if it works

Huzzah!!!, It works, let us move to the next problem.

Binding instances of different classes, or same classes but different attrnames or different classes and different attrnames

This is by far the most annoying of all problems and to tackle it we will need to get rid of all of our assumptions that we made earlier and start thinking of how we will deal with all the different cases. To make this easier, we will first define a check called completeMatch. completeMatch will be True when both instances are of the same parent class and attrnames match. We will test for completeMatch at the very beginning of the Wire.solder() code.

Since we got rid of all our assumptions, we now have a complete mess. Here are some cases that we can encounter.

1. Both instance/attrname combinations might have not been binded before, and therefore will not have a BindingDescriptor, getter, and setter.
2. One of the instance/attrname combinations might have a BindingDescriptor, getter, and setter functions but the other does not have these.
3. One or both of the instance/attrname combinations might have a BindingDescriptor but no getter or setter functions; This might happen if another instance from the same parent class was binded before on the same attrname.

First, we will try to get the BindingDescriptor of instance1 and instance2. We will call them potential_melt1 and potential_melt2 respectively. If the BindingDescriptor was not found we will set its potential_melt to None.

To clean things up, we will first define a method on Wire called retrieve_getter_and_setter(). This method will try to retrieve the getter and setter from the BindingDescriptor of instance1, if it fails, it will try to get them from the BindingDescriptor of instance2, if it fails, it will just request_new_getter_and_setter() and return them.

Now let us use our newly created method to attain getters and setters.

Note: The getter and setter functions will always be the same for both classes, this is the main concept that makes this idea work in the first place.

The updated solder method will now work as follows.

Okay so let us test this against lots of ciritcal cases and normal cases.

And here are the test results.

Once instance attributes are soldered, they cannot be desoldered.

This problem is an easy pick, because desoldering an instance is as easy as removing its getter and setter functions from the respective descriptor. So let’s define define a Wire.desolder() method on the Wire class. The Wire.desolder() method should be given the instance, the attribute name and an optional argument of whether we want to maintain the soldered value, or restored to the value this instance had before being desoldered. Here is the definition I wrote for the desolder method.

Now we need to go ahead and define the remove instance method on the BindingDescriptor. So here is how we will do this.

And that’s it!! First we will retrieve the original value using the getter of the instance in case we need it. We will then proceed to delete the getter and setter function of the instance. Afterwards, if the user wishes to maintain the soldered value on the desoldered instance, we will update the instance’s __dict__ with the value that was previously soldered on this instance.

If the BindingDescriptor did not exist on the class, trying to getattr it in the Wire.desolder() method would have resulted in an AttributeError Exception. If the BindingDescriptor existed but did not have this instance as a key in either its getters or setters dictionary, this will raise a KeyError exception. We will catch both exceptions in the Wire.desolder() method we defined earlier and ignore them.

Let us try and see if this works. Here are the tests and the test results.

Performance

After reading part 1 of this article and this part, you might be wondering which one is the faster method? I performed quick tests to get an overview of which method would be faster in a real life application and found this new method to perform consistently faster.

Conclusion

Being able to bind or solder python class attributes together opens up an entire range of possibilities. Imagine being able to get rid of callback functions that do nothing except update values. But before you get too excited, this is Python and not magic. Instead of updating the variables yourself, you’re letting Python do it for you. So do not expect this to have any less update time for the variables you are trying to access.

Please tell me what you think of this article, what do you want to see me write about, and if you have any question or feedback leave it in the comments section below and I will respond as soon as I am able.