What happens when you instantiate it (create an instance of that class)?
f = Foo(1, y=2)
That call to Foo- what function or method is being called there? Most beginners and even many experienced Python programmers will immediately answer that __init__is called. If you stop to think about it for a second, this is far from being a correct answer.
__init__doesn’t return an object, but calling Foo(1, y=2) doesreturn an object. Also, __init__expects a selfparameter, but there is no such parameter when calling Foo(1, y=2). There is something more complex at work here. In this post we’ll investigate together what happens when you instantiate a class in Python.
Instantiating an object in Python consists of a few stages, but the beauty of it is that they are Pythonic in themselves - understanding the steps gives us a little bit more understanding of Python in general. Foois a class, but classes in Python are objects too! Classes, functions, methods and instances are all objects and whenever you put parentheses after their name, you invoke their __call__method. So Foo(1, y=2)is equivalent to Foo.__call__(1, y=2). That __call__is the one defined by Foo’s class. What is Foo’s class?
So Foois an object of type typeand calling __call__returns an object of class Foo. We want to see how does the __call__method for typelooks like. This method looks fairly complicated, but we’ll try to simplify it. Below I have pasted both the CPython Cand the PyPy Python implementation. I find that looking at the original source code is very interesting, but feel free to skip to my simplification of it below:
objis then initialized by calling obj.__init__(*args, **kwargs).
Now we turn our attention to the __new__method. Essentially, it is the method responsible for actual object creation. We won’t go in detail into the base implementation of __new__. The gist of it is that it allocates space for the object and returns it. The interesting thing about __new__is that once you realize what it does, you can use it to customize instance creation in interesting ways. It should be noted that while __new__is a static method, you don’t need to declare it with @staticmethod- it is special-cased by the Python interpreter.
A nice example of the power of __new__is using it to implement a Singleton class:
Notice that in this Singleton implementation, __init__will be called each time we call Singleton(), so care should be taken.
Another similar example is implementing the Borg design pattern:
_dict = None
def __new__(cls, *args, **kwargs):
obj = super().__new__(cls, *args, **kwargs)
if cls._dict is None:
cls._dict = obj.__dict__
obj.__dict__ = cls._dict
>>> b1 = Borg()
... b2 = Borg()
... b1 is b2
>>> b1.x = 8
One final note - the examples above show the power of __new__, but just because you canuse it, doesn’t mean you should:
__new__is one of the most easily abused features in Python. It’s obscure, riddled with pitfalls, and almost every use case I’ve found for it has been better served by another of Python’s many tools. However, when you do need __new__, it’s incredibly powerful and invaluable to understand.
– Arion Sprague, Python’s Hidden New
It is rare to come across a problem in Python where the best solution was to use __new__. The trouble is that if you have a hammer, every problem starts to look like a nail - and you might “suddenly” come across many problem that __new__can solve. Always prefer a better design over a shiny new tool. __new__is not always better.
The Python Language Reference / Data Model
Eli Bendersky / Python Object Creation Sequence
Thanksto Hannan Aharonov, Yonatan NakarandRam Rachum for reading drafts of this.
See All Posts