Python Callables: The Basics and the Secrets

PYTHON PROGRAMMING

In programming languages, callable objects are typically associated with functions, and for good reason. Functions are perhaps the best examples of callable objects, but they are not the only ones. In Python, there are many other callable types, which can be incredibly useful and powerful. You can also create your own callable objects. This article is about both.

---

A callable is an object that can be called using a pair of parentheses, like for example below, where we use the built-in function sum():

Python-repl">>>> sum([1, 1, 1])
3

A call to a callable, depending on its definition, can be

• without any arguments, as in no_args_callable()
• or a series of positional and/or keyword arguments, as in args_callable(arg1, arg2), args_callable(arg1, arg2=value2) or args_callable(arg1=value1, arg2=value2)

Above, I described a callable as a noun. The word callable, however, is also used as an adjective, meaning being a callable. Therefore, a callable is the same as a callable object.

Python has a built-in function , callable(), that checks if an object is callable, or – in other words – if it's a callable. Consider the following examples of actual callables:

>>> callable(lambda x: x + 1)
True
>>> callable(print)
True
>>> def foo(): ...
>>> callable(foo)
True

The below objects are not callables:

>>> callable(None)
False
>>> callable(10)
False
>>> callable("hello")
False

The positive examples above were about functions, which are what most people associate with callables. However, in fact, every Python class is callable. If you know the basics of object-oriented programming in Python, you know that to create an instance of a class, you do the following:¹

>>> class Empty: ...

This looks exactly like a call, and it is – and that is why Python classes are callables.

This code shows that the Empty class is callable, but the truth is, every single Python class is callable. However, in Python terminology, the term "callable classes" is usually used to indicate something different: a class whose instances are callable.

Our Empty class is callable, but its instances are not:

>>> empty_instance = Empty()
>>> empty_instance()
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'Empty' object is not callable

This code throws TypeError, because instances of the Empty class are not callable:

>>> callable(empty_instance)
False

To make a class's instances callable, you need to implement a .__call__() method. We do so below, although the method is empty – it does nothing:

>>> class EmptyCallable:
...     def __call__(self): ...
>>> empty_call_instance = Empty()
>>> callable(empty_call_instance)
True
>>> empty_call_instance()

As you see, nothing happened – which basically means that Empty() return None. This time, however, no error was thrown, as instances of EmptyCallable are indeed callable.

Note: Every Python class is callable, meaning that calling it creates a new instance of the class. However, not all instances of Python classes are callable. To make an instance callable, you must implement the __call__() method in the class body. We typically name a class as callable only when its instances are callable, even if theoretically this is not fully correct.

Examples of callables

Sometimes the best way of showing something is… to do it. Hence, I will provide a list of examples of callables, and then we will discuss when it is useful to make an object callable.

Let's start with the most obvious examples and work our way down to less common ones.

Regular and lambda functions

>>> def foo(): ...
>>> callable(foo)
True
>>> callable(lambda: ...)
True

Classes and class instances

>>> class Empty:
...     def __call__(self): ...
...     def method(self): ...
...     @staticmethod
...     def static_method(): ...
...     @classmethod
...     def class_method(cls): ...
>>> callable(Empty)
True
>>> callable(Empty.class_method)
True
>>> instance = Empty()
>>> callable(instance)
True
>>> callable(instance.method)
True
>>> callable(instance.static_method)
True

Functions from the operator module

>>> from operator import mul, itemgetter, or_
>>> callable(mul), callable(itemgetter), callable(or_)
(True, True, True)

Partial objects

>>> from functools import partial
>>> def foo(x, y, z): return f"{x = }, {y = }, {z = }"
>>> foo(1, 2, 3)
'x = 1, y = 2, z = 3'
>>> fooxy5 = partial(foo, z=5)
>>> fooxy5(1, 2)
'x = 1, y = 2, z = 5'
>>> callable(fooxy5)
True

Decorators

>>> def decorator(func):
...     def inner():
...         print("I'm a callable and I return one!")
...         return func
...     return inner
>>> callable(decorator)
True
>>> callable(decorator(foo))
True

Closures

>>> callable(lambda x: lambda y:x*y)
True

Let's pause here for a moment. This code line requires some explanation.

I would never recommend writing this code. The only advantage to doing so is that it demonstrates your understanding of Python intricacies. However, it should never be written in production code.

Let's see how it works:

>>> mult = lambda x: lambda y:x*y
>>> mult_by_5 = mult(5)
>>> mult_by_5(2)
10

Do you see what's happening? The mult function is an example of a closure: a function that returns a function, and the inner function can access free variables in the outer function's scope, even after the outer function, mult(), has returned.

If you are familiar with the operator module, you know that some of its functions are closures. Examples include mul, add, itemgetter, and methodcaller. It's good to be familiar with closures and to understand how they work, as understanding them elevates your Python skills to a deeper level.

Closures can be very useful in Python and are used in many different ways, but that is not the topic of this article. We will discuss them another time. What matters to us now is that a closure is callable and returns a callable, which is the same thing we observed for the decorator decorator above.

Let's rewrite the closure to make it more readable; it will be easier to see what's happening and why closures both are and return callable objects:

>>> def defmult(x):
...     def inner(y):
...         return x * y
...     return inner
>>> defmult_by_5 = mult(5)
>>> defmult_by_5(2)
10

Definitely cleaner. If you noticed that decorators are just specific examples of closures, you're right. Anyway:

>>> callable(defmult)
True
>>> callable(defmult_by_5)
True

So, closures are and closures return callables.

For a short moment, I'd like to return to functools.partial. Any discussion of callables in Python must not ignore them – as partial objects constitute an extremely useful tool.

It's yet another subject that deserves a dedicated article, so I will only show some simple examples of these powerful callables. The official Python documentation explains partial objects as follows:

The partial() is used for partial function application which "freezes" some portion of a function's arguments and/or keywords resulting in a new object with a simplified signature.

Hence, you can achieve similar things as you do with closures. Remember this?

>>> mult = lambda x: lambda y: x*y 
>>> mult_by_5 = mult(5)
>>> mult_by_5(2)
10

We can achieve the same in the following way:

>>> from functools import partial
>>> def mult(x, y): return x*y
>>> partialmult_by_5 = partial(mult, y=5)
>>> partialmult_by_5(2)
10

You can create a new callable with some (even all, if that's what you need) of the arguments being assigned particular values. You can use it also to change the default values of a function's arguments:

>>> def multiply_str(s: str, n: int = 2) -> str:
...     return s*n
>>> multiply_str("abc")
'abcabc'
>>> multiply_str_5 = partial(multiply_str, n=5)
>>> multiply_str_5("abc")
'abcabcabcabcabc'

Technically, partial objects are not functions:

>>> type(multiply_str)

>>> type(multiply_str_5)

>>> callable(multiply_str_5)
True

They are callables, and they are partial objects. It's worth to mention also functools.partialmethod(), which creates partial objects to be used as class methods, unlike functools.partial(), whose objects are used as functions. If you're interested, I hope to publish the dedicated article in the near future; and for the moment, you can read the official documentation.

When to make an object callable?

As shown above, not only is Python full of callable objects, but also we can create them with ease. The following article shows this:

A Callable Float? Fun and Creativity in Python

I show there how to implement a callable class, Float, that inherits from float. The class's instances are callable, so we're talking about callable floating-point numbers.

Why bother at all? What can you achieve that way? Consider this example taken from the above article, and if you're interested in the implementation of the Float class, you'll find it there:

>>> i = Float(12.105)
>>> 2*i
24.21
>>> i(round)
12
>>> i(lambda x: 200)
200
>>> i(lambda x: x + 1)
13.105
>>> def square_root_of(x):
...     return x**.5
>>> i(square_root_of)
3.479224051422961
>>> i(lambda x: round(square_root_of(x), 5))
3.47922
>>> i = Float(12345.12345)
>>> i(lambda x: Float(str(i)[::-1]))
54321.54321

So, you can call a Float number and provide a function as an argument, and the function will be applied to the number kept by the instance. The article in question did not make a point that such a callable class is useful; nor do I want to make it here. Not trying to prove a class like that makes sense, the article discussed creativity in Python and showed the fun of Python coding.

Why am I mentioning this, then? Because the article does show something else. It shows that you can create callable objects with ease. The point is to know not only how to do it, but also – if not mainly – when to do it.

Some examples when you may want to create a callable are as provided below. I will omit the most obvious one, like a need of creating a function or a class.

• Function-like objects. For example, you might want to create a function-like object that takes arguments and returns a value, but that also has additional state or behavior. Closures, decorators and context managers are good examples.
• Design patterns that use callable objects. The Strategy pattern is a perfect example; it allows you to define a family of algorithms, encapsulate each one, and make them interchangeable.
• Dynamic functions and dynamic callables. This means creating a function or a callable at runtime. Callable objects allow you to do this easily.

This was pretty technical and theoretical, so let's analyze a practical example. Imagine you have a class ClassifyTextTo that aims to classify a text into several categories. Let's ignore implementation details, instead focusing on the class's design. We can write the following prototype of this class:

class ClassifyTextTo:
    def __init__(self, config, path):
        self.config = config
        self.path = path
    def read_text(self):
        ...
    def preprocess_text(self):
        ...
    def classify(self):
        ...
    def diagnose(self):
        ...
    def report(self):
        ...
    def pipeline(self):
        self.read_text()
        self.preprocess_text()
        self.classify()
        self.report()

The .pipeline() method explains the whole process:

• the text is read from self.path
• the text is preprocessed, to make it ready for the classification model
• the classification model is run; it's configured in self.config
• the model is diagnosed
• a report is created and logged

The class works for a particular text – or more specifically – for a particular file located in path. So, for each file you create an instance of the class and run the pipeline, like here:

>>> classify1 = ClassifyTextTo("texts/text1.txt", configuration)
>>> classify1.pipeline()
>>> classify2 = ClassifyTextTo("texts/text2.txt", configuration)
>>> classify2.pipeline()

If you have more such texts, you can do it in a loop:

>>> for text in texts:
...     classify = ClassifyTextTo(text, configuration)
...     classify.pipeline()

or, simpler:

>>> for text in texts:
...     ClassifyTextTo(text, configuration).pipeline()

Note that we're creating as many instances of ClassifyTextTo as we have texts. Do we need to do so?

When you have a pipeline to run in a class, oftentimes creating a callable class is a natural thing to do. Consider this alternative prototype:

class CallClassifyTextTo:
    def __init__(self, config):
        self.config = config
    def read_text(self, path):
        ...
    def preprocess_text(self):
        ...
    def classify(self):
        ...
    def diagnose(self):
        ...
    def report(self):
        ...
    def __call__(self, path):
        self.read_text(path)
        self.preprocess_text()
        self.classify()
        self.report()

Although the implementation does not look all that different, the difference is significant and lies in the design. While ClassifyTextTo needed to create an instance per path, CallClassifyTextTo doesn't. If the configuration is the same for all texts, we can use just one instance. In fact, we could do so even if the configuration needed to change from path to path, but that way we would lose the big advantage of the design – using the very same instance for each path:

>>> classify = CallClassifyTextTo(configuration)
>>> classify("texts/text1.txt")
>>> classify("texts/text2.txt") 

or:

>>> classify = ClassifyTextTo(configuration)
>>> for text in texts:
...     classify(text)

This is a natural thing because now we have the classify object, which is callable, thanks to the .__call__() method in the ClassifyTextTo class. It's a little cheaper than the first approach, as it creates only one instance of the class.

The most important thing for me is that calling classify() per path is a natural thing to do, as it means running the whole pipeline for path after path. I like the simplicity of this design:

• The instance of the CallClassifyTextTo class represents a particular model, not a path.
• Running a model is an action, and calling the instance does represent this action, just like calling a function does.

I am not saying this is the only correct approach. In programming, oftentimes several approaches will be correct. In such cases, I use several criteria to decide which one to use:

• code readability and simplicity
• how well the code design reflects the actual objects and actions
• performance
• the team's and my own preferences

Conclusion

We discussed the basics and intricacies of callables in Python Programming. I'd say that you need to know both if you want to be an advanced Python developer. Lucky for us, they are not as difficult as they may seem at first glance.

On the other hand, I'm just talking about understanding how callables work in Python. This is not the same as understanding every single scenario in which they can be used. For instance, closures are quite a complex topic, and understanding them is a totally different thing from understanding how to create classes with callable instances. Often, the only closures you will use are decorators, but one day you may need to use them for other scenarios. One example is the rounder package and the _do()function:

rounder/rounder/rounder.py at main · nyggus/rounder

Nevertheless, I hope this article helped you understand the basics of Python callables and take a look at some of their intricacies. From now on, in your Python work, remember that Python callables can be quite powerful and consider whether using them can improve the code design of your project.

---

Footnotes

¹ The Empty class does nothing. However, this does not mean that it cannot have any sensible uses. For example, it could serve as a sentinel, that is, an object that is used to indicate a specific state or condition. None is the best-known example of a sentinel.

---

Thanks for reading. If you enjoyed this article, you may also enjoy other articles I wrote; you will see them here. And if you want to join Medium, please use my referral link below:

Join Medium with my referral link – Marcin Kozak

Tags: Data Science Function Functional Programming Python Python Programming

Comment