Before I started writing Python code, I spent most of my time working with PHP. There are a few things that PHP makes first class citizens to the language. One of them is dynamic class inclusion, and the other is the ability to dynamically call a function.
Python on the other hand does not make either of these easy. There is good reason. Both autoloading classes and dynamically executing functions lean toward the magical side of programming. Code that uses dynamic function calls can become confusing very fast.
Simple is better than complex. Complex is better than complicated... Readability counts.
What is a Dynamic Function Call?
You may be wondering what I mean by "dynamically" call a function. Let's look at a quick example in PHP. Even if you have never looked at PHP, you should be able to understand what is going on.
function area( int $length, int $width ) {
print( $length * $width );
}
$area_func_name = 'area';
// Method 1
call_user_func( $area_func_name, 4, 5 );
// Output: 20
// Method 2
$area_func_name( 3, 5 );
// Output: 15Note: in this example the type declarations are optional, but they are a good practice.
In this example, the function area() can be called as long as we know the name of the function i.e. area. As long as we know the name of the function we want to call, we can execute the function and even pass in arguments.
This is the idea behind dynamic function execution. The function that will be executed is not determined when the code is written. It is determined at runtime.
Doing this in Python is a little more complicated. As we said earlier, that is not necessarily bad. To do this in Python you have to access the global namespace. Python makes you do this explicitly where PHP is implicit.
def area(length: int, width: int):
print(length * width)
area_func = globals()["area"]
area_func(5, 5)
# Output: 25In this example we use the globals() function to access the global namespace. globals() returns a dictionary that includes area as a key and the value is a reference to the area() function. If you are not familiar with globals() a simple change to our code can give you a clear idea of what we are doing.
def area(length: int, width: int):
print(length * width)
print(area)
# Output: <function area at 0x00B2F1D8>
print(globals()["area"])
# Output: <function area at 0x00B2F1D8>You can see that both area and globals()["area"] point to the same address in memory 0x00B2F1D8. This means that in the current scope calling area(5, 5) and globals()["area"](5, 5) would run the exact same function.
Don't Abuse Globals
Just because you can do something does not mean you should. Using globals() in this way is often frowned on as neither pythonic nor safe. It is safer to use locals() or vars() to access just the local scope, but still not ideal.
Fortunately, the objects in scope in this way is not as dangerous as it could be. Python built in functions and classes are available in the "__builtins__" key. This is very important, since It means calling globals()["exec"](malicous_code) will raise a KeyError. You would actually need to call globals()["__builtins__"]["exec"](malicous_code).
Let's be honest Python's built in functions and classes are not the only exploitable objects in any program.
A better way is to use a class to encapsulate the methods you want to make available to execute at runtime. You can create each function as a method of the class. When the name of a function is provided you can test to see if it is an attribute of the class and callable.
import math
class Rectangle:
length: int
width: int
def __init__(self, length: int, width: int):
self.length = length
self.width = width
def do_area(self):
area = self.length * self.width
print(f"The area of the rectangle is: {area}")
def do_perimeter(self):
perimeter = (self.length + self.width) * 2
print(f"The perimeter of the rectangle is: {perimeter}")
def do_diagonal(self):
diagonal = math.sqrt(self.length ** 2 + self.width ** 2)
print(f"The diagonal of the rectangle is: {diagonal}")
def solve_for(self, name: str):
do = f"do_{name}"
if hasattr(self, do) and callable(func := getattr(self, do)):
func()
rectangle = Rectangle(3, 5)
rectangle.solve_for('area')
rectangle.solve_for('perimeter')
rectangle.solve_for('diagonal')
# Output:
# The area of the rectangle is: 15
# The perimeter of the rectangle is: 16
# The diagonal of the rectangle is: 5.830951894845301If you are looking at the if statement in the solve_for() method and getting a little confused. Don't worry. I am using some syntax that is new in Python 3.8.
The assignment expression := allows you to both set the value of a variable and evaluate the result of an expression in a single line.
The function is equivalent to the following...
def solve_for(self, name: str):
do = f"get_{name}"
if hasattr(self, do) and callable(getattr(self, do)):
func = getattr(self, do)
func()The difference is that getattr(self, do) does not need to be evaluated twice.
Let's get back to our example.
The solve_for() method is really doing most of the heavy lifting here. It first takes the name and appends "do_" to the front. This is often a good idea. Your class will likely have few extra methods and attributes that you don't want to expose.
Once the name of the function has been determined we can check to make sure it is a valid attribute and a function that we can call. The last thing we do is simply call the function.
Dynamic Function Arguments
Passing arguments to the dynamic function is straight forward. We simply can make solve_for() accept *args and **kwargs then pass that to func().
def solve_for(self, name: str, *args, **kwargs):
do = f"get_{name}"
if hasattr(self, do) and callable(func := getattr(self, do)):
func(*args, **kwargs)Of course, you will need to handle the arguments in the function that will be called. Currently, none of the do_ methods in our example accept arguments.
One of the complexities of using arguments with dynamic function execution is the need for each function to handle the same arguments. This requires you to standardize your function arguments.
Dynamic Function Uses in Python
Two of the common uses for dynamic function execution in PHP are callbacks and hooks. However, in Python neither of these are very common.
I think Python's focus on being explicit keeps developers from moving toward meta and magical programming. Python also provides decorators that can be used to explicitly hook or wrap a function. These two things limit the use of dynamic function calls.
I think the best example of dynamic function execution in Python is form validation in Django. Let's say we had a form like this...
from django import forms
class RegisterForm(forms.Form):
username = forms.CharField()
email = forms.EmailField()
When Django validates and parses the values to native Python data types it will call a function for each field. The function is clean_<fieldname>() where <fieldname> is the name of the field.
In this example we could add a method to ensure the value of name is valid and formatted the way we want.
from django import forms
class RegisterForm(forms.Form):
username = forms.CharField()
email = forms.EmailField()
def clean_username(self):
data = self.cleaned_data['username']
return data.lower()
This is a very simple example. We are just taking whatever value is given in username and making it lower case. clean_username() is called by Django's BaseForm._clean_fields() method.
The _clean_fields() method from Django is a good example of how to execute a function dynamically.
class BaseForm:
...
def _clean_fields(self):
for name, field in self.fields.items():
# value_from_datadict() gets the data from the data dictionaries.
# Each widget type knows how to retrieve its own data, because some
# widgets split data over several HTML fields.
if field.disabled:
value = self.get_initial_for_field(field, name)
else:
value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
try:
if isinstance(field, FileField):
initial = self.get_initial_for_field(field, name)
value = field.clean(value, initial)
else:
value = field.clean(value)
self.cleaned_data[name] = value
if hasattr(self, 'clean_%s' % name):
value = getattr(self, 'clean_%s' % name)()
self.cleaned_data[name] = value
except ValidationError as e:
self.add_error(name, e)The reason _clean_fields() exists is not because it is imposible to do this any other way. But because it provides a clean interface for developers building forms in Django.
I think this is a good example of why you may want to use this pattern. In general, I would recommend avoiding it, but there are times when it makes a library or API easy for developers to use. In those instances, don't shy away from doing it, but make sure you do it safely!
