Double underscore (dunder) methods, also known as special methods in Mojo🔥 are used to define how objects behave in specific situations. Mojo🔥 is a member of the Python family and it embraces Python's dunder method syntax and extends it in new ways. You can use dunder methods to customize the behavior of common operations such as +, -, *, **, ==, %, @ and others on custom data structures, initialize and delete objects, get string representation, enable indexing on objects and more. For example, you can use dunder methods to define __add__() to define element-wise addition operation using the + operator, on custom objects that represent vectors or tensors.

Mojo🔥 supports several dunder methods and in this blog post we'll dive deep into every single dunder method listed below with code examples. They are broadly categorized into the following types:

1. Initialize, copy, move, delete
2. Getters and Setters: obj[idx], obj[idx]=val
3. Unary arithmetic operators: -obj, +obj, ~obj
4. Comparison operators: <, <=, ==, ~=, >, >=
5. Normal, reversed and in-place arithmetic operators: +, -, *, @, /, //, %, ** , <<, >>, &, |, ^
6. Defined by built-in Traits: len(obj), int(obj), print(obj)

Note: 1-5 are compiler dunder methods and 6 are dunder methods defined by built-in Traits that enable functions like len(), int() and print() to work on custom objects.

Dunder methods in Mojo🔥

Let’s say you want to implement the following equation in Mojo:

$z = x^3 + 3x - \frac{x}{5} + 0.3 + y^2$

It's easy to do that in Mojo if you’re working with scalar floating point variables:

Output:

What if you wanted to extend this capability to custom objects that implement vectors? We’ll have to explicitly define the behavior of these three operations **, *, / for two cases:

1. When used between a scalar and a vector
2. When used between a vector and a vector

You can define these behaviors in the dunder methods __pow__(), __mul__() and __truediv__() with overloaded versions for scalar-vector and vector-vector operations. Through the rest of this blog post I'll walk you through creating an example Mojo struct (similar to a class in Python) that implements most of the dunder method supported by Mojo.

Note: This example was built with Mojo SDK v0.7. Future releases may add, change or remove supported dunder methods and/or Traits.

DunderArray: An example Mojo struct that demonstrates how to use Mojo dunder methods 

The hero of this blog post is the DunderArray. We’ll implement a DunderArray struct which has the following qualities:

1. DunderArray is an array data structure that is parameterized by a type (Float64, Float32, Bool, etc.)
2. DunderArray implements all Mojo dunder methods shown in the figure above.
3. DunderArray also implements several dunder methods defined by built-in Traits

Using DunderArray you can do scalar-vector and vector-vector math operations like the example below, making it easy to implement mathematical equations. The complete example is on GitHub.

DunderArray struct has the following skeleton structure:

We’ll work our way through every dunder method defined in DunderArray struct, and for each dunder method in DunderArray we’ll discuss its usage and implementation.

1. Initialize, copy, move, delete

__init__()

The __init__() method is used to initialize a new object when it is created, set its initial attributes or perform additional setup. You can also overload it to allow different types of initializations. 

Usage:

Output:

Implementation:

The two different initializations seen above DunderArray(5) and DunderArray(1,2,3,4,5) call two different __init__ functions:

You can have any number of overloads for __init__() to customize how you want to initialize your struct. The complete example on GitHub also includes an __init__() function that can load NumPy arrays into DunderArray.

Since my goal is to introduce you to dunder methods in Mojo, I’ve implemented these methods in DunderArray.

__copyinit__()

The __copyinit__() dunder method is invoked when the assignment operator = is used to make a copy of an object.

Usage:

You can assign a DunderArray variable a to a new variable b:

Output:

Implementation:

When a is assigned to b, __copyinit__() is invoked, which creates a copy of the underlying data in DunderArray and assigns it to other which in this case is variable b

You can learn more about copy constructors in the Mojo programming manual.

__moveinit__()

Similarly, when we want to move the value of a variable, its ownership is transferred from one variable to another using __moveinit__()

Usage:

Output:

Implementation:

You can learn more about move constructors in the Mojo programming manual.

Note: __copyinit__ and __moveinit__ are unique to Mojo🔥 and are not supported in Python. Mojo is Pythonic in syntax but also lets you incrementally adopt systems-programming features like strong type checking and memory management. You can also avoid implementing __init__(), __copyinit__() and __moveinit__() completely by using the @value decorator, as discussed in this doc page.

__del__()

The __del__ dunder method is used by Mojo to end a value’s lifetime. 

Usage:

Mojo uses static compiler analysis to find when a value is last used and calls __del__ to perform the clean up. 

Implementation:

In DunderArray this function simply frees the pointer. You can add any additional steps necessary to cleanup if needed.

You can learn more about destruction behavior in the programming manual.

2. Getters and Setters

In Mojo and in Python, the __getitem__() method is used to define how an object should behave when accessed with square brackets arr[index]. The __setitem__ method is used to specify the behavior when assigning a value to an item using square brackets arr[index] = value.

__getitem__()

Usage:

In DunderArray we can get an element from the array by using square bracket indexing:

Output:

Implementation:

We implement this logic in the __getitem__() method to load and return a value at a specific index.

You can also overload __getitem__() to implement slicing using the syntax arr[a:b] to retrieve values between a and b. I wrote an earlier blog post on how to implement NumPy style matrix slicing in Mojo🔥and you can find the code and explanation on how to do that in that blog post.

__setitem__()

DunderArray, lets you access specific elements in the array using square bracket indexing:

Usage:

Output:

Implementation:

The __setitem__() dunder method lets you define the behavior of the square bracket assignment arr[idx] = value operation.

3. Unary operators

Unary arithmetic operators in Mojo are used to perform operations on a single operand, and they include + for positive, - for negation, and ~ for boolean inversion.

__pos__() and __neg__()

Usage:

Output:

Implementation:

In the __pos__() and __neg__() dunder methods, we multiply each element of the DunderArray with +1.0 or -1.0. These functions assume the multiplication operation * has already been defined on DunderArray. We’ll discuss that in the arithmetic operations section.

__invert__()

Usage:

Since DunderArray is parameterized on DType, we can also initialize a bool array. The __invert__() dunder method allows us to define the behavior of boolean inversion

Output:

Implementation:

To invert a DunderArray we have to first check that the array is of type bool. We do that using the compile time conditional _mlirtype_is_eq[Scalar[dtype], Bool]() which returns True if the array is boolean. If it is a boolean array, we use the elementwise function to invert the array in a fast vectorized way.

4. Comparison operators

__lt__(), __le__(), __eq__(), __ne__(), __gt__(), __ge__()

DunderArrays can be compared with each other using these comparison dunder methods. When comparing vectors, you have to define a metric on the vector that you wish to compare between vectors. In DunderArray struct we compute the euclidean norm and compare the norm values of two arrays.

Usage:

Output:

Implementation:

DunderArray struct implements the following comparision dunder methods:

• __lt__: Less than comparison.
• __le__: Less than or equal to comparison.
• __eq__: Equality comparison.
• __ne__: Inequality comparison.
• __gt__: Greater than comparison.
• __ge__: Greater than or equal to comparison.

5. Normal, reflected and in-place arithmetic operators

In Mojo, normal arithmetic operators perform the standard arithmetic operation between two operands, such as a + b for addition, a - b for subtraction, a * b for multiplication, and a / b for division. These are implemented with __add__(), __sub__(), __mul__() and __div__() dunder methods. 

Reflected arithmetic operators are used when the left operand does not support the corresponding normal operator. For example, a float Scalar does not support addition with DunderArray, because it doesn’t know what a DunderArray is. The DunderArray struct can implement a reflected addition operator specified by the __radd__() that can perform the operation. 

Finally, in-place arithmetic operators combine the current value of the left operand with the right operand and assign the result to the left operand. For instance, a += b is an in-place addition, and it is equivalent to a = a + b. These operators are specified by methods such as __iadd__(), __isub__(), __imul__(), and __itruediv__().

__add__(), __radd__(), __iadd__()

You can add a scalar to a DunderArray or add two DunderArrays together.

Usage:

Output:

Implementation:

In DunderArray we use two helper functions to calculate elementwise operations between floating point scalars and DunderArray (_elemwise_scalar_math) and DunderArray and DunderArray _elemwise_array_math

__sub__(), __rsub__(), __isub__()

Similarly, you can subtract DunderArrays and scalars

Usage:

Output:

Implementation:

__mul__(), __rmul__(), __imul__()

Usage:

Output:

Implementation:

__matmul__(), __imatmul__()

You can also define a special behavior for the @ operators, which is typically reserved for matrix multiplication. Since we're working with vectors only, and * already defines elementwise vector multiplications, we'll use the @ operators to calculate dot products between DunderArrays:

Usage:

Output:

Implementation:

The implementation is similar to * but we also reduce the result using the helper function _reduce_sum

__truediv__(), __rtruediv__(), __itruediv__()

Usage:

Output:

Implementation:

__floordiv__(), __rfloordiv__(), __ifloordiv__()

Floor division is similar to true division operation, bit it rounds down the result to the nearest integer, yielding the largest integer that is less than or equal to the exact result. DunderArray implements the //

Usage:

Output:

Implementation:

__mod__(), __rmod__(), __imod__()

DunderArray implements the modulus operation using the % operator, which returns the remainder of the elementwise division of two arrays or between one array and a scalar.

Usage:

Output:

Implementation:

__pow__(), __ipow__()

DunderArrays can be raised to an integer power:

Usage:

Output:

Implementation:

To calculate the power of a floating point variable we'll implement a helper function called _elemwise_pow() shown below.

__lshift__(), __rshift__(), __ilshift__(), __irshift__() 

DunderArrays can be left shifted (<< n) which performs elementwise shifts the bits which is effectively multiplying it by $2^n$. Right shift (>> n) performs elementwise bits to the right, akin to dividing the number by $2^n$

Usage:

Output:

Implementation:

DunderArray must be of the type Int32

__and__(), __iand__(),__or__(), __ior__(),__xor__(), __ixor__()

DunderArray can also be initialized as a boolean array by using the dtype=DType.bool. With a boolean DunderArray, you can perform standard boolean operations such as and, or, and xor described below.

Usage:

Output:

Implementation:

• __and__: Bitwise AND operation between DunderArray objects.
• __iand__: In-place bitwise AND operation.
• __or__: Bitwise OR operation between objects.
• __ior__: In-place bitwise OR operation.
• __xor__: Bitwise XOR operation between objects.
• __ixor__: In-place bitwise XOR operation.

6. Defined by built-in traits

__len__(), __int__()

DunderArray conforms to the Sized and Intable traits as shown here:

This means DunderArray must implement __len__() and __int__() which lets you can call the len() and int() functions on DunderArray objects. For the purpose of illustration, we return the same values for both functions.

Usage:

Output:

Implementation:

__str__()

DunderArray also conforms to the Stringable trait, so we must implement the __str__() that returns a String that can be printed using the print() function. You've seen this in action throughout this blog post, and here you'll can see how it's implemented.

Usage:

Output:

Implementation:

Conclusion

That's it folks, I hope you enjoyed this deep dive into all the dunder methods. To sum up, using dunder methods in Mojo🔥 provides a way to customize struct object behaviors. On GitHub you'll find both a Jupyter notebook that shows you how to use DunderArray and a Mojo file which implements the DunderArray.

If you're new to Mojo🔥 here are some additional getting started resources:

• Get started with downloading Mojo
• Head over to the docs to read the programming manual and learn about APIs
• Explore the examples on GitHub
• Join our Discord community
• Contribute to  discussions on the Mojo GitHub
• Read and subscribe to Modverse Newsletter 
• Read Mojo blog posts, watch developer videos and past live streams
• Report feedback, including issues on our GitHub tracker

Until next time🔥!

‍