MAX is built for production inference at maximum speed. We recently redesigned our Module API to help developers build fast models even faster. Module is a standard building block in machine learning frameworks, and this post explains how and why we rebuilt this API, diving into a few key aspects:
- How the new design speeds up development
- Production inference with the new
Module - Support for distributed computation
- Simplification of the
ModuleAPIs - Familiar PyTorch-like syntax
- Where the new
Modulesits in the stack compared to its predecessor
Rebuilding MAX’s AI framework
Modular has focused on production inference with state-of-the-art performance. MAX's advantage is its compilation stack: cross-architecture, optimized Mojo kernels, plus graph compilation and execution that aggressively fuse kernels and plan memory.
The legacy Module was built directly on our graph stack. The graph-forward approach delivered excellent performance, but the developer experience left room for improvement. We closed the gap by introducing a new Eager layer on top of the graph stack, then rebuilding Module on top of it.

Familiar, PyTorch-like syntax
If you've written a PyTorch model, the new Module should feel familiar. Here is the same layer in MAX and in PyTorch:
The differences are intentional simplifications of the design: no super().__init__() call, and plain Tensors instead of nn.Parameter.
Eager execution enables faster development
Agents have fundamentally changed the development cycle. While model performance remains critical for production workloads, enabling fast iteration during development is arguably more important than ever. Paying a compilation cost on every unit test and debug log is painful.
Eager execution avoids expensive graph compilation and instead starts computing results immediately. When you first run Gemma 4 using the new Module your first token appears much more quickly.

Debugging models also gets a lot faster!

Production inference with the new Module
The redesigned Module still serves our primary use case, production inference. The built-in compile method traces the model and compiles it as a graph, which lets MAX optimize the computation using knowledge about the whole model. Our graph compiler performs:
- Operation fusion, to reduce launch overhead.
- Constant folding and other initialization-time lifting, to avoid repeating the same constant compute on every loop.
- Execution and memory planning, so MAX can plan storage and reuse buffers instead of allocating new ones.
- Shape inference and reasoning about dynamic shapes, which supports all of the above.
Module directly supports both eager and compiled execution, so model authors get the best of both worlds:
Distributed computation: split work across many devices
The redesign was a chance to rethink how we write distributed models. The framework's role is to structure the scaling of a computation from a single GPU to many.
There is no single right way to do that. Many parallelism strategies exist because GPUs differ in compute and memory limits, models differ in layers and dimensions, and inputs differ in shape. Here are several ways to distribute a model:



Lower-level abstractions in MAX can now express data parallelism, pipeline parallelism, or tensor parallelism. A Tensor is still a logical N-D array, but it can now live on more than one device. Operations on distributed tensors behave according to specific sharding rules.
A future post will dive deeper into the specifics of our distributed computation support. For now, see below how a row parallel linear layer is implemented:

Tree utilities
A Module is now a Tree, a new API in MAX inspired by JAX’s Pytrees. Trees are made of Modules and Tensors which simplifies two things:
Module.parametersandModule.descendantsuse tree traversal to collect connectedModules andTensors.Module.traceandModule.compileusetree.mapto convertTensors into graph weights.
These APIs also indirectly help with serialization: tree flatten produces both the flattened leaves and a TreeDef object that stores the tree structure.

As an example, we use Tree.map to quantize only the expert layers in a mixture-of-experts model.

Simpler APIs for handling graphs, weight tensors and dtypes
We also removed abstractions and boilerplate that were no longer needed.
No symbolic Weight class. Weight existed to track and create weights in the graph. Now you use Tensor directly in a Module.

Default dtype and device. The layer at the top of this post defines neither. A default dtype and device context manager means a model that uses one device or dtype everywhere doesn't need to pass them to every layer. You can still set them per layer when needed.

Built-in compile. Graph construction boilerplate is gone. Every Module now has a compile method, covered in the production inference section above.

Final thoughts
The legacy Module implementation now lives in max.nn.legacy and will be removed in Q1 2027, so now is a good time to migrate your existing models. We’re also restructuring max.nn around the new max.tensor and max.functional libraries.
The best place to get started is the Model Bringup Workflow tutorial, which walks through building a model end to end with the new APIs.
Looking ahead, 2027 brings quantization, training, and multi-node support. Next in this series is a framework overview showing where MAX fits in the ML framework landscape, followed by posts on eager execution, distributed tensors, serialization, and more.



