November 16, 2026

Modular at LLVM Developers' Meeting 2026

Michael Dunn-OConnor

Community

Modular recently spent three days at the 2026 LLVM Developers' Meeting in Santa Clara, California (October 26 to 28), where we ran a full-day MAX and Mojo workshop, gave a technical talk on device graphs, and talked with folks in the LLVM community in the hallways and at our booth. This was the first time we could share details about the entire Modular stack from end to end, with all of the source code available for attendees to follow along. We showed the internal architecture of the Mojo language, how we use MLIR across the stack, and how we layer Mojo with our graph compiler, among other topics. This post covers what we presented, what we heard, and what collaboration we see ahead.

Mojo and MAX workshop poster
Mojo and MAX workshop poster

Workshop: Mojo Internals Deep Dive and Roadmap

On Monday, October 26th, we ran "Mojo: Internals Deep Dive and Roadmap," an all-day session for compiler engineers familiar with LLVM and MLIR. The goal was to give that audience a working map of how Mojo is built, from the compiler and type system to GPU back-ends and the standard library. Despite the name, this was not just a session on Mojo. We built MAX on the foundation that Mojo provides, so we also covered how MAX targets new accelerator hardware, and how it compiles and optimizes computational graphs for complex AI models. The newly source-available MAX let us show, for the first time, how the entire Modular stack works together.

Mojo is built on MLIR (a subproject of LLVM), so this audience of active users and contributors had a unique perspective on the language, though many were new to Mojo and MAX. We were able to go much deeper into the internals of both Mojo and MAX than we typically do for a general audience. Unfortunately, LLVM workshops are not recorded, but the slides are included below.

What each talk covered

  • Mojo's place in the LLVM ecosystem. Billy Zhu, Lukas Hermann, and Weiwei Chen framed Mojo as the user-facing layer over MLIR for accelerators, where hardware specialization is performed by libraries that domain experts can write, not locked inside a compiler or a vendor binary. Mojo gives programmers the full power of direct MLIR and LLVM IR access, acting as a unified LLVM-style layer for AI.
  • Mojo compiler architecture overview. Billy Zhu and Weiwei Chen went deep into the unique MLIR-based design of the Mojo compiler. They discussed how the Mojo parser emits parametric MLIR directly, with no AST, and resolves declarations lazily. They also covered how the elaborator concretizes generic code asynchronously, and how the plugin system allows new backends to be added without forking the entire compiler.
  • Plugin architecture: separating the compiler and SDK from hardware-specific parts. Connor Gray showed how Modular is creating a portable and performant AI stack through programmable plugins. Anyone can extend the standard library and MAX accelerator library with optimized support for new hardware by implementing simple Mojo PluginHooks and MaxPluginHooks traits. These plugins are concise, with the backend for Qualcomm Cloud AI 100 only 263 lines. The full plugin architecture will enable plugins to bring the entire Modular stack to new hardware, a key technical enabler for the Modular hardware alliance.
  • Value ownership, lifetimes and origins: Mojo's memory safety model. Chris Lattner explained the benefits of Mojo's origins system: memory safety without a garbage collector, systems-level control, and an escape to unsafe APIs when needed. Generally, programmers don’t have to think about references until they define an API that returns one. The compiler checks lifetimes and destroys each value right after its last use, which gives smaller capture sets, more tail calls and copy-to-move optimization, and linear types that compose with the rest of the type system.
  • How Mojo supports non-LLVM back-ends (TPU and Trainium). Kolya Panchenko explained why TPU and Trainium are difficult hardware backends for a generalpurpose language. They expect typed, tiled, structured code, and Mojo has no memref type. The first milestone was a vector add written in Mojo that compiles to Mosaic MLIR for the TPU.
  • MAX graph compiler and the MLIR dialects behind kernel fusion. Feras Boulala walked an RMS-norm example through the whole MAX pipeline. Along the way, he showed the optimizations applied at each stage and features that set MAX apart for debugging a model, including Python stack traces and tracing.
  • How we write Mojo in Mojo. Joe Loser and Nathan Ward built an Int from scratch in MLIR, then showed how the types in the Mojo standard library are built the same way. A few leaf types wrap MLIR (like Bool, SIMD, and Pointer), then other types are built in Mojo using these leaves.
  • Case study: metaprogramming that abstracts away hardware specifics. Kern Handa described bringing up a 32-bit device behind a 64-bit host. The case study detailed three problems (a fixed C entry point, packing arguments into a flexible array, and serializing closures), each solved with Mojo library code. Kern demonstrated how bringing up an accelerator is mostly library work.

Technical talk: Moving Beyond Record & Replay

On Wednesday, October 28, Spenser Bauman from our Graph Execution team presented Moving Beyond Record & Replay, a case for building device graphs in a compiler, such as the one inside MAX, instead of capturing them at runtime. If you’re not familiar, both CUDA and HIP (as well as other accelerators) offer a means of sending a graph of operations to an accelerator, instead of sending each operation by itself.

As GPUs get faster, the cost of CPU-to-accelerator communication grows until it takes up much of the inference time. Device graphs address that by submitting a whole sequence of work to the device at once. MAX uses them, and so does every widely used serving stack. The usual way to build one is record and replay: capture a stream of commands at runtime, then replay it. CUDA and HIP both offer this.

Spenser laid out two problems with that approach:

  • Record and replay spreads complexity across the graph executor, async runtime, memory pools, kernels, and drivers.
  • It depends on a capture mode that not every vendor provides. Several accelerators expose device-graph or command-buffer construction APIs but have no record and replay equivalent, which puts them out of reach for a capture-based stack.

The alternative Modular is pursuing is to use a graph compiler to build those device graphs directly. Rather than capturing and then replaying a graph, the graph compiler within MAX already knows the fundamental structure of the final device graph. The graph compiler can construct the device graph and emit direct calls to the target's graph-construction APIs. The talk walked through four pieces of that design:

  • How kernels declare their device-graph capabilities.
  • The front-end IR that marks device-graph-capable regions.
  • The middle-end dialects and operations that model device graphs.
  • The Mojo-level APIs for working with them.

The approach is already working in practice. A production Gemma 4 model now runs end-to-end through device graph synthesis on one or more GPUs, and a single model can contain multiple device-graph regions. Building the graph directly is about 40% faster than recording it. DeviceGraphBuilder implementations exist for CUDA, HIP, and Qualcomm AI100, all behind one interface, so the same approach extends to new targets.

What comes next

The workshop closed with the Mojo roadmap and an invitation to contribute.

Roadmap. Denis Gurchenkov outlined the near-term priority of Mojo language features for kernel programming (enumerations and pattern matching, more compile-time metaprogramming, syntax for hardware concepts such as address spaces) and a longer-term goal for 2027 of reaching developers who write performant CPU code, with the necessary development of async/await, dynamic dispatch, threading, and networking. On the compiler side, the plan is to make the GPU and NPU backends source available, support pluggable MLIR and LLVM backends, enable a WASM backend, and to continue improving compilation speed. Tooling and community items include automatic Mojo and C interoperability, a debugger, a testing framework, a package manager built on pixi, and a Bazel-based build system.

Modular's open source journey. Brad Larson and Joe Loser traced three years of progressive open-sourcing of Mojo and MAX, starting with the Mojo standard library in March of 2024, all the way to open-sourcing Mojo and making MAX fully source-available just a few weeks ago. The whole Modular stack now builds from the public modular/modular repository (which now has over 30,000 GitHub stars!), a tremendously exciting milestone. We’ve also learned a lot of lessons during our process of open-sourcing Mojo and MAX, which we get into in this presentation.

With all of the source code for Mojo and MAX now publicly available, the team left attendees with one message: please clone the modular/modular repository, build Mojo and MAX from source, and experiment with extending the platform. Check out the contributor guides for Mojo as well as the newly added guide for MAX and see where and how you can upstream fixes and additions for both.

Modular (and Qualcomm) are proud long-time supporters of LLVM Foundation events, and this year was particularly special: we finally took everyone behind the curtains of our work, and are excited to see what the community will build.

Read more from Modular

View all blogs

Build the future of AI with Modular

View Editions
  • Person with blonde hair using a laptop with an Apple logo.

    Sign up today

    Signup to our Cloud Platform today to get started easily.

    Sign Up
  • Magnifying glass emoji with black handle and round clear lens.

    Browse open models

    Browse our model catalog, or deploy your own custom model

    Browse models
No items found.