Blog

AI is powered by a virtuous circle of data, algorithms (“models”), and compute. Growth in one pushes needs in the others and can grossly affect the developer experience on aspects like usability and performance. Today, we have more data and more AI model research than ever before, but compute isn’t scaling at the same speed due to … well, physics.

At Modular, we are rebuilding AI infrastructure for the world. Our goal is to move past AI tools that are themselves research projects and into a future where AI development and deployment are orders of magnitude more efficient for everyone. You should be able to do this without trading off performance or having to rewrite your entire code base.

The technological progress that has been made in AI over the last ten years is breathtaking — from AlexNet in 2012 to the recent release of ChatGPT, which has taken large foundational models and conversational AI to another level.

The first four requirements address one fundamental problem with how we've been using MLIR: weights are constant data, but shouldn't be managed like other MLIR attributes. Until now, we've been trying to place a square peg into a round hole, creating a lot of wasted space that's costing us development velocity (and, therefore, money for users of the tools).

Here in November 2022, we see a continuing onslaught of bad news: significant layoffs of incredible people as companies tighten their belts; companies that raised too much money, too fast, without core fundamentals are dying; and a changing climate where over-tightening rather than under-tightening is seemingly the new normal.

What makes a brand? When you see an apple with a bite out of it, you immediately associate it with the famous technology company — but what about the logo elicits the feelings it does? 

Machine learning models are getting larger and larger — some might even say, humongous. The world’s most advanced technology companies have been in an arms race to see who can train the largest model (MUM, OPT, GPT-3, Megatron), while other companies focused on production systems have scaled their existing models to great effect. Through all the excitement, what’s gone unsaid is the myriad of practical challenges larger models present for existing AI infrastructure and developer workflows.

AI promised to profoundly change the world, so why hasn’t it?From healthcare to manufacturing, finance, climate, communication, and travel, to how we live and work. AI can help solve any problem that can be represented by data, assuming the right algorithms and enough computational resources.

Platforms like TensorFlow, PyTorch, and CUDA do not focus on modularity - there, we said it! They are sprawling technologies with thousands of evolving interdependent pieces that have grown organically into complicated structures over time. AI software developers must deal with this sprawl while deploying workloads to server, mobile devices, microcontrollers, and web browsers using multiple hardware platforms and accelerators.