Qwen3.8 27B: Hybrid-Attention Inference on Modular

Qwen3.8 27B

Deploy dedicated endpoint

Try in Playground

Example Usage

Output

  Speculative decoding uses a smaller draft model to predict multiple
  tokens ahead, then verifies them against the full model in a single
  pass. Accepted tokens skip individual generation steps, improving
  throughput without sacrificing accuracy. It's most effective when the
  draft model closely matches the target model's distribution.

Code to use

Python


  from openai import OpenAI
  
  client = OpenAI(
      base_url="https://model.api.modular.com",
      api_key="<your_api_token>",
  )
  
  response = client.chat.completions.create(
      model="qwen/qwen3.8-27b",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "Explain speculative decoding in 3 sentences."},
      ],
      stream=True,
  )
  
  for chunk in response:
      if chunk.choices[0].delta.content:
          print(chunk.choices[0].delta.content, end="")

Get your API token

Output

  “The image shows a fox wearing a backpack in a forest.”

Input image

Cute orange cartoon fox with a small brown backpack sitting on grass in a bright, colorful forest.

Code to use

  import base64
  from openai import OpenAI
  
  client = OpenAI(
      base_url="https://model.api.modular.com",
      api_key="<your_api_token>",
  )
  
  with open("fox.png", "rb") as image_file:
      image_data = base64.b64encode(image_file.read()).decode("utf-8")
  
  response = client.chat.completions.create(
      model="qwen/qwen3.8-27b",
      messages=[
          {
              "role": "user",
              "content": [
                  {
                      "type": "text",
                      "text": "Describe this image in one sentence.",
                  },
                  {
                      "type": "image_url",
                      "image_url": {
                          "url": f"data:image/png;base64,{image_data}"
                      },
                  },
              ],
          }
      ],
  )
  
  print(response.choices[0].message.content)

Model Details

Developed by
Qwen
Model family
qwen/qwen3.8-27b
Input Modality
Text, Image
Output Modality
Text
Context Window
256K
Total Params
27B
Precision
NVFP4
Deployment options
Shared, Dedicated, Self-hosted

Why choose Qwen3.8 27B NVFP4 on Modular?

High performance, out of the box
Run leading open models with strong default performance and the ability to optimize down to the kernel — extracting more from every GPU.
Lower Infrastructure Costs
Deploy efficiently across NVIDIA and AMD hardware to reduce GPU count, increase throughput, and avoid expensive closed-model licensing.
Easy Integration
Integrate through an OpenAI-compatible endpoint, swap models freely, and scale across clouds or hardware without redesigning your application stack.

Want to self-host this model with our open source infrastructure?

Read How

🔥 Trending models

DeepSeek V4.1 Flash 1017

DeepSeek-V4.1-Flash-1017 is a minor update to DeepSeek-V4.1-Flash, delivering substantial improvements across reasoning, coding, agentic tasks, and visual benchmarks. Alongside these gains, the model reduces task execution time and output length, enabling users to complete tasks faster and at lower cost and advancing the performance–cost Pareto frontier across multiple agent harnesses.

LLM

FLUX.2 Klein 4B

FLUX.2 [klein] 4B is a 4 billion parameter rectified flow transformer capable of generating images from text descriptions and supports multi-reference editing capabilities.

Image

GLM 5.3 Flash

GLM-5.3-Flash is the first natively multimodal model in Z.ai's GLM-5 series, with 320B total parameters and 18B active. A hybrid architecture combining sparse and linear attention cuts attention compute and KV cache by 3.0x and 4.4x against GLM-5.3, supporting a 1M-token context window at low serving cost for visual coding and agentic workflows.

LLM

Vision

MiniMax M3.1 Flash

MiniMax M3.1 Flash delivers a generational leap in agentic tasks, coding, Computer Use and cybersecurity, while making online inference nearly three times faster and reducing usage costs by 70%. M3.1 Flash’s inference efficiency has improved through more aggressive sparsity. In everyday use, output speeds can reach 200+ TPS (tokens per second), with median (P50) speeds consistently reaching 170+ TPS.

LLM

Vision

Similar models

MiMo-V2.5-Pro

MiMo-V2.5-Pro is an open-source Mixture-of-Experts (MoE) language model with 1.02T total parameters and 42B active parameters

LLM

Nemotron 3 Ultra

Nemotron-3-Ultra-550B-A55B-BF16 is a frontier-scale large language model (LLM) trained by NVIDIA, designed to deliver strong agentic, reasoning, and conversational capabilities. It is optimized for the most demanding workloads, including complex multi-step agents, long-context analysis, and high-accuracy reasoning over code, math, and science.

LLM

DeepSeek V3.2

DeepSeek V3.2 is a 685B MoE model with 37B active parameters, excelling at code, math, and general reasoning tasks.

LLM

Qwen3.5-397B-A17B

Qwen3.5-397B-A17B by Alibaba is a 397B MoE model with 17B active parameters, supporting text, vision, and video with hybrid reasoning.

LLM

Vision

View all models

Get started with Modular

Request a demo
Schedule a demo of Modular and explore a custom end-to-end deployment built around your models, hardware, and performance goals.
- Distributed, large-scale online inference endpoints
- Highest-performance to maximize ROI and latency
- Deploy in Modular cloud or your cloud
- View all features with a custom demo
Book a demo
Talk with our sales lead Adam!
30min demo. Evaluate with your workloads. Ask us anything.

Talk to us!
Book a demo for a personalized walkthrough of Modular in your environment. Learn how teams use it to simplify systems and tune performance at scale.
- Custom 30 min walkthrough of our platform
- Cover specific model or deployment needs
- Flexible pricing to fit your specific needs
Book a demo
Talk with our sales lead Jay!
Start using MAX
( FREE )
Run any open source model in 5 minutes, then benchmark it. Scale it to millions yourself (for free!).
Install MAX
What is MAX?
Start using Mojo
( FREE )
Install Mojo and get up and running in minutes. A simple install, familiar tooling, and clear docs make it easy to start writing code immediately.
Install Mojo🔥
What is Mojo🔥?

Qwen3.8 27B: Hybrid-Attention Inference on Modular

Example Usage

Why choose Qwen3.8 27B NVFP4 on Modular?

🔥 Trending models

Similar models

Get started with Modular

Start using Mojo