Gemma 4 31B: Dense Vision-Language Inference on Modular

  Speculative decoding uses a smaller draft model to predict multiple
  tokens ahead, then verifies them against the full model in a single
  pass. Accepted tokens skip individual generation steps, improving
  throughput without sacrificing accuracy. It's most effective when the
  draft model closely matches the target model's distribution.

  “The image shows a fox wearing a backpack in a forest.”

  import base64
  from openai import OpenAI
  
  client = OpenAI(
      base_url="https://model.api.modular.com",
      api_key="<your_api_token>",
  )
  
  with open("fox.png", "rb") as image_file:
      image_data = base64.b64encode(image_file.read()).decode("utf-8")
  
  response = client.chat.completions.create(
      model="google/gemma-4-31B-it",
      messages=[
          {
              "role": "user",
              "content": [
                  {
                      "type": "text",
                      "text": "Describe this image in one sentence.",
                  },
                  {
                      "type": "image_url",
                      "image_url": {
                          "url": f"data:image/png;base64,{image_data}"
                      },
                  },
              ],
          }
      ],
  )
  
  print(response.choices[0].message.content)