Workers AI – Clef-omni adds audio and video input, Clef-flash is now cheaper, and Clef is faster

@cf/cloudflare/clef-omni is now available on Workers AI. Clef-omni is a decision model that takes audio (WAV or MP3) and video (MP4 or WebM) input alongside text and images. It joins Clef and Clef-flash in the Clef family of open-weight decision models. We also cut the price of Clef-flash, so it now costs less than Jev, and made Clef faster.

Previously, making a decision about a voice recording or a video meant chaining models together: transcribe the speech, split the audio and visual tracks, then pass the results to a text decision model. Clef-omni reads every modality directly in one request. A video’s soundtrack is aligned with its frames, so the model can reason over what is seen and heard at the same time.

Clef-omni is built on a 30B-parameter mixture-of-experts (MoE) backbone with 3B active parameters. Like the rest of the Clef family, it does not generate text. It scores every allowed answer in a single pass, so decisions return quickly:

• Text requests: about 20 ms
• Image or audio inputs: under 100 ms
• A 21-second video clip with sound: about 300 ms

Pass media as base64 data URLs in the images, audio, and videos fields:

const response = await env.AI.run("@cf/cloudflare/clef-omni", {
	model: "clef-omni",
	state:
		"Review the installation: a photo of the unit, an audio recording of it running, and a video of the fan.",
	images: ["data:image/png;base64,<base64-png>"],
	audio: ["data:audio/mpeg;base64,<base64-mp3>"],
	videos: ["data:video/mp4;base64,<base64-mp4>"],
	questions: {
		label_visible: {
			type: "noul",
			instructions:
				"Is the model and serial number label visible in the photo?",
		},
		sounds_normal: {
			type: "noul",
			instructions:
				"Does the unit sound like it is running smoothly, without rattling or grinding?",
		},
		fan_running: {
			type: "noul",
			instructions: "Is the fan running in the video?",
		},
	},
});

const response = await env.AI.run("@cf/cloudflare/clef-omni", {
	model: "clef-omni",
	state:
		"Review the installation: a photo of the unit, an audio recording of it running, and a video of the fan.",
	images: ["data:image/png;base64,<base64-png>"],
	audio: ["data:audio/mpeg;base64,<base64-mp3>"],
	videos: ["data:video/mp4;base64,<base64-mp4>"],
	questions: {
		label_visible: {
			type: "noul",
			instructions:
				"Is the model and serial number label visible in the photo?",
		},
		sounds_normal: {
			type: "noul",
			instructions:
				"Does the unit sound like it is running smoothly, without rattling or grinding?",
		},
		fan_running: {
			type: "noul",
			instructions: "Is the fan running in the video?",
		},
	},
});

Clef-omni scores highest of the Clef family on BANKING77, CLINC150+OOS, and Amazon ESCI:

Clef-flash now costs $0.038 per million input tokens, down from $0.090, which makes it cheaper than Jev. To offer this price, the hosted Clef-flash context window is now 24K tokens, down from 64K. Based on usage data, only 0.24% of requests exceed 24K input tokens. If you need a larger context window, use Clef, which keeps its 64K context window.

The Clef-flash weights on Hugging Face are unchanged and support up to a 256K context window if you self-host.

All Clef models convert image inputs to input tokens, and Clef-omni does the same for audio and video. For details on how each input type is tokenized, refer to the Clef, Clef-flash, and Clef-omni model pages.

We optimized how Clef is served on Workers AI, so it now returns decisions up to 2x faster. The model weights are unchanged.

Part of this speedup comes from moving Clef to SGLang ↗︎. Clef support is coming to SGLang in version 0.5.22 (PR #42721 ↗︎). If you self-host Clef, launch commands are available in the Clef collection on Hugging Face ↗︎.

Clef-omni follows the same System One API as Clef and Clef-flash, and works with AI Gateway. To try it, change the model ID to @cf/cloudflare/clef-omni and set the model selector to clef-omni.

For more information, refer to the Clef-omni model page and pricing.

Source: Cloudflare