ai/lfm3-omni repository overview

API⁠ Try⁠ Docs⁠ Blog⁠

⁠LFM3-Omni-2B-GGUF

GGUF version of LFM3-Omni-2B⁠ for local inference on CPUs and GPUs.

LFM3-Omni-2B is an omni-modal model that understands text, images, video and audio, and answers in text and speech. It is built for on-device use.

Find architecture and usage details in the original model card⁠.

⁠How to run LFM3-Omni-2B

Recommended sampling: greedy decoding (temperature 0) for text output. For speech output, audio temperature 0.4 and audio repetition penalty 1.2.

• Llama.app⁠ (Use this model > Llama in the menu), for macOS and Windows: image, audio and video input. Requires llama.cpp version v0.8.0⁠ or newer.
• LM Studio⁠ (Use this model > LM Studio in the menu), for macOS, Windows and Linux: image input. Requires LM Studio with llama.cpp engine 2.65.1 or newer.
• Ollama⁠ (ollama run lfm3-omni), for macOS: image and audio input. Requires Ollama version v0.40.12⁠ or newer.

⁠Use with llama.cpp

Add a quantization tag to pick a file. With -hf, llama.cpp also downloads the vision and audio projector.

LFM3-Omni-2B is a non-reasoning model: with llama serve, use --reasoning off. Start a web chat UI and an OpenAI-compatible API at http://localhost:8080:

llama serve -hf LiquidAI/LFM3-Omni-2B-GGUF:Q4_K_M --reasoning off --temp 0 --port 8080

To ask one question from the command line, use llama cli.

Speech recognition: use the system prompt Perform ASR in <language>.

llama cli -hf LiquidAI/LFM3-Omni-2B-GGUF:Q4_K_M \
  --audio speech.wav -sys "Perform ASR in English." -p "<__media__>" --temp 0

Image understanding:

llama cli -hf LiquidAI/LFM3-Omni-2B-GGUF:Q4_K_M \
  --image photo.png -p "Describe this image." --temp 0

Video understanding: the model sees the frames and hears the soundtrack of the video, use the system prompt Answer questions about the video using what you see and hear. Video input needs ffmpeg in PATH.

curl -sL -o reading.mp4 https://huggingface.co/datasets/raushan-testing-hf/videos-test/resolve/4cba700bd771f44d72b549253da025c32e944d42/sample_demo_1.mp4  # a child reading a book

llama cli -hf LiquidAI/LFM3-Omni-2B-GGUF:Q4_K_M \
  --video reading.mp4 -sys "Answer questions about the video using what you see and hear." \
  -p "What happens here, and what can you hear?" --temp 0

Speech synthesis: --tts-lang takes a language code (default en) and --tts-voice is female (default) or male. The audio sampling parameters of the original model card are stored in the GGUF.

llama-tts -m LFM3-Omni-2B-Q4_K_M.gguf -mm mmproj-LFM3-Omni-2B-Q8_0.gguf \
  -p "Hello world" -o out.wav

More options in the llama.cpp guide⁠.