liveBook · Manning

LLaVA

As explained in LLMs in Production

LLaVA is a multimodal large language model that combines language understanding with visual input processing.

LLaVA extends language-model interaction beyond text: provide an image, then ask questions about what it shows. It combines language understanding with visual input processing, making multimodal applications possible while retaining an LLM at its core. section 11.4.3

A practical deployment requires:

  • Adding the vision component through a model projector; converting only the LLaVA language portion to GGUF is insufficient. section 11.4.3
  • Treating runtime dependencies cautiously: llama.cpp multimodality support changed abruptly during development, creating production reproducibility and debugging risks. section 11.4.3
$ wget https://huggingface.co/cjpais/llava-1.6-mistral-7b-
 gguf/resolve/main/mmproj-model-f16.gguf
$ mv mmproj-model-f16.gguf ./models/llava/mmproj.gguf

Download the multimodal projector and place it beside the LLaVA model files. section 11.4.3

$./server -m ./models/llava/llava-v1.6-mistral-7b-q4_k_m.gguf --host 
 $PI_IP_ADDRESS --api-key $API_KEY --MMPROJ ./models/llava/mmproj.gguf

Launch the llama.cpp server with the language model, host settings, API key, and projector path. section 11.4.3

How LLMs in Production covers it

How deep it goes
Explains multimodality and projector-based deployment, but stops short of detailed vision-model internals.
Examples
Uses image questions, projector download commands, and a llama.cpp server launch configuration.
What you can do after
Operate a LLaVA deployment through llama.cpp while accounting for unstable multimodal server support.
How it connects
Connects natural language processing, large language models, computer vision, model projectors, GGUF, and llama.cpp.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.