This article explores the capabilities of two prominent AI vision models, Qwen-VL 2.5 and Moondream, by comparing their performance on various prompts and images. The analysis aims to provide insights into how these models interpret visual data and how factors like model size and prompting techniques influence their output.

Understanding Vision Models

Vision models are artificial intelligence systems designed to interpret and understand visual information from images, videos, or other visual inputs. Unlike traditional language models that process text, vision models can identify objects, scenes, and actions within visual data.

Qwen-VL 2.5

Qwen-VL 2.5, developed by Alibaba's AI division, is presented as a flagship, state-of-the-art vision model. Key features highlighted by its developers include:

• Visual Understanding: The ability to comprehend visual content in images and video streams.
• Agentic Capabilities: Potential for use in creating AI agents that can interact with digital interfaces, such as screenshots of computer or phone screens, to perform actions like pressing buttons.
• Video Analysis: Capacity to understand long videos, capture events, and provide summaries of content like tutorial videos.
• Visual Localization: The ability to identify and locate specific objects within an image when prompted (e.g., "where is the cake," "where are the wheels").
• Structured Output Generation: Producing results in a format that can be used for further processing, enabling applications like monitoring power meter readings from a camera feed.
• Multilingual Support: Including the ability to process and understand Chinese text.
• Document Analysis: Capability to interpret documents, including PDFs.
• Model Sizes: Available in different parameter sizes, including 7 billion and 14 billion parameters. The specific model tested in this comparison was a 3 billion parameter version.
• Licensing: Released under an Apache 2.0 license, allowing for free use, modification, and commercial application.

The model's capabilities are further demonstrated through examples such as recognizing over 10,000 bird species, identifying landmarks, and recognizing celebrities.

Moondream

Moondream is described as a highly efficient and fast vision model, developed by a smaller team. Despite its smaller size, it is noted for its accuracy. Key features include:

• Object Detection: Ability to detect and identify objects like cars, including specific models and license plates.
• Component Understanding: Comprehension of individual components within an object, such as car parts.
• Agentic Capabilities: Similar to Qwen-VL 2.5, it can be used to create agents capable of interacting with digital interfaces.
• Model Size: The specific model tested was a 2 billion parameter version, making it smaller than the Qwen-VL 2.5 model used.

Performance Comparison

The comparison involved using the same prompts and images for both Qwen-VL 2.5 (3 billion parameters) and Moondream (2 billion parameters) on a Windows workstation equipped with an NVIDIA RTX A5000 graphics card.

Test 1: Birthday Cake Image

Prompt: "What do you see in this image?"

• Qwen-VL 2.5 (3B): Described two chocolate cakes decorated to resemble goggles, noting their circular shape and central hole. It identified chocolate bars, gummy bears, and what it interpreted as straps (though they were toy cars). It also correctly identified the wooden cutting board.
• Moondream (2B): Described a large chocolate-covered cake resembling a double infinity sign, placed on a wooden table and decorated with various chocolate and candy toppings.

Analysis: Both models provided reasonable descriptions. Moondream's description of the cake's shape as an "infinity sign" was considered more fitting by the observer. Moondream was also faster, completing the task in approximately 11 seconds compared to Qwen-VL 2.5's 23 seconds.

Test 2: Girl with Drawing

Prompt: "What do you see in this image?"

• Qwen-VL 2.5 (3B): Identified a young girl at a table holding up a drawing of a beach scene. It noted a notebook and sketchbook, a plate with food (toast and eggs), and suggested a casual dining setting.
• Moondream (2B): Described a young girl at a dining table holding up a drawing, noting she was smiling and proud. It also mentioned the table setting with a plate of food (egg and sandwich) and other people in the background.

Analysis: The responses were similar, with Moondream providing slightly more detail about the girl's expression. Both models performed comparably in speed and accuracy for this prompt.

Test 3: Cats and Child in a Park

Prompt 1: "What do you see in this image?"

• Qwen-VL 2.5 (3B): Described a person on a paved path next to a bench, with two cats sitting and resting. It noted greenery, bushes, trees, and a bicycle parked nearby, with someone riding it.
• Moondream (2B): Identified a young boy sitting on a park bench with three cats. It noted the boy was positioned in the middle of the bench and the scene appeared to be in a park.

Analysis: Moondream correctly identified three cats, while Qwen-VL 2.5 only detected two. This highlighted Moondream's accuracy in counting objects.

Prompt 2: "Create a detailed long caption of this image in a neutral tone without using mood-like descriptions. Be specific."

• Qwen-VL 2.5 (3B): Provided a more detailed description, including a person on a paved pathway adjacent to a wall, a black metal bench with wood, and noted two cats on the bench. It also mentioned background elements like a building and street signs. However, it failed to mention the child.
• Moondream (2B): Offered a more detailed description, including the presence of three cats, a small pink plastic container, a light-colored long-sleeved shirt worn by the child, and even identified the cat types as Calico and Tabby. It also detected the child.

Analysis: For this more complex prompt, Moondream provided a more comprehensive and accurate description, including details missed by Qwen-VL 2.5, such as the child and the plastic container. This suggested that Moondream might be more robust in extracting fine-grained details.

Prompt 3: "Find and describe in detail."

• Qwen-VL 2.5 (3B): Described a scene on a paved sidewalk, a wooden bench near a green area, and two cats resting. It also noted a building with multiple stories, people walking, and street signs in the background. It failed to mention the child.
• Moondream (2B): Provided a detailed description of a park bench with three cats, a child in a white shirt, a small pink plastic container, and identified the cat breeds (Calico and Tabby).

Analysis: Moondream continued to excel in detailed object and subject identification, including the child and specific cat breeds, which Qwen-VL 2.5 missed.

Prompt 4: "Create a detailed long caption of this image, a neutral tone without using mood-like descriptions, including distant objects as well."

• Qwen-VL 2.5 (3B): Described a person seated on a wooden bench near a green area, holding a bucket. It also noted a concrete feature and a path.
• Moondream (2B): The model failed to respond to this prompt.

Analysis: This prompt demonstrated a limitation for Moondream, which was unable to process the complex request. Qwen-VL 2.5, while not as detailed as in previous attempts, provided a response.

Prompt 5: "Tell me all you can see in this image. Do not miss anything."

• Qwen-VL 2.5 (3B): Provided a good description of a park bench surrounded by three cats on a brick sidewalk.
• Moondream (2B): Provided a more detailed description, including the bench's placement and surrounding elements.

Analysis: Both models performed well with this simpler, direct prompt, with Moondream offering slightly more detail.

Test 4: Urban Street Scene

Prompt: "Do you see in this image? Be detailed and specific. Do not miss a thing."

• Qwen-VL 2.5 (3B): Described a busy urban street scene with a white truck labeled "Lantern," red with white stripes, and a readable license plate. It identified a double-decker bus (number 29, destination Woodgreen), pedestrians, traffic lights, and trees.
• Moondream (2B): Described a white truck with the word "Lantern" on the front, red with white stripes, and a readable license plate (missing the letter 'D'). It also identified a red double-decker bus with number 29 and destination Wood Green, along with other vehicles.

Analysis: Both models demonstrated strong capabilities in reading text on vehicles and identifying specific details like bus numbers and destinations. Qwen-VL 2.5 was slightly more accurate in reading the license plate. Qwen-VL 2.5 took approximately 40 seconds, while Moondream completed it in 11 seconds.

Prompt 2: "Tell me everything you see and make me a list of things. Give me even the image quality if it's noisy and tell me which season it was, was it in winter, fall, summer and the time of day."

• Qwen-VL 2.5 (3B): Provided a detailed description and a list of objects. It identified the photo style, time of day, and image quality but could not determine the season.
• Moondream (2B): Successfully extracted the last part of the prompt, providing a list of objects including "land vehicle, tow truck," and was very fast.

Analysis: Qwen-VL 2.5 handled the complex prompt more comprehensively, providing a list and attempting to identify environmental factors. Moondream, while faster, only managed to extract a partial list.

Conclusion

The comparison between Qwen-VL 2.5 (3 billion parameters) and Moondream (2 billion parameters) revealed that model size is not the sole determinant of performance.

• Accuracy: Moondream demonstrated superior accuracy in specific tasks, such as correctly identifying three cats in a scene where Qwen-VL 2.5 consistently identified only two. Moondream also provided more detailed and nuanced descriptions in several instances.
• Speed: Moondream, being a smaller model, was significantly faster in processing images and generating responses.
• Prompting: The effectiveness of prompts played a crucial role. More detailed and specific prompts often yielded better results, but the optimal prompting strategy varied between models. Moondream sometimes struggled with overly complex prompts, while Qwen-VL 2.5 showed more resilience to intricate instructions, albeit with slower processing times.
• Capabilities: Both models exhibited strong capabilities in object recognition, text reading on vehicles, and understanding complex scenes. Qwen-VL 2.5 showed a slight edge in reading license plates accurately and handling more complex, multi-faceted prompts.

The study suggests that while larger models like Qwen-VL 2.5 may offer broader capabilities and better handling of complex instructions, smaller, well-trained models like Moondream can achieve remarkable accuracy and speed, making them highly efficient for specific applications. The quality of training data and the model's architecture are as critical as its parameter count.