The new MiniCPM-V version 4.7 model represents a significant breakthrough in local artificial intelligence systems capable of processing visual data. Developed with an emphasis on minimal hardware requirements, this 1-billion-parameter model enables computer vision tasks to be performed directly on edge devices, eliminating the need for expensive cloud servers or powerful GPUs.

Key Features and Benefits

The primary focus during the development of MiniCPM-V 4.7 was on optimizing performance and reducing resource consumption.

• Compact Size: The total model download size is just over 1 GB, making it easy to store on standard hard drives.
• Efficient Token Compression: The model uses 4x to 16x fewer tokens to compress visual data, significantly speeding up processing.
• Google Architecture: It utilizes a state-of-the-art Google development as its base architecture, ensuring accurate visual data mapping.
• Low Hardware Requirements: The model is optimized to run on consumer devices, requiring only 2 GB of RAM for the 8-bit quantized version and not needing a GPU. For mobile devices, a 4-bit quantized version is available, under 1 GB in size, with 1 GB of computational power consumption. For maximum precision, a 16-bit version is provided, requiring over 3 GB of VRAM and a dedicated graphics card.

Image Processing Pipeline

MiniCPM-V 4.7 employs a multi-stage process to analyze input images:

1. Tiling: The input image is divided into 36 distinct tiles to prevent distortion and maintain aspect ratios.
2. Feature Extraction: Using Google's Transformer architecture, semantic features and visual patterns are extracted from each tile and then converted into structured data.
3. Redundancy Filtering: The system discards unnecessary information, reducing processing load and saving computational resources and memory.
4. Text Generation: The Qwen 3.6 base language model converts visual tokens into coherent sentences, providing a response in a short time, typically under 1.5 seconds.

Operating Modes

The model offers two primary modes for data compression, tailored to different use cases:

• Speed Mode: Provides aggressive data compression up to 16x, resulting in maximum generation speed at the cost of some minor detail loss. This mode is ideal for processing long video streams or security camera data feeds, as it reduces system memory usage by more than half and prevents crashes on standard computers.
• Precision Mode: Retains 4x more visual information, delivering higher detail and accuracy. This mode is recommended for scanning dense documents, including complex text and code. It eliminates model "hallucinations" common in standard image processing tools and ensures accurate numerical data retrieval. Activating this mode is done by adding a specific flag to your Python script.

Local Deployment and Testing

For local execution, the Ollama service is utilized. After downloading the model file in GGML format (quantized versions range from 600 MB to 2 GB, with the original model weighing 3 GB) and creating a custom model file that references the GGML file and defines the chat structure, the model is registered under the name MiniCPM v4.7.

As part of the model's testing, four tasks were assigned:

1. Receipt Processing: Extracting the store name, date, list of items with prices, and the total amount from the receipt image store_receipt.jpg.
2. Optical Phenomenon Explanation: Interpreting an image depicting optical refraction, describing the physical principle, and the behavior of light rays at the interface between media (air-water).
3. Benchmark Analysis: Interpreting a performance graph, identifying the leading model within the 1 billion parameter scale based on Pareto efficiency, and explaining the reasons.
4. Image Comparative Analysis: Comparing an image of a city street with an image of optical refraction, identifying key objects, and applying refraction principles to real-world cameras.

The results of the analysis are saved to the file outputs.markdown.

Performance and Testing Results

The model demonstrated impressive results:

• Speed: The total processing time for four images was only 5 seconds, with response generation taking approximately 1 second. The first generated word latency is 40% lower than previous competitors.
• Accuracy: The model successfully handled all tasks.
  • Receipt: The store name (Coop Supermarket AG), date (2024-05-18, 14:32:11), receipt number (49208812), item positions with prices and total amount (15.5), and payment method (contactless debit card Visa) were correctly extracted.
  • Refraction: Optical refraction and Snell's law were accurately described, including the bending of light when transitioning from a less dense to a denser medium (refraction angle θ2 is less than incidence angle θ1) and partial reflection obeying the law of reflection.
  • Benchmark: MiniCPM-V 4.7 was identified as a leader in Pareto efficiency, outperforming Qwen 3.5 B in performance with significantly lower inference time (40% less). The model shows better accuracy and real-time generation speed compared to larger models.
  • Comparative Analysis: The model correctly described an urban street with moving vehicles, reflective glass, and street lighting. An analogy was drawn to optical reflection and refraction of light in glass (e.g., a car's windshield), explaining how raindrops on the windshield can distort the perception of distance to objects due to light refraction, and emphasizing the need for sensor calibration for accurate distance estimation.

Limitations and Recommendations

Despite its high performance, MiniCPM-V 4.7 has some limitations:

• Complex Mathematical Tasks: The model is not designed to solve complex mathematical problems that require architectures with more than 35 billion parameters.
• Prompt Format: Standard prompt templates must be used to achieve the best results.
• Video Processing: Direct processing of video files requires the use of specialized Python libraries that are incompatible with standard Torch media processing packages.

Conclusion

MiniCPM-V 4.7 is a powerful and accessible tool for local visual data processing. Its compact size, low hardware requirements, and high accuracy make it an ideal solution for developers aiming to create private offline computer vision systems. The model successfully transforms unstructured images and videos into structured data, opening up new possibilities for practical AI applications.