Translating sensitive data via cloud APIs is a costly and time-consuming process. Tencent has introduced an on-premises alternative that runs entirely offline. This review delves into the key details, test results, and practical applications of the Hy-MT2 model.

Introduction to Hy-MT2

Hy-MT2 is a new translation model from Tencent, designed to run locally on the user's computer. The primary focus is on high-speed processing, enabling the translation of many languages locally and handling documents faster than many commercial alternatives. The model is built on an architecture optimized for instruction following and precise rule adherence. The license permits the use of the model in commercial projects without restrictions.

Model Variants and Hardware Requirements

The Hy-MT2 model is available in different sizes:

• 2 Billion Parameters (Light Version): Minimal processing requirements, can run on a Central Processing Unit (CPU). Minimum memory is 440MB.
• 30 Billion Parameters (Server Version): Activates 3 billion parameters for a specific task.

For 1.8 billion parameters with 4-bit quantization, approximately 1.2GB of Video Random Access Memory (VRAM) is required. This makes the model accessible for running on entry-level computers and even on CPUs.

Architecture and Workflow

The original model file was compressed from 3.6 GB to 440 MB, reducing memory consumption and speeding up generation. The model's workflow includes the following stages:

1. Data Input: Standard text files or data extracted via OCR from images are accepted.
2. Formatting Rules Application: The user can provide their own dictionary or rules that the model will consider.
3. Processing: The model, with parameters of 1.8 billion or 30 billion, processes the data.
4. Output: Clean dialogue text is generated in the requested translation format.

This approach allows for the creation of structured JSON files, which is convenient for application interfaces and prevents formatting errors.

Practical Applications

Hy-MT2 can be used in various scenarios:

• Video Editors: Translating subtitles while preserving original timestamps, ensuring synchronization with the video.
• VLM Integration: Collaborating with models like LLaVA 1.5 2B or LiquidAI's LFM-2.5 for translating text extracted from screenshots.
• Technical Translation: The ability to enforce the use of specific terms, which standard models often miss.
• Deployment on Edge Devices: Low memory consumption (less than 0.5GB RAM) allows the model to be used on mobile devices supporting Apple, Qualcomm, and MediaTek platforms.
• Data Privacy: Translation is performed locally, ensuring the security of confidential files, such as code blocks and table layouts, without alteration.
• Cost Savings: The absence of cloud computing eliminates additional expenses.

Test Results

The Hy-MT2 model demonstrates high performance:

• First place in instruction-following benchmarks: Covers 33 languages and 5 regional dialects.
• Superiority over cloud services: A compact version (1.8 billion parameters) using less than 0.5 GB of memory outperforms Microsoft Translate and DeepL API, achieving 88% in tests.
• 30-billion parameter model: Achieves 94% performance, being a state-of-the-art open-source machine translation model.

Practical Test: Subtitle Translation

For the test, a 1.8 billion parameter model with 4-bit quantization was used. The subtitle file (SRT) of a previous video, approximately 1 minute in length, was translated into Hindi and Spanish.

Process:

1. Load the model's GGUF file into the Ollama directory.
2. Register the model using the Ollama command.
3. Extract text and timestamps from the SRT file.
4. Formulate a prompt for the model, specifying translation into the target languages (Hindi - HI, Spanish - ES).
5. Send the prompt to the local Ollama URL.
6. Save the translated files and performance metrics (number of blocks, characters, execution time, generation speed).

Results:

• Metrics: 28 blocks and 3000 characters were processed. The total execution time was 6 seconds. The average latency per block was 0.2 seconds. The total translation time for both languages was 2-3 seconds.
• Hindi Translation: The model successfully translated the majority of the text; however, some technical terms, such as "lightweight system," remained untranslated or were transliterated.
• Spanish Translation: The results of the Spanish translation were presented with preserved timestamps.

The main advantage of the model is its high generation speed, making it useful for local workflows requiring low latency.

Hardware Requirements for Different Variants

• 1.8 Billion Parameters, 16-bit Quantization: Requires 4GB VRAM (suitable for RTX 3050).
• Lower Bit Quantizations (8-bit, 2-bit, 1.25-bit): Reduce model size and memory consumption.
• 30 Billion Parameters (3 Billion Active): 14GB model size, 8GB VRAM consumption (requires a high-end GPU or Mac Studio).

Even with an 8GB RAM and 4GB VRAM configuration, when integrated with vision models, latency of less than one second per suggestion is achieved.

Limitations

• Runtime Required: A suitable runtime engine is necessary to execute the model.
• Ollama Integration: Highly compressed versions require setting up integration with Ollama.
• Rare Dialects: The 1.8 billion parameter model performs better with common languages. For rare dialects, the 30 billion parameter version is recommended for smoother translation.

Conclusion

Tencent's Hy-MT2 is a powerful solution for on-device machine translation, offering high speed, low resource requirements, and data privacy. Its compact size and ability to run on basic hardware make it an attractive alternative to cloud-based services.