dindang/FloorMaterialAnalyzer_AI_Model
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Upload README.md with huggingface_hub

Browse Source
This commit is contained in:
Weiyun Wang 2025-04-12 11:31:21 +00:00 committed by system
parent 89ef9a8235
commit 04b6ad96f2
No known key found for this signature in database
GPG Key ID: 6A528E38E0733467
1 changed files with 24 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
README.md
View File

@ -146,7 +146,30 @@ We compare InternVL3 with Qwen2.5 Chat models, whose corresponding pre-trained
Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series.
Please note that the evaluation scores of Qwen2.5 series may differ from those officially reported, as we have adopted the prompt versions provided in the table across all datasets for OpenCompass evaluation.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/geqAQiiganmQiDY65VIpm.png)
![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/XHOC-GZO7qAtG8seR05ig.png)
## Ablation Study
### Native Multimodal Pre-Training
We conduct experiments on the InternVL2-8B model while keeping its architecture, initialization parameters, and training data entirely unchanged. Traditionally, InternVL2-8B employs a training pipeline that begins with an MLP warmup phase for feature alignment followed by an Instruction Tuning stage. In our experiments, we substitute the conventional MLP warmup phase with a native multimodal pre-training process. This modification isolates the contribution of native multimodal pre-training to the overall multimodal capability of the model.
The evaluation results in the Figure below shows that the model with native multimodal pre-training exhibits performance on most benchmarks that is comparable to the fully multi-stage-trained InternVL2-8B baseline. Furthermore, when followed by instruction tuning on higher-quality data, the model demonstrates further performance gains across evaluated multimodal tasks. These findings underscore the efficiency of native multimodal pre-training in imparting powerful multimodal capabilities to MLLMs.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/X4PC_JDyPGWJF6gYMPSH6.png)
### Mixed Preference Optimization
As shown in the table below, models fine-tuned with MPO demonstrate superior reasoning performance across seven multimodal reasoning benchmarks compared to their counterparts without MPO. Specifically, InternVL3-78B and InternVL3-38B outperform their counterparts by 4.1 and 4.5 points, respectively. Notably, the training data used for MPO is a subset of that used for SFT, indicating that the performance improvements primarily stem from the training algorithm rather than the training data.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/bZuOd8yy_sFHcrkYoRERG.png)
### Variable Visual Position Encoding
As reported in the table below, the introduction of V2PE leads to significant performance gains across most evaluation metrics. In addition, our ablation studies—by varying the positional increment \\( \delta \\)—reveal that even for tasks primarily involving conventional contexts, relatively small \\( \delta \\) values can achieve optimal performance. These findings provide important insights for future efforts aimed at refining position encoding strategies for visual tokens in MLLMs.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/K6D0OP5ZiNLTJB51-HOWa.png)
## Quick Start