From 89ef9a823500c14df45efe9e1f673b1b8e978e65 Mon Sep 17 00:00:00 2001 From: Weiyun Wang Date: Sat, 12 Apr 2025 07:23:24 +0000 Subject: [PATCH] Upload README.md with huggingface_hub --- README.md | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index f0749de..b323ca5 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ base_model: - Qwen/Qwen2.5-7B base_model_relation: merge datasets: - - OpenGVLab/MMPR-v1.1 + - OpenGVLab/MMPR-v1.2 language: - multilingual tags: @@ -31,7 +31,7 @@ tags: We introduce InternVL3, an advanced multimodal large language model (MLLM) series that demonstrates superior overall performance. Compared to InternVL 2.5, InternVL3 exhibits superior multimodal perception and reasoning capabilities, while further extending its multimodal capabilities to encompass tool usage, GUI agents, industrial image analysis, 3D vision perception, and more. -Additionally, benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series, which serves as the initialization of the langauge component in InternVL3. +Additionally, we compare InternVL3 with Qwen2.5 Chat models, whose corresponding pre-trained base models are employed as the initialization of the langauge component in InternVL3. Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/fMRWSzB8ysrafQ2XJW9WR.png) @@ -49,6 +49,8 @@ In the following table, we provide an overview of the InternVL3 series. | InternVL3-38B | [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5) | [Qwen2.5-32B](https://huggingface.co/Qwen/Qwen2.5-32B) | [🤗 link](https://huggingface.co/OpenGVLab/InternVL3-38B) | | InternVL3-78B | [InternViT-6B-448px-V2_5](https://huggingface.co/OpenGVLab/InternViT-6B-448px-V2_5) | [Qwen2.5-72B](https://huggingface.co/Qwen/Qwen2.5-72B) | [🤗 link](https://huggingface.co/OpenGVLab/InternVL3-78B) | +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/9wf54ERGoiM3-QICkj3Oc.png) + ## Model Architecture As shown in the following figure, [InternVL3](https://internvl.github.io/blog/2025-04-11-InternVL-3/) retains the same model architecture as [InternVL 2.5](https://internvl.github.io/blog/2024-12-05-InternVL-2.5/) and its predecessors, InternVL 1.5 and 2.0, following the "ViT-MLP-LLM" paradigm. In this new version, we integrate a newly incrementally pre-trained InternViT with various pre-trained LLMs, including InternLM 3 and Qwen 2.5, using a randomly initialized MLP projector. @@ -104,35 +106,45 @@ In this work, we use the Best-of-N evaluation strategy and employ [VisualPRM-8B] ### Multimodal Reasoning and Mathematics -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/Vg18MtN8tpX-eF4ls6m26.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/uVIhB9BKlirAc6zlQYI0a.png) ### OCR, Chart, and Document Understanding -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/cx3DG7RfFg5LvHfbjqyWg.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/ivWJi3Rx_snJTfq--_9kD.png) ### Multi-Image & Real-World Comprehension -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/po72y3ypO6_RmtXty0b0U.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/y1OnFvJxei4dd9ZzCZ6yo.png) ### Comprehensive Multimodal & Hallucination Evaluation -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/RFykBfuGY8sJ3paZpmj39.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/XeBACd5_k_1lBbT70c6rp.png) ### Visual Grounding -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/N2V8VPGmcivyKvOAS7F5H.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/LFBrfb6amBxL_WKOqs5gr.png) ### Multimodal Multilingual Understanding -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/wPKL20-qJj4HZGuLn1bmf.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/1wc3dA3KwIeMWOVja2GF0.png) ### Video Understanding -![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/NuC0HDxJyW-JmvckMAqE8.png) +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/qU5-6VDYtFcXxSq2bNH7T.png) + +### GUI Grounding + +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/FRmXQGxgjbp1O_fYeJ2he.png) + +### Spatial Reasoning + +![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/J4AmWoDuJ6JwILdtsNJcN.png) ## Evaluation on Language Capability -Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series, which serves as the initialization of the langauge component in InternVL3. +We compare InternVL3 with Qwen2.5 Chat models, whose corresponding pre-trained base models are employed as the initialization of the langauge component in InternVL3. +Benefitting from Native Multimodal Pre-Training, the InternVL3 series achieves even better overall text performance than the Qwen2.5 series. +Please note that the evaluation scores of Qwen2.5 series may differ from those officially reported, as we have adopted the prompt versions provided in the table across all datasets for OpenCompass evaluation. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/619507e7b74b6c591f794340/geqAQiiganmQiDY65VIpm.png)