243 lines
6.1 KiB
Markdown
243 lines
6.1 KiB
Markdown
|
# Material Analyzer
|
|||
|
|
|
|||
|
|
用于将 FloorVisualizer 的地板 SKU 离线预处理为可复现 `MaterialAssets` 的流水线。
|
|||
|
|
|
|||
|
|
## 运行
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . -limit 10 -workers 4 -verbose
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
CLI 默认读取：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
..\FloorVisualizer\data\products
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
并输出到：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
MaterialAssets\<safe-sku>\
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
常用参数示例：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . -data-dir "D:\go-demo\FloorVisualizer\data\products" -output-dir MaterialAssets
|
|||
|
|
go run . -sku "I966106LP" -force -verbose
|
|||
|
|
go run . -limit 100 -workers 8
|
|||
|
|
go run . -retry 2 -strict
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Provider 参数示例：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . -embedding-provider local -semantic-provider local
|
|||
|
|
go run . -embedding-provider http -embedding-url http://127.0.0.1:5200
|
|||
|
|
go run . -semantic-provider http -semantic-url http://127.0.0.1:5300 -semantic-model gemini-3-pro-image
|
|||
|
|
go run . -semantic-provider local,internvl3 -semantic-url http://127.0.0.1:5300
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
当配置多个 semantic provider 时，分析器会按字段做语义融合，并根据置信度加权。
|
|||
|
|
|
|||
|
|
## 模型依赖
|
|||
|
|
|
|||
|
|
本项目代码仓库不包含大模型权重。`InternVL3-8B` 应作为外部模型单独管理，例如放在独立 Git LFS 仓库、Hugging Face 仓库或团队对象存储中。
|
|||
|
|
|
|||
|
|
推荐本地目录结构：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
D:\go-demo\
|
|||
|
|
Material Analyzer\ # 本项目代码仓库
|
|||
|
|
data\
|
|||
|
|
InternVL3-8B\ # 外部模型目录，不提交到本项目仓库
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
导入模型时，将模型仓库或下载后的权重放到：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
D:\go-demo\data\InternVL3-8B
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
模型目录至少需要包含：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
config.json
|
|||
|
|
generation_config.json
|
|||
|
|
model.safetensors.index.json
|
|||
|
|
model-00001-of-00004.safetensors
|
|||
|
|
model-00002-of-00004.safetensors
|
|||
|
|
model-00003-of-00004.safetensors
|
|||
|
|
model-00004-of-00004.safetensors
|
|||
|
|
tokenizer.json
|
|||
|
|
tokenizer_config.json
|
|||
|
|
vocab.json
|
|||
|
|
merges.txt
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
如果模型放在其他位置，通过环境变量指定：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
set INTERNVL3_MODEL_PATH=D:\path\to\InternVL3-8B
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
模型目录中有本项目专用说明：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
D:\go-demo\data\InternVL3-8B\README_MaterialAnalyzer.md
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 输出
|
|||
|
|
|
|||
|
|
每个处理完成的 SKU 会生成：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
preview.jpg
|
|||
|
|
thumbnail.jpg
|
|||
|
|
material.json
|
|||
|
|
histogram.json
|
|||
|
|
embedding.bin
|
|||
|
|
manifest.json
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`material.json` 包含：
|
|||
|
|
|
|||
|
|
- 分析器版本信息：`analyzer_version`、`feature_schema`、`generator`、`generated_at`
|
|||
|
|
- 通用颜色特征：主/次 RGB、LAB、HSV、亮度、对比度、饱和度
|
|||
|
|
- 纹理特征：熵、Sobel 频率、边缘密度、方向方差、LBP 均匀度、GLCM 统计
|
|||
|
|
- 规范化统计：平均/中位 RGB、颜色方差、纹理方差、亮度分布、梯度直方图
|
|||
|
|
- 语义特征：描述、材质类型、表面处理、纹理类型、颜色族、风格、变化程度、光泽等级、标签
|
|||
|
|
- 材质适配器输出，以及木材、瓷砖、仿石瓷砖、乙烯基地板和通用材质的字段置信度
|
|||
|
|
- 校验状态
|
|||
|
|
- 二进制向量的 embedding manifest
|
|||
|
|
|
|||
|
|
`manifest.json` 会列出所有文件和资产校验状态。
|
|||
|
|
|
|||
|
|
批量运行会写入：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
MaterialAssets/benchmark.json
|
|||
|
|
MaterialAssets/failures.json
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`embedding.bin` 存储两个拼接的 float32 向量。默认 local provider 会生成确定性的 fallback 向量：
|
|||
|
|
|
|||
|
|
- `dino_v2_texture_local`：颜色与纹理指纹
|
|||
|
|
- `clip_semantic_local`：元数据与语义哈希指纹
|
|||
|
|
|
|||
|
|
如果使用可选模型服务，则会生成：
|
|||
|
|
|
|||
|
|
- `dinov2_texture`：`facebook/dinov2-large`，1024 维，已归一化
|
|||
|
|
- `clip_visual`：`ViT-L/14`，768 维，已归一化
|
|||
|
|
|
|||
|
|
## 可选模型服务
|
|||
|
|
|
|||
|
|
Go CLI 可以调用本地 DINOv2/CLIP 模型服务：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd tools/model_server
|
|||
|
|
python -m pip install -r requirements.txt
|
|||
|
|
python server.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
然后运行：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . -sku I966106LP -force -embedding-provider http -embedding-url http://127.0.0.1:5200
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
第一次请求会通过 Hugging Face 下载模型权重，可能需要一些时间。
|
|||
|
|
|
|||
|
|
## InternVL3 语义服务
|
|||
|
|
|
|||
|
|
`InternVL3-8B` 用于本项目的视觉语义分析。它不会生成 embedding，而是读取 `preview.jpg` 和产品元数据，输出结构化材质语义字段。
|
|||
|
|
|
|||
|
|
启动本地 InternVL3 语义服务：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cd tools/vision_server
|
|||
|
|
python -m pip install -r requirements.txt
|
|||
|
|
set INTERNVL3_MODEL_PATH=D:\go-demo\data\InternVL3-8B
|
|||
|
|
python server.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
服务默认监听：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
http://127.0.0.1:5300/semantic
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
然后让分析器连接该服务：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . -sku I966106LP -semantic-provider internvl3 -semantic-url http://127.0.0.1:5300
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
也可以和本地规则 provider 组合使用：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . -semantic-provider local,internvl3 -semantic-url http://127.0.0.1:5300
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
常用环境变量：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
INTERNVL3_MODEL_PATH 模型目录，默认 D:\go-demo\data\InternVL3-8B
|
|||
|
|
INTERNVL3_PORT 服务端口，默认 5300
|
|||
|
|
INTERNVL3_IMAGE_SIZE 输入图块尺寸，默认 448
|
|||
|
|
INTERNVL3_MAX_TILES 最大图块数，默认 4
|
|||
|
|
INTERNVL3_MAX_NEW_TOKENS 最大生成 token 数，默认 512
|
|||
|
|
INTERNVL3_TEMPERATURE 生成温度，默认 0.0
|
|||
|
|
INTERNVL3_LOAD_IN_8BIT 是否 8bit 量化加载
|
|||
|
|
INTERNVL3_LOAD_IN_4BIT 是否 4bit 量化加载
|
|||
|
|
INTERNVL3_USE_FLASH_ATTN 是否启用 flash attention
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Material Intelligence
|
|||
|
|
|
|||
|
|
基于已有 `MaterialAssets` 构建 v3 知识层：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . intelligence -assets-dir MaterialAssets -output-root . -k 5
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
该命令会创建：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
KnowledgeBase/knowledge.db
|
|||
|
|
KnowledgeBase/similarity.json
|
|||
|
|
PromptDataset/prompt_dataset.json
|
|||
|
|
Statistics/statistics.json
|
|||
|
|
Embeddings/embeddings.json
|
|||
|
|
Fingerprints/fingerprints.json
|
|||
|
|
GroundTruth/ground_truth.json
|
|||
|
|
Regression/regression_report.json
|
|||
|
|
Benchmarks/benchmark_vision.json
|
|||
|
|
material_intelligence_summary.json
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
每个资产也会收到：
|
|||
|
|
|
|||
|
|
```text
|
|||
|
|
semantic.json
|
|||
|
|
manifest.json
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
`manifest.json` 包含 SHA256、文件大小和文件时间戳，用于资产完整性校验。
|
|||
|
|
|
|||
|
|
使用基线资产库做回归对比：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
go run . intelligence -assets-dir MaterialAssets -baseline-dir OldMaterialAssets
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 房间图识别
|
|||
|
|
|
|||
|
|
如果产品主材质图看起来像房间场景，分析器会标记：
|
|||
|
|
|
|||
|
|
```json
|
|||
|
|
"room_like": true
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
并使用图像下方的地面区域裁剪做特征提取。这样可以让流水线继续运行，同时也便于后续审计和清理这些资产。
|