First-hand information for everyone
A new preprint on arXiv examines whether training multimodal large language models (MLLMs) on in‑domain industrial inspection data is enough to match human performance across the range of tasks needed for practical anomaly understanding. The short answer from the authors: not yet — but a different architectural approach helps close the gap.
The paper studies ‘‘multimodal industrial anomaly understanding’’ (MM‑IAU), a set of closely related problems that together matter in manufacturing inspection: detecting whether an object is defective, localizing the defect in an image, describing it in words, and reasoning about its likely cause or severity. The authors report that a single MLLM, even when specialized with in‑domain training, struggles to excel at all four tasks simultaneously. On the commonly used MMAD benchmark, trained specialist models achieve at most 75.5% accuracy in defect localization, compared with 92.3% for human experts. Alarmingly, some trained specialists even detect anomalies less accurately than their untrained base model, showing that fine‑tuning can sometimes harm certain fine‑grained perceptual abilities.
The study also finds that different MLLMs tend to offer complementary strengths: one model might be good at spotting anomalies, another at producing detailed text descriptions. However, these models share a common weakness in ‘‘fine‑grained perception’’ — the ability to distinguish subtle visual differences that matter for industrial defects — so simply combining multiple models without a different design does not remove the shortfall.
To address this, the authors propose SiGMA, a spatially grounded multi‑agent framework that divides labor among heterogeneous components. In their implementation: a multimodal searcher supplies relevant industrial knowledge and retrieves ‘‘normal’’ reference images; a dedicated visual defect expert performs careful query‑to‑reference comparison and converts differences into calibrated anomaly evidence; and a label‑free reliability controller weighs each agent’s output according to how competent it is for a given task and how strong the evidence looks for that particular query. The design is deliberately modular so new MLLMs can be plugged in without retraining the whole system.
On MMAD, SiGMA achieves an average accuracy of 85.2%, which the authors note is 4.0 percentage points higher than the strongest single trained specialist and one high‑end model they tested (Gemini‑2.5‑Pro), and within 1.5 percentage points of human experts. The paper also shows a compact three‑agent configuration (agents at most 9B parameters each) reaching 84.4%, suggesting the approach can be effective without relying solely on the largest models.
The results are promising but the authors — and the paper’s framing — are careful about the remaining gaps and limitations. MMAD is a widely used benchmark, but real factory floors present wider variability in lighting, materials, sensor types, and rare or evolving defect modes. The need for a dedicated visual defect expert and a controller that evaluates evidence quality points to the current reality: purely end‑to‑end multimodal language models still struggle with very fine visual discrimination required in industrial contexts.
Practical implications are clear. A modular, multi‑agent system that combines domain retrieval, specialized visual comparison, and query‑level reliability estimation could make automated inspection systems more accurate and easier to update: new language or vision agents can be added without full retraining, and specialization helps avoid tradeoffs that hurt some tasks. At the same time, companies should treat the results as an intermediate step, not a drop‑in replacement for human inspectors: human experts still outperform models on localization, and the system’s robustness in real production lines remains to be validated.
Future work suggested by the study includes testing across more diverse datasets and sensors, evaluating long‑term deployment under changing defect distributions, and developing stronger techniques for fine‑grained visual perception that do not rely on extensive per‑task labeled data. The preprint is available on arXiv (arXiv:2610.12310v1) for readers who want full technical details and experimental results.