PlantVLM‑Edge shows crops can be diagnosed on a tiny edge chip with almost full‑precision quality
arXiv: 2611.03557v1
Scientists have built a vision‑language model that can run entirely on a single NVIDIA Jetson AGX Orin, a small computing board that does not need internet access. The system, called PlantVLM‑Edge, adapts a pre‑trained vision‑language model to plant‑disease and insect‑pest question‑answer data using LoRA, then compresses the language backbone to 4‑bit or 8‑bit precision. The compressed model is served with vLLM, and farmers can chat with it via a local web page.
Figure 2. PlantVLM‑Edge pipeline schematic showing LoRA adaptation of the language backbone, four post‑training quantization schemes, and deployment on a Jetson AGX Orin.Wagle, Gaurab, Adhikari, Bishal, Li, Jiajia, Huang, Xiaolei, Chen, Dong · arXiv:2611.03557v1 · Figure 2: · CC BY 4.0
The framework works with any VLM. In experiments the authors used two backbones: a small Qwen3‑VL‑2B model and a larger LLaVA‑1.5‑7B model. After LoRA training, the models reached similar answer quality – about 78 % ROUGE‑L on plant‑disease questions and 54 % on pest questions – showing that the domain‑adaptation step does not require the larger backbone to win.
Quantization was applied only to the linear layers of the language backbone; the vision encoder and fusion layers stayed at 16‑bit, so the visual features fed to the language part were unchanged. Four compression schemes were tested: W8A16‑RTN, W8A8‑SQ, W4A16‑RTN, and W4A16‑AWQ. Memory usage dropped dramatically: the 2‑B model shrank from 3.96 GiB to 2.01 GiB with 4‑bit AWQ, a 49 % reduction. The 7‑B model fell from 13.16 GiB to 4.20 GiB, a 68 % reduction.
On the Jetson device, the 4‑bit AWQ model returned its first token in about 0.12 s and finished a 64‑token answer in 0.78 s, the lowest end‑to‑end latency among all variants. 8‑bit quantization changed answer quality only slightly compared to full precision. The 4‑bit AWQ kept 98.6 % of the full‑precision ROUGE‑L score on both disease and pest datasets.
However, the study also found limits. Exact‑match scores stayed below 5 % even after fine‑tuning, showing that crops’ free‑form diagnostic answers do not align to templates. When 4‑bit quantization was used without AWQ, the “Comprehensive Description” category suffered a loss of 3.67 ROUGE‑L points – the worst degradation seen – indicating that highly explanatory answers are most sensitive to aggressive compression.
Table listing ROUGE‑L scores for each category under full precision and 4‑bit quantization without AWQ, highlighting the 3.67‑point drop in Comprehensive Description.
Qwen3-VL-2B
LLaVA-1.5-7B
Category
𝐧
16-bit
W8
SQ
W4
AWQ
16-bit
W8
SQ
W4
AWQ
Causal Reasoning
2,180
66.02
−0.26
−0.58
−1.41
−1.44
65.74
−0.03
−0.09
−0.52
−0.25
Comprehensive Descr.
1,767
64.74
−0.21
+0.41
−3.67
−0.93
63.75
+0.05
0.00
−0.19
−0.26
Counterfactual Reas.
1,202
74.00
−0.15
+0.26
−1.84
−0.41
74.58
−0.06
−0.11
+0.41
−0.73
Detailed Verification
1,689
79.79
−0.14
+0.08
−0.65
−0.79
80.38
0.00
−0.20
−0.24
−0.51
Existence & Sanity
72
100.00
0.00
0.00
−0.32
−1.57
100.00
0.00
0.00
0.00
0.00
General Health
1,337
97.18
−0.03
−0.07
−0.69
−0.30
97.28
+0.01
−0.19
−0.33
−0.14
Plant Species ID
2,147
89.38
−0.05
−0.23
−2.36
−1.80
89.48
−0.06
−0.09
−0.38
−0.26
Specific Disease ID
3,972
76.87
−0.12
−0.02
−1.80
−0.95
77.09
−0.02
−0.04
0.00
−0.36
Visual Attribute Gr.
716
87.56
−0.13
−0.16
−0.28
−1.86
86.98
+0.01
+0.19
−0.58
−0.07
Table 6. Category‑wise ROUGE‑L loss for 4‑bit quantization without AWQ, showing a worst‑case decrease of 3.67 points in the “Comprehensive Description” category.Wagle, Gaurab, Adhikari, Bishal, Li, Jiajia, Huang, Xiaolei, Chen, Dong · arXiv:2611.03557v1 · Table 6: · CC BY 4.0
The results demonstrate that crop‑health diagnosis can be performed offline on a modest edge device, but the evaluation was limited to two VLM backbones, two benchmark datasets, and a single Jetson hardware model. Future work should test more architectures and devices and extend the benchmarks to further capture real‑world question diversity.
