Researchers have created MoI‑Bench, a benchmark focused on ​mobile image generation and editing. It contains 64 tasks and 1,150 examples, built around 16 operations (such as removal or replacement) and eight visual target types, from portraits to text.

To judge each task, the team introduced MoI‑Eval. The evaluation asks task‑specific questions about how well a model follows the instruction, the visual quality of its output and how well it keeps details. Scores for every example are folded into two axes: overall quality and capability, shown across operation, target and criterion.

MoI‑Bench lets the authors plot models on a Quality‑Deployability map. This two‑dimensional chart shows both how well a model performs and how expensive it is to run on a phone. Using the map, they compared representative on‑device models to server‑side ones.

The results reveal that current on‑device models can run efficiently for some tasks but they cover fewer operations. Improving the range of tasks and the quality of edits—especially keeping a person’s identity and rendering text accurately—should be higher priorities than merely tightening resource use.

The benchmark therefore points to concrete gaps like text‑replacement that deserve attention before overall text‑editing quality can be widely improved.