lm3_specimen_segmenter__yolo26x_seg_1280

YOLO26x-seg trained to segment plant material on a herbarium specimen sheet; the union of its instance masks is the foreground used for background removal. Alternative to the LeafMachine3 default specimen_segmenter model (lm3_specimen_segmenter__unetpp_effb7_1024).

LM3 metadata

Field	Value
LM3 action	specimen_segmenter
LM3_default_model	false
LM3_version	TBD: supported from the LM3 release that adds model keys to the specimen_segmenter stage
Model key	yolo26x_seg_1280 (training run yolo26x-seg-1280-paperclean)
Architecture	YOLO26x segment, end-to-end (NMS-free)
Base weights	yolo26x-seg.pt (Ultralytics, AGPL-3.0)
Input	whole sheet image, 1280 letterbox, RGB
Output	instance boxes + masks for one class; foreground = union of instance masks at confidence >= 0.25
Classes (1)	plant
Trained	2026-07-31

Files

Folder	File	Use
pytorch/	best.pt	Ultralytics, retraining
onnx/	model.onnx	onnxruntime (same export form as the LeafMachine3 YOLO26 leaf segmenter)
torchscript/	model.torchscript	libtorch
openvino/	model_openvino_model/	Intel CPU/iGPU
coreml/	model.mlpackage/	Apple devices
docs/	compare_*.jpg	example panels (see Examples)

SHA-256 of every file is in manifest.json.

Training data

There is no hand-drawn ground truth for this task. The training targets are machine-generated masks, refined in three steps:

1. Boxes to masks. Specimen boxes from the LeafMachine3 plant detection set prompt SAM 3; the per-box masks are unioned, holes filled, and failed masks gated out. 21,098 sheets kept.
2. Self-training. A first UNet++ trained on those masks re-predicts every herbarium sheet; its cleaner predictions replace the SAM 3 masks. 2,193 field photographs are excluded as out of domain. 18,904 sheets.
3. Paper removal. A color rule samples the mounting paper around each mask and deletes large in-mask regions that match it.

The result is dataset LM3_Specimen_Segmentation-specimen-unetMasksFromSam3-paperclean: 18,904 sheet images from 75 labeling projects, one plant class. Masks were converted to instance polygons for this model. Images and labels are not distributed.

Split	Images
train	15,154
val	1,910
test	1,840

Training

300 epochs planned with early-stopping patience 50; stopped at epoch 151, best checkpoint from epoch 101. Batch 16, 1280 letterbox, cosine LR from 0.01, close_mosaic=15, overlap_mask, mask_ratio=4. Augmentation: hsv 0.015/0.7/0.4, degrees 15, translate 0.1, scale 0.5, flipud 0.5, fliplr 0.5, mosaic 1.0, mixup 0.15, copy_paste 0.3 (flip), erasing 0.4, randaugment.

Evaluation

Scores are agreement with the machine-generated masks described above, not with human annotation. They measure how well the model reproduces its training targets on held-out sheets; a model that corrects a flawed target is penalized for it. Read them as a consistency check and judge quality from the examples. Every LeafMachine3 specimen-segmenter card is scored on the same random 150-sheet sample of each split, so the numbers are comparable across models.

Split	Sheets scored	Dice (mean)	Dice (median)	IoU (mean)
val, LM3 production path	150 of 1,910	0.915	0.961	0.865
test, LM3 production path	150 of 1,840	0.919	0.959	0.870
val, plus paper removal	150 of 1,910	0.943	0.979	0.913
test, plus paper removal	150 of 1,840	0.948	0.978	0.916

Scored at native sheet resolution on the union mask, through LeafMachine3's own YOLO path (full-resolution instance masks); 2% of test sheets score below 0.5 Dice. LeafMachine3's paper-removal follow-up lifts test Dice from 0.919 to 0.948, because it deletes the mounting paper the polygon masks enclose. Validation during training at the best epoch (instance metrics against the same machine-generated polygons): box mAP50 0.864, mask mAP50 0.860, mask mAP50-95 0.725. Polygon masks tend to fill paper enclosed by stems and branches, which keeps more of the sheet than the pixel-wise models (see Examples).

Examples

Background removal on the same sheets by this model (YOLO26x-seg) and by the other LeafMachine3 specimen segmenters trained on the same data, for comparison. Black marks what each model removed; the percentage is the share of the sheet kept. Raw model masks, before LeafMachine3's paper-removal follow-up. File names give each sheet's dataset split. Panels 1 to 12 are a standing comparison set that includes deliberately hard sheets (several from the training split); panels 13 to 20 are random held-out test sheets.

Acer saccharum, CMN_1804514859 · test split · kept: UNet++ 40.1%, BiRefNet 40.2%, YOLO26x-seg 47.1%

Acer saccharum, CM_2859177966 · train split · kept: UNet++ 28.5%, BiRefNet 28.5%, YOLO26x-seg 32.4%

Cassia fistula, FLAS_1457816931 · val split · kept: UNet++ 23.0%, BiRefNet 23.3%, YOLO26x-seg 23.8%

Chrysophyllum gonocarpum, NY_1928830727 · val split · kept: UNet++ 22.6%, BiRefNet 22.2%, YOLO26x-seg 30.8%

CHC21_009_9 · val split · kept: UNet++ 2.6%, BiRefNet 2.6%, YOLO26x-seg 2.7%

Macaranga saccifera, MNHN_730851102 · val split · kept: UNet++ 44.4%, BiRefNet 44.6%, YOLO26x-seg 49.3%

Macaranga magnifolia, MNHN_731821562 · train split · kept: UNet++ 11.0%, BiRefNet 0.5%, YOLO26x-seg 47.6%

Macaranga gracilis, UM_1807486051 · train split · kept: UNet++ 31.2%, BiRefNet 34.0%, YOLO26x-seg 37.8%

Torricellia angulata, K_912592861 · train split · kept: UNet++ 22.7%, BiRefNet 24.2%, YOLO26x-seg 27.5%

Quercus grisebachii, CLF_1437883182 · train split · kept: UNet++ 22.5%, BiRefNet 23.5%, YOLO26x-seg 23.8%

Quercus semilanuginosa, CLF_1437893278 · train split · kept: UNet++ 20.5%, BiRefNet 32.1%, YOLO26x-seg 29.5%

Quercus pinnatiloba, MNHN_731018123 · train split · kept: UNet++ 33.0%, BiRefNet 32.7%, YOLO26x-seg 33.7%

Lonicera maackii, KYO_2443051967 · test split · kept: UNet++ 15.1%, BiRefNet 15.5%, YOLO26x-seg 19.2%

Macaranga waturandangii, BO1284103_4067113860 · test split · kept: UNet++ 33.8%, BiRefNet 34.1%, YOLO26x-seg 35.6%

Melanophylla alnifolia, L_2517447339 · test split · kept: UNet++ 22.2%, BiRefNet 21.9%, YOLO26x-seg 23.7%

Raphanus raphanistrum, ASU_2270499210 · test split · kept: UNet++ 10.0%, BiRefNet 10.2%, YOLO26x-seg 15.7%

Posoqueria longiflora, UM_1807432107 · test split · kept: UNet++ 29.4%, BiRefNet 29.5%, YOLO26x-seg 31.4%

1D41D09F-BC60-4E11-9283-3A0F23787BA7_1_105_c · test split · kept: UNet++ 24.9%, BiRefNet 24.9%, YOLO26x-seg 27.8%

Piper salinasanum, MNHN_438671634 · test split · kept: UNet++ 12.1%, BiRefNet 12.0%, YOLO26x-seg 13.3%

lab_tilia_tomentosa_ny1110-08-4 · test split · kept: UNet++ 7.5%, BiRefNet 7.5%, YOLO26x-seg 7.7%

Use in LeafMachine3

Supported from the LM3 release that adds model keys to the specimen_segmenter stage (version TBD). The stage then runs this model with its own training-time input (1280 letterbox) and joins the instance masks into one specimen mask before paper removal. Install and select it:

lm3 models install --model specimen_segmenter=yolo26x_seg_1280

modules:
  specimen_segmenter:
    model: { key: "yolo26x_seg_1280", path: "models/specimen_segmenter/yolo26x_seg_1280/model.onnx", format: "onnx" }
    yolo: { conf: 0.25, iou: 0.5, max_det: 300 }
    paperclean: true

from huggingface_hub import hf_hub_download
path = hf_hub_download("phyloforfun/lm3_specimen_segmenter__yolo26x_seg_1280", "onnx/model.onnx")

Citation

Weaver, W. N. (2026) LeafMachine3. https://leafmachine.org