A segmentation mask is an image-sized array in which every pixel carries a label that says which class, object or region it belongs to. In computer vision, it is the output format of image segmentation models and the ground-truth format used to train them, giving a pixel-accurate outline of shapes that a bounding box can only approximate.
Masks appear in medical imaging, autonomous driving, photo editing, satellite analysis and robotics. Their exact encoding depends on the task: semantic segmentation labels classes, instance segmentation separates individual objects, and panoptic segmentation combines both in a single map.
How a segmentation mask encodes pixel labels
A segmentation mask encodes labels as integer or binary values aligned to the image grid, so the pixel at row r and column c in the mask describes the pixel at the same position in the image. The mask has the same height and width as the image, or a known scale factor relative to it.
Three encodings dominate practice. A binary mask stores 0 for background and 1 for the object and is common for single-object tasks. A class-index mask stores one integer per pixel, such as 0 for road and 7 for pedestrian. A one-hot or probability tensor stores one channel per class, which is the form a neural network outputs before an argmax turns it into a class-index mask.
The following table summarizes the common mask encodings and where each is used.
| Encoding | Shape and values | Typical use |
|---|---|---|
| Binary mask | H x W, values 0 or 1 | One object, foreground extraction |
| Class-index mask | H x W, integers 0 to K-1 | Semantic segmentation labels |
| Probability tensor | K x H x W, floats 0 to 1 | Raw model output, soft masks |
| Instance ID map | H x W, unique ID per object | Instance and panoptic tasks |
| Run-length encoding | Compressed pixel runs | Storage in annotation files |
Datasets rarely store masks as raw arrays. They use indexed PNG images, polygon coordinates that are rasterized on load, or run-length encoding (RLE), which the widely used COCO annotation format adopts for compact storage of irregular shapes.
Mask types across segmentation tasks
The task defines what a mask label means, so the same pixel array can carry very different information. A semantic mask cannot tell two adjacent cars apart, while an instance mask can, and that difference decides which downstream analysis is possible.
| Task | What each label means | Limitation |
|---|---|---|
| Semantic segmentation | Class of the pixel | Touching objects of one class merge |
| Instance segmentation | Specific object, plus class | Background regions often unlabeled |
| Panoptic segmentation | Class plus instance ID for every pixel | More complex annotation and metrics |
| Video object segmentation | Object identity across frames | Errors accumulate over time |
| Interactive segmentation | Object chosen by user clicks | Depends on prompt quality |
Panoptic masks distinguish countable objects such as people and vehicles from amorphous regions such as sky, grass or road. Countable objects receive instance IDs, while amorphous regions receive only a class label, which gives a complete, non-overlapping description of the scene.
Segmentation masks compared with other annotations
A segmentation mask is the most spatially precise annotation among common computer vision labels, and also the most expensive to create. Choosing it over lighter formats is justified when exact shape, area or boundary matters to the application.
The comparison below places masks next to the neighboring annotation types.
| Annotation | Spatial detail | Relative labeling effort |
|---|---|---|
| Image-level label | None, whole image | Seconds per image |
| Bounding box | Rectangle around object | A few seconds per object |
| Keypoints | Selected landmark points | Moderate, depends on point count |
| Polygon outline | Vertices along the boundary | Tens of seconds per object |
| Segmentation mask | Every pixel labeled | Highest, minutes for complex scenes |
Because full masks are costly, teams often use weak supervision: boxes or scribbles are converted into approximate masks, or a promptable segmentation model proposes masks that annotators only correct. This can cut annotation time substantially, although the final quality still depends on human review of boundaries.
Measuring mask quality
Mask quality is measured by comparing predicted pixels with ground-truth pixels, most often through intersection over union (IoU), the area of overlap divided by the area of union. For a whole dataset, the per-class IoU values are averaged into mean IoU (mIoU), the standard semantic segmentation metric.
Other metrics capture aspects that IoU underweights. The Dice coefficient, equal to twice the overlap divided by the sum of both areas, is preferred in medical imaging. Boundary IoU focuses on pixels near edges, where most errors occur, and mask average precision extends mean average precision from boxes to masks for instance tasks. Panoptic tasks use panoptic quality (PQ), which combines recognition and segmentation accuracy.
A known pitfall is that large objects dominate pixel-level scores. A model can reach a high mIoU while missing thin structures such as poles, wires or blood vessels, so per-class and boundary metrics should be reported alongside the headline number.
Practical issues when working with masks
Most errors with segmentation masks come from data handling rather than model design. The table below lists frequent problems, their causes and the usual fixes.
| Symptom | Likely cause | Fix |
|---|---|---|
| Blurred or invalid labels | Mask resized with bilinear interpolation | Use nearest-neighbor resizing |
| Shifted masks | Augmentation applied to image only | Apply identical transforms to both |
| Wrong class colors | PNG palette read as RGB | Load as indexed or single-channel |
| Ignored edges in training | Boundary pixels marked as void | Check the ignore index, often 255 |
| Poor small-object results | Class imbalance in pixel counts | Use weighted or Dice loss |
Resolution is another trade-off. Many models predict masks at one quarter or one eighth of input resolution and upsample them, which saves memory but softens fine boundaries. Applications that need crisp edges, such as background removal, often add a refinement step or a separate matting model.
Explained without jargon
A segmentation mask is like a coloring-book page laid exactly over a photo. Every tiny dot of the picture gets colored according to what it shows: one color for the road, another for people, another for the sky.
A computer uses this colored overlay to know precisely where each thing is, down to its exact outline, instead of just drawing a rough box around it. That precision is what lets a phone blur the background behind a person or a car tell the sidewalk from the street.
A typical use case
A radiology team trains a model to measure liver tumors on CT scans. Specialists annotate about 300 scans by painting binary masks over each tumor slice by slice, and the model learns to output a mask of the same size for every new scan.
Because each mask pixel corresponds to a known physical size, for example 0.7 by 0.7 millimeters with a 2.5-millimeter slice thickness, the software counts mask pixels to compute tumor volume in milliliters. Validated with a Dice score of around 0.85 against expert annotations, the tool lets radiologists track whether a tumor shrinks between treatment cycles in seconds instead of manual measurement that takes several minutes per scan.