open-mmlab/mmsegmentation

Introduction

MMSegmentation is an open source semantic segmentation toolbox based on PyTorch. It is a part of the OpenMMLab project.

The main branch works with PyTorch 1.6+.

🎉 Introducing MMSegmentation v1.0.0 🎉

We are thrilled to announce the official release of MMSegmentation's latest version! For this new release, the main branch serves as the primary branch, while the development branch is dev-1.x. The stable branch for the previous release remains as the 0.x branch. Please note that the master branch will only be maintained for a limited time before being removed. We encourage you to be mindful of branch selection and updates during use. Thank you for your unwavering support and enthusiasm, and let's work together to make MMSegmentation even more robust and powerful! 💪

MMSegmentation v1.x brings remarkable improvements over the 0.x release, offering a more flexible and feature-packed experience. To utilize the new features in v1.x, we kindly invite you to consult our detailed 📚 migration guide, which will help you seamlessly transition your projects. Your support is invaluable, and we eagerly await your feedback!

Major features

• Unified Benchmark

  We provide a unified benchmark toolbox for various semantic segmentation methods.

• Modular Design

  We decompose the semantic segmentation framework into different components and one can easily construct a customized semantic segmentation framework by combining different modules.

• Support of multiple methods out of box

  The toolbox directly supports popular and contemporary semantic segmentation frameworks, e.g. PSPNet, DeepLabV3, PSANet, DeepLabV3+, etc.

• High efficiency

  The training speed is faster than or comparable to other codebases.

What's New

v1.2.0 was released on 10/12/2023, from 1.1.0 to 1.2.0, we have added or updated the following features:

Highlights

• Support for the open-vocabulary semantic segmentation algorithm SAN

• Support monocular depth estimation task, please refer to VPD and Adabins for more details.

  

• Add new projects: open-vocabulary semantic segmentation algorithm CAT-Seg, real-time semantic segmentation algofithm PP-MobileSeg

Installation

Please refer to get_started.md for installation and dataset_prepare.md for dataset preparation.

Get Started

Please see Overview for the general introduction of MMSegmentation.

Please see user guides for the basic usage of MMSegmentation. There are also advanced tutorials for in-depth understanding of mmseg design and implementation .

A Colab tutorial is also provided. You may preview the notebook here or directly run on Colab.

To migrate from MMSegmentation 0.x, please refer to migration.

Tutorial

Get Started	MMSeg Basic Tutorial	MMSeg Detail Tutorial	MMSeg Development Tutorial
MMSeg overview MMSeg Installation FAQ	Tutorial 1: Learn about Configs Tutorial 2: Prepare datasets Tutorial 3: Inference with existing models Tutorial 4: Train and test with existing models Tutorial 5: Model deployment Deploy mmsegmentation on Jetson platform Useful Tools Feature Map Visualization Visualization	MMSeg Dataset MMSeg Models MMSeg Dataset Structures MMSeg Data Transforms MMSeg Dataflow MMSeg Training Engine MMSeg Evaluation	Add New Datasets Add New Metrics Add New Modules Add New Data Transforms Customize Runtime Settings Training Tricks Contribute code to MMSeg Contribute a standard dataset in projects NPU (HUAWEI Ascend) 0.x → 1.x migration 0.x → 1.x package

Benchmark and model zoo

Results and models are available in the model zoo.

Supported backbones	Supported methods	Supported Head	Supported datasets	Other
ResNet(CVPR'2016) ResNeXt (CVPR'2017) HRNet (CVPR'2019) ResNeSt (ArXiv'2020) MobileNetV2 (CVPR'2018) MobileNetV3 (ICCV'2019) Vision Transformer (ICLR'2021) Swin Transformer (ICCV'2021) Twins (NeurIPS'2021) BEiT (ICLR'2022) ConvNeXt (CVPR'2022) MAE (CVPR'2022) PoolFormer (CVPR'2022) SegNeXt (NeurIPS'2022)	SAN (CVPR'2023) VPD (ICCV'2023) DDRNet (T-ITS'2022) PIDNet (ArXiv'2022) Mask2Former (CVPR'2022) MaskFormer (NeurIPS'2021) K-Net (NeurIPS'2021) SegFormer (NeurIPS'2021) Segmenter (ICCV'2021) DPT (ArXiv'2021) SETR (CVPR'2021) STDC (CVPR'2021) BiSeNetV2 (IJCV'2021) CGNet (TIP'2020) PointRend (CVPR'2020) DNLNet (ECCV'2020) OCRNet (ECCV'2020) ISANet (ArXiv'2019/IJCV'2021) Fast-SCNN (ArXiv'2019) FastFCN (ArXiv'2019) GCNet (ICCVW'2019/TPAMI'2020) ANN (ICCV'2019) EMANet (ICCV'2019) CCNet (ICCV'2019) DMNet (ICCV'2019) Semantic FPN (CVPR'2019) DANet (CVPR'2019) APCNet (CVPR'2019) NonLocal Net (CVPR'2018) EncNet (CVPR'2018) DeepLabV3+ (CVPR'2018) UPerNet (ECCV'2018) ICNet (ECCV'2018) PSANet (ECCV'2018) BiSeNetV1 (ECCV'2018) DeepLabV3 (ArXiv'2017) PSPNet (CVPR'2017) ERFNet (T-ITS'2017) UNet (MICCAI'2016/Nat. Methods'2019) FCN (CVPR'2015/TPAMI'2017)	ANN_Head APC_Head ASPP_Head CC_Head DA_Head DDR_Head DM_Head DNL_Head DPT_HEAD EMA_Head ENC_Head FCN_Head FPN_Head GC_Head LightHam_Head ISA_Head Knet_Head LRASPP_Head mask2former_Head maskformer_Head NL_Head OCR_Head PID_Head point_Head PSA_Head PSP_Head SAN_Head segformer_Head segmenter_mask_Head SepASPP_Head SepFCN_Head SETRMLAHead_Head SETRUP_Head STDC_Head Uper_Head VPDDepth_Head	Cityscapes PASCAL VOC ADE20K Pascal Context COCO-Stuff 10k COCO-Stuff 164k CHASE_DB1 DRIVE HRF STARE Dark Zurich Nighttime Driving LoveDA Potsdam Vaihingen iSAID Mapillary Vistas LEVIR-CD BDD100K NYU HSIDrive20	Supported loss boundary_loss cross_entropy_loss dice_loss focal_loss huasdorff_distance_loss kldiv_loss lovasz_loss ohem_cross_entropy_loss silog_loss tversky_loss

Please refer to FAQ for frequently asked questions.

Projects

Here are some implementations of SOTA models and solutions built on MMSegmentation, which are supported and maintained by community users. These projects demonstrate the best practices based on MMSegmentation for research and product development. We welcome and appreciate all the contributions to OpenMMLab ecosystem.

Contributing

We appreciate all contributions to improve MMSegmentation. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMSegmentation is an open source project that welcome any contribution and feedback. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible as well as standardized toolkit to reimplement existing methods and develop their own new semantic segmentation methods.

Citation

If you find this project useful in your research, please consider cite:

@misc{mmseg2020,
    title={{MMSegmentation}: OpenMMLab Semantic Segmentation Toolbox and Benchmark},
    author={MMSegmentation Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmsegmentation}},
    year={2020}
}

License

This project is released under the Apache 2.0 license.

OpenMMLab Family

• MMEngine: OpenMMLab foundational library for training deep learning models.
• MMCV: OpenMMLab foundational library for computer vision.
• MMPreTrain: OpenMMLab pre-training toolbox and benchmark.
• MMagic: OpenMMLab Advanced, Generative and Intelligent Creation toolbox.
• MMDetection: OpenMMLab detection toolbox and benchmark.
• MMYOLO: OpenMMLab YOLO series toolbox and benchmark.
• MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
• MMRotate: OpenMMLab rotated object detection toolbox and benchmark.
• MMTracking: OpenMMLab video perception toolbox and benchmark.
• MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
• MMOCR: OpenMMLab text detection, recognition, and understanding toolbox.
• MMPose: OpenMMLab pose estimation toolbox and benchmark.
• MMHuman3D: OpenMMLab 3D human parametric model toolbox and benchmark.
• MMFewShot: OpenMMLab fewshot learning toolbox and benchmark.
• MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
• MMFlow: OpenMMLab optical flow toolbox and benchmark.
• MMDeploy: OpenMMLab Model Deployment Framework.
• MMRazor: OpenMMLab model compression toolbox and benchmark.
• MIM: MIM installs OpenMMLab packages.
• Playground: A central hub for gathering and showcasing amazing projects built upon OpenMMLab.