Image Segmentation to Apply Pixel Masks to Scenes | Fritz

Image Segmentation

Use Image Segmentation to recognize objects and identify exactly which pixels belong to each object. Great for creating pixel-level masks, performing photo compositing and more.
 

Getting Started

import Fritz

var visionModel: FritzVisionPeopleSegmentationModel?

let image = FritzVisionImage(image: uiImage)

visionModel.predict(image) { [weak self] (mask, error) in
    guard let mask = mask else { return }
    let maskImage = mask.toImageMask(of: FritzVisionPeopleClass.person)
    DispatchQueue.main.async {
        self?.maskView.image = maskImage
    }
}
Image Segmentation

The Swift code sample here illustrates how simple it can be to use image segmentation in your app. Use the links below to access additional documentation, code samples, and tutorials that will help you get started.

Mobile Image Segmentation

People

Mask One Person or Many
Example Use Cases
  • Portrait Mode
  • Composite Images
  • Bokeh Effect
Model Variants
  • High-performance
  • High-accuracy

Portrait Tutorial (iOS) Portrait Tutorial (Android) Superimpose X Case Study Momento Case Study

Image Segmentation for Hair

Hair

Mask Hair to Alter Color and Appearance
Example Use Cases
  • Change Color
  • Brighten or Darken
  • Switch Style
Model Variants
  • High-performance
  • High-accuracy

Hair Color Tutorial (iOS) Hair Color Tutorial (Android)

Image Segmentation for Pets

Pets

Mask Dogs and Cats
Example Use Cases
  • Pet Portraits
  • Composite Images
  • Pet Stickers
Model Variants
  • High-performance
  • High-accuracy

Pet Sticker Tutorial

Image Segmentation for Sky

Sky

Mask the Sky or Foreground
Example Use Cases
  • Nature Photography
  • Day/Night Composite Images
Model Variants
  • High-performance
  • High-accuracy

Image Segmentation for Indoor

Indoor

Mask Indoor Items and Furnishings
Indoor Labels
  • Chair
  • Wall
  • Coffee Table
  • Ceiling
  • Floor
  • Bed
  • Lamp
  • Sofa
  • Window
  • Pillow

Image Segmentation for Outdoor

Outdoor

Mask Streetscape and Landscape Objects
Outdoor Labels
  • Buildings
  • Sky
  • Tree
  • Pavement / Sidewalk
  • Ground
  • Car
  • Water
  • House
  • Fence
  • Signs
  • Skyscrapers
  • Bridge
  • River
  • Bus
  • Tree
  • Truck / Motortruck
  • Van
  • Minibike / Motorbike
  • Bicycle
  • Traffic Light
  • Person

Features

Multiple Scenes

Several Image Segmentation models are available for different scenes:

  • People
  • Hair
  • Pets
  • Indoor
  • Outdoor
Runs On-Device

All predictions / model inferences are made completely on-device.

No internet connection is required to interpret images or video.

No internet dependency means super-fast performance.

Customize Scenes

Our team can create custom image segmentation scenes upon request.

Cross-Platform SDKs

Supported mobile platforms:

  • Android Image Segmentation
  • iOS Image Segmentation
Live Video Performance

Runs on live video with a fast frame rate.

Exact FPS performance varies depending on device, but it is possible to run this feature on live video on modern mobile devices.

Technical Specifications

Architecture

MobileNet and ICNet variants

Model Size

5 to 25 MB

FLOPS

300 to 8,750 M

Input

Image, video, or camera

Output

Height and width of the mask

Number of classes the model predicts

Probability that pixel belongs to class

Formats

Core ML, TensorFlow, TensorFlow Mobile, TensorFlow Lite, Keras

Benchmarks

~30 FPS on iPhone X

~10 FPS on Pixel 2