Rigid 3D Pose Estimation | Fritz AI

Rigid 3D Pose Estimation

Create augmented and virtual reality apps using Rigid 3D Pose Estimation to track the position and pose of any object in real-world coordinates. Use Rigid 3D Pose Estimation to build experiences where physical objects interact with virtual ones.
 

Custom Pose Estimation New

Quickly move from an idea to a production-ready Pose Estimation model with Fritz AI Studio.
import Fritz

let poseModel = FritzVisionPoseModel()

let image = FritzVisionImage(buffer: sampleBuffer)

poseModel.predict(image) { result, error in
    guard error == nil, let poseResult = result else { return }

    // Overlays pose on input image
    let imageWithPose = poseResult.drawPose()
}
Rigid 3D Pose Estimation

The Swift code sample here illustrates how simple it can be to use Pose Estimation in your app. Use the links below to access additional information about Fritz AI Studio and documentation that will help you get started.

Discover Studio

Features

Detect Custom Keypoints

Coordinates of predicted keypoints are provided for each skeleton detected.

Use pose lifting in order to infer the 3D pose from 2D coordinates.

Runs On-Device

All predictions / model inferences are made completely on-device.

No internet connection is required to interpret images or video.

No internet dependency means super-fast performance.

Model Variants

Fast: Optimized for speed, best for processing video streams in real-time or on older devices.

Accurate: Optimized for higher accuracy where prediction quality is more important than speed.

Small: Optimized for size, keep your application bundle size low and conserve bandwidth.

Cross-Platform SDKs

Supported mobile platforms:

  • Android Pose Estimation
  • iOS Pose Estimation
Live Video Performance

Runs on live video with a fast frame rate.

Exact FPS performance varies depending on device, but it is possible to run this feature on live video on modern mobile devices.

Technical Specifications

Architecture

Uses a MobileNet backbone

Model Size

~2 MB

FLOPS

1,600 M

Input

353x257-pixel image

Output

3D Coordinates of each keypoint detected

Number of keypoints detected

The confidence associated with each detection

Formats

Core ML, TensorFlow, TensorFlow Mobile, TensorFlow Lite, Keras

Benchmarks

25 FPS on iPhone X

7 FPS on Pixel 2