Custom Pose Estimation
Create augmented and virtual reality apps using Custom Pose Estimation to track the 2D position of keypoints on any object. Use Custom Pose Estimation to build experiences where physical objects interact with virtual ones.
Pose Estimation Platform New
Quickly move from an idea to a production-ready Pose Estimation model with Fritz AI Studio.
import Fritz
let poseModel = FritzVisionPoseModel()
let image = FritzVisionImage(buffer: sampleBuffer)
poseModel.predict(image) { result, error in
guard error == nil, let poseResult = result else { return }
// Overlays pose on input image
let imageWithPose = poseResult.drawPose()
}Custom Pose Estimation
The Swift code sample here illustrates how simple it can be to use Pose Estimation in your app. Use the links below to access additional information about Fritz AI Studio and documentation that will help you get started.
Discover StudioFeatures
Detect Custom Keypoints
2D Coordinates of predicted keypoints are provided for each skeleton detected.
Runs On-Device
All predictions / model inferences are made completely on-device.
No internet connection is required to interpret images or video.
No internet dependency means super-fast performance.
Model Variants
Fast: Optimized for speed, best for processing video streams in real-time or on older devices.
Accurate: Optimized for higher accuracy where prediction quality is more important than speed.
Small: Optimized for size, keep your application bundle size low and conserve bandwidth.
Cross-Platform SDKs
Supported mobile platforms:
- Android Pose Estimation
- iOS Pose Estimation
Live Video Performance
Runs on live video with a fast frame rate.
Exact FPS performance varies depending on device, but it is possible to run this feature on live video on modern mobile devices.
Technical Specifications
Architecture
Uses a MobileNet backbone
Model Size
~2 MB
FLOPS
1,600 M
Input
353x257-pixel image
Output
2D Coordinates of each keypoint detected
Number of keypoints detected
The confidence associated with each detection
Formats
Core ML, TensorFlow, TensorFlow Mobile, TensorFlow Lite, Keras
Benchmarks
25 FPS on iPhone X
7 FPS on Pixel 2
