PoseEstimator

scalacv.PoseEstimator

Human-pose (skeleton) estimation over a keypoint network run through Dnn.

MediaPipe's models ship as TFLite; OpenCV's inference path — and therefore scalacv's — is ONNX, so this is built the way FaceDetect and Dnn are: you bring the model (Dnn.fromOnnx), and scalacv provides the typed result and the decode. The two common output layouts are both handled (PoseEstimator.decode), so a MoveNet or an OpenPose export drops in by naming its KeypointLayout and PoseTopology.

// With a caller-loaded Net (see Dnn):
val pose = Dnn.blobFromImage(image.mat, size = Some(Size(192, 192)), swapRB = true).use { blob =>
 Dnn.forward(net, blob).use { out =>
   PoseEstimator.decode(out.mat, image.size, KeypointLayout.Regression)
 }
}

For hand and head pose see PoseTopology.Hand21 and HeadPose.

Attributes

Source: Pose.scala
Graph
Supertypes: class Object

trait Matchable

class Any
Self type: PoseEstimator.type

Members list

Value members

Concrete methods

Decodes a network's output tensor into a Pose in image pixels.

Value parameters

imageSize: the size of the image the keypoints should be scaled to.
layout: how the tensor encodes keypoints — see KeypointLayout.
output: the Mat from Dnn.forward.
topology: the naming/connectivity; its size must match the model's keypoint count.

Attributes

Source: Pose.scala

In this article

Generated with