scalacv

package scalacv

Members list

Type members

Classlikes

How adaptive thresholding weights each pixel's neighbourhood. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Animation — a drawing that is a function of the frame number, rendered to a video.

The natural extension of a composable Picture: describe frame i as frame(i), and record draws every frame onto a fresh canvas and writes them out through a Recorder. Creative coding, a rendered data animation, a synthetic test clip — all fall out of the same graphics vocabulary.

// `.avi`, not `.mp4`: the default codec is MJPG, which opens only in an AVI container — see [[Codec.Mjpg]].
Animation.record("spin.avi", frames = 60, width = 320, height = 240) { i =>
 Picture.regularPolygon(Point(160, 120), sides = 5, radius = 80, rotation = i * 6)
   .strokeColor(Color.hsl(i * 6, 0.8, 0.6)).strokeWidth(3)
}

Attributes

Source: Animation.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Animation.type

Marker-based augmented reality: recover a marker's 3D pose from its detected corners, then project model geometry back onto the image to draw on top of it.

The flow is the classic one. estimatePose runs solvePnP (with the square-planar IPPE_SQUARE solver, which is both faster and more stable for a flat tag than the general iterative one) against the four corners an Aruco detection gives you, yielding a Pose3D. project then maps any Point3 model — a set of axes, a cube — through that pose and the camera Intrinsics to pixel coordinates you can draw with the ordinary Draw verbs. The high-level image.drawMarkerAxes and image.drawMarkerCube wire all three steps together.

Attributes

Source: Ar.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Ar.type

ArUco marker detection and generation over org.opencv.objdetect.ArucoDetector. Follows the same build-use-free ownership model documented on Qr — detectors are never handed to the caller.

Attributes

Source: Detectors.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Aruco.type

The predefined ArUco dictionaries.

The name encodes the tag's bit grid and the dictionary size — Dict5x5_250 is 250 distinct 5x5 markers. Fewer markers in a dictionary means a larger Hamming distance between them and therefore more robust detection, so pick the smallest one that has enough ids for the job rather than the largest.

Objdetect.DICT_* are the underlying constants; the AprilTag families are addressable here too because OpenCV's ArUco detector reads them through the same path.

Attributes

Source: Detectors.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

One detected ArUco marker: its dictionary id and its four corners, clockwise from the top left in the marker's own frame.

Attributes

Source: Detectors.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Video-conferencing background effects — blur or replace the background behind a person.

The compositing here is scalacv's and needs no model: given a foreground mask (white over the person, black over the background), it feathers the edge and alpha-blends. The mask is the part a model produces — run a selfie-segmentation ONNX network through Dnn and turn its output into a mask with Segmenter, or bring any binary mask you already have (even a colour Image.inRange key for a green screen).

The effects are extension methods on Image (defined below) — image.blurBackground(mask) and image.replaceBackground(mask, bg) — so they chain like any other transform; this object holds the shared compositing they delegate to.

Attributes

Source: BackgroundEffect.scala
Supertypes: class Object

trait Matchable

class Any
Self type: BackgroundEffect.type

Border extrapolation.

Plain rather than an enum-with-modifiers: BORDER_ISOLATED is a modifier, but it only means anything for ROI-based calls that scalacv does not expose yet, so it is deliberately omitted rather than offered and ignored.

==One type, two domains==

OpenCV packs two different sets of accepted values into this one int, and this enum is the union of them. copyMakeBorder (behind pad and border) and warpAffine (behind rotated) honour all five modes. The imgproc filter family — gaussianBlur, boxBlur, sobel, laplacian — does not: it rejects BorderType.Wrap, see that case and BorderType.requireFilterSupport.

Splitting the type (a FilterBorder without Wrap, widening into a TransformBorder with it) is the end state that would make the mistake unrepresentable, but it is a breaking change to four public signatures. Until then BorderType.requireFilterSupport is the check every filter operation must run on its border parameter, so the rejection lands at the Scala boundary instead of as an assertion failure inside OpenCV. BORDER_TRANSPARENT stays out of the enum entirely for a related reason: copyMakeBorder throws on it, and warpAffine with it leaves the freshly-allocated destination uninitialised, so the value can only ever produce a crash or garbage pixels.

Attributes

Companion: object
Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: enum
Source: Enums.scala
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: BorderType.type

The axis-aligned bounding box of a Picture — what the layout combinators (Picture.beside, Picture.above) measure to place pictures next to each other.

Attributes

Source: Graphics.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Reports how this artifact was built. Present from the first commit so the build has a real compilation unit and so a consumer can report an accurate version in a bug report.

Every value here is generated from build.mill's Deps block rather than typed out a second time — see core.generatedSources. That is not tidiness: a version written down in two places is a version that eventually disagrees with itself, and the failure is silent. A dependency bump used to move the build and leave this object (and the "add these lines" help text in OpenCv and Cascades) quoting the previous release, which is precisely the number a bug report or a broken classpath depends on being right.

Attributes

Source: Build.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Build.type

The result of a camera calibration: the recovered Intrinsics, the image size they were measured at, and the RMS reprojection error (in pixels) the solver achieved.

The reprojection error is the headline quality number: it is the root-mean-square distance, over every corner of every view, between where a corner actually sat and where the recovered model says it should be. Under ~1px is a good calibration; several pixels means blurry captures, a bad board, or too few angles.

The intrinsics are the same Intrinsics that Ar, HeadPose and Localizer take — so a real calibration drops straight into the pose stack in place of Intrinsics.approx's field-of-view guess, and an Image can be undistorted with Image.undistort.

Attributes

Companion: object
Source: Calibration.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Camera calibration from chessboard captures — the foundation the metric parts of the vision stack were built to sit on.

Everything that turns pixels into geometry (VisualOdometry, Localizer, Ar, HeadPose, StereoDepth) needs to know the camera's focal length, optical centre and lens distortion. Without a calibration they run on Intrinsics.approx's "focal ≈ image width, no distortion" guess — enough to see an overlay track, not enough to measure. This turns the guess into measured numbers: show the camera a chessboard from a handful of angles, and fromChessboard recovers the Intrinsics.

val board = ChessboardPattern(columns = 9, rows = 6, squareSize = 0.025) // 25 mm squares
val views = (1 to 15).flatMap(i => Image.read(s"calib/$i.jpg").toOption)
Calibration.fromChessboard(views, board) match
 case Right(calib) =>
   println(f"calibrated to ${calib.reprojectionError}%.3f px RMS")
   val straight = frame.undistort(calib)         // lens distortion removed
   Ar.estimatePose(marker, 0.05, calib.intrinsics) // metric marker pose
 case Left(err) => System.err.println(err.getMessage)

Attributes

Companion: class
Source: Calibration.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Calibration.type

Attributes

Companion: class
Source: Camera.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Camera.type

High-level video capture — a camera or a video file, walked as owned Images.

Camera is the high-level counterpart to Video. Where Video.frames hands you one reused, borrowed Mat for zero-copy speed, Camera hands you a fresh owned Image per frame — one you can transform, detect on, annotate, or keep, on the same terms as any other Image. The price is one frame copy per iteration; when that matters, drop to Video.frames on the borrowed capture.

import scalacv.*
OpenCv.load()

// Process every frame of a file into an edge video. Reading `.mp4` is fine — the container restriction
// is the writer's: the default codec is MJPG, which opens only in an `.avi` (see [[Codec.Mjpg]]).
Camera.usingFile("clip.mp4") { cam =>
 cam.recordTo("edges.avi")(_.gray.canny(80, 160).convert(ColorConversion.GrayToBgr))
}

// Grab a single webcam snapshot:
Camera.using(0)(_.snapshot().flatMap(_.write("shot.png")))

The capture is caller-owned: close it, or acquire it through Camera.using / Camera.usingFile, which close for you. Camera is AutoCloseable.

Attributes

Companion: object
Source: Camera.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

The camera's motion between two frames: a 3×3 rotation and a translation direction, with the inlier count.

From a single camera the translation is only known up to scale (you cannot tell a small nearby motion from a large distant one), so translation is a unit direction, not metres. Fuse it with wheel odometry, IMU, or a known baseline to recover scale.

Attributes

Source: VisualOdometry.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A camera's absolute pose: the 3×3 rotation and 3-vector translation that map world points into the camera frame (x_cam = R·x_world + t).

Attributes

Source: Localizer.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Which videoio backend to ask for.

Any is the right answer almost always: OpenCV tries its registered backends in priority order and uses the first that can read the source. Naming one is for when that choice is wrong — forcing CaptureBackend.FFmpeg on a file that the image-sequence reader would otherwise claim, or forcing CaptureBackend.V4L2 on Linux so that a camera's native pixel format is honoured.

A backend that is not compiled into the OpenCV build on the classpath simply cannot open anything, so naming one turns a working open into a failing one. The bytedeco 4.13.0 builds do not all carry the same set — this is a portability decision, not a tuning knob.

Attributes

Source: Video.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

What the backend claims about an open capture.

Every field is a CAP_PROP_* query, and every one of them is advisory. A live camera usually reports frameCount == 0 (or -1) because the question is meaningless; some containers report a frameCount that is off by a frame or two from what actually decodes; fps can be 0 for a camera that has not delivered a frame yet. Use these to size a org.opencv.videoio.VideoWriter or to show progress — never as a loop bound. The frame count that is true is the one frames hands you.

Attributes

Source: Video.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

How a capture should be opened.

==Timeouts are best-effort, and off by default==

VideoCapture.read has no timeout overload and blocks in native code, so a stream that stops delivering hangs the calling thread with nothing scalacv can do about it from the JVM side. OpenCV's only lever is CAP_PROP_OPEN_TIMEOUT_MSEC / CAP_PROP_READ_TIMEOUT_MSEC, which is:

Backend-dependent. FFMPEG and GStreamer honour them for network sources. V4L2, AVFoundation and the built-in MJPEG reader ignore them entirely. Nothing in the API reports which you got.
Only settable at open time. VideoCapture.set on a not-yet-opened capture returns false (measured), so the values have to travel through the open(source, backend, params) overload.
Rejected outright by backends that do not understand them. Measured on this build: opening a local .avi with the timeout parameters attached yields isOpened == false, where the same file opens fine without them. Video therefore retries without the parameters rather than reporting a failure that is really "your backend has no timeout support".

They default to None because of the third point: paying a failed open, plus OpenCV's stderr noise, on every local file to configure something local files never need is the wrong default. Set them for network sources — RTSP, HTTP — where a hang is the failure mode you actually face.

==Why a camera needs warming up and a file does not==

A webcam is not ready the instant open returns. Auto-exposure, auto-white-balance and auto-gain are closed loops running on the device, and they need a handful of real frames to converge — which is why a naive open-then-snapshot so often yields a black or badly-under-exposed image and reports it as a success. There is no property to poll for "converged", so the only fix is to pull some frames and throw them away.

warmupFrames is how many to discard before the capture is handed back. It defaults to None, which means "let the source decide": Video.open(index, …) discards 5 and Video.open(source, …) discards 0. That split is the point — a file or an RTSP URL has no exposure loop, its first frame is exactly as correct as its hundredth, and discarding frames there would silently skip real content. Set it explicitly to override either default (Some(0) disables warm-up on a camera).

Value parameters

backend: which videoio backend to ask for; see CaptureBackend.
openTimeout: best-effort cap on how long opening the source may block.
readTimeout: best-effort cap on how long a single frame read may block.
warmupFrames: how many frames to grab and discard immediately after opening; None takes the per-source default described above.

Attributes

Companion: object
Source: Video.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Video.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: CaptureOptions.type

One of the Haar cascades shipped inside the bytedeco OpenCV classifier jar.

Typed rather than a raw filename because the failure mode of a typo is silent: CascadeClassifier does not throw for a path it cannot read, it constructs an empty classifier that then detects nothing, forever. A name that cannot be misspelled removes the most common way to reach that state.

The lbpcascades and the two frontalcatface files are deliberately omitted: LBP is a different model family with different tuning, and the cat detectors are a novelty. Both are still reachable through Cascades.loadFrom.

Attributes

Source: Cascades.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Haar cascade resolution and loading.

Two things here are load-bearing, and both are about failing loudly.

CascadeClassifier never reports a bad path. new CascadeClassifier("/nope.xml") succeeds, prints nothing useful, and hands back an object whose empty() is true. Every subsequent detectMultiScale then returns zero rectangles, which is indistinguishable from "there was nothing in the frame". Cascades.load and loadFrom check empty() and return a Left instead.
The cascades are a classpath resource, not a file on disk. They live in the per-platform classifier jar under share/opencv4/haarcascades/, so they have to be extracted before OpenCV — a C++ library that only knows filesystem paths — can read them. Loader.cacheResource does that and caches it, and notably needs no native load: resolve works before OpenCv.load().

windows-x86_64 is the exception that has to be handled by name: its jar ships an empty share/ directory and no cascades at all, so resolve can only fail there. It says so in those words rather than surfacing a null or an opaque IO error.

Attributes

Source: Cascades.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Cascades.type

Minimal data visualisation, built from Picture — proof that the graphics layer composes into charts, and a handy way to overlay a plot on an image (a histogram beside a detection, a signal on a frame).

Each returns a Picture sized to a width×height box with its origin at the top-left, so it composes and transforms like any other picture: Chart.bars(counts, 200, 80).at(Point(10, 10)) drops a chart into a corner.

Attributes

Source: Chart.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Chart.type

A planar chessboard calibration target, described by its inner-corner grid.

The number that matters to OpenCV is the count of corners between squares, not the number of squares: a board of 10×7 squares has a 9×6 inner grid, so that is what you pass here. squareSize is the real edge length of one square, in whatever unit you want the calibration expressed in — millimetres and metres are both common. It only scales the translation part of the recovered geometry; the intrinsics (fx, fy, cx, cy) are in pixels regardless.

Value parameters

columns: inner corners across (squares-across − 1).
rows: inner corners down (squares-down − 1).
squareSize: the physical side length of one square.

Attributes

Source: Calibration.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A video container/codec, as a FOURCC.

The four-character code is packed in pure Scala — the same bit layout as OpenCV's CV_FOURCC — so naming a codec needs no native call and the enum can be referenced before OpenCv.load(). Whether a codec actually works still depends on what the platform's videoio build links (FFmpeg, the OS frameworks); an unavailable one surfaces as a Left from Recorder.open, never a silent black file.

Attributes

Companion: object
Source: Camera.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: enum
Source: Camera.scala
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: Codec.type

Attributes

Companion: class
Source: Color.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Color.type

A colour with an alpha channel — the palette for Picture graphics.

Doodle-inspired: colours are values you build and transform (.lighten, .fadeOut, .blend) rather than raw BGR triples. Stored as RGBA in [0, 255]; the renderer converts to OpenCV's BGR at the boundary, and alpha drives per-shape transparency when a picture is drawn over an image.

Attributes

Companion: object
Source: Color.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Colour space conversions. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A false-colour map — turns a single-channel image (a depth map, a motion field, any data) into a colour heatmap. The perceptually-uniform ones (Colormap.Viridis, Magma, Inferno, Plasma, Turbo) are the honest choice for data; Jet is the classic-but-misleading rainbow.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

One connected outline, copied out of native memory.

OpenCV hands contours back as a java.util.List[MatOfPoint] — a list of live native handles the caller is expected to free individually. That is the single most reliable leak in the OpenCV Java API: nothing in the signature says the list owns anything, and the objects survive every reasonable-looking use of the result. So findContours copies the points across the boundary and frees the natives before it returns, and this type is ordinary immutable Scala data with no pointer behind it. It stays valid after the source Mat is released.

The measurements (Point carries Double) come back whole: findContours produces CV_32SC2, so every coordinate is an integer that happens to be widened.

Attributes

Companion: object
Source: Contours.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Contours.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Contour.type

How findContours compresses each contour. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Which contours findContours reports. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

The error policy.

The core cannot be "total". org.opencv.core.CvException escapes from ordinary in-memory operations, including on the empty Mat that a failed imread hands back, and no wrapper can prevent that. So scalacv draws the line deliberately:

Either[CvError, A] where failure is data-dependent and expected — a file that is not there, bytes that do not decode, a model that will not load.
Thrown IllegalArgumentException for precondition violations, which are programmer errors and should not be pattern-matched.
Propagated CvError.NativeCall for everything OpenCV throws at us that we did not anticipate. Wrapped so the operation is named, never swallowed.

Attributes

Source: Cv.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Cv.type

Attributes

Companion: class
Source: CvError.scala
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: CvError.type

Everything scalacv can fail with.

Modelled as an exception hierarchy rather than a pure ADT because it has to interoperate with a JNI boundary that throws: org.opencv.core.CvException escapes from ordinary Imgproc calls, and no wrapper can make the core total. The API returns Either[CvError, A] where failure is data-dependent and expected — a missing file, an undecodable image — and throws for programmer errors.

Attributes

Companion: object
Source: CvError.scala
Supertypes: class RuntimeException

class Exception

class Throwable

trait Serializable

class Object

trait Matchable

class Any
Show all
Known subtypes: class CalibrationFailed

class DecodeFailed

class EncodeFailed

class LoadFailed

class NativeCall

class NativesMissing
Show all

A stroke dash pattern — alternating on/off run lengths in pixels. OpenCV has no dashed line, so Picture draws them by hand from this.

Attributes

Companion: object
Source: Graphics.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Graphics.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Dash.type

ORB keypoints and their binary descriptors for one image. Owns a native descriptor Mat, so it is caller-owned — close it (or use Using).

Attributes

Source: Features.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Deep-network inference over org.opencv.dnn, narrowed to the ONNX path.

Only ONNX is exposed. OpenCV can also read Caffe, Darknet, TensorFlow, TFLite and Torch graphs, but every one of those importers has its own set of unsupported-layer failure modes, and offering seven entry points would imply a level of support this library cannot honestly give. ONNX is the format the other frameworks export to, so one importer covers the realistic cases.

==Ownership==

Net is one of the 185 generated types with no public release(), so it is freed through the delete(long) bridge — see Releasable.handle. Both the Net from fromOnnx and the Mats from blobFromImage and forward are caller-owned: release them, or take them with Managed.use. Nothing here takes ownership of an argument.

==Statefulness==

A Net is not a pure function. setInput mutates it and forward reads that mutation back, so a single Net cannot be driven from two threads concurrently — forward does both in one call precisely so that the window between them is not something a caller can accidentally widen, but it is still not a lock. One Net per thread, or serialise access yourself.

That statefulness reaches the results too, and this is where OpenCV would otherwise hand back a nasty surprise: Net.forward returns a header onto the layer's own output buffer, which the next pass overwrites. forward copies before it returns, so holding two outputs from one Net — this frame's heatmap against the previous frame's, or frames.map(f => Dnn.forward(net, blob(f))) — means what it looks like it means.

Attributes

Source: Dnn.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Dnn.type

Text measurement — the one part of drawing that answers a question instead of changing an image.

Attributes

Source: Draw.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Draw.type

One face reported by FaceDetect.detect.

Plain immutable Scala data, copied out of OpenCV's result Mat, so it stays valid after every native object involved has been freed — see Geometry for why that copy is the right trade.

Value parameters

box: the face's bounding box. It is not clipped to the image: YuNet regresses boxes from anchors, so a face at the edge of the frame legitimately yields a negative x/y or a box running past cols/rows. Image.crop and Mat.submat both reject such a rectangle outright, so clip it with clippedBox before cropping — that is the intersection with the frame, and it answers None for a box that lies entirely outside it.
landmarks: exactly five points, always in this order: right eye, left eye, nose tip, right mouth corner, left mouth corner. "Right" is the subject's right, so it appears on the left of the image.
score: the model's confidence in [0, 1]. Only faces at or above the detector's scoreThreshold are reported.

Attributes

Source: FaceDetect.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

YuNet face detection over org.opencv.objdetect.FaceDetectorYN.

A small CNN (232 kB) that is both far more accurate and far faster than the Haar cascades in Cascades, and unlike them it returns five facial landmarks per face. It is the detector to reach for; the cascades remain for heritage and for environments where no model file can be shipped.

Four things about FaceDetectorYN are easy to get wrong, and each is handled here rather than left to the caller. All four were verified against the 4.13.0 bindings and the running library, not read off a blog:

The input size is fixed at construction and enforced at detect time. detect runs CV_CheckEQ(input_image.size(), input_size) and throws a CvException if a later frame differs by a single pixel — which is exactly what happens the first time you feed it a resized frame, or a webcam that renegotiated its resolution. detect therefore calls setInputSize for every frame, so any Mat works. The cost is that a detector is stateful and not safe to share across threads; give each thread its own.
detect returns an int status flag, not a face count. It is 1 when the network ran and 0 when the input was empty. Reading it as a count silently reports one face for every successful call. The count is faces.rows().
No faces means a 0x0 Mat, not an Nx15 Mat with zero rows. Any decode loop that trusts cols() without checking empty() first will read column 14 of a Mat that has no columns.
A detection row has 15 columns, all CV_32F: x, y, w, h, then five (x, y) landmark pairs, then the score. detect fails loudly if a future model emits a different width rather than decoding garbage.

The model itself is not shipped with scalacv — see downloadModel.

Attributes

Source: FaceDetect.scala
Supertypes: class Object

trait Matchable

class Any
Self type: FaceDetect.type

A face's identity as a fixed-length embedding — a 128-dimensional vector produced by FaceRecognizer.

Two faces are compared by the angle between their embeddings (cosineSimilarity, higher is more alike) or their l2Distance (lower is more alike). The vector is plain immutable data, so it outlives every native object and is cheap to store in a Gallery or a database.

Attributes

Source: FaceRecognizer.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A named best match from a Gallery: who it is and how strong the cosine similarity was.

Attributes

Source: FaceRecognizer.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Face recognition via org.opencv.objdetect.FaceRecognizerSF (SFace) — turns an aligned face into an embedding you can compare or look up in a Gallery.

The model is yours to supply, exactly as with the YuNet detector: download the SFace ONNX (face_recognition_sface_2021dec.onnx, ~37 MB, from the OpenCV Zoo) and hand FaceRecognizer.load its path. Recognition builds on detection: embed takes a Face (from image.faces) and the image it came from, aligns and crops the face using its five landmarks, then extracts the embedding.

for recognizer
   val enrolled = Gallery.empty.enroll("ada", rec.embed(refImage, refFace))
   enrolled.identify(rec.embed(frame, face)) match
     case Some(FaceMatch(name, s)) => println(f"$name ($s%.2f)")
     case None                     => println("stranger")

Owns a native recognizer — caller-owned, close it (or use Using).

Attributes

Companion: object
Source: FaceRecognizer.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: class
Source: FaceRecognizer.scala
Supertypes: class Object

trait Matchable

class Any
Self type: FaceRecognizer.type

One descriptor-to-descriptor correspondence: the two keypoint indices and how far apart the descriptors are (Hamming distance for ORB — smaller is better).

Attributes

Source: Features.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

ORB feature detection and matching — the recognition front end for localization and loop closure.

Detect repeatable keypoints and their binary descriptors in each frame (detect), then match them across frames (matches) to find what the camera is looking at again. With OpticalFlow and VisualOdometry this is the visual front end of a SLAM/localization pipeline; the map and the global optimisation that make it SLAM live in a back end beyond OpenCV — see the navigation guide.

Attributes

Source: Features.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Features.type

A named, composable photo filter — an Image => Image transform you can name, chain, and apply.

The catalog in the companion is a set of ready-made "looks" built from the Image tone, colour, and stylisation operations; each is a Filter you apply with image.filter(Filter.vintage) or compose with andThen. Because a filter is just a named transform, your own are first-class: Filter("mine")(_.gamma(1.2).saturate(1.3)).

Like every Image transform, applying a filter consumes the image and returns a new one.

Attributes

Companion: object
Source: Filter.scala
Supertypes: class Object

trait Matchable

class Any

Attributes

Companion: class
Source: Filter.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Filter.type

How to mirror an image. Named by the visible effect, not OpenCV's axis-centric flip code.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Hershey fonts for putText. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: FaceRecognizer.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Gallery.type

An immutable set of enrolled faces — a "who is this?" lookup. Enroll named embeddings, then identify a fresh one against them; the highest-scoring entry above the threshold wins, or None for a stranger.

Immutable and value-like: enroll returns a new gallery, so a gallery is safe to share and snapshot.

Attributes

Companion: object
Source: FaceRecognizer.scala
Supertypes: class Object

trait Matchable

class Any

Static hand-gesture recognition from a PoseTopology.Hand21 hand pose.

This is the rule-based layer on top of hand-landmark estimation: given the 21 landmarks (from a hand model run through Dnn and decoded with PoseEstimator), it decides which fingers are extended and names the gesture. No model of its own — pure geometry — so it is deterministic and testable.

It is also the entry point for sign language: static fingerspelling shapes are exactly this (extend the ruleset, or classify the landmark vector with your own model); dynamic signs add a temporal classifier over a sequence of these poses — see the docs.

val gesture = GestureRecognizer.recognize(handPose) // handPose: Pose over PoseTopology.Hand21

Attributes

Source: Gesture.scala
Supertypes: class Object

trait Matchable

class Any
Self type: GestureRecognizer.type

The navigator's read of the scene: the chosen Steering, how clear the path ahead is (0 blocked … 1 wide open), and the raw near-ness of each third.

Attributes

Source: Navigator.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A recognised static hand gesture.

Attributes

Source: Gesture.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Head-pose estimation from a detected Face's five landmarks, via solvePnP against a canonical 3D face model. No extra model file — it reuses what FaceDetect already gives you.

The 3D reference is an approximate generic head, so the angles are indicative rather than metric: good for "looking left / up / tilted", not for a calibrated measurement. For that, a dedicated head-pose network (run through Dnn) and a calibrated camera matrix are the way.

Attributes

Companion: class
Source: Pose.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: HeadPose.type

Head orientation in degrees — the classic yaw / pitch / roll.

Attributes

Companion: object
Source: Pose.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

The high-level, fluent face of scalacv — an owned image you transform by chaining.

Image is the layer to reach for first. It wraps a single native Mat and lets you express the common OpenCV shape — read, transform, detect, annotate, write — as one readable chain:

import scalacv.*
OpenCv.load()

for _ &lt;- Image.read("photo.jpg").flatMap(_.gray.blur(2).canny(80, 160).write("edges.png"))
yield ()

==Move semantics: a transform consumes the image==

Every transform (gray, blur, canny, resize, crop, a draw*) returns a new Image and spends the one it was called on — using the old handle afterwards throws IllegalStateException rather than reading freed memory. That is what makes the chain leak-free without a scope: each step frees (or hands on) the previous Mat, so a long pipeline holds exactly one live Mat at a time, never a pile of intermediates. It is Mats.chain's guarantee, surfaced as a type.

The trade is that you cannot use one Image twice. To branch, take a copy first, or drop to the mid-level API on a borrowed mat.

==Queries borrow, terminals consume==

A query (width, faces, qrCodes, contours) only reads, so it leaves the image alive. A terminal (write, bytes, close) consumes it and releases the Mat. If a value escapes the chain without ever reaching a terminal it leaks, exactly as a stray Managed would — so prefer Image.reading, which closes for you even when the body already consumed the image (release is idempotent).

==Not a wall==

Image never hides the library underneath it. mat borrows the raw org.opencv.core.Mat for any org.opencv.* call this type does not wrap; managed hands the whole Managed over. The high-level API is the pleasant default, not a ceiling. Domain verbs that only happen to start from an image — face and marker detection, pose and track overlays, OCR preparation, background replacement — are extension methods brought in with import scalacv.*, not members of this class, which is why they are absent below; they read image.faces(detector) all the same.

==Failures==

A transform does not return an Either: when OpenCV itself rejects the pixels (a data-dependent failure this library cannot foresee), the op throws CvError.NativeCall, naming the operation — an unchecked throw, so it is invisible at the call site. Argument mistakes this library can see are rejected up front with IllegalArgumentException. Only the Either-returning boundary methods (Image.read, write, bytes, Image.decode) turn failure into a value. To fold a transform's throw into an Either too, wrap it with Cv.attempt. Reusing an already-consumed image throws IllegalStateException; see Managed for -Dscalacv.trackOwnership=true, which points the error at the consuming call.

Image is AutoCloseable, so scala.util.Using manages it too.

Attributes

Companion: object
Source: Image.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: class
Source: Image.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Image.type

Reading, writing, encoding and decoding images — the boundary between OpenCV and everything else.

This is the one place in the library where OpenCV's error reporting is genuinely inconsistent, and the whole point of the object is to flatten that into a single Either. Two distinct failure shapes come out of org.opencv.imgcodecs.Imgcodecs:

imdecode never throws for bytes that are not an image. It returns a Mat with empty() == true (and logs a findDecoder warning to stderr). Anyone who forgets the empty() check gets a CvException several call frames later, from an Imgproc operation that had nothing to do with the mistake.
imencode throws CvException when the extension names no known encoder — so haveImageWriter is consulted first and that case is returned as CvError.EncodeFailed before the throw can happen, rather than being recovered from OpenCV's error text.

Every function here returns Either[CvError, ?] covering both. The encode failures share the one CvError.EncodeFailed type, so case EncodeFailed(...) catches them all.

==The file I/O is done by the JVM, not by OpenCV== read and write do not call imread/imwrite. They open the file with java.nio.file and leave OpenCV only the codec work, through decode and encode. The reason is the path. The JNI layer narrows a Java String with GetStringUTFChars, so the native side receives modified UTF-8 bytes, and OpenCV's imgcodecs hands those bytes straight to fopen. On Windows the C runtime interprets them in the process's ANSI code page, so any non-ASCII character in the path resolves to a different, nonexistent name: imread then returns an empty Mat and imwrite returns false — indistinguishable from "the file is not there" and "the directory is not writable". Upstream has not fixed this (opencv#4292 is still open in 4.13) and exposes no wide-character entry point to call instead. Reading and writing the bytes on the JVM side avoids the narrowing altogether, and as a side effect lets both functions report which of the causes they used to lump together actually happened.

The price is that the encoded file passes through a JVM byte array: peak heap grows by its size, and a file above 2 GB is out of reach, because Files.readAllBytes cannot return an array that long. Routing around that with a memory-mapped buffer was rejected — imdecode needs a MatOfByte built from a JVM array anyway, so the copy is not avoidable here.

The same narrowing hazard applies to every other String-path native call in the library — VideoCapture, VideoWriter, CascadeClassifier.load, Dnn.readNet — and none of those has an in-memory equivalent to reroute through, so they remain ASCII-path-only on Windows.

==Ownership== A returned Managed[Mat] is caller-owned: nothing else holds a reference and nothing else will free it. Prefer Images.read(p).map(_.use(...)) over holding one. Mats created internally — the MatOfByte staging buffers, and the empty Mat a failed read hands back — are released here.

Attributes

Source: Images.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Images.type

How imread/imdecode should decode a pixel's colour.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Enums.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: ImreadFlags.type

Image reading flags — a total model, not a bitmask.

OpenCV's IMREAD_* constants look like OR-able bits but are not: each IMREAD_REDUCED_* value already bakes in its colour bit, and IMREAD_UNCHANGED is -1, whose bits swamp everything else. OR-ing a colour with a reduced-size flag therefore silently decodes the wrong image. So the (colour, scale) pair maps totally onto exactly one named constant instead of composing, and only ignoreOrientation (bit 128) is a genuinely independent flag that may be OR-ed on top.

Reduced-size decode exists only for ImreadColor.Grayscale and ImreadColor.Color, and ImreadColor.Unchanged (-1) can carry no extra bit at all; the two requires reject the combinations OpenCV has no constant for, rather than quietly OR-ing them into something else.

Attributes

Companion: object
Source: Enums.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

The fraction of full resolution to decode at. OpenCV's reduced-size decode is cheaper than a full read followed by a resize, because the codec skips the discarded detail rather than producing it first.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Interpolation for resize and warps. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A pinhole camera's intrinsics — what turns a pixel measurement into a metric one.

fx/fy are the focal length in pixels, cx/cy the principal point (usually near the image centre). distortion is OpenCV's radial/tangential coefficients (k1, k2, p1, p2[, k3 …]); leave it empty for an ideal lens. A real camera's numbers come from a chessboard calibration; when you have not calibrated, Intrinsics.approx gives a serviceable guess from the image size and a field-of-view estimate — good enough to see an augmented overlay track, not good enough to measure with.

This is the core camera model the vision layer builds on: Ar, HeadPose and Localizer all take an Intrinsics, Calibration produces one, and Image.undistort consumes one.

Attributes

Companion: object
Source: Intrinsics.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Intrinsics.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Intrinsics.type

Attributes

Companion: class
Source: Tracking.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Kalman.type

A constant-velocity Kalman filter over a 2D point — the smoother behind ObjectTracker, useful on its own to steady a jittery detection or to coast through a frame where the measurement dropped out.

The state is position and velocity (x, y, vx, vy); you predict the next position, then correct it with a fresh measurement (or skip the correction if you have none this frame and trust the model). Owns a native filter — caller-owned, close it. Stateful and not safe to share across threads; give each thread its own, as with the detectors.

Attributes

Companion: object
Source: Tracking.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

One named landmark of a Pose — a point in image pixels and the model's confidence in it.

Attributes

Source: Pose.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

How a pose network encodes its keypoints in the tensor forward returns — see PoseEstimator.

Attributes

Source: Pose.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Line rasterisation. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Absolute localization — where the camera is, given a map of known 3D points and their matches in the current frame.

This is the "I recognise these landmarks, so I must be here" step: match the frame's Features to a map, then hand the 3D↔2D correspondences here and solvePnP recovers the 6-DoF pose. Unlike VisualOdometry (relative, up-to-scale, drifts), this is absolute and metric — it is what stops a SLAM trajectory from drifting once a map exists.

Attributes

Source: Localizer.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Localizer.type

A detected loop closure: the earlier keyframe this frame revisits, how many features matched, and a score (matched features as a fraction of the current frame's).

Attributes

Source: LoopDetector.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Loop-closure detection — recognising a place the camera has already been.

This is the piece that turns drifting Odometry into something map-like: keep a keyframe's ORB Features as you go, and when a new frame matches an old keyframe strongly, you have closed a loop — the signal a SLAM back end uses to correct accumulated drift. (Doing that correction — re-optimising the pose graph — is the back end itself, beyond OpenCV; this detects the opportunity.)

It is appearance-based brute-force matching against every stored keyframe, which is fine for hundreds of keyframes; a city-scale system would swap in a bag-of-words index, but the contract would be the same.

Stateful and caller-owned — it holds a descriptor set per keyframe, so close it. Not thread-safe.

==Bounding memory==

Each keyframe owns a native ORB descriptor Mat, so an unbounded run accumulates native memory. Pass maxKeyframes to cap the number kept live: once exceeded, the oldest keyframes are evicted and their descriptors freed. Eviction leaves a tombstone in place of the evicted slot rather than renumbering the survivors, so a LoopClosure.keyframe index handed out earlier stays valid — it just refers to a slot that may since have been evicted (matching against it is skipped). The default is unbounded, preserving the original behaviour; a bounded detector trades old-place recall for a fixed memory ceiling.

Attributes

Companion: object
Source: LoopDetector.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: class
Source: LoopDetector.scala
Supertypes: class Object

trait Matchable

class Any
Self type: LoopDetector.type

Attributes

Companion: class
Source: Managed.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Managed.type

A native OpenCV object with a release that happens exactly once.

Two guarantees, both of which exist because getting them wrong is a JVM crash rather than an exception. Calling a method on a freed OpenCV object segfaults from native code — no stack trace, no catch, no test report; a double delete is undefined behaviour that merely often happens to survive. Measured, both.

Release is a compare-and-set, so a second release is a no-op rather than a double free.
Access after release throws IllegalStateException on the Scala side, before anything crosses JNI.

Prefer use over holding one of these. The scoped form is the only one where the compiler helps you.

==Diagnosing use-after-move==

The move semantics of Image mean the commonest mistake is reusing a handle a transform already consumed, and the resulting IllegalStateException fires at the reuse, which is rarely the interesting line. Start the JVM with -Dscalacv.trackOwnership=true and the exception carries, as its cause, the stack of the transform or terminal that actually spent the handle. It is off by default because it allocates a Throwable every time a handle is spent; the check that reads it lives only on the already-failing path, so a program that never misuses a handle pays nothing.

Attributes

Companion: object
Source: Managed.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

A detected marker together with the pose recovered for it — what image.arMarkers returns.

Attributes

Source: Ar.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Helpers that do not belong on a Mat.

Attributes

Source: Ops.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Mats.type

Attributes

Companion: class
Source: Models.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: ModelSpec.type

A downloadable model file: its fixed name, the mirror URLs to try in order, the SHA-256 the fetched bytes must match, and optionally the exact size they must have.

Integrity checking is the default: build a spec with ModelSpec.apply and its pinned hash is verified on every download and on every cache hit. Skipping the check is a deliberate, named opt-out — ModelSpec.unverified — that loses the tamper/corruption guard, so reach for it only for a model with no published checksum.

sizeBytes is not redundant with the hash; it changes the message, and only ever in the direction of being more useful. The failure it catches is the common one: a mirror that answers a model request with an HTML error page, or a Git LFS host that serves a 131-byte pointer file, both with HTTP 200. Those hash wrong, of course — but "SHA-256 mismatch" invites the reader to suspect tampering, whereas "expected 232589 bytes, got 131" says what actually happened. It is also checked first, so it costs one stat rather than a full digest of a file that was never the model.

Attributes

Companion: object
Source: Models.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A small registry and downloader for the model files scalacv's detectors need. It is the downloader: FaceDetect.downloadModel is a one-line alias for fetch(FaceDetect.modelSpec, into).

fetch downloads to a temp file beside the target and moves it into place only after it verifies, so an interrupted run never leaves a truncated model for the next load to trip over. It is idempotent: a target that already exists (and, if a hash or size is pinned, still matches) is returned without touching the network. URLs may be http(s):// or file://, so a model you already have on disk is just another source.

The detector model specs live next to their detectors (FaceDetect.modelSpec and FaceRecognizer.modelSpec); supply your own ModelSpec for anything else.

Attributes

Source: Models.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Models.type

Compound morphological operations (morphologyEx). Erosion and dilation have their own methods.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

The structuring-element shape for morphology. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Motion.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Motion.type

What one frame of motion detection found — plain immutable data, valid after the frame is freed.

Value parameters

moving: true when ratio crosses the detector's motionRatio threshold.
ratio: the fraction of the frame that changed, in [0, 1].
regions: bounding boxes of the moving blobs, largest first, already filtered by the detector's minArea.

Attributes

Companion: object
Source: Motion.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Detects motion across a sequence of frames from a (usually static) camera.

This is the piece for the classic surveillance / trail-cam / ESP32-CAM job: a fixed camera streaming frames — often low frame-rate MJPEG, i.e. a run of independent JPEGs — where you want to know when something moved and where, cheaply. Feed frames in order; the detector is stateful (it remembers the previous frame or an adaptive background) and not thread-safe.

import scalacv.*
OpenCv.load()

val detector = MotionDetector.frameDifference()
try
 for jpeg
     if motion.moving then
       println(s"motion in ${motion.regionCount} region(s), ${(motion.ratio * 100).round}% of frame")
finally detector.close()

Or drive it from a Camera — an ESP32 MJPEG endpoint opens like any other source:

Camera.usingFile("http://esp32-cam.local:81/stream") { cam =>
 cam.foreach(frame => if detector.detect(frame).moving then alert())
}

Two strategies, both reached through the factories:

MotionDetector.frameDifference — compares each frame to the one before. Cheap and immediate; the right default for a static camera and low frame rates.
MotionDetector.backgroundSubtraction — an adaptive background model (OpenCV's MOG2). Heavier and needs a few frames to settle, but shrugs off gradual lighting changes and repetitive background motion.

A detector holds native memory (a retained frame, or the background model), so it is AutoCloseable: close it when done.

Attributes

Companion: object
Source: Motion.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: trait
Source: Motion.scala
Supertypes: class Object

trait Matchable

class Any
Self type: MotionDetector.type

Reactive obstacle avoidance — turning a depth reading into a steering suggestion.

The simplest useful navigation primitive: split the view ahead into left / centre / right, measure how near the closest thing is in each (from a StereoDepth disparity map, brighter = nearer), and steer toward the clearest when something looms in the centre. It is memoryless and reflexive — a Braitenberg-style avoider, not a planner; a planner layers a map and a goal on top (see the navigation guide).

Attributes

Source: Navigator.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Navigator.type

One tracked object as reported by ObjectTracker.update: a stable id that persists across frames, the current box, and how long the track has lived.

Attributes

Source: Tracking.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Tracking.scala
Supertypes: class Object

trait Matchable

class Any
Self type: ObjectTracker.type

Tracking-by-detection: turns a per-frame stream of detections (from any detector — faces, motion boxes, a DNN) into tracks with stable identities. This is the "SORT-lite" pattern — the piece that lets you say "person #3" frame after frame, or count how many distinct objects have passed.

Each frame it ObjectTracker.updates: every live track is advanced by its own Kalman filter, detections are matched to tracks by bounding-box overlap (IoU, greedily best-first), matched tracks are corrected toward their detection, unmatched detections spawn new tracks, and tracks unseen for maxAge frames are retired. Stateful and caller-owned — close it to free the per-track filters.

It is detector-agnostic by design: it never looks at the image, only at the boxes, so it composes with whatever produced them.

Build one with ObjectTracker.create and close it when done, the same shape as Tracker.create and Kalman.point. Like those, it is stateful and not safe to share across threads — update mutates the live-track buffer — so keep the detect-and-track loop on one thread, or serialise access.

Attributes

Companion: object
Source: Tracking.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

One detected obstacle: where it is in the frame and how near it is (0 far … 1 right in front).

Attributes

Source: Depth.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Obstacle detection from a depth/disparity map.

Attributes

Source: Depth.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Obstacles.type

Attributes

Companion: class
Source: OccupancyGrid.scala
Supertypes: class Object

trait Matchable

class Any
Self type: OccupancyGrid.type

A 2D occupancy grid — a top-down map of free vs. occupied space, accumulated from range/obstacle observations over time.

This is the map Navigator's reflex lacks and a planner needs. Each cell holds a log-odds estimate that it is occupied: an obstacle reading nudges a cell toward occupied, seeing through empty space nudges the cells along the way toward free, and repeated evidence accumulates and clamps. World coordinates (in metres, say) are quantised to cells by resolution, with the grid centred on the origin.

Feed it from stereo/obstacle readings — a robot at a known pose turns each Obstacle into a ray via observe. It is a plain in-memory structure (no native memory); toImage renders it for viewing.

Attributes

Companion: object
Source: OccupancyGrid.scala
Supertypes: class Object

trait Matchable

class Any

High-level OCR: the OpenCV preprocessing, then an engine.

The recognition step is delegated to an OcrEngine you provide; everything up to it — the part that actually decides whether the text comes out clean — is scalacv's.

Attributes

Source: Ocr.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Ocr.type

A pluggable OCR engine.

scalacv owns the OpenCV half of OCR — the grayscale → denoise → threshold → deskew preprocessing that makes or breaks recognition (image.forOcr) — and this contract. The engine is yours to supply, because Tesseract (and the cloud OCRs) are heavy, separately-licensed native dependencies that do not belong in a thin OpenCV wrapper. Implementing it is a few lines over tess4j or bytedeco's tesseract preset — see the OCR guide.

val engine: OcrEngine = myTesseractEngine
val text = Image.read("scan.jpg").map(img => try Ocr.read(img, engine).text finally img.close())

Attributes

Source: Ocr.scala
Supertypes: class Object

trait Matchable

class Any

The text an OcrEngine read out of an image.

Attributes

Source: Ocr.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

One recognised word (or line/block, depending on the engine), with where it sits and how sure the engine is. Plain immutable data.

Attributes

Source: Ocr.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A running visual-odometry pipeline: feed frames in order, get the camera's motion each step.

This is the front-end loop of visual SLAM, composing the per-frame primitives: it keeps the previous frame and its tracked points, follows them into each new frame with OpticalFlow, and estimates the step's motion with VisualOdometry. Chaining the steps is dead-reckoning — it drifts, and correcting that drift (loop closure, global optimisation) is the SLAM back end, beyond OpenCV.

Monocular, so each step's translation is a unit direction (scale is unobservable from one camera). The pipeline retains a frame's worth of native memory between calls, so it is AutoCloseable — close it.

val odometry = Odometry.monocular(Intrinsics(fx = 500, fy = 500, cx = 320, cy = 240))
try camera.foreach(frame => odometry.update(frame).foreach(step => track(step)))
finally odometry.close()

Not thread-safe: feed one frame at a time.

Attributes

Companion: object
Source: Odometry.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: class
Source: Odometry.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Odometry.type

Loads the OpenCV native libraries, without ever requiring a GUI toolkit.

The obvious approach — Loader.load(classOf[opencv_java]) — does not work on a headless machine. javacpp eagerly initialises the whole preset graph, and opencv_highgui is GTK2-linked on Linux, so on a box without GTK it throws and takes objdetect, calib3d, features2d and video down with it. objdetect is precisely what this library needs most.

libopencv_java itself links no GUI toolkit. So we bring javacpp up through a GUI-free preset, extract the platform payload, and then load the JNI shim, resolving its dependencies on demand — see satisfy for why loading them speculatively is not merely wasteful but unsafe. The result needs no apt-get install libgtk2.0-0t64 on any runner.

Attributes

Source: OpenCv.scala
Supertypes: class Object

trait Matchable

class Any
Self type: OpenCv.type

Sparse optical flow — following points from one frame to the next.

The tracking primitive under visual odometry and visual navigation: seed some good-to-track corners, follow them frame to frame with pyramidal Lucas–Kanade, and read motion off the survivors. Combined with Features and VisualOdometry it is the front end of a visual-SLAM pipeline (the back end — mapping, loop closure, bundle adjustment — is beyond OpenCV; see the navigation guide).

Attributes

Source: OpticalFlow.scala
Supertypes: class Object

trait Matchable

class Any
Self type: OpticalFlow.type

The destination depth for the operators that can change it — the derivative operators, and normalize.

Worth a type of its own rather than a bare int because OutputDepth.SameAsSource is a trap on the commonest input: Sobel on an 8-bit unsigned image with ddepth = -1 clips every negative derivative to zero, so half of each edge silently disappears. Signed16 then convertScaleAbs is the standard fix.

Attributes

Source: Ops.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: trait
Source: Graphics.scala
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: Picture.type

An immutable, composable 2D drawing — the graphics layer, inspired by Doodle and adapted to image space.

A Picture is a value: build primitives, style them, compose them, and only then render onto an image. Because it composes, the same vocabulary annotates a detection, draws a chart, or makes generative art.

import scalacv.*
OpenCv.load()

// A dashed green box with a label — an overlay for a detected face:
val overlay =
 Picture.rectangle(face.box).strokeColor(Color.Green).strokeWidth(2).dashed
   .on(Picture.text("face", Point(face.box.x, face.box.y - 6)).strokeColor(Color.Green))

image.draw(overlay) // draw it on

Coordinates are image pixels (origin top-left, y down). Styling is contextual: a style set on a group is the default its members inherit unless they set their own. Alpha in a Color gives real transparency when drawn over an image.

Attributes

Companion: object
Source: Graphics.scala
Supertypes: class Object

trait Matchable

class Any

Attributes

Companion: class
Source: Geometry.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Point.type

A 2D point in pixel coordinates, origin top-left, x right and y down — the sub-pixel form OpenCV uses for feature and contour work, so both fields are Double.

Attributes

Companion: object
Source: Geometry.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Geometry.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Point3.type

A point in 3D space — a model coordinate for Ar pose work, in the same units you give a marker's side length (metres is the usual choice). z points out of the marker plane toward the camera.

Attributes

Companion: object
Source: Geometry.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A line in Hesse normal form, as HoughLines reports it: infinite, with no endpoints.

Value parameters

rho: distance in pixels from the image origin (top-left) to the line, along the normal.
theta: angle of that normal in radians. 0 is a vertical line, Pi/2 a horizontal one — the angle describes the normal, not the line, which is the usual source of confusion.

Attributes

Source: Hough.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A PolarLine plus its accumulator score.

HoughLinesWithAccumulator exists precisely so the votes are visible; they are the only way to rank results, since the plain transform already returns them sorted but discards the magnitudes. Votes are whole numbers stored in a float channel, hence the narrowing.

Attributes

Source: Hough.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A detected pose — an ordered set of Keypoints for a known PoseTopology, as plain immutable data that stays valid after the frame and the network output are freed.

Attributes

Source: Pose.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A rigid pose: where a marker (or any known object) sits relative to the camera.

rvec is the rotation in OpenCV's Rodrigues (axis-angle) form and tvec the translation, both in the marker's units. You rarely read these directly — hand the pose back to Ar.project to draw with it — but distance (the length of tvec) is the camera-to-marker distance and is often all you want.

Attributes

Source: Ar.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Human-pose (skeleton) estimation over a keypoint network run through Dnn.

MediaPipe's models ship as TFLite; OpenCV's inference path — and therefore scalacv's — is ONNX, so this is built the way FaceDetect and Dnn are: you bring the model (Dnn.fromOnnx), and scalacv provides the typed result and the decode. The two common output layouts are both handled (PoseEstimator.decode), so a MoveNet or an OpenPose export drops in by naming its KeypointLayout and PoseTopology.

// With a caller-loaded Net (see Dnn):
val pose = Dnn.blobFromImage(image.mat, size = Some(Size(192, 192)), swapRB = true).use { blob =>
 Dnn.forward(net, blob).use { out =>
   PoseEstimator.decode(out.mat, image.size, KeypointLayout.Regression)
 }
}

For hand and head pose see PoseTopology.Hand21 and HeadPose.

Attributes

Source: Pose.scala
Supertypes: class Object

trait Matchable

class Any
Self type: PoseEstimator.type

A keypoint naming and connectivity scheme — the "which landmark is which, and which bones connect them" that a pose model implies but does not carry.

Value parameters

edges: index pairs that form the skeleton's bones (for drawing).
names: the keypoint names, in the model's output order.

Attributes

Companion: object
Source: Pose.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Pose.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: PoseTopology.type

QR code detection over org.opencv.objdetect.QRCodeDetector.

Both this and Aruco report their geometry through Mats that the caller would otherwise own, and neither QRCodeDetector nor ArucoDetector has a public release() — they are two of the 185 types that only expose a private delete(long). So each detector is created, used and freed inside a single call here, and the results cross the boundary as ordinary immutable Scala data (Point case classes) rather than as live native handles. That is what makes these signatures return Seq instead of Managed[…]: there is nothing left to own.

Attributes

Source: Detectors.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Qr.type

One decoded QR code.

text is empty when OpenCV located a symbol but could not decode it — a common outcome for a blurred or partially occluded code, and deliberately not filtered out, because the corners are still useful (to draw an overlay, or to re-crop and retry).

Attributes

Source: Detectors.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Writes Images to a video file — the counterpart to Camera for output.

A recorder is fixed at open time to one frame size, fps and codec; every frame written must match that size and be 8-bit. VideoWriter is one of the three OpenCV types with a real public release(), and the recorder is caller-owned — close it, or use Recorder.using.

Attributes

Companion: object
Source: Camera.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: class
Source: Camera.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Recorder.type

An axis-aligned integer rectangle: top-left corner (x, y) and non-negative width/height. The origin may be negative (a region of interest can extend past the top-left of the image); the extent may not.

Attributes

Companion: object
Source: Geometry.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Geometry.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Rect.type

How a native OpenCV object gets freed.

There are two regimes, and which one applies is not a style choice — it is dictated by what the generated Java binding exposes. Of the 188 org.opencv.* types that hold a native pointer, exactly three have a public release(): Mat, VideoCapture and VideoWriter. The other 185 — including every detector this library wraps — expose only a private static native void delete(long) plus a finalize().

Relying on that finalize() is not viable. It is not disabled (a common myth), but it only runs when the collector runs, and the collector sees ~40 bytes of Java header per multi- megabyte native buffer. Measured: 2000 unreleased 1000x1000 Mats reach 5.8 GB RSS against 144 MB when released.

Attributes

Companion: object
Source: Releasable.scala
Supertypes: class Object

trait Matchable

class Any

Attributes

Companion: trait
Source: Releasable.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Releasable.type

Lossless quarter-turn rotations — no interpolation, exact pixels. A true enumeration.

Attributes

Source: Enums.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A pixel value: up to four channel components, in whatever channel order the Mat uses. OpenCV's default is BGR, not RGB, so Scalar.Red is Scalar(0, 0, 255). Unset channels default to 0.

Attributes

Companion: object
Source: Geometry.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Geometry.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Scalar.type

Screen (and screenshot) analysis: finding a known sub-image, and spotting what changed between two captures.

The staple of screen automation and visual testing — "is this button on screen, and where?", "what changed since the last frame?". It is ordinary template matching and differencing, no model involved.

import scalacv.*
OpenCv.load()

for
 screen   &lt;- Image.read("screenshot.png")
 template &lt;- Image.read("button.png")
yield
 try Screen.locate(screen, template) // Option[TemplateMatch]
 finally { screen.close(); template.close() }

Attributes

Source: Screen.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Screen.type

A line segment with real endpoints, as HoughLinesP reports it.

Integer, because the underlying Mat is CV_32SC4. Rounding here would invent precision OpenCV never produced.

Attributes

Source: Hough.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Turns a selfie-segmentation network's output into a person mask.

The model is yours (run it through Dnn — a MODNet, U²-Net, or MediaPipe-selfie ONNX export); this decodes its output tensor into the CV_8UC1 mask that BackgroundEffect wants: white over the person, black over the background. The decode is testable on a synthetic tensor, so no download is needed to prove it.

Attributes

Source: BackgroundEffect.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Segmenter.type

A width/height extent in pixels. Neither side may be negative — a zero extent is allowed (an empty size), a negative one throws.

Attributes

Companion: object
Source: Geometry.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Geometry.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Size.type

A suggested steering action from what is ahead.

Attributes

Source: Navigator.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Depth from a rectified stereo pair — the basis of obstacle detection on a robot or drone.

A disparity map encodes how far each pixel shifts between the left and right cameras, which is inverse to distance: nearer things shift more. Obstacles.fromDisparity then reads the near-field blobs off it.

The pair must already be rectified (row-aligned) — that is a one-time camera-calibration step OpenCV also provides (stereoRectify), done off the hot path, so it is not wrapped here.

Attributes

Source: Depth.scala
Supertypes: class Object

trait Matchable

class Any
Self type: StereoDepth.type

One template-match hit — where a template was found and how well it matched.

Attributes

Source: Screen.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

What a string will occupy once drawn, from Imgproc.getTextSize.

Value parameters

baseline: how far the descenders reach below the baseline, in pixels. It is returned separately because drawText's anchor is the baseline's left end, not the top-left corner: a background box has to be size.height + baseline tall to enclose the text, and forgetting it clips every g and y.
size: the bounding box of the glyphs.

Attributes

Source: Draw.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

How wide a stroke is, or that a shape is filled instead.

OpenCV encodes "filled" as a thickness of -1, a sentinel that ordinary arithmetic on a thickness will happily produce by accident. Worse, it is only meaningful for closed shapes: cv::line asserts 0 < thickness, so passing the sentinel to a line or to text aborts in native code. Splitting the two cases into distinct types lets the shapes that can be filled accept Thickness while lines and text accept Thickness.Stroke only — the mistake stops compiling rather than crashing.

Attributes

Companion: object
Source: Draw.scala
Supertypes: class Object

trait Matchable

class Any
Known subtypes: object Filled

class Stroke

Attributes

Companion: trait
Source: Draw.scala
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: Thickness.type

Thresholding — a bitmask, not an enumeration.

Imgproc.threshold takes a mode OR-ed with at most one automatic-threshold modifier. Modelled as enum the useful combinations would be unrepresentable, and THRESH_MASK would leak into a public API where it means nothing.

Attributes

Companion: object
Source: Enums.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Source: Enums.scala
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Threshold.type

What threshold actually returns.

Imgproc.threshold returns a double that most wrappers discard. For Otsu and Triangle it is the threshold OpenCV chose — frequently the reason you called it at all.

Attributes

Source: Enums.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

One tracked point across two frames: where it started, where it ended up, and whether the tracker kept hold of it.

Attributes

Source: OpticalFlow.scala
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

A single-object tracker: told where an object is in one frame (init), it finds it in the next (update) without re-detecting. This is model-free tracking — it learns the object's appearance from the box you give it, so it works on anything, not just a class a detector knows.

Owns a native tracker, so it is caller-owned — close it (or use Using). A tracker is stateful and single-object; for many objects that come and go, use ObjectTracker instead.

Using.resource(Tracker.create(TrackerKind.Csrt)): tracker =>
 tracker.init(firstFrame, box)
 for frame  frame.drawRect(b).write(...))

Attributes

Companion: object
Source: Tracking.scala
Supertypes: trait AutoCloseable

class Object

trait Matchable

class Any

Attributes

Companion: class
Source: Tracking.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Tracker.type

Which single-object tracking algorithm to run. All three ship in this OpenCV build.

TrackerKind.Csrt — the accuracy pick: discriminative correlation filter with channel/spatial reliability. Slower, but it handles scale change and partial occlusion well.
Kcf — the speed pick: kernelised correlation filter. Fast and steady, but it does not adapt its box to scale.
Mil — multiple-instance learning. Robust to small appearance changes; no failure detection.

Attributes

Source: Tracking.scala
Supertypes: trait Enum

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Video capture: opening a source, and walking its frames without leaking one per iteration.

==Why there is no frame LazyList==

The obvious shape for "the frames of a video" is a lazy sequence, and it is wrong here. LazyList memoises: once evaluated, a cell holds its head forever so that a second traversal is cheap. Applied to frames, that means every Mat the list has ever produced stays reachable — so either nothing is ever released (an unbounded native leak; a 1080p BGR frame is ~6 MB, so a minute at 30 fps is over 10 GB) or frames are released as they are consumed and the list is a field of dangling handles that the next traversal hands back as empty Mats. There is no version of the API where memoisation and per-frame release are both correct. Same argument, verbatim, for Stream, and for any Iterator combinator that retains what it has seen.

So the frame source here is an Iterator that owns exactly one Mat and decodes into it in place. It is created inside a scope, it is released when that scope ends, and it holds one frame's worth of native memory no matter how long the video is.

==The borrowing contract==

This is the one place in scalacv where a Mat you are handed is not yours, and it is the exact opposite of the contract in Ops.scala:

The Mat from frames is borrowed. It is valid from the next() that returned it until you next ask the iterator for anything, and it is released when the frames block returns. The iterator is retired at that point, so keeping one is inert rather than dangerous.
Do not retain it. Do not put it in a collection. it.toList compiles and gives you N references to one Mat holding the last frame — not N frames.
Do read it, and do run the Ops extensions over it: those allocate their own destination and never alias the receiver, so frame.cvtColor(...) inside the loop is correct and yields a Mat you own.
Need to keep a frame? framesCopied, which clones per frame and hands you a caller-owned Managed.

Video.open("clip.mp4").map { capture =>
 capture.use { c =>
   Video.frames(c) { frames =>
     frames.map(_.cvtColor(ColorConversion.BgrToGray).use(_.findContours().size)).sum
   }
 }
}

==Exception mode==

VideoCapture.setExceptionMode(true) turns a silent false into a CvException carrying OpenCV's own message, and open uses it: a missing file becomes CvError.NativeCall quoting the path instead of a bare "it did not open". frames deliberately turns it off for the duration of the loop, because OpenCV reports end-of-file through the identical exception it uses for a broken stream — cap.cpp:533 error: (-2:Unspecified error) in function 'grab', measured on a clean five-frame file. With exception mode on there is no way to tell "the video ended" from "the camera was unplugged", so the loop would have to treat every real failure as a normal end. Off, read returning false ends the stream and a genuine decode error still surfaces as CvError.NativeCall.

Attributes

Source: Video.scala
Supertypes: class Object

trait Matchable

class Any
Self type: Video.type

Monocular visual odometry — estimating how the camera moved between two frames from matched point correspondences, via the essential matrix and recoverPose.

Pair it with OpticalFlow (track points frame to frame) or Features (detect and match): those give the correspondences, this turns them into motion. Chaining the per-frame motions is dead-reckoning odometry; making it drift-free SLAM needs a back end (keyframes, loop closure, bundle adjustment) that is beyond OpenCV — see the navigation guide.

Attributes

Source: VisualOdometry.scala
Supertypes: class Object

trait Matchable

class Any
Self type: VisualOdometry.type

Extensions

extension (img: Image)

Prepares this image for OCR — the OpenCV half of the pipeline: grayscale → denoise → adaptive threshold → deskew, producing a clean, upright, binarised image an OcrEngine can read well. Feed the result to Ocr.read with preprocess = false, or just call Ocr.read(image, engine) which does this for you.

Value parameters

blockSize: the adaptive-threshold neighbourhood (odd, ≥ 3).
c: the adaptive-threshold bias — raise it to keep less ink.
denoise: median-blur radius applied before thresholding; 0 skips it.

Attributes

Source: Ocr.scala

extension (img: Image)

Video-conferencing blur: keeps the person (where mask is white) sharp and blurs the background, feathering the edge. mask is a borrowed CV_8UC1 foreground mask (from Segmenter or any keying).

Attributes

Source: BackgroundEffect.scala

Replaces the background (where mask is black) with background, resized to fit and feathered at the edge — a virtual background. mask and background are borrowed.

Attributes

Source: BackgroundEffect.scala

Runs a selfie-segmentation Net over this image and decodes the person mask — the one-call form of blob → forward → Segmenter.decodeMask, ready to hand to blurBackground or replaceBackground. The image is only read from (it stays alive, so you can segment then composite in two lines), and the network is borrowed, not released. The returned mask is a new owned CV_8UC1 Image.

The blob knobs mirror Dnn.blobFromImage and are model-specific; the defaults suit a MediaPipe-selfie or MODNet-style export (RGB input, [0, 1] range). When you need the raw output tensor, drop to Dnn and Segmenter.decodeMask directly.

Value parameters

inputSize: the spatial size the network expects, e.g. Size(256, 256).
net: a caller-owned segmentation network (see Dnn.fromOnnx). Stateful — one per thread.
threshold: the foreground probability above which a pixel is the person.

Attributes

Source: BackgroundEffect.scala

extension (img: Image)

Detects every marker and recovers its 3D Pose3D in one step — the query behind marker AR. markerLength is the tag's real side length (metres, conventionally); intrinsics is the camera model (Intrinsics.approx if you have not calibrated). Markers whose pose fails to solve are dropped.

Attributes

Source: Ar.scala

Draws a 3D coordinate frame at every marker's pose — the classic "is my pose right?" overlay. X is red, Y green, Z blue (pointing out of the tag toward the camera). markerLength is the tag's real side; the axes are drawn at axisLength (defaulting to half the side).

Attributes

Source: Ar.scala

Draws a wireframe cube standing on every marker, sized to the marker's side by default — the "hello world" of marker AR. Consumes this image and returns the annotated one.

Attributes

Source: Ar.scala

extension (img: Image)

Rectangles via a Haar CascadeClassifier you supply (see Cascades). Borrowed, not released.

Takes the raw classifier, not the Managed Cascades.load returns — a defaulted overload for the Managed is not expressible alongside this one — so keep the classifier inside its scope to keep the spent-handle guard: Cascades.load(name).map(_.use(c => image.detectHaar(c))). (The FaceDetectorYN detector, faces, does take a Managed directly, since it has no defaults to collide.)

Attributes

Source: Cascades.scala

extension (img: Image)

Every ArUco marker from dictionary. Self-contained — builds and frees its own detector.

Attributes

Source: Detectors.scala

Every QR code in the image, decoded. Self-contained — builds and frees its own detector.

Attributes

Source: Detectors.scala

extension (img: Image)

Undistort an Image straight from a Calibration — unwraps its Intrinsics and delegates to Image.undistort(intrinsics). An extension method (in the vision layer) rather than a member of the core Image, so the core need not depend on this camera-calibration type. import scalacv.* gives frame.undistort(calib).

Attributes

Source: Calibration.scala

extension (img: Image)

Annotates ObjectTracks: a box and an #id label per track — the one-call "show me what the tracker is following". Consumes this image and returns the annotated one.

Attributes

Source: Tracking.scala

extension (img: Image)

Draws a Pose skeleton: a line per bone and a dot per confident keypoint.

Attributes

Source: Pose.scala

Runs a keypoint Net over this image and decodes a Pose — the one-call form of the blob → forward → decode dance, the pose counterpart to image.faces(detector). The image is only read from (it stays alive), and the network is borrowed, not released.

The blob knobs mirror Dnn.blobFromImage and are model-specific — the defaults suit a MoveNet-style export (RGB input, [0, 1] range); pass the values your model documents. For a network whose output layout or keypoint scheme differs, name the KeypointLayout and PoseTopology. When you need the intermediate blob or output tensor, drop to Dnn and PoseEstimator.decode directly.

Value parameters

inputSize: the spatial size the network expects, e.g. Size(192, 192) for MoveNet Lightning.
layout: how the output tensor encodes keypoints — see KeypointLayout.
net: a caller-owned pose network (see Dnn.fromOnnx). Stateful — do not share one across threads.

Attributes

Source: Pose.scala

extension (img: Image)

Faces via a YuNet FaceDetectorYN you supply — the model is yours to build (see FaceDetect). The detector is borrowed and mutated (its input size is set to this image), never released here.

Attributes

Source: FaceDetect.scala

Annotates detected faces: a box per face and a dot per landmark. The one-call "show me what YuNet found" convenience.

Attributes

Source: FaceDetect.scala

extension (img: Image)

Draws a composable Picture onto the image — the high-level graphics layer (shapes, dashed strokes, text, transforms, transparency). Consumes this image and returns the annotated one.

Attributes

Source: Graphics.scala

extension (mat: Mat)

The standard Hough transform: every line is infinite and expressed in polar form.

Value parameters

maxTheta: upper bound on the reported angle, in radians.
minTheta: lower bound on the reported angle, in radians.
rho: accumulator distance resolution, in pixels.
srn: divisor for a coarse-to-fine rho; 0 (with stn) selects the classic transform.
stn: divisor for a coarse-to-fine theta.
theta: accumulator angle resolution, in radians.
threshold: minimum accumulator votes for a line to be reported.

Attributes

Throws: IllegalArgumentException
if the receiver is not a non-empty 8-bit single-channel image.
Source: Hough.scala

The probabilistic Hough transform: finite segments with endpoints in image coordinates.

Value parameters

maxLineGap: largest gap, in pixels, that will still be bridged into one segment.
minLineLength: segments shorter than this are discarded. 0 keeps everything.
rho: accumulator distance resolution, in pixels.
theta: accumulator angle resolution, in radians.
threshold: minimum accumulator votes for a line to be reported.

Attributes

Throws: IllegalArgumentException
if the receiver is not a non-empty 8-bit single-channel image.
Source: Hough.scala

As houghLines, but keeps each line's accumulator score.

Attributes

Throws: IllegalArgumentException
if the receiver is not a non-empty 8-bit single-channel image.
Source: Hough.scala

extension (mat: Mat)

Draws a line with an arrowhead at to. Mutates the receiver.

Value parameters

tipLength: the arrowhead's length as a fraction of the whole line, so the head stays in proportion however long the line is. OpenCV's own default is 0.1.

Attributes

Source: Draw.scala

Draws a circle of radius pixels about center. Mutates the receiver.

Attributes

Source: Draw.scala

Draws every contour in contours. Mutates the receiver.

This is the renderer for what findContours returns. Thickness.Filled fills them, which is the usual way to turn a set of contours back into a mask.

Attributes

Source: Draw.scala

Draws a straight line from from to to. Mutates the receiver.

Coordinates outside the image are clipped, not rejected — that is OpenCV's behaviour and it is what makes drawing a detection that runs off the edge of a frame safe.

Attributes

Source: Draw.scala

Draws a connected run of line segments through points. Mutates the receiver.

Value parameters

closed: whether to draw the closing edge from the last point back to the first. Defaults to true, matching what Contour and polygon data mean. An empty points draws nothing rather than failing: a polyline is frequently the result of a filter, and filtering everything out is a legitimate outcome, not a programming error.

Attributes

Source: Draw.scala

Draws an axis-aligned rectangle. Mutates the receiver.

Pass Thickness.Filled for a solid block — useful as a label background or for building a mask.

Attributes

Source: Draw.scala

Draws every segment in segments. Mutates the receiver.

The renderer for houghLinesP, whose results are otherwise invisible.

Attributes

Source: Draw.scala

Draws text with its baseline's left end at at. Mutates the receiver.

at is not the top-left corner: OpenCV anchors text on the baseline, so a y of 0 puts almost the whole string above the image and draws nothing visible. Draw.textSize gives the box to place it by.

Only the Hershey vector fonts exist — OpenCV cannot render a system font, and non-ASCII characters are drawn as ?.

Attributes

Source: Draw.scala

Fills the polygon described by points with color. Mutates the receiver.

The outline is implicitly closed. Self-intersecting outlines are filled by OpenCV's even-odd rule.

Attributes

Source: Draw.scala

extension (mat: Mat)

Finds contours in a binary image.

The input must be single-channel 8-bit (or CV_32SC1); anything else raises CvError.NativeCall. Unlike the C++ API this does not modify mat — the Java binding copies internally — but treating a thresholded image as consumed is still the safer habit.

Every MatOfPoint OpenCV allocates, and the hierarchy Mat it fills, are released before this returns. The hierarchy itself is not exposed: it is only meaningful for the nesting-aware retrieval modes, and handing back a raw Nx1 CV_32SC4 Mat of indices would be exactly the untyped, unmanaged shape this library exists to remove. A typed nesting API can be added later without breaking this one.

Value parameters

approximation: how each outline is compressed. ContourApproximation.Simple collapses straight runs to their endpoints, so an axis-aligned rectangle comes back as 4 points rather than its full pixel chain.
retrieval: which contours to return and how to relate them. Defaults to ContourRetrieval.External — outermost only, which is what callers who ignore the hierarchy almost always mean.

Attributes

Returns: the contours, in OpenCV's order, as plain Scala data. Empty when the image is uniform.
Source: Contours.scala

extension (mat: Mat)

Runs a cascade over this image and returns the detections as immutable Scala values.

The receiver is neither modified nor released. The MatOfRect OpenCV fills in is internal and is released before returning, which is why the result is Seq[Rect] and not a live Mat — the rectangles are copied out at the native boundary and stay valid after everything here is freed.

Best results come from a single-channel, histogram-equalised image; a colour Mat works but is slower. An empty Mat makes OpenCV throw, and that throw is deliberately not caught: it is a programmer error, not a data-dependent failure.

Value parameters

minNeighbors: how many overlapping detections a candidate needs to survive. Higher is stricter.
minSize: objects smaller than this are ignored. Setting it is the cheapest speed-up available.
scaleFactor: how much the detection window grows per pyramid level. Just above 1 is slower and finds more.

Attributes

Source: Cascades.scala

extension (self: Mat)

Absolute per-element difference |self - other|. other is borrowed. The basis of frame-difference motion detection — see MotionDetector.

Attributes

Source: Ops.scala

Adaptive threshold — a threshold computed per neighbourhood rather than once for the whole image, which is what makes it hold up under uneven lighting (document scans, OCR pre-processing). CV_8UC1 only.

Value parameters

blockSize: the neighbourhood side; must be odd and ≥ 3.
c: a constant subtracted from the local mean/Gaussian — raise it to keep less.

Attributes

Source: Ops.scala

Weighted sum: self * alpha + other * beta + gamma.

other is borrowed, exactly like the receiver — it is neither released nor aliased.

Attributes

Source: Ops.scala

Edge-preserving bilateral filter: smooths flat regions while keeping edges crisp. Markedly slower than a Gaussian. diameter ≤ 0 lets OpenCV derive it from sigmaSpace.

Attributes

Source: Ops.scala

Bitwise NOT — inverts every pixel (255 - v for 8-bit).

Attributes

Source: Ops.scala

Adds a border (padding) of the given pixel widths on each side.

Attributes

Source: Ops.scala

Normalised box filter. anchor defaults to Point(-1, -1), OpenCV's spelling of "the kernel centre".

Named boxBlur, not blur, on purpose: the high-level Image.blur is a radius-based Gaussian, and a mid-level method sharing that name would silently switch filter families (and output hash) the moment a caller drops from image.blur(2) to image.mat.blur(...). The two are different algorithms; the names say so.

border may not be BorderType.Wrap — see BorderType.requireFilterSupport.

Attributes

Source: Ops.scala

Canny edge detection. The result is always CV_8UC1 regardless of the source type.

OpenCV accepts only 3, 5 and 7 for apertureSize — the Sobel aperture used internally — and aborts in native code for anything else, so it is checked here instead.

Attributes

Source: Ops.scala

Applies a false-colour map — turns a single-channel image (a depth map, a motion field, any data) into a colour heatmap. See Colormap.

Attributes

Source: Ops.scala

Scales, takes the absolute value, and saturating-casts to 8-bit unsigned.

The companion to a OutputDepth.Signed16 sobel: it is what turns a signed derivative back into something displayable without losing the negative lobe.

Attributes

Source: Ops.scala

Converts between colour spaces. The channel count of the result follows the conversion, not the source.

Attributes

Source: Ops.scala

Detects the dominant text skew and rotates the image upright — the classic OCR pre-step. Works on any image: it binarises internally to find the text pixels, fits a minimum-area rectangle to them, and rotates by that tilt. The exposed corners are filled white, and a detected skew beyond maxAngle is treated as a misread and left alone (a page of large graphics can fool the estimate).

Attributes

Source: Ops.scala

Detail enhancement — boosts local contrast and texture. Needs 8-bit 3-channel input.

Attributes

Source: Ops.scala

Morphological dilation — grows bright regions, fills small dark gaps.

Attributes

Source: Ops.scala

Edge-preserving smoothing — flattens texture while keeping edges (the basis of the painterly filters).

Attributes

Source: Ops.scala

Emboss, via a directional convolution.

Attributes

Source: Ops.scala

Histogram equalisation. OpenCV accepts CV_8UC1 only; anything else fails in native code.

Attributes

Source: Ops.scala

Morphological erosion with a radius-derived structuring element — shrinks bright regions, removes small bright specks. iterations applies it repeatedly.

Attributes

Source: Ops.scala

Extracts a single channel as its own image.

Attributes

Source: Ops.scala

Mirrors the image across an axis — see Flip.

Attributes

Source: Ops.scala

Gamma correction: g < 1 darkens the mid-tones, g > 1 lifts them.

Attributes

Source: Ops.scala

Gaussian blur.

kernel may be Size(0, 0), in which case OpenCV derives the kernel from the sigmas; otherwise both extents must be positive and odd. A sigmaY of 0 means "same as sigmaX", which is OpenCV's own default and not a degenerate value.

border may not be BorderType.Wrap — see BorderType.requireFilterSupport.

Attributes

Source: Ops.scala

A binary mask (CV_8UC1, 0 or 255) of the pixels whose every channel lies within [lo, hi]. The core of colour segmentation — usually run on an HSV image.

Attributes

Source: Ops.scala

Inpaints the region under mask (CV_8UC1, non-zero = repair) from its surroundings — remove a scratch, an object, or a watermark. mask is borrowed.

Attributes

Source: Ops.scala

Laplacian. kernelSize of 1 is the 3x3 aperture OpenCV special-cases, and is its default.

border may not be BorderType.Wrap — see BorderType.requireFilterSupport.

Attributes

Source: Ops.scala

Keeps this image only where mask (CV_8UC1) is non-zero; the rest becomes black. mask is borrowed.

Attributes

Source: Ops.scala

Median blur — each pixel becomes the median of its ksize×ksize neighbourhood. The standard cure for salt-and-pepper noise, and unlike a Gaussian it does not smear edges. ksize must be odd and ≥ 3.

Attributes

Source: Ops.scala

A compound morphological operation (open/close/gradient/top-hat/black-hat) — see MorphOp.

Attributes

Source: Ops.scala

Linearly rescales values into [alpha, beta] (min-max normalisation) and hands the result back at depth. Useful for stretching contrast, and the standard way of bringing a non-8-bit result — a disparity map, a distance transform, a float Sobel response — into a displayable range.

depth defaults to OutputDepth.Unsigned8 rather than to OpenCV's own dtype = -1, which means "same depth as the source". With -1 a CV_32F input rescaled to [0, 255] comes back as a CV_32F holding the values 0..255, so the second half of the job — making it displayable — never happened: Image.toBufferedImage rejects it, applyColorMap (colorMap) aborts in native code because it takes CV_8UC1/CV_8UC3 only, and imwrite only survives it by silently coercing behind our back. For an already-8-bit source Unsigned8 and -1 are the same conversion, so a plain contrast stretch is unaffected by the default.

Pass OutputDepth.SameAsSource for a stretch that must keep the source's precision — rescaling a float image into [0, 1] for a model's input, for instance, where 8-bit would collapse the range onto 256 levels.

Attributes

Source: Ops.scala

A colour pencil-sketch rendering. Needs 8-bit 3-channel input.

Attributes

Source: Ops.scala

Posterises to levels tones per channel.

Attributes

Source: Ops.scala

Resizes to an absolute size, given here as a Size whose two Double extents are truncated toward zero on the way into native code: Size(1.9, 1.9) asks for a 1×1 image.

That truncation is why the check below is on the truncated integers and not on the doubles. A computed target such as Size(width * factor, height * factor) with a small factor lands between 0 and 1, which is positive as a Double but empty as a cv::Size, and OpenCV then aborts with CV_Assert(inv_scale_x > 0) — a CvError.NativeCall quoting a C++ expression, in place of the IllegalArgumentException naming the caller's own argument that this file promises for a zero target size. Checking after truncation is what Mats.requireKernel already does for kernels.

Attributes

Source: Ops.scala

A lossless quarter-turn rotation — exact pixels, no interpolation. See Rotation.

Attributes

Source: Ops.scala

Rotates by an arbitrary angle (degrees, counter-clockwise) about the centre, expanding the canvas so no corner is clipped. scale zooms at the same time. The exposed border is filled per border.

Attributes

Source: Ops.scala

Adjusts saturation: factor > 1 is more vivid, < 1 toward grey, 0 fully grey (still 3-channel).

Attributes

Source: Ops.scala

Resizes by independent x and y scale factors. Rejects a pair of factors that would round this Mat's own size down to an empty one.

A separate method rather than an overload because OpenCV distinguishes the two modes by passing Size(0, 0) — a sentinel that has no business in a typed API.

Positive factors are not on their own enough to know the call is legal: OpenCV derives the destination from the receiver as cvRound(cols * fx) × cvRound(rows * fy) and then asserts !dsize.empty(), so shrinking a small image hard enough (a 100-pixel sprite at fx = 0.005) dies in native code. The check therefore has to be against the receiver's extent, not against the factors.

math.rint and not .toInt or math.round, because cvRound rounds half to even: on a 100-wide source fx = 0.006 legitimately yields a 1-pixel result that truncation would reject, and fx = 0.025 yields 2 where math.round says 3. Note the asymmetry with resize, where the destination arrives as a cv::Size and is truncated instead — the two native paths genuinely round differently, so one shared rule would be wrong for one of them.

Attributes

Source: Ops.scala

Seamlessly clones this image (the foreground object) into background at center, blending gradients so the paste is invisible (Poisson editing). mask (CV_8UC1) marks the object; background and mask are borrowed. The result is background-sized.

Attributes

Source: Ops.scala

Sepia tone, via a colour matrix.

Attributes

Source: Ops.scala

Unsharp-mask sharpening: adds back amount × (image − its blur). amount 0 is a no-op; ~1 is a firm sharpen. Overdo it and haloes appear at edges.

Attributes

Source: Ops.scala

Sobel derivative.

See OutputDepth before leaving depth at its default on an 8-bit image. border may not be BorderType.Wrap — see BorderType.requireFilterSupport.

Attributes

Source: Ops.scala

Stylisation — a smooth, painterly cartoon look via edge-aware smoothing. Needs 8-bit 3-channel input.

Attributes

Source: Ops.scala

Colour temperature: shift > 0 warms (more red), < 0 cools (more blue), in [-1, 1].

Attributes

Source: Ops.scala

Thresholding.

Returns the thresholded image and the double OpenCV computed. Most wrappers drop that number; for Threshold.Auto.Otsu and Threshold.Auto.Triangle it is the threshold OpenCV chose, which is frequently the reason the call was made. For a fixed threshold it is just value handed back.

Imgproc.threshold has a single 5-argument overload with no defaults, so every argument is spelled out here rather than being layered over Java defaults that do not exist.

Attributes

Source: Ops.scala

Removes lens distortion using calibrated camera Intrinsics — the barrel/pincushion bend a real lens adds is mapped back out, so straight edges in the world come back straight. A no-op (a plain copy) when intrinsics.distortion is empty. See Calibration.

Attributes

Source: Ops.scala

extension (self: Scalar)

The reverse of Color.toScalar: read an OpenCV BGR Scalar into the RGBA Color palette, for handing an OpenCV colour to the Picture graphics layer. Channels are rounded and clamped to [0, 255]; the result is fully opaque (a Scalar's fourth channel is not a reliable alpha, so it is not carried over). An extension rather than a method on Scalar so the core Scalar type stays free of any dependency on this graphics-layer palette.

Attributes

Source: Color.scala

extension (self: Managed[Mat])

Hands the wrapped Mat to f and releases it once f has produced its own result.

This is the whole reason the ownership contract above is safe to write down. Each op returns a Mat the caller owns, so a chain of them produces one owned Mat per stage, and every stage but the last is garbage the moment the next one returns. pipe makes that the default rather than something the caller has to remember: self is consumed, and using it afterwards throws IllegalStateException instead of reading freed memory.

val edges = src.gaussianBlur(Size(5, 5), 1.5).pipe(_.canny(50, 150))

The release happens in a finally, so a stage that throws does not leak its input either. For a terminal stage that produces something other than a Mat — a count, a Seq[Rect] — use Managed.use, which has the same shape and the same guarantee.

Attributes

Source: Ops.scala

In this article

Generated with