Vision performs computer-vision analysis on images and video, including detection, recognition, body and animal pose estimation, and aesthetics scoring. You configure request objects for the analysis you need and run them over an image or video frame: detect faces and their landmarks with VNDetectFaceRectanglesRequest and VNDetectFaceLandmarksRequest, find barcodes, contours, rectangles, and text with requests such as VNDetectBarcodesRequest and VNDetectTextRectanglesRequest, estimate human and animal pose using VNDetectHumanBodyPoseRequest, VNDetectHumanHandPoseRequest, VNDetectHumanBodyPose3DRequest, and VNDetectAnimalBodyPoseRequest, and generate segmentation masks, saliency, optical flow, and aesthetics scores. You can also classify images and run your own Core ML models through VNClassifyImageRequest, VNCoreMLModel, and VNCoreMLRequest, and read results through observation and point types like VNHumanBodyPoseObservation, VNRecognizedPoint, and VNPoint3D. Vision is available on iOS, macOS, tvOS, watchOS, and visionOS.
Performing Requests 18
The handlers and base request types that execute computer-vision analysis on still images and video sequences.
- ClVNImageRequestHandleriOS 11.0+@brief Performs requests on a single image.
- ClVNSequenceRequestHandleriOS 11.0+@brief Performs requests on a sequence of images.
- ClVNVideoProcessoriOS 14.0+@abstract A controller object that is used to perform one or more requests on a video stream.
- ClVNRequestiOS 11.0+@class VNRequest
- ClVNImageBasedRequestiOS 11.0+@brief A request that will process the contents of a reference image.
- ClVNTargetedImageRequestiOS 11.0+@brief A request that requires an initial image as a starting point for its work.
- ClVNStatefulRequestiOS 14.0+@brief A request that builds evidence over time by being reused on multiple images.
- ClImageRequestHandleriOS 18.0+A Swift handler that performs analysis requests on a single image.
- ClTargetedImageRequestHandleriOS 18.0+A Swift handler that performs requests comparing an analyzed image to a target image.
- ClVideoProcessoriOS 18.0+A Swift object that runs analysis requests over the frames of a video.
- PrVisionRequestiOS 18.0+A protocol for Vision analysis requests.
- PrImageProcessingRequestiOS 18.0+An image analysis request that operates on a region of interest and produces observations.
- PrTargetedRequestiOS 18.0+A request that can be used with a `TargetedImageHandler` to analyze two images together.
- PrStatefulRequestiOS 18.0+A request type that builds evidence of a condition over time.
- PrDownloadableAssetsRequestiOS 27.0+A protocol for requests whose models require downloadable assets.
- StDownloadableAssetsProgressA value that reports the download progress of a request's model assets.
- EnDownloadableAssetsRequestStatusiOS 27.0+Constants that describe the download status of a request's model assets.
- EnRequestDescriptoriOS 18.0+A descriptor that identifies a request type and its supported revisions.
Face and Body Detection 18
Requests and observations for detecting faces, facial landmarks, capture quality, and human regions.
- ClVNDetectFaceRectanglesRequestiOS 11.0+@brief A request that will detect faces in an image.
- ClVNDetectFaceLandmarksRequestiOS 11.0+@brief A request that will produce face landmark information.
- ClVNDetectFaceCaptureQualityRequestiOS 13.0+@brief A request that will produce or update a VNFaceObservation's property faceCaptureQuality with a float value.
- ClVNDetectHumanRectanglesRequestiOS 13.0+@brief A request that will detect human Torsos in an image.
- ClVNFaceObservationiOS 11.0+@class VNFaceObservation
- ClVNHumanObservationiOS 15.0+@class VNHumanObservation
- ClVNFaceLandmarksiOS 11.0+@class VNFaceLandmarks
- ClVNFaceLandmarks2DiOS 11.0+@class VNFaceLandmarks2D
- ClVNFaceLandmarkRegioniOS 11.0+@class VNFaceLandmarkRegion
- ClVNFaceLandmarkRegion2DiOS 11.0+@class VNFaceLandmarkRegion2D
- StDetectFaceRectanglesRequestiOS 18.0+A request that finds faces within an image.
- StDetectFaceLandmarksRequestiOS 18.0+An image analysis request that finds facial features like eyes and mouth in an image.
- StDetectFaceCaptureQualityRequestiOS 18.0+A request that determines the capture quality of faces in a photo.
- StDetectHumanRectanglesRequestiOS 18.0+A request that finds rectangular regions that contain people in an image.
- StFaceObservationiOS 18.0+A Swift observation describing a detected face and its attributes.
- StHumanObservationiOS 18.0+An object that represents a person that the request detects.
- PrVNFaceObservationAcceptingiOS 11.0+@brief A protocol that allows a user to provide an input collection of VNFaceObservations as part of a request. The protocol can be adopted by request objects that request additional information about
- EnVNRequestFaceLandmarksConstellation@brief Constellation type defines how many landmark points are used to map a face. Revisions 1, 2, and 3 support 65 points, where Rev3 also supports 76 points.
Pose Estimation 21
Requests and observations that estimate human body, hand, and animal poses in two and three dimensions.
- ClVNDetectHumanBodyPoseRequestiOS 14.0+@brief Detects specific landmark points on human bodies.
- ClVNDetectHumanBodyPose3DRequestiOS 17.0+@brief A request that detects specific landmark points on human bodies in 3D space relative to the camera.
- ClVNDetectHumanHandPoseRequestiOS 14.0+@brief Detects specific landmark points on human hands.
- ClVNDetectAnimalBodyPoseRequestiOS 17.0+@brief Detects specific landmark points on animal bodies.
- ClVNHumanBodyPoseObservationiOS 14.0+An observation that describes the detected joint positions of a human body pose.
- ClVNHumanBodyPose3DObservationiOS 17.0+An observation that describes a human body pose in three-dimensional space.
- ClVNHumanHandPoseObservationiOS 14.0+An observation that describes the detected joint positions of a human hand pose.
- ClVNAnimalBodyPoseObservationiOS 17.0+An observation that describes the detected joint positions of an animal body pose.
- ClVNHumanBodyRecognizedPoint3DiOS 17.0+@brief
- StDetectHumanBodyPoseRequestiOS 18.0+A request that detects a human body pose.
- ClDetectHumanBodyPose3DRequestiOS 18.0+A request that detects a human body pose.
- StDetectHumanHandPoseRequestiOS 18.0+A request that detects a human hand pose.
- StDetectAnimalBodyPoseRequestiOS 18.0+A request that detects an animal body pose.
- StHumanBodyPoseObservationiOS 18.0+An observation that provides the body points the analysis recognized.
- StHumanBodyPose3DObservationiOS 18.0+An observation that provides the three-dimensional body points the request recognizes.
- StHumanHandPoseObservationiOS 18.0+An observation that provides the hand points the analysis recognized.
- StAnimalBodyPoseObservationiOS 18.0+An observation that provides the animal body points the analysis recognizes.
- StJointiOS 18.0+A body pose joint represented as a normalized point in an image, along with a joint name label and a confidence value.
- StJoint3DiOS 18.0+A named three-dimensional joint within a detected pose.
- EnChiralityiOS 18.0+Constants that specify the left or right handedness of a detected hand.
- PrPoseProvidingiOS 18.0+An observation that provides a collection of joints that make up a pose.
Detecting Shapes and Trajectories 17
Requests and observations that locate rectangles, contours, the horizon, and moving objects in a scene.
- ClVNDetectRectanglesRequestiOS 11.0+@brief A request that will detect rectangles in an image.
- ClVNDetectContoursRequestiOS 14.0+@brief A request that will detect the contours for the edges in an image.
- ClVNDetectHorizonRequestiOS 11.0+@brief Determine what the horizon tilt of an image is.
- ClVNDetectTrajectoriesRequestiOS 14.0+@brief A request that detects trajectories of shapes (even small ones) that follow a parabolic path in a sequence of images.
- ClVNRectangleObservationiOS 11.0+@class VNRectangleObservation
- ClVNContoursObservationiOS 14.0+An observation that contains the detected contours of an image.
- ClVNContouriOS 14.0+@brief The VNContour class describes a contour provided by a VNContoursObservation.
- ClVNHorizonObservationiOS 11.0+@class VNHorizonObservation
- ClVNTrajectoryObservationiOS 14.0+@class VNTrajectoryObservation
- StDetectRectanglesRequestiOS 18.0+An image-analysis request that finds projected rectangular regions in an image.
- StDetectContoursRequestiOS 18.0+A request that detects the contours of the edges of an image.
- StDetectHorizonRequestiOS 18.0+An image analysis request that determines the horizon angle in an image.
- ClDetectTrajectoriesRequestiOS 18.0+An image analysis request that determines the horizon angle in an image.
- StRectangleObservationiOS 18.0+An object that represents the four vertices of a detected rectangle.
- StContoursObservationiOS 18.0+An observation that provides all of the detected contours in an image.
- StHorizonObservationiOS 18.0+The horizon angle information that an image analysis request detects.
- StTrajectoryObservationiOS 18.0+An observation that describes a detected trajectory.
Reading Text, Barcodes, and Documents 26
Requests and observations for recognizing text, scanning barcodes, segmenting documents, and detecting lens smudges.
- ClVNRecognizeTextRequestiOS 13.0+@brief A request that will detect regions of text and recognize the containing text in an image.
- ClVNDetectTextRectanglesRequestiOS 11.0+@brief A request that will detect regions of text in an image.
- ClVNDetectBarcodesRequestiOS 11.0+@brief A request that detects barcodes in an image.
- ClVNDetectDocumentSegmentationRequestiOS 15.0+@brief Performs document detection and segmentation in an image.
- ClVNRecognizedTextiOS 13.0+@class VNRecognizedText
- ClVNRecognizedTextObservationiOS 13.0+@class VNRecognizedTextObservation
- ClVNTextObservationiOS 11.0+@class VNTextObservation
- ClVNBarcodeObservationiOS 11.0+@class VNBarcodeObservation
- StRecognizeTextRequestiOS 18.0+An image-analysis request that recognizes text in an image.
- StDetectTextRectanglesRequestiOS 18.0+An image analysis request that finds regions of visible text in an image.
- StDetectBarcodesRequestiOS 18.0+A request that detects barcodes in an image.
- StDetectDocumentSegmentationRequestiOS 18.0+A request that detects rectangular regions that contain text in the input image.
- StRecognizeDocumentsRequestiOS 26.0+An image-analysis request to scan an image of a document and provide information about its structure.
- StRecognizedTextiOS 18.0+The recognized text string and confidence for a detected text region.
- StRecognizedTextObservationiOS 18.0+A Swift observation containing recognized text candidates for a text region.
- StTextObservationiOS 18.0+A Swift observation locating a region of text without recognizing its contents.
- StBarcodeObservationiOS 18.0+A Swift observation describing a detected barcode and its decoded payload.
- StDetectedDocumentObservationiOS 18.0+An observation that contains a detected document.
- StDocumentObservationiOS 26.0+Information about the sections of content that an image-analysis request detects in a document.
- StDetectLensSmudgeRequestiOS 26.0+A request that detects a smudge on a lens from an image or video frame capture.
- StSmudgeObservationiOS 26.0+An observation that provides a confidence score indicating the presence of a smudge in an image or video frame capture.
- StBarcodeReaderToolA tool that scans machine-readable codes in an image.
- StOCRToolA tool that recognizes text in an image.
- StVNBarcodeSymbology@brief Barcode symbologies that are supported by the Vision framework.
- EnBarcodeSymbologyiOS 18.0+Constants that identify the barcode symbologies Vision can detect.
- EnVNRequestTextRecognitionLevel@brief Text recognition level options to favor speed over recognition accuracy. The VNRequestTextRecognitionLevelAccurate is the default option used by VNRecognizeTextRequest.
Classification and Core ML 23
Requests, models, and observations for classifying images, recognizing animals, and running custom Core ML models.
- ClVNClassifyImageRequestiOS 13.0+@brief A request for classifying an image.
- ClVNRecognizeAnimalsRequestiOS 13.0+@brief A request that will recognize various animals in an image. The list of animals supported by the recognition algorithm can be queried by -supportedIdentifiersAndReturnError:
- ClVNCoreMLRequestiOS 11.0+@brief The VNCoreMLRequest uses a VNCoreMLModel, that is based on a CoreML MLModel object, to run predictions with that model. Depending on the model the returned
- ClVNCoreMLModeliOS 11.0+@brief The VNCoreMLModel uses an CoreML based model and prepares it for use with VNCoreMLRequests.
- ClVNClassificationObservationiOS 11.0+@class VNClassificationObservation
- ClVNRecognizedObjectObservationiOS 12.0+@class VNRecognizedObjectObservation
- ClVNCoreMLFeatureValueObservationiOS 11.0+@class VNCoreMLFeatureValueObservation
- ClVNDetectedObjectObservationiOS 11.0+@class VNDetectedObjectObservation
- ClVNGenerateImageFeaturePrintRequestiOS 13.0+@brief A request for generating a feature print of an image.
- ClVNFeaturePrintObservationiOS 13.0+An observation that contains a feature-print descriptor for comparing images.
- StClassifyImageRequestiOS 18.0+A request to classify an image.
- StRecognizeAnimalsRequestiOS 18.0+A request that will recognize various animals in an image.
- StCoreMLRequestiOS 18.0+An image analysis request that uses a Core ML model to process images.
- StCoreMLModelContaineriOS 18.0+A model container to use with an image analysis request.
- StClassificationObservationiOS 18.0+An object that represents classification information that an image analysis request produces.
- StRecognizedObjectObservationiOS 18.0+An observation with an array of classification labels that classify the recognized object.
- StCoreMLFeatureValueObservationiOS 18.0+A Swift observation wrapping a Core ML feature value produced by a model.
- StDetectedObjectObservationiOS 18.0+An observation that provides the position and extent of an image feature that an image analysis request detects.
- StGenerateImageFeaturePrintRequestiOS 18.0+An image-based request to generate feature prints from an image.
- StFeaturePrintObservationiOS 18.0+An observation that provides the recognized feature print.
- StVNAnimalIdentifierConstants that identify the kinds of animals Vision can recognize.
- EnVNImageCropAndScaleOptionConstants that specify how an image is cropped and scaled before analysis.
- EnImageCropAndScaleActioniOS 18.0+Constants that specify how an image is cropped and scaled before analysis.
Segmentation, Saliency, and Aesthetics 24
Requests and observations that generate segmentation masks, saliency and optical-flow maps, and aesthetics scores.
- ClVNGeneratePersonSegmentationRequestiOS 15.0+@brief Performs person segmentation on an image generating a mask.
- ClVNGeneratePersonInstanceMaskRequestiOS 17.0+@brief A request that generates an instance mask of individual people found in the image.
- ClVNGenerateForegroundInstanceMaskRequestiOS 17.0+@brief A request that generates an instance mask of salient objects that can be separated from the background.
- ClVNGenerateAttentionBasedSaliencyImageRequestiOS 13.0+@brief Generates an image that identifies which part(s) of a given image is most interesting (i.e. something that a human is likely to look at - hence attention based).
- ClVNGenerateObjectnessBasedSaliencyImageRequestiOS 13.0+@brief Generates an image that identifies which part(s) of a given image are most likely to be objects (i.e. something that a human is likely to see as an object).
- ClVNGenerateOpticalFlowRequestiOS 14.0+@brief VNGenerateOpticalFlowRequest will determine directional change vectors for each pixel in the targeted image to transform it into the image processed
- ClVNCalculateImageAestheticsScoresRequestiOS 18.0+@brief Analyzes an image for aesthetically pleasing attributes and returns a VNImageAestheticsScoresObservation.
- ClVNSaliencyImageObservationiOS 13.0+@class VNSaliencyImageObservation
- ClVNInstanceMaskObservationiOS 17.0+@brief An observation resulting from an instance mask generation request. It contains an instance mask that labels instances in the mask that labels per pixel an instance.
- ClVNPixelBufferObservationiOS 11.0+@class VNPixelBufferObservation
- ClVNImageAestheticsScoresObservationiOS 18.0+@class VNImageAestheticsScoresObservation
- ClGeneratePersonSegmentationRequestiOS 18.0+A request that produces a matte image for a person it finds in the input image.
- StGeneratePersonInstanceMaskRequestiOS 18.0+Generates an instance mask of individual people found in the image.
- StGenerateForegroundInstanceMaskRequestiOS 18.0+A request that generates an instance mask of noticeable objects to separate from the background.
- StGenerateSelectedObjectSegmentationRequestGenerates a segmentation mask based on the provided points, rectangle, mask or scribble
- ClGenerateIterativeSegmentationRequestGenerates a segmentation mask based on the provided points, rectangle, or scribble
- StGenerateAttentionBasedSaliencyImageRequestiOS 18.0+An object that produces a heat map that identifies the parts of an image most likely to draw attention.
- StGenerateObjectnessBasedSaliencyImageRequestiOS 18.0+A request that generates a heat map that identifies the parts of an image most likely to represent objects.
- StCalculateImageAestheticsScoresRequestiOS 18.0+A request that analyzes an image for aesthetically pleasing attributes.
- StSaliencyImageObservationiOS 18.0+An observation that contains a grayscale heat map of important areas across an image.
- StInstanceMaskObservationiOS 18.0+An observation that contains an instance mask that labels instances in the mask.
- StPixelBufferObservationiOS 18.0+An object that represents an image that an image analysis request produces.
- StOpticalFlowObservationiOS 18.0+An object that represents an optical flow that an image analysis request produces.
- StImageAestheticsScoresObservationiOS 18.0+A Swift observation reporting the computed aesthetics score for an image.
Tracking and Image Registration 20
Requests and observations for tracking objects across frames and aligning images by translation or homography.
- ClVNTrackingRequestiOS 11.0+@abstract A base class for all tracking requests.
- ClVNTrackObjectRequestiOS 11.0+@brief VNTrackObjectRequest tracks an object in a sequence of images.
- ClVNTrackRectangleRequestiOS 11.0+@brief VNTrackRectangleRequest tracks a rectangle in a sequence of images.
- ClVNTrackOpticalFlowRequestiOS 17.0+@brief `VNTrackOpticalFlowRequest` will determine directional change vectors for each pixel from a previous to current image, reporting this result with a single `VNPixelBufferObservation`.
- ClVNImageRegistrationRequestiOS 11.0+@brief A request that will calculate a transformation for morphing a "floating" image onto an unchanging "reference" image.
- ClVNTranslationalImageRegistrationRequestiOS 11.0+@brief An image registration request that will calculate a translational transformation for morphing a "floating" image onto an unchanging "reference" image.
- ClVNHomographicImageRegistrationRequestiOS 11.0+@brief An image registration request that will calculate a homographic transformation for morphing a "floating" image onto an unchanging "reference" image.
- ClVNTrackTranslationalImageRegistrationRequestiOS 17.0+@brief An image registration request that will produce a translational transformation which will morph one image to another.
- ClVNTrackHomographicImageRegistrationRequestiOS 17.0+@brief An image registration request that will produce a homographic transformation that can morph one image to another.
- ClVNImageAlignmentObservationiOS 11.0+@class VNImageAlignmentObservation
- ClVNImageTranslationAlignmentObservationiOS 11.0+@class VNImageTranslationAlignmentObservation
- ClVNImageHomographicAlignmentObservationiOS 11.0+@class VNImageHomographicAlignmentObservation
- ClTrackObjectRequestiOS 18.0+An image analysis request that tracks the movement of a previously identified object across multiple images or video frames.
- ClTrackRectangleRequestiOS 18.0+An image analysis request that tracks movement of a previously identified rectangular object across multiple images or video frames.
- ClTrackOpticalFlowRequestiOS 18.0+This request generates an OpticalFlowObservation object that describes the directional change from image to image.
- ClTrackTranslationalImageRegistrationRequestiOS 18.0+An image analysis request, as a stateful request you track over time, that determines the affine transform necessary to align the content of two images.
- ClTrackHomographicImageRegistrationRequestiOS 18.0+An image analysis request, as a stateful request you track over time, that determines the perspective warp matrix necessary to align the content of two images.
- StImageTranslationAlignmentObservationiOS 18.0+Affine transform information that an image alignment request produces.
- StImageHomographicAlignmentObservationiOS 18.0+An object that represents a perspective warp transformation.
- EnVNRequestTrackingLevel@brief Tracking level options to favor speed or location accuracy. The VNRequestTrackingLevelFast is the default option used by trackers.
Geometry and Points 17
Value and geometry types describing points, vectors, circles, and recognized landmark coordinates.
- ClVNPointiOS 14.0+@class VNPoint
- ClVNPoint3DiOS 17.0+@class VNPoint3D
- ClVNVectoriOS 14.0+@class VNVector
- ClVNCircleiOS 14.0+@class VNCircle
- ClVNDetectedPointiOS 14.0+@class VNDetectedPoint
- ClVNRecognizedPointiOS 14.0+@class VNRecognizedPoint
- ClVNRecognizedPoint3DiOS 17.0+@class VNRecognizedPoint3D
- ClVNGeometryUtilsiOS 14.0+A set of utilities for computing geometric measurements on detected contours.
- ClVNRecognizedPointsObservationiOS 14.0+@class VNRecognizedPointsObservation
- ClVNRecognizedPoints3DObservationiOS 14.0+@class VNRecognizedPoints3D
- StNormalizedPointiOS 18.0+A 2D point with x and y coordinates in the range [0, 1].
- StNormalizedRectiOS 18.0+A rectangle with normalized coordinates in the range [0, 1].
- StNormalizedCircleiOS 18.0+A circle defined in normalized image coordinates.
- EnCoordinateOriginiOS 18.0+Constants that specify the origin used for normalized coordinates.
- PrBoundingBoxProvidingiOS 18.0+A protocol for objects that have a bounding box.
- PrBoundingRegionProvidingiOS 26.0+A protocol for observations that expose a bounding region in an image.
- PrQuadrilateralProvidingiOS 18.0+An protocol for objects that have a bounding quadrilateral.
Observation Base Types 4
The root observation type and protocols shared by the results that Vision requests produce.
- ClVNObservationiOS 11.0+@class VNObservation
- PrVisionObservationiOS 18.0+A protocol that defines the common interface for Vision analysis results.
- PrVNRequestProgressProvidingiOS 13.0+A protocol for requests that report incremental progress during analysis.
- PrVNRequestRevisionProvidingiOS 12.0+A protocol for types that report the algorithm revision used to produce a result.
Options, Keys, and Errors 11
Configuration options, dictionary keys, identifiers, and error types used throughout the framework.
- StVNImageOption@brief Options keys passed into the VNImageRequestHandler creations or requests that take an auxiliary image. These are options that either describe specific properties of an image like the VNImageOpt
- StVNVideoProcessingOptionKeys that configure how a video processor runs analysis requests.
- StVNComputeStageiOS 17.0+Constants that identify the compute stages a request can run on.
- EnComputeStageiOS 18.0+Types that represent the compute stage.
- StVNRecognizedPointKeyConstants that identify individual recognized points such as body joints.
- StVNRecognizedPointGroupKeyConstants that identify groups of related recognized points.
- EnVNElementTypeiOS 13.0+Constants that specify the element data type of an observation's results.
- EnElementTypeiOS 18.0+An enumeration of the type of element in feature print data.
- EnVNErrorCodeError codes that Vision reports when a request fails.
- EnVisionErroriOS 18.0+An error type that describes failures encountered while performing Vision requests.
- EnVisionResultiOS 18.0+A type that represents the outcome of a Vision request.
Enumerations 3
- EnVNChiralityiOS 15.0+
- EnVNPointsClassificationiOS 16.0+
- EnVNBarcodeCompositeTypeiOS 17.0+
Type Aliases 6
- TyVNRequestCompletionHandler@abstract A block that is executed at the completion of a request.
- TyVNRequestProgressHandler@abstract A block that is executed at intervals during the processing of a request.
- TyVNConfidence
- TyVNAspectRatio
- TyVNDegrees
- TyNormalizedRegion