What's New / Apple Intelligence, ML & Evaluation

What's new in MediaIntelligence

+16 NewiOS · macOS · tvOS

MediaIntelligence is an Apple framework for analyzing images and video. Its API covers detecting and grouping faces and other entities found in media assets.

The 27 SDK adds 16 APIs and removes none. New types include the MediaIntelligenceError enum, the MediaIntelligenceImageAsset and MediaIntelligenceVideoAsset structs, MIManagedFace and its ID type, FaceGroupAnalyzer.State, and RawValue types for Face.ID, Entity.ID, and State.

New

16
extension

FaceGroupAnalyzer.State

NewiOSmacOStvOS
extension FaceGroupAnalyzer.State : Equatable
Declaration
extension FaceGroupAnalyzer.State : Equatable {
}
extension

FaceGroupAnalyzer.State

NewiOSmacOStvOS
extension FaceGroupAnalyzer.State : Hashable
Declaration
extension FaceGroupAnalyzer.State : Hashable {
}
extension

FaceGroupAnalyzer.State

NewiOSmacOStvOS
extension FaceGroupAnalyzer.State : RawRepresentable
Declaration
extension FaceGroupAnalyzer.State : RawRepresentable {
}
enum

MediaIntelligenceError

NewiOSmacOStvOS
public enum MediaIntelligenceError : LocalizedError

An error that indicates a media analysis operation failed.

Framework methods throw this error when they encounter a problem processing a request.

Declaration
public enum MediaIntelligenceError : LocalizedError {

    /// The framework can't access the working directory.
    ///
    /// This error occurs when the working directory you
    /// provide to ``FaceGroupAnalyzer/init(workingDirectory:)`` or
    /// ``FaceGroupAnalyzer/purge(workingDirectory:)`` doesn't exist or
    /// isn't accessible. Verify that the URL points to a valid, writable
    /// directory before creating an analyzer.
    case workingDirectory

    /// The framework can't process a media asset.
    ///
    /// This error occurs when the framework can't read or process an
    /// image or video asset. This can happen when the asset's URL is
    /// unreachable, the file format is unsupported, or the asset data is
    /// corrupt.
    case mediaProcessing

    /// The framework can't complete a face grouping operation.
    ///
    /// This error occurs when the framework fails to detect faces,
    /// insert or delete face data, or update face cluster assignments. If
    /// this error occurs during
    /// ``FaceGroupAnalyzer/insertOrUpdateAssets(_:)`` or
    /// ``FaceGroupAnalyzer/deleteAssets(_:)``, the framework automatically
    /// rolls back any partial changes.
    case faceGroupProcessing

    /// The framework can't retrieve analysis results.
    ///
    /// This error occurs when the framework can't find a result that
    /// matches the type expected by a ``VideoAnalyzer/Request``. Confirm
    /// that you're passing the correct request type to
    /// ``VideoAnalyzer/analyze(_:for:)``.
    case resultFetching

    /// A localized message describing what error occurred.
    public var errorDescription: String? { get }

    /// Returns a Boolean value indicating whether two values are equal.
    ///
    /// Equality is the inverse of inequality. For any values `a` and `b`,
    /// `a == b` implies that `a != b` is `false`.
    ///
    /// - Parameters:
    ///   - lhs: A value to compare.
    ///   - rhs: Another value to compare.
    public static func == (a: MediaIntelligenceError, b: MediaIntelligenceError) -> Bool

    /// Hashes the essential components of this value by feeding them into the
    /// given hasher.
    ///
    /// Implement this method to conform to the `Hashable` protocol. The
    /// components used for hashing must be the same as the components compared
    /// in your type's `==` operator implementation. Call `hasher.combine(_:)`
    /// with each of these components.
    ///
    /// - Important: In your implementation of `hash(into:)`,
    ///   don't call `finalize()` on the `hasher` instance provided,
    ///   or replace it with a different instance.
    ///   Doing so may become a compile-time error in the future.
    ///
    /// - Parameter hasher: The hasher to use when combining the components
    ///   of this instance.
    public func hash(into hasher: inout Hasher)

    /// The hash value.
    ///
    /// Hash values are not guaranteed to be equal across different executions of
    /// your program. Do not save hash values to use during a future execution.
    ///
    /// - Important: `hashValue` is deprecated as a `Hashable` requirement. To
    ///   conform to `Hashable`, implement the `hash(into:)` requirement instead.
    ///   The compiler provides an implementation for `hashValue` for you.
    public var hashValue: Int { get }
}
extension

MediaIntelligenceError

NewiOSmacOStvOS
extension MediaIntelligenceError : Equatable
Declaration
extension MediaIntelligenceError : Equatable {
}
extension

MediaIntelligenceError

NewiOSmacOStvOS
extension MediaIntelligenceError : Hashable
Declaration
extension MediaIntelligenceError : Hashable {
}
struct

MediaIntelligenceImageAsset

NewiOSmacOStvOS
public struct MediaIntelligenceImageAsset : Sendable

An image asset to analyze.

Create a MediaIntelligenceImageAsset to identify an image you want FaceGroupAnalyzer to process. Each asset has a unique identifier you assign, and a Kind value that describes how the framework accesses the image data.

Use the same ID value consistently for a specified image. The framework uses this identifier to match new submissions against existing data, so changing the identifier causes the framework to treat the image as a new asset.

Declaration
public struct MediaIntelligenceImageAsset : Sendable {

    /// A unique identifier for an image asset.
    ///
    /// ``MediaIntelligenceImageAsset/ID`` is a string-backed identifier
    /// you assign to each image asset. The framework uses this value to
    /// track assets across calls to
    /// ``FaceGroupAnalyzer/insertOrUpdateAssets(_:)`` and
    /// ``FaceGroupAnalyzer/deleteAssets(_:)``.
    ///
    /// Choose identifiers that remain constant for a specified image. For
    /// example, a photo library asset identifier or a file path that
    /// doesn't change.
    public struct ID : Hashable, RawRepresentable, Codable, Sendable {

        /// The corresponding value of the raw type.
        ///
        /// A new instance initialized with `rawValue` will be equivalent to this
        /// instance. For example:
        ///
        ///     enum PaperSize: String {
        ///         case A4, A5, Letter, Legal
        ///     }
        ///
        ///     let selectedSize = PaperSize.Letter
        ///     print(selectedSize.rawValue)
        ///     // Prints "Letter"
        ///
        ///     print(selectedSize == PaperSize(rawValue: selectedSize.rawValue)!)
        ///     // Prints "true"
        public let rawValue: String

        /// Creates a new instance with the specified raw value.
        ///
        /// If there is no value of the type that corresponds with the specified raw
        /// value, this initializer returns `nil`. For example:
        ///
        ///     enum PaperSize: String {
        ///         case A4, A5, Letter, Legal
        ///     }
        ///
        ///     print(PaperSize(rawValue: "Legal"))
        ///     // Prints "Optional(PaperSize.Legal)"
        ///
        ///     print(PaperSize(rawValue: "Tabloid"))
        ///     // Prints "nil"
        ///
        /// - Parameter rawValue: The raw value to use for the new instance.
        public init(rawValue: String)

        public init(_ rawValue: String)

        /// The raw type that can be used to represent all values of the conforming
        /// type.
        ///
        /// Every distinct value of the conforming type has a corresponding unique
        /// value of the `RawValue` type, but there may be values of the `RawValue`
        /// type that don't have a corresponding value of the conforming type.
        @available(macOS 27.0, iOS 27.0, tvOS 27.0, *)
        public typealias RawValue = String
    }

    /// A value that describes the source of an image asset's data.
    ///
    /// ``MediaIntelligenceImageAsset/Kind`` describes how the framework
    /// reads an image.
    public enum Kind : Sendable {

        /// An image located at the specified URL.
        case url(URL)
    }

    /// A unique identifier for the asset.
    public let id: MediaIntelligenceImageAsset.ID

    /// A value that describes how the framework accesses the image data.
    public let kind: MediaIntelligenceImageAsset.Kind

    /// Creates an image asset with the specified identifier and kind.
    ///

Truncated.

struct

MediaIntelligenceVideoAsset

NewiOSmacOStvOS
public struct MediaIntelligenceVideoAsset : Sendable

A video asset to analyze.

Create an instance of this type to identify a video for VideoAnalyzer to process. Each asset has a unique identifier you assign, and a Kind value that describes how the framework accesses the video data.

Declaration
public struct MediaIntelligenceVideoAsset : Sendable {

    /// A unique identifier for a video asset.
    ///
    /// This value is a string-backed identifier that you
    /// assign to each video asset. Choose a value that uniquely identifies
    /// the video in your app, such as a file name or a database key.
    public struct ID : Hashable, RawRepresentable, Codable, Sendable {

        /// The corresponding value of the raw type.
        ///
        /// A new instance initialized with `rawValue` will be equivalent to this
        /// instance. For example:
        ///
        ///     enum PaperSize: String {
        ///         case A4, A5, Letter, Legal
        ///     }
        ///
        ///     let selectedSize = PaperSize.Letter
        ///     print(selectedSize.rawValue)
        ///     // Prints "Letter"
        ///
        ///     print(selectedSize == PaperSize(rawValue: selectedSize.rawValue)!)
        ///     // Prints "true"
        public let rawValue: String

        /// Creates a new instance with the specified raw value.
        ///
        /// If there is no value of the type that corresponds with the specified raw
        /// value, this initializer returns `nil`. For example:
        ///
        ///     enum PaperSize: String {
        ///         case A4, A5, Letter, Legal
        ///     }
        ///
        ///     print(PaperSize(rawValue: "Legal"))
        ///     // Prints "Optional(PaperSize.Legal)"
        ///
        ///     print(PaperSize(rawValue: "Tabloid"))
        ///     // Prints "nil"
        ///
        /// - Parameter rawValue: The raw value to use for the new instance.
        public init(rawValue: String)

        public init(_ rawValue: String)

        /// The raw type that can be used to represent all values of the conforming
        /// type.
        ///
        /// Every distinct value of the conforming type has a corresponding unique
        /// value of the `RawValue` type, but there may be values of the `RawValue`
        /// type that don't have a corresponding value of the conforming type.
        @available(macOS 27.0, iOS 27.0, tvOS 27.0, *)
        public typealias RawValue = String
    }

    /// A value that describes the source of a video asset's data.
    ///
    /// ``MediaIntelligenceVideoAsset/Kind`` describes how the framework reads
    /// a video.
    public enum Kind : Sendable {

        /// A video located at the specified URL.
        case url(URL)
    }

    /// A unique identifier for the asset.
    public let id: MediaIntelligenceVideoAsset.ID

    /// A value that describes how the framework accesses the video data.
    public let kind: MediaIntelligenceVideoAsset.Kind

    /// Creates a video asset with the specified identifier and kind.
    ///
    /// - Parameters:
    ///   - id: A unique identifier for the asset.
    ///   - kind: A value that describes how the framework accesses the video data.
    public init(id: MediaIntelligenceVideoAsset.ID, kind: MediaIntelligenceVideoAsset.Kind)
}
extension

MIManagedFace

NewiOSmacOStvOS
extension MIManagedFace
Declaration
extension MIManagedFace {

    @nonobjc public class func fetchRequest() -> NSFetchRequest<MIManagedFace>

    @objc @NSManaged nonisolated dynamic public var assetID: String

    @objc @NSManaged nonisolated dynamic public var assetUUID: String

    @objc @NSManaged nonisolated dynamic public var entityID: String?

    @objc @NSManaged nonisolated dynamic public var faceHeight: Float

    @objc @NSManaged nonisolated dynamic public var faceID: String

    @objc @NSManaged nonisolated dynamic public var facePixels: Data?

    @objc @NSManaged nonisolated dynamic public var faceWidth: Float

    @objc @NSManaged nonisolated dynamic public var faceX: Float

    @objc @NSManaged nonisolated dynamic public var faceY: Float

    @objc @NSManaged nonisolated dynamic public var vuObservationID: Int64

    /** Attribute name constants for use in NSPredicate and fetch requests.
     *  Using #keyPath provides compile-time verification that attributes exist. */
    public enum AttributeName {

        public static let assetID: String

        public static let assetUUID: String

        public static let entityID: String

        public static let faceHeight: String

        public static let faceID: String

        public static let facePixels: String

        public static let faceWidth: String

        public static let faceX: String

        public static let faceY: String

        public static let vuObservationID: String
    }
}
extension

MIManagedFace

NewiOSmacOStvOS
extension MIManagedFace : Identifiable
Declaration
extension MIManagedFace : Identifiable {

    /// A type representing the stable identity of the entity associated with
    /// an instance.
    @available(macOS 27.0, iOS 27.0, tvOS 27.0, *)
    public typealias ID = ObjectIdentifier
}
typealias

Entity.ID.RawValue

NewiOSmacOStvOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

typealias

Face.ID.RawValue

NewiOSmacOStvOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

typealias

MediaIntelligenceImageAsset.ID.RawValue

NewiOSmacOStvOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

typealias

MediaIntelligenceVideoAsset.ID.RawValue

NewiOSmacOStvOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

typealias

MIManagedFace.ID

NewiOSmacOStvOS
public typealias ID = ObjectIdentifier

A type representing the stable identity of the entity associated with an instance.

typealias

State.RawValue

NewiOSmacOStvOS
public typealias RawValue = Int

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

No APIs match your filter.

← More in Apple Intelligence, ML & Evaluation