What's New / Media, Audio & Capture

What's new in CoreMedia

+29 NewiOS · macOS · tvOS · watchOS

CoreMedia defines the low-level media types and time representations shared by audio and video pipelines on Apple platforms, including sample buffers, format descriptions, CMTime-based timing, and the attachment metadata carried with media samples.

29 new APIs, no deprecations or removals. New types include CMCameraIntrinsicMatrix, CMCustomNotificationInfo, the enums CMDroppedFrameReason and CMStillImageLensStabilization, and the attachment-key enums CMBlockBufferAttachmentKeyDefinitions and CMSampleBufferAttachmentKeyDefinitions. A new attachment-access surface adds CMAttachmentAccess, CMAttachmentContainer, CMAttachmentKeyDefinitions, and CMAttachmentPropagationMode. Other additions cover CMTime and CMTimeRange plus metadata identifier and base-data-type constants such as kCMMetadataIdentifier_ITUT_T35MetadataSMPTE2094_50, kCMMetadataBaseDataType_MacRoman, and kCMMetadataBaseDataType_ISOLatin1.

New

29
typealias

CMAttachmentAccess

NewiOSmacOStvOSwatchOS
public typealias CMAttachmentAccess = CVAttachmentAccess
typealias

CMAttachmentContainer

NewiOSmacOStvOSwatchOS
public typealias CMAttachmentContainer = CVAttachmentContainer
typealias

CMAttachmentKeyDefinitions

NewiOSmacOStvOSwatchOS
public typealias CMAttachmentKeyDefinitions = CVAttachmentKeyDefinitions
typealias

CMAttachmentPropagationMode

NewiOSmacOStvOSwatchOS
public typealias CMAttachmentPropagationMode = CVAttachmentMode
typealias

CMAttachmentRawValue

NewiOSmacOStvOSwatchOS
public typealias CMAttachmentRawValue = CVAttachmentRawValue
typealias

CMAttachmentValueRepresentable

NewiOSmacOStvOSwatchOS
public typealias CMAttachmentValueRepresentable = CVAttachmentValueRepresentable
enum

CMBlockBufferAttachmentKeyDefinitions

NewiOSmacOStvOSwatchOS
public enum CMBlockBufferAttachmentKeyDefinitions : CMAttachmentKeyDefinitions

A namespace for block buffer attachment keys.

Declaration
public enum CMBlockBufferAttachmentKeyDefinitions : CMAttachmentKeyDefinitions {
}
struct

CMCameraIntrinsicMatrix

NewiOSmacOStvOSwatchOS
public struct CMCameraIntrinsicMatrix : Sendable, Equatable, CMAttachmentValueRepresentable

A matrix that describes the camera's intrinsic properties.

This matrix has the following content

 fx  0   ox
 0   fy  oy
 0   0   1

fx and fy are the focal length in pixels. For square pixels, they will have the same value. ox and oy are the coordinates of the principal point. The origin is the upper left of the frame.

Declaration
public struct CMCameraIntrinsicMatrix : Sendable, Equatable, CMAttachmentValueRepresentable {

    public var matrix: simd_float3x3

    public init(matrix: simd_float3x3)

    public static func makeFromRawAttachmentValue(_ repr: CMAttachmentRawValue) -> CMCameraIntrinsicMatrix?

    public var rawAttachmentValueRepresentation: CMAttachmentRawValue { get }

    /// Returns a Boolean value indicating whether two values are equal.
    ///
    /// Equality is the inverse of inequality. For any values `a` and `b`,
    /// `a == b` implies that `a != b` is `false`.
    ///
    /// - Parameters:
    ///   - lhs: A value to compare.
    ///   - rhs: Another value to compare.
    public static func == (a: CMCameraIntrinsicMatrix, b: CMCameraIntrinsicMatrix) -> Bool
}
struct

CMCustomNotificationInfo

NewiOSmacOStvOSwatchOS
public struct CMCustomNotificationInfo : Sendable, ExpressibleByDictionaryLiteral, CMAttachmentValueRepresentable

Holds custom userInfo for notification sent for a buffer event.

Declaration
public struct CMCustomNotificationInfo : Sendable, ExpressibleByDictionaryLiteral, CMAttachmentValueRepresentable {

    public var userInfo: [String : any Sendable]

    public init(userInfo: [String : any Sendable])

    /// Creates an instance initialized with the given key-value pairs.
    public init(dictionaryLiteral elements: (String, any Sendable)...)

    public static func makeFromRawAttachmentValue(_ repr: CVAttachmentRawValue) -> CMCustomNotificationInfo?

    public var rawAttachmentValueRepresentation: CVAttachmentRawValue { get }

    /// The key type of a dictionary literal.
    @available(macOS 27.0, iOS 27.0, tvOS 27.0, watchOS 27.0, *)
    public typealias Key = String

    /// The value type of a dictionary literal.
    @available(macOS 27.0, iOS 27.0, tvOS 27.0, watchOS 27.0, *)
    public typealias Value = any Sendable
}
enum

CMDroppedFrameReason

NewiOSmacOStvOSwatchOS
public enum CMDroppedFrameReason : String, Sendable, Hashable, CMAttachmentValueRepresentable
Declaration
public enum CMDroppedFrameReason : String, Sendable, Hashable, CMAttachmentValueRepresentable {

    /// The frame was dropped because it was late.
    ///
    /// When a video capture client has indicated that late video frames should be dropped and the current frame is
    /// late. This condition is typically caused by the client's processing taking too long.
    case frameWasLate

    /// The frame was dropped because the module providing frames is out of buffers.
    ///
    /// When the module providing sample buffers has run out of source buffers. This condition is typically caused by
    /// the client holding onto buffers for too long and can be alleviated by returning buffers to the provider by
    /// releasing the buffers.
    case outOfBuffers

    /// An unknown number of frames were dropped.
    ///
    /// When the module providing sample buffers has experienced a discontinuity, and an unknown number of frames have
    /// been lost. This condition is typically caused by the system being too busy.
    case discontinuity

    /// Creates a new instance with the specified raw value.
    ///
    /// If there is no value of the type that corresponds with the specified raw
    /// value, this initializer returns `nil`. For example:
    ///
    ///     enum PaperSize: String {
    ///         case A4, A5, Letter, Legal
    ///     }
    ///
    ///     print(PaperSize(rawValue: "Legal"))
    ///     // Prints "Optional(PaperSize.Legal)"
    ///
    ///     print(PaperSize(rawValue: "Tabloid"))
    ///     // Prints "nil"
    ///
    /// - Parameter rawValue: The raw value to use for the new instance.
    public init?(rawValue: String)

    /// The corresponding value of the raw type.
    ///
    /// A new instance initialized with `rawValue` will be equivalent to this
    /// instance. For example:
    ///
    ///     enum PaperSize: String {
    ///         case A4, A5, Letter, Legal
    ///     }
    ///
    ///     let selectedSize = PaperSize.Letter
    ///     print(selectedSize.rawValue)
    ///     // Prints "Letter"
    ///
    ///     print(selectedSize == PaperSize(rawValue: selectedSize.rawValue)!)
    ///     // Prints "true"
    public var rawValue: String { get }

    /// Provides additional information regarding the dropped video frame.
    public enum Info : String, Sendable, Hashable, CMAttachmentValueRepresentable {

        /// A discontinuity was caused by a camera mode switch.
        ///
        /// When the module providing sample buffers has experienced a discontinuity due to a camera mode switch. Short
        /// discontinuities of this type can occur when the session is configured for still image capture on some devices.
        case cameraModeSwitch

        /// Creates a new instance with the specified raw value.
        ///
        /// If there is no value of the type that corresponds with the specified raw
        /// value, this initializer returns `nil`. For example:
        ///
        ///     enum PaperSize: String {
        ///         case A4, A5, Letter, Legal
        ///     }
        ///
        ///     print(PaperSize(rawValue: "Legal"))
        ///     // Prints "Optional(PaperSize.Legal)"
        ///
        ///     print(PaperSize(rawValue: "Tabloid"))
        ///     // Prints "nil"
        ///

Truncated.

extension

CMDroppedFrameReason

NewiOSmacOStvOSwatchOS
extension CMDroppedFrameReason : RawRepresentable
Declaration
extension CMDroppedFrameReason : RawRepresentable {
}
extension

CMDroppedFrameReason.Info

NewiOSmacOStvOSwatchOS
extension CMDroppedFrameReason.Info : RawRepresentable
Declaration
extension CMDroppedFrameReason.Info : RawRepresentable {
}
extension

CMMutableDataBlockBuffer

NewiOSmacOStvOSwatchOS
extension CMMutableDataBlockBuffer
Declaration
extension CMMutableDataBlockBuffer {

    public typealias Attachments = CVAttachmentAccess<CMBlockBufferAttachmentKeyDefinitions>

    public var attachments: CMMutableDataBlockBuffer.Attachments
}
extension

CMReadOnlyDataBlockBuffer

NewiOSmacOStvOSwatchOS
extension CMReadOnlyDataBlockBuffer
Declaration
extension CMReadOnlyDataBlockBuffer {

    public typealias Attachments = CVAttachmentAccess<CMBlockBufferAttachmentKeyDefinitions>

    public var attachments: CMReadOnlyDataBlockBuffer.Attachments
}
extension

CMReadySampleBuffer

NewiOSmacOStvOSwatchOS
extension CMReadySampleBuffer
Declaration
extension CMReadySampleBuffer {

    public typealias Attachments = CVAttachmentAccess<CMSampleBufferAttachmentKeyDefinitions>

    public var attachments: CMReadySampleBuffer<Content>.Attachments
}
enum

CMSampleBufferAttachmentKeyDefinitions

NewiOSmacOStvOSwatchOS
public enum CMSampleBufferAttachmentKeyDefinitions : CMAttachmentKeyDefinitions

A namespace for sample buffer attachment keys.

Declaration
public enum CMSampleBufferAttachmentKeyDefinitions : CMAttachmentKeyDefinitions {

    /// Marks an intentionally empty interval in the sequence of samples.
    ///
    /// The sample buffer’s output presentation timestamp indicates when the empty interval begins. Marker sample
    /// buffers with this attachment are used to announce the arrival of empty edits. The getter returns the default
    /// value of false if this attachment is not present.
    public static let emptyMedia: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Marks the end of the sequence of samples.
    ///
    /// Marker sample buffers with this attachment in addition to ``emptyMedia`` are used to indicate that no further
    /// samples are expected. The getter returns the default value of false if this attachment is not present.
    public static let permanentEmptyMedia: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Indicates that the empty marker should be dequeued immediately regardless of its timestamp.
    ///
    /// Marker sample buffers with this attachment in addition to ``emptyMedia`` are used to tell that the empty sample
    /// buffer should be dequeued immediately regardless of its timestamp. This attachment should only be used with
    /// sample buffers with the ``emptyMedia`` attachment. The getter returns the default value of false if this
    /// attachment is not present.
    public static let displayEmptyMediaImmediately: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Indicates whether the sample buffer should be drained after decoding.
    ///
    /// This attachment is used at run time to indicate that a sample precedes a break in decode sequence and that it is
    /// appropriate to drain the decoder after decoding this sample. This attachment is not written to media files. The
    ///  getter returns the default value of false if this attachment is not present.
    public static let drainAfterDecoding: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Indicates the reason the current video frame was dropped.
    ///
    /// Sample buffers with this attachment contain no image or data buffer. They mark a dropped video frame. This
    /// attachment identifies the reason the frame was dropped.
    public static let droppedFrameReason: CVAttachmentKeyDefinition<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, CMDroppedFrameReason>

    /// Indicates additional information regarding the dropped video frame.
    ///
    /// Sample buffers with this attachment contain no image or data buffer. They mark a dropped video frame. If
    /// present, this attachment provides additional information about the reason described by the
    /// ``droppedFrameReason`` key.
    public static let droppedFrameReasonInfo: CVAttachmentKeyDefinition<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, CMDroppedFrameReason.Info>

    /// Indicates that sample buffer’s decode timestamp may be used to define the previous sample buffer’s duration.
    ///
    /// Marker sample buffers with this attachment may be used in situations where sample buffers are transmitted before
    /// their duration is known. In such situations, normally the recipient may use each sample buffer’s timestamp to
    /// calculate the duration of the previous sample buffer. The marker sample buffer with this attachment is sent to
    /// provide the timestamp for calculating the final sample buffer’s duration. The getter returns the default value of
    /// false if this attachment is not present.
    public static let endsPreviousSampleDuration: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Fill the difference between discontiguous sample buffers with silence.
    ///
    /// If a sample buffer enters a buffer queue and the presentation time stamp between the previous buffer and the
    /// buffer with this attachment are discontiguous, handle the discontinuity by generating silence for the time
    /// difference. The getter returns the default value of false if this attachment is not present.
    public static let fillDiscontinuitiesWithSilence: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Provides the 3x3 camera intrinsic matrix applied to the current sample buffer.
    public static let cameraIntrinsicMatrix: CVAttachmentKeyDefinition<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, CMCameraIntrinsicMatrix>

    /// Indicates that the current or next video sample buffer should be forced to be encoded as a key frame.
    ///
    /// If this attachment is `true` on a sample buffer with a video frame, that video frame will be forced to become
    /// a key frame. If the sample buffer for which this is `true` does not have a valid video frame, the next sample
    /// buffer processed that contains a valid video frame will be encoded as a key frame. The getter returns the
    /// default value of false if this attachment is not present.
    ///
    /// Unless the origin and destination of a sample buffer is known, the general recommended practice is to synthesize
    /// an empty sample buffer with this attachment alone and insert it into the sample buffer stream ahead of the
    /// concrete sample buffer rather than setting this attachment on the concrete sample buffer itself.
    public static let forceKeyFrame: CVAttachmentKeyDefinitionWithDefault<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Bool>

    /// Indicates the decoder refresh count.
    ///
    /// Sample buffers with this attachment may be used to identify the audio decoder refresh count.
    public static let gradualDecoderRefresh: CVAttachmentKeyDefinition<CMSampleBufferAttachmentKeyDefinitions.ShouldPropagate, Int16>

    /// Indicates that decode pipelines should post a notification when consuming the sample buffer.

Truncated.

enum

CMStillImageLensStabilization

NewiOSmacOStvOSwatchOS
public enum CMStillImageLensStabilization : String, Sendable, Hashable, CMAttachmentValueRepresentable
Declaration
public enum CMStillImageLensStabilization : String, Sendable, Hashable, CMAttachmentValueRepresentable {

    /// The lens stabilization module was active for the duration this buffer.
    case active

    /// The motion of the device or duration of the capture was outside of what the stabilization mechanism could support.
    ///
    /// The value of kCMSampleBufferAttachmentKey_StillImageLensStabilizationInfo if the module stabilizing the lens
    /// was unable to compensate for the movement.
    case outOfRange

    /// The lens stabilization module was unavailable for use.
    ///
    /// The value of kCMSampleBufferAttachmentKey_StillImageLensStabilizationInfo if the lens stabilization module is
    /// unavailable to compensate for the motion of the device. The module may be available at a later time.
    case unavailable

    /// The lens stabilization module was not used during this capture.
    ///
    /// The value of kCMSampleBufferAttachmentKey_StillImageLensStabilizationInfo if the lens stabilization module was
    /// not used for this capture.
    case off

    /// Creates a new instance with the specified raw value.
    ///
    /// If there is no value of the type that corresponds with the specified raw
    /// value, this initializer returns `nil`. For example:
    ///
    ///     enum PaperSize: String {
    ///         case A4, A5, Letter, Legal
    ///     }
    ///
    ///     print(PaperSize(rawValue: "Legal"))
    ///     // Prints "Optional(PaperSize.Legal)"
    ///
    ///     print(PaperSize(rawValue: "Tabloid"))
    ///     // Prints "nil"
    ///
    /// - Parameter rawValue: The raw value to use for the new instance.
    public init?(rawValue: String)

    /// The corresponding value of the raw type.
    ///
    /// A new instance initialized with `rawValue` will be equivalent to this
    /// instance. For example:
    ///
    ///     enum PaperSize: String {
    ///         case A4, A5, Letter, Legal
    ///     }
    ///
    ///     let selectedSize = PaperSize.Letter
    ///     print(selectedSize.rawValue)
    ///     // Prints "Letter"
    ///
    ///     print(selectedSize == PaperSize(rawValue: selectedSize.rawValue)!)
    ///     // Prints "true"
    public var rawValue: String { get }

    /// The raw type that can be used to represent all values of the conforming
    /// type.
    ///
    /// Every distinct value of the conforming type has a corresponding unique
    /// value of the `RawValue` type, but there may be values of the `RawValue`
    /// type that don't have a corresponding value of the conforming type.
    @available(macOS 27.0, iOS 27.0, tvOS 27.0, watchOS 27.0, *)
    public typealias RawValue = String
}
extension

CMStillImageLensStabilization

NewiOSmacOStvOSwatchOS
extension CMStillImageLensStabilization : RawRepresentable
Declaration
extension CMStillImageLensStabilization : RawRepresentable {
}
extension

CMTime

NewiOSmacOStvOSwatchOS
extension CMTime : CVAttachmentValueRepresentable
Declaration
extension CMTime : CVAttachmentValueRepresentable {

    public static func makeFromRawAttachmentValue(_ repr: CVAttachmentRawValue) -> CMTime?

    public var rawAttachmentValueRepresentation: CVAttachmentRawValue { get }
}
extension

CMTimeRange

NewiOSmacOStvOSwatchOS
extension CMTimeRange : CVAttachmentValueRepresentable
Declaration
extension CMTimeRange : CVAttachmentValueRepresentable {

    public static func makeFromRawAttachmentValue(_ repr: CVAttachmentRawValue) -> CMTimeRange?

    public var rawAttachmentValueRepresentation: CVAttachmentRawValue { get }
}
let

kCMMetadataBaseDataType_ISOLatin1

NewiOSmacOStvOSwatchOS
public let kCMMetadataBaseDataType_ISOLatin1: CFString
let

kCMMetadataBaseDataType_MacRoman

NewiOSmacOStvOSwatchOS
public let kCMMetadataBaseDataType_MacRoman: CFString
let

kCMMetadataDataType_QuickTimeMetadataSMPTE2094_50

NewiOSmacOStvOSwatchOS
public let kCMMetadataDataType_QuickTimeMetadataSMPTE2094_50: CFString
let

kCMMetadataIdentifier_ITUT_T35MetadataSMPTE2094_50

NewiOSmacOStvOSwatchOS
public let kCMMetadataIdentifier_ITUT_T35MetadataSMPTE2094_50: CFString
typealias

CMCustomNotificationInfo.Key

NewiOSmacOStvOSwatchOS
public typealias Key = String

The key type of a dictionary literal.

typealias

CMCustomNotificationInfo.Value

NewiOSmacOStvOSwatchOS
public typealias Value = any Sendable

The value type of a dictionary literal.

typealias

CMDroppedFrameReason.RawValue

NewiOSmacOStvOSwatchOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

typealias

CMDroppedFrameReason.Info.RawValue

NewiOSmacOStvOSwatchOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

typealias

CMStillImageLensStabilization.RawValue

NewiOSmacOStvOSwatchOS
public typealias RawValue = String

The raw type that can be used to represent all values of the conforming type.

Every distinct value of the conforming type has a corresponding unique value of the RawValue type, but there may be values of the RawValue type that don't have a corresponding value of the conforming type.

No APIs match your filter.

← More in Media, Audio & Capture