What's New / Graphics & Metal

What's new in CoreImage

+11 NewiOS · macOS · tvOS

Core Image is Apple's framework for GPU-accelerated image processing. It provides built-in filters, custom kernels, and processing pipelines that operate on CIImage data backed by Metal.

The 27 SDK adds 11 APIs with no deprecations or removals. New CIImageOption keys cover image loading: subsampleFactor, typeIdentifierHint, and useHardwareAcceleration. CIImageProcessorKernel gains apply, CIImageProcessorOutput adds temporarySurface and temporaryPixelBuffer. Pipeline reporting fields plannedPixelsProcessed, plannedPixelsOverdrawn, and plannedPassCount are added, along with supportedCameraModels.

New

11
var

plannedPassCount

NewiOSmacOStvOS
open var plannedPassCount: Int { get }
var

plannedPeakMemory

NewiOSmacOStvOS
open var plannedPeakMemory: Int { get }
var

plannedPixelsOverdrawn

NewiOSmacOStvOS
open var plannedPixelsOverdrawn: Int { get }
var

plannedPixelsProcessed

NewiOSmacOStvOS
open var plannedPixelsProcessed: Int { get }
func

supportedCameraModels

NewiOSmacOStvOS
open class func supportedCameraModels(with version: CIRAWDecoderVersion) -> [String]
let

CIImageOption.subsampleFactor

NewiOSmacOStvOS
public static let subsampleFactor: CIImageOption

The factor by which to scale down a returned images.

The value of this key should be an NSNumber containing the integer value 2, 4, or 8. It can be used to improve performance and reduce memory usage when working with large images.

When you specify this key, the retured image will be scaled down the image data by the specified numerical factor. If the image format doesn’t support the specified scale factor, a larger or full-size normal image is returned.

This option is only supported by JPEG, HEIF, TIFF, PNG and RAW images formats.

This option is only supported by these APIs:

  • imageWithContentsOfURL:options:
  • initWithContentsOfURL:options:
  • imageWithData:options:
  • initWithData:options:
  • imageWithCGImageSource:index:options:
  • initWithCGImageSource:index:options:

Note: the kCGImageSourceSubsampleFactor key can also be used for this purpose.

let

CIImageOption.typeIdentifierHint

NewiOSmacOStvOS
public static let typeIdentifierHint: CIImageOption

The uniform type identifier string to use in cases where a file's format cannot be conclusively determined based solely on its contents.

The value of this key should be an NSString containing a hint. It is most commonly needed for some RAW file formats which can also be interpreted as TIFF files.

This option is only supported by these APIs:

  • imageWithContentsOfURL:options:
  • initWithContentsOfURL:options:
  • imageWithData:options:
  • initWithData:options:

Note: the key kCGImageSourceTypeIdentifierHint key can also be used for this purpose.

let

CIImageOption.useHardwareAcceleration

NewiOSmacOStvOS
public static let useHardwareAcceleration: CIImageOption

A Boolean value specifying that using hardware is preferred when decoding.

If the value for this option is:

  • True: The image will be decoded using dedicated hardware if possible.
  • False: The image will be decoded using the CPU is possible.
  • Not specified: The default behavior is True.

This option is only supported by JPEG and HEIF images formats.

This option is only supported by these APIs:

  • imageWithContentsOfURL:options:
  • initWithContentsOfURL:options:
  • imageWithData:options:
  • initWithData:options:
  • imageWithCGImageSource:index:options:
  • initWithCGImageSource:index:options:

Note: the kCGImageSourceUseHardwareAcceleration key can also be used for this purpose.

func

CIImageProcessorKernel.apply

NewiOSmacOStvOS
open class func apply(withTiledExtent tileExtents: [CIVector], inputs: [CIImage]?, arguments args: [String : Any]?) throws -> CIImage

Call this method on your Core Image Processor Kernel subclass to create a new image based on an array of tile extents that together cover the output.

Each tile is a CGRect encoded as a CIVector using +[CIVector vectorWithCGRect:]. The overall output extent is computed as the union of all tile extents.

This method will return nil and an error if:

  • calling outputFormat on your subclass returns an unsupported format.
  • calling formatForInputAtIndex: on your subclass returns an unsupported format.
  • your subclass does not implement processWithInputs:arguments:output:error:

Parameters

tileExtents
The array of bounding rectangles that the CIImageProcessorKernel can produce. Each rectangle in the array is an object created using vectorWithCGRect: This method will return CIImage.emptyImage if the rectangles in the array have gaps or overlaps.
inputs
An array of CIImage objects to use as input.
arguments
This dictionary contains any additional parameters that the processor needs to produce its output. The argument objects can be of any type but in order for CoreImage to cache intermediates, they must be of the following immutable types: NSArray, NSDictionary, NSNumber, NSValue, NSData, NSString, NSNull, CIVector, CIColor, CGImage, CGColorSpace, or MLModel.
error
Pointer to the NSError object into which processing errors will be written.

ReturnsAn autoreleased CIImage

func

CIImageProcessorOutput.temporaryPixelBuffer

NewiOSmacOStvOS
func temporaryPixelBuffer(withIdentifier identifier: String, format: OSType, width: Int, height: Int, attributes: [AnyHashable : Any]? = nil) -> Unmanaged<CVPixelBuffer>?

Returns a temporary CVPixelBuffer that your Core Image Processor Kernel can use as scratch storage during processing.

Use this method when your processor needs an intermediate CVPixelBuffer to stash data between stages of its work. Core Image manages the lifetime of the returned buffer and reuses the underlying allocation across multiple invocations when possible. This is more efficient than allocating a fresh CVPixelBuffer on each invocation of your processor.

The returned pixel buffer is valid only for the duration of the processWithInputs:arguments:output:error: call that requested it. Don't retain it beyond the scope of that method or use it after the method returns.

Calling this method multiple times within the same processor invocation with the same identifier, format, width, and height returns a pixel buffer backed by the same IOSurface. Otherwise it returns a distinct pixel buffer. This lets a processor request several independent pixel buffers by giving each one a unique name.

Parameters

identifier
A name that uniquely identifies this scratch buffer within the processor invocation.
format
The pixel format for the buffer. Must be a non-zero OSType pixel format constant.
width
The width of the buffer in pixels. Must be greater than zero.
height
The height of the buffer in pixels. Must be greater than zero.
attributes
An optional dictionary of CVPixelBuffer creation attributes.

ReturnsA non-retained CVPixelBuffer of the requested size and format, or nil if the buffer could not be created.

func

CIImageProcessorOutput.temporarySurface

NewiOSmacOStvOS
func temporarySurface(withIdentifier identifier: String, format: OSType, width: Int, height: Int) -> IOSurface?

Returns a temporary IOSurface that your Core Image Processor Kernel can use as scratch storage during processing.

Use this method when your processor needs an intermediate IOSurface to stash data between stages of its work. Core Image manages the lifetime of the returned surface and reuses the underlying allocation across multiple invocations when possible. This is more efficient than allocating a fresh IOSurface on each invocation of your processor.

The returned surface is valid only for the duration of the processWithInputs:arguments:output:error: call that requested it. Don't retain it beyond the scope of that method or use it after the method returns.

Calling this method multiple times within the same processor invocation with the same identifier, format, width, and height returns the same surface. Otherwise it returns a distinct surface. This lets a processor request several independent surfaces by giving each one a unique name.

Parameters

identifier
A name that uniquely identifies this scratch surface within the processor invocation.
format
The pixel format for the surface. Must be a non-zero OSType pixel format constant.
width
The width of the surface in pixels. Must be greater than zero.
height
The height of the surface in pixels. Must be greater than zero.

ReturnsAn autoreleased IOSurface of the requested size and format, or nil if the surface could not be created.

No APIs match your filter.

← More in Graphics & Metal