General

• New in iOS 11.0 - Support for binary (non-text) barcodes.

  • Added APIs to AV Foundation, Core Image, and SiriKit to support detection, decoding, and creation of barcodes with binary content.

  • Added CIBarcodeDescriptor, a new barcode descriptor object to Core Image to provide interoperability with AV Foundation and the Vision APIs.

App Frameworks

• New in iOS 11.0 - Support for drag and drop.

  • Drag and drop in iOS lets a user drag items from one location to another onscreen; either within a single app or in different apps.

  • Added functionally to identify views as a drag sources and destinations.

  • Added customizable previews and set-down animations.

  • For more information, see Drag and Drop in the documentation.

• New in iOS 11.0 - Browsing local and iCloud documents.

  • Added view controllers for browsing documents stored locally and in the cloud. For more information, see UIDocumentBrowserViewController and UIDocumentBrowserTransitionController.

  • Added UIDocumentBrowserAction, an object for creating a custom action for the document browser.

  • Added the FileProvider and FileProviderUI frameworks for adding third party storage services.

  • For more information, see Adding a Document Browser to Your App.

• Improved Dynamic Type Support.

  • Added UIFontMetrics, an object for creating custom fonts that scale based on the currently selected text size.

  • Updated Auto Layout to dynamically adjust spacing based on the font sizes when a baseline anchor is part of a constraint that uses the system spacing.

  • Added an attribute to preserve vector data for PDF assets to enable smooth scaling. This can be used to show a larger version of bar items and segmented control items, as well as to adjust image sizes to match the user's text size. In the asset catalog, select "Preserve vector data" for the PDF to enable scaling.

  • Added UIAccessibilityContentSizeCategoryImageAdjusting, a protocol for scaling images for accessibility text sizes.

• Improved Auto Layout support for Dynamic Type.

  • Updated NSLayoutXAxisAnchor and NSLayoutYAxisAnchor to provide factory methods that create constraints using the system spacing between two anchors. Previously the only way to create such a constraint was with the dash (-) in the Visual Format Language.

  • Added an option to NSLayoutFormatOptions for creating a Visual Format Language string that uses baseline-to-baseline spacing.

  • Updated UIStackView to enable system spacing and customized spacing.

• Updated text content to work with App Password autofill.

  • Added username and password properties to UITextContentType.

• Updates to keyboard extensions.

  • Added selectedText, a property of UITextDocumentProxy that returns the currently selected text in the document.

  • Added documentIdentifier, a property of UITextDocumentProxy that specifies if the user is navigating to a new text widget.

  • Added hasFullAccess, a property of UIInputViewController for checking keyboard permissions.

  • Added needsInputModeSwitchKey, a property of UIInputViewController to control the display of the input-mode switch key.

  • Added new system permissions in Settings for app access to included Keyboard Extensions.

• Improved Available Storage Space API.

  • Added new keys to URL for different usage scenarios.

  • volumeAvailableCapacityForImportantUsageKey returns total amount of bytes available for operations explicitly requested by user or essential to proper functioning of your apps.

  • volumeAvailableCapacityForOpportunisticUsageKey returns total amount of bytes available for storing non-essential items, such as content pre-downloaded for performance that may or may not get used by the user.

Graphics and Games

• New in iOS 11.0 - Support for augmented reality.

  • Added the ARKit framework that combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.

• New in iOS 11.0 - High performance image analysis.

  • Added the Vision framework for detecting faces, barcodes, text, image horizon, and rectangular regions.

  • Integrates with Core ML to run custom models on images.

  • Tracks objects in video.

  • Includes support for image registration.

• New in iOS 11.0 - Write custom image blending kernels for Core Image.

  • Added CIBlendKernel, a special type of CIColorKernel to blend two images (supported by CIRenderDestination and CIImageAccumulator).

  • Added init(functionName:fromMetalLibraryData:) to CIKernel for writing kernels using Metal to benefit from the modern language features and the reduced compile time.

• New in iOS 11.0 - Lightweight render destination.

  • Added CIRenderDestination, an object for creating renderers that return to the caller after the work has been issued. Functionality includes specifying all the destination attributes of the renderer for different destinations including IOSurface, CVPixelBuffer, GL textures, Metal textures, and memory.

• Extended ReplayKit framework.

  • Updated RPScreenRecorder for screen capture and back camera support.

• Added new Core Image filters CITextImageGenerator, CIColorCurves, CILabDeltaE, CIBokehBlur, CIMinMaxRed, and CIBicubicScaleTransform.

Metal 2

Metal 2 contains significant additions and updates to Metal, the Metal Shading Language, and the Metal Performance Shaders framework. Items below indicate where the updates occur:

  – MTL: An update in the Metal framework.

  – MSL: An update in the Metal Shading Language.

  – MPS: An update in the Metal Performance Shaders framework.

• MPS: New in Metal 2 - Cross-platform Metal Performance Shaders support.

  • All of the Metal Performance Shaders functionality is available in iOS 11.0, tvOS 11.0, and macOS 10.13.

• MPS: New in iOS 11.0 - Neural network support.

  • Added support for neural networks to the Metal Performance Shaders framework.

  • Added graphs to offer a higher level API for simplifying the creation of neural networks, including objects that allow state to be transferred between nodes in a neural network.

  • Added Convolutional Neural Networks (CNN) to support implementing and running deep learning using previously obtained training data.

  • Added Recurrent Neural Networks for implementing inference on images and matrices.

• New in iOS 11.0 - Indirect Argument Buffers.

  • Group your resources into an indirect argument buffer (IAB) to reduce CPU overhead.

  • MSL: Added the [[id(n)]] attribute qualifier to identify resources in an IAB structure.

  • MTL: Added the MTLIndirectArgumentEncoder protocol to encode resources into an IAB.

• MTL: New in iOS 11.0 - Programmable sample positions.

  • Configure the position of samples when rendering to a multisampled render target.

  • Updated the MTLRenderPassDescriptor class to set and get sample positions for a render pass.

• MSL: New in iOS 11.0 - Uniform type.

  • Added the uniform type to declare variables that are uniform for all threads executing the graphics or compute function of a draw or dispatch call.

• MSL: New in iOS 11.0 - Array of samplers.

  • Added the array<sampler, size_t N> type to store an array of samplers.

• MPS: Added new filters.

  • Added filters for image statistics, such as computing the mean and variance for an image region.

  • Added filters for combining two images together, such as an element-wise sum and finding key points.

  • Added filters for matrix decomposition and solving, such as decomposition using Cholesky or LU (Lower Upper) factorization.

• MSL: Extended function specialization.

  • Extended [[color(n)]] and [[raster_order_group(index)]] attribute qualifiers to work with function constants.

  • Members of a structure used in a graphics, compute, or user function can be used with function constants.

• MTL: Extended vertex formats.

  • Added new MTLVertexFormat values for small formats such as char, short, and half.

• Added dual-source blending support to iOS.

  • Output two source colors for a fixed-function blending operation to a single render target.

  • MSL: Added a new [[index(i)]] attribute qualifier to the [[color(n)]] attribute qualifier to output a second source color.

  • MTL: Updated MTLBlendFactor to operate on a second source color.

App Services

• New in iOS 11.0 - Support for machine learning models.

  • Added the Core ML framework for easily integrating machine learning models into apps.

• New in iOS 11.0 - SiriKit support for visual codes.

  • Added the Visual Codes domain to Sirkit to support showing visual codes for exchanging payment and contact information.

• New in iOS 11.0 - SiriKit support for notes and to-do lists.

  • Added the Lists and Notes domain to SiriKit to support using Siri for adding notes, interacting with to-do lists, and interacting with reminders.

• Added intents in SiriKit domains.

  • Added ride canceling and feedback to the Ride Booking domain.

  • Added transferring money and searching for accounts to the Payments domain.

• New in iOS 11.0 - Find the heading of the device.

  • Added heading to CMDeviceMotion, a property that returns the heading angle with respect to the CMAttitudeReferenceFrame. The returned value is the heading in degrees as a double. A negative value is returned when the reference frame is xArbitraryZVertical or xArbitraryCorrectedZVertical.

• New in iOS 11.0 - Multipath TCP.

  • Added support for using multiple interfaces, such as Wi-Fi and Cellular, for the transmission of a single data stream by extending URLSessionConfiguration to support Multipath TCP as defined in IETF RFC 6824. For more information, see URLSessionConfiguration.MultipathServiceType.

• New in iOS 11.0 - DNS Proxy.

  • Added a new DNS Proxy app extension type to the Network Extension framework.

• Enhanced end user transaction flow in Apple Pay.

  • Added PKPaymentError to PassKit, a structure for detailed reporting of errors in a users' shipping and payment information, and for authorization errors. Developers can use the information to provide a customized error string.

  • Updated the handler methods in PKPaymentAuthorizationControllerDelegate to receive a PKPaymentError.

  • Updated PKPaymentRequest to use PKContactField for contact information.

  • Added supportedCountries to PKPaymentRequest for specifying supported countries for a transaction.

  • You can present payment buttons even if there are no supported payment methods in Wallet. Apple Pay now provisions payment without leaving your app, and then returns to checkout.

• New in iOS 11.0 - Promoting in-app purchases on the App Store.

  • Developers can promote up to 20 in-app purchases on their App Store product page. Customers can start their purchase on the App Store, and then be taken to the app to complete the transaction.

  • Added paymentQueue(_:shouldAddStore:for:), a new method of SKPaymentTransactionObserver for promoted in-app purchases. Apps need to support this delegate for promoted in-app purchases to display on the App Store.

• New in iOS 11.0 - Live messages.

  • Added MSMessageLiveLayout, a new message layout for showing live messages that can display dynamic content, such as games. Each live message has its own MSMessagesAppViewController, and there can be multiple active live messages on the screen at once.

    The following code shows adding a live message to a message stream including an alternate layout for devices that do not support live messages.

    guard let conversation = activeConversation else {

    fatalError("No active conversation")

    }

    let alternateLayout = MSMessageTemplateLayout()

    alternateLayout.image = UIImage(named: "SuperSweetGameImage")

    alternateLayout.caption = "$(\(conversation.localParticipantIdentifier)) wants to play a game!"

    let layout = MSMessageLiveLayout(alternateLayout: alternateLayout)

    let message = MSMessage()

    message.layout = layout

    conversation.insert(message, completionHandler: nil)

• Enhanced triggers for HomeKit.

  • Enhanced time-based conditions for triggers. HMSignificantTimeEvent specifies an offset from sunrise and sunset. HMCalendarEvent specifies a date and time. HMDurationEvent specifies a time interval.

  • Added HMCharacteristicThresholdRangeEvent to support tracking the state of an accessory within a range, such as running an automation when the temperature is between 68 and 72 degrees.

  • Added HMPresenceEvent for adding a condition based on the presence or absence of users.

  • Updated HMEventTrigger to enable multiple recurrences of the event.

• Receive updates of the home hub state by implementing home:didUpdateHomeHubState:..

• Updated MapKit for clearer display of developer data.

  • Added mutedStandard, a new map display mode that emphasizes developer data.

  • Added properties to customize how annotations behave when collisions occur. Developers use a combination of displayPriority, collisionMode, and clusteringIdentifier to influence which annotations remain on the map.

Media and Web

• New in iOS 11.0 - Support for High Efficiency Video Coding (HEVC).

  • High Efficiency Video Coding (HEVC) is a new standard for video encoding that offers substantially better compression than H.264 at the same level of visual quality.

  • Use AV Foundation to playback movies containing HEVC encoded tracks, and to capture and export videos.

  • VideoToolbox clients can encode and decode HEVC video bitstreams.

• New in iOS 11.0 - Support for High Efficiency Image Format (HEIF).

  • High Efficiency Image Format (HEIF) is a new standard of image compression that nearly doubles current data compression ratios for the same level of image quality.

  • Added functionality to the Photos and Core Image frameworks to display, encode, and export HEIF images.

• New in iOS 11.0 - Support for capturing and manipulating Depth data, and enhanced photo capture.

  • Added objects to AV Foundation for capturing and representing depth data. For more information, see AVCaptureDepthDataOutput, AVDepthData, and related APIs.

  • Added AVCapturePhoto, an object that encapsulates the information for a captured photo and supports HEVC and HEIC encoded images.

  • Updated AVCapturePhotoOutput to provide more information.

• New in iOS 11.0 - Automatic storage management.

  • Added automatic storage management of HTTP live streaming assets to AVAssetDownloadTask. The system can automatically purge expired or unneeded downloads when space is required. Use priorities to influence the purging policy.

• New in iOS 11.0 - AirPlay 2.

  • Improved AirPlay reliability for some audio playback interfaces in AV Foundation. To take advantage of the increased reliability, play audio using AVPlayer or the new AVSampleBufferAudioRenderer object.

  • Added multiple speaker support to AirPlay for long-form audio, such as music and podcasts. To mark your application as presenting long-form audio, invoke the AVAudioSession method setCategory(_:mode:routeSharingPolicy:options:) and use AVAudioSessionRouteSharingPolicyLongForm as the parameter value.

• FairPlay streaming key management.

  • Improved the functionality of AVContentKeySession. Use AVContentKeySession to initiate content key requests independent of playback or downloading of media assets. Objects conforming to the AVContentKeyRecipient protocol, such as AVURLAsset, can be added as a recipient to AVContentKeySession to obtain access to existing content keys and initiate new content key requests

• Added more Live Photo adjustments.

  • Added a collection of Live Photo adjustments, called effects, that render the live photo as an Loop, Bounce, or Long Exposure. Unlike regular live photos, Loop and Bounce videos will play in a continuous loop.

  • Added a playbackStyle, a new property that identifies how to present the PHAsset to the user.

System

• New in iOS 11.0 - Hotspot configuration.

  • Added a network extension for hotspot configuration. For more information, see NEHotspotConfiguration.

• Updated the Core Bluetooth framework.

  • Added support for L2CAP Channels.

  • Extended session restoration to work across Bluetooth resets and device reboots.

  • Updated the Core Bluetooth framework to match across iOS, tvOS, watchOS, and macOS, and marked call availability based on platform.

• New in iOS 11.0 - Detect NFC tags and read messages that contain NDEF data.

  • Added Core NFC, a new framework for reading Near Field Communications (NFC) tags and data in NFC Data Exchange Format (NDEF.)

• APFS is now the default filesystem.

  • APFS now has normalization-insensitive support for a case sensitive filesystem.