709 Whats New in Core ML Part 2

#WWDC18
What’s New in Core ML

•
Part two
Aseem Wadhwa, Core ML

Sohaib Qureshi, Core ML
© 2018 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.
Core ML
Part One Recap
Float weights Quantized weights

•
Model size
•
Performance
•
Customization 60 MB 15 MB
•
Part One Recap
Model size
•
Performance
•
Customization
•
Part One Recap
Model size
•
Performance
•
Customization
•
Part One Recap
Model size
•
Performance
•
Customization
•
Part Two
Agenda
Agenda
Core ML Tools ecosystem

Agenda

•
Quantization utilities
Agenda

•
•
Custom conversion
•

•
•
Custom conversion
?
Download
https://developer.apple.com/machine-learning/
NEW
Create ML
Last Year
Core ML Tools
https://github.com/apple/coremltools
Last Year
Core ML Tools
Today
Core ML Tools
Today
Core ML Tools
TensorFlow Converter
https://github.com/tf-coreml/tf-coreml
In collaboration with Google

Support for custom layers

Support for Quantized TensorFlow models

(coming soon!)
https://github.com/onnx/onnx-coreml
In collaboration with Facebook and Prisma

https://github.com/onnx/onnx-coreml
•

•
•
Custom conversion
•
Core ML Tools 2.0 

Core ML Tools 2.0
https://github.com/apple/coremltools
Support for latest Core ML .mlmodel specification
Flexible shape utilities

Quantization Utilities
Core ML Tools
Float32 weights Quantize weights 1-8 bit weights
Post-training Quantization
Quantization
Peeking under the hood
W W W
Quantization
W
Quantization
W
Quantization
0.14
0.15
0.35
W 0.26
0.34
0.87
0.13
…
Weight Quantization
Min Max
Float weights
Weight Quantization
Min Max
Float weights
Quantized weights
Linear
Three-bit example
Min Max
Float weights
Quantized weights
0 1 2 3 4 5 6 7
Linear
Three-bit example
Min Max
Float weights
scale*quantized + bias
Quantized weights
0 1 2 3 4 5 6 7
Lookup Table
Three-bit example
Min Max
Float weights
Quantized weights
0 1 2 3 4 5 6 7
Lookup Table
Three-bit example
Min Max
Float weights
lookupTable[quantized]
Quantized weights
0 1 2 3 4 5 6 7
Core ML Tools

Core ML Tools
Decide on precision and algorithm

Core ML Tools
Decide on precision and algorithm
Let Core ML Tools work its magic!

•
Demo
Quantization in Core ML Tools
// Quantize model using KMeans Lookup Table
quantized_model = quantize_weights(model, 8, ‘kmeans’)
// Compare Quantized model with original

compare_model(model, quantized_model, ‘./sample_data/’)


Model Size Versus Agreement
Model 1
Agreement Model 2
Metric
Float32 One bit
Model Size
Model 1
Agreement Model 2
Metric
Float32 One bit
Model Size
Metrics
• Accuracy Model 1
• Signal-to-noise ratio Agreement Model 2

Metric
• Visual inspection
• Model-specific metrics
Float32 One bit
Model Size
32 bit - 6.7 MB 16 bit - 3.4 MB 8 bit (linear) - 1.7 MB
4 bit (linear) - 857 KB 3 bit (linear) - 647 KB 2 bit (linear) - 437 KB

•
Core ML tools ecosystem

•
•
Custom conversion
•
Custom Conversion
Aseem Wadhwa, Core ML

New Neural Network Layer
New Model Architecture

?
New Neural Network Layer Customization
New Model Architecture

Customization
New Neural Network Layer
Model Conversion
Converters: Simple API

Model Conversion
import coremltools
coremltools.converters.keras.convert(keras_model)
Model Conversion
import onnx_coreml
onnx_coreml.convert(onnx_model)
Model Conversion
import tfcoreml
tfcoreml.convert(tf_model_path=tf_model_path,
mlmodel_path=mlmodel_path,
output_feature_names=['output:0'])
Model Conversion
import tfcoreml
output_feature_names=[‘output:0'],
)
Model Conversion
import tfcoreml
output_feature_names=[‘output:0'],
)
Use Custom Layers!

•
Custom Layer Examples

Image Classifier
Beach
Opening the Hood
Beach
Neural Network
Convertible to Core ML
ReLU ReLU
Convolution Convolution Convolution
Activation Activation
Supported by Core ML
Not convertible to Core ML
New New
Not Supported by Core ML

Custom Layer
New New
Custom Layer in Core ML Model

Image Classifier
Beach
With Custom Activation Layer

Image Classifier
Beach
With Custom Activation Layer

Image Classifier
A Simple Classifier
Classifier Net
6
Localization Net
sha1_base64="8ziVlcrM346ng4sDFlX68r05NSY=">AAAB7nicjVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmQ2dllplcISz7CiwdFvPo93vwbJ4+DioIFDUVVN91dUaqkJd//8Aorq2vrG8XN0tb2zu5eef+gaZPMCGyIRCWmHXGLSmpskCSF7dQgjyOFrWh8PfNb92isTPQdTVIMYz7UciAFJye1ujRC4qVeuRJU/TnY36QCS9R75fduPxFZjJqE4tZ2Aj+lMOeGpFA4LXUziykXYz7EjqOax2jDfH7ulJ04pc8GiXGlic3VrxM5j62dxJHrjDmN7E9vJv7mdTIaXIa51GlGqMVi0SBTjBI2+531pUFBauIIF0a6W5kYccMFuYT+GULzrBr41eD2vFK7WsZRhCM4hlMI4AJqcAN1aICAMTzAEzx7qffovXivi9aCt5w5hG/w3j4B3uSPPw==</latexit>
sha1_base64="8ziVlcrM346ng4sDFlX68r05NSY=">AAAB7nicjVDLSgNBEOyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOOsmQ2dllplcISz7CiwdFvPo93vwbJ4+DioIFDUVVN91dUaqkJd//8Aorq2vrG8XN0tb2zu5eef+gaZPMCGyIRCWmHXGLSmpskCSF7dQgjyOFrWh8PfNb92isTPQdTVIMYz7UciAFJye1ujRC4qVeuRJU/TnY36QCS9R75fduPxFZjJqE4tZ2Aj+lMOeGpFA4LXUziykXYz7EjqOax2jDfH7ulJ04pc8GiXGlic3VrxM5j62dxJHrjDmN7E9vJv7mdTIaXIa51GlGqMVi0SBTjBI2+531pUFBauIIF0a6W5kYccMFuYT+GULzrBr41eD2vFK7WsZRhCM4hlMI4AJqcAN1aICAMTzAEzx7qffovXivi9aCt5w5hG/w3j4B3uSPPw==</latexit><latexit
latexit
<
✓
Spatial Transformer Network
Grid Sampler
(Custom layer)
Jaderberg, Max, Karen Simonyan, and Andrew Zisserman. "Spatial transformer networks." Advances in neural information processing systems. 2015.
Classifier Net
6
•
Demo
•
Conversion with Custom Layer

Custom Layer
(Parameters)
gridSampler.swift
Custom Layer
Layer in a Neural Network

Custom Model
New Model
Core ML Custom Model Core ML

•
Summary
Core ML Tools 2.0
Core ML Tools 2.0
Rich Core ML Tools ecosystem

Core ML Tools 2.0

•
Easy-to-use quantization utilities

Core ML Tools 2.0

•
Easy-to-use quantization utilities

•
Integrate new layers

More Information
https://developer.apple.com/wwdc18/709
Machine Learning Lab Technology Lab 2 Wednesday 4:00PM
Machine Learning Lab Technology Lab 12 Friday 2:00PM

709 Whats New in Core ML Part 2

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

709 Whats New in Core ML Part 2

Hochgeladen von

Copyright:

#WWDC18

What’s New in Core ML

Aseem Wadhwa, Core ML

Float weights Quantized weights

Core ML Tools ecosystem

Core ML Tools ecosystem

Core ML Tools ecosystem

Core ML Tools ecosystem

In collaboration with Google

Support for custom layers

Support for Quantized TensorFlow models

In collaboration with Facebook and Prisma

Core ML Tools ecosystem

Core ML Tools 2.0

Support for latest Core ML .mlmodel specification

Flexible shape utilities

Float32 weights Quantize weights 1-8 bit weights

Float32 weights Quantize weights 1-8 bit weights

Float32 weights Quantize weights 1-8 bit weights

Decide on precision and algorithm

Float32 weights Quantize weights 1-8 bit weights

Decide on precision and algorithm

Let Core ML Tools work its magic!

// Compare Quantized model with original

// Compare Quantized model with original

// Compare Quantized model with original

Float32 One bit

Float32 One bit

• Signal-to-noise ratio Agreement Model 2

Float32 One bit

4 bit (linear) - 857 KB 3 bit (linear) - 647 KB 2 bit (linear) - 437 KB

Core ML tools ecosystem

Aseem Wadhwa, Core ML

New Model Architecture

New Model Architecture

Converters: Simple API

Converters: Simple API

Converters: Simple API

Converters: Simple API

Converters: Simple API

Converters: Simple API

Use Custom Layers!

Custom Layer Examples

Not Supported by Core ML

Custom Layer in Core ML Model

With Custom Activation Layer

With Custom Activation Layer

Conversion with Custom Layer

Layer in a Neural Network

Core ML Custom Model Core ML

Rich Core ML Tools ecosystem

Rich Core ML Tools ecosystem

Easy-to-use quantization utilities

Rich Core ML Tools ecosystem

Easy-to-use quantization utilities

Integrate new layers

Machine Learning Lab Technology Lab 2 Wednesday 4:00PM

Machine Learning Lab Technology Lab 12 Friday 2:00PM

Das könnte Ihnen auch gefallen

Core ML Tools 2.0 