“Core ML is awesome! There’s so much ideas that I can do with it!” You might exclaim following WWDC 2017. “But I only have one mac. I can’t just install a beta OS on my main machine — It’ll probably brick and I’ll lose work”

Fortunately you don’t have to go to the High Sierras (pun intended) just to “have a go” on machine learning with Core ML. All thanks to the iOS Simulator.

Yes, iOS 11 Simulator has Core ML too. You don’t need to install a pre-production macOS nor install a buggy OS to your phone either. Just run the beta iOS Simulator with a production macOS alongside your regular Xcode and iOS Simulators.

What’s also great is that both iOS Simulator and Core ML works on Swift Playgrounds — making it easier for you for exploratory testing of ML models (and ensuring that it survived conversion). Well, sort-of for Core ML. You will need the “compiled” Core ML model and make some modifications on the corresponding generated stub sources. Hats off to Andrew C Bancroft who describes a solution to a similar issue with Core Data.

Here’s what you need to do to get your hands dirty on Core ML. You will need Xcode 9 Beta for this, but you can run it on current production macOS 10.12 — beta macOS optional.

1. Get a Core ML Model.
2. Compile the model using a dummy project. This will create a compiled .mlmodelc model and a corresponding Swift source file containing stub classes for the model.
3. Use Xcode 9 (beta) to create a Swift Playground with the platform set to iOS
4. Copy the generated .mlmodelc folder into the Playground’s Resources folder.
5. Copy the generated Swift stub into the playground’s Sources model.
6. Modify the generated Swift stub file and make the classes therein public as well as any other properties that you might use in your main Playground file.
7. Start playing with Core ML!

Getting a Core ML model

The easiest way to obtain an ML model is to download a pre-made one from Apple’s machine learning overview page. If any model suffice — just for getting your hands dirty — taking one from their Core ML sample code would be better. Otherwise you’ll need to create one yourself and convert it to Core ML’s format using the provided tools.

Compiling the model.

The easiest way right now to compile a Core ML model and generate Swift files is via a dummy project. Create a blank Swift project and set the deployment target to iOS 11. Then copy the .mlmodel file into the project and add it to the main target. Start a build for iOS Simulator and then you should get the model file compiled.

If you’d like to play around with the command line, there is the mlkitc command line program inside Xcode’s application package under Contents/Developer/usr/bin. However I was only able to generate Objective-C stub sources and couldn’t find how to make it generate Swift files. I found these by guessing and there’s no help text on how to use it as of Xcode 9 beta 3.

Use the word compile as the first parameter of mlkitc followed by the source .mlmodel file and a target where to place the compiled models.

./mlkitc compile {source-model} {target-folder} 

For example, the following command would compile VGG16.mlmodel from the Downloads folder and then create a folder VGG16.mlmodelc containing the compiled model data under the Downloads/VGG folder.

./mlkitc compile ~/Downloads/VGG16.mlmodel ~/Downloads/VGG 

To create Objective-C stub classes for easy access to the model, use the word generate as the first parameter to mlkitc. This would create a pair of .h and .m files that contains three classes — one for the ML model itself, an input class for specifying input parameters and an output class that contains the model’s inference result.

./mlkitc generate {source-model} {target-folder} 

For example, the following command would take the VGG16.mlmodel model file and create the stub Objective-C sources in the Downloads/VGG folder.

./mlkitc generate ~/Downloads/VGG16.mlmodel ~/Downloads/VGG 

In case you’re curious, VGG16.mlmodel is Visual Geometry Group’s convolutional model to identify the dominant object of an image. The Core ML model file was downloaded from Apple’s Machine Learning overview page

Locating the generated files

After building the target, the Core ML stub sources should be present inside your project’s derived data folder. Try looking for a folder named CoreMLGenerated from the derived data folder and generated the stub files should be two level down from that, enclosed by a folder derived from the model file name.

Another way easiest way to find the stub sources is by referencing the model stub class from one of your own sources and let Xcode show you where it was defined.

For example, write code to instantiate the model stub from one of your view controllers like so:

let model = MarsHabitatPricer() 

Build the target then command-click on the word MarsHabitatPricer and select Jump to Definition to find out where it came from. With that file open, ctrl-click on a black area in the source file and select Show in Finder to open a Finder window that highlights its location. Then you can easily copy it out to the playground.

However the compiled Core ML model would be found inside the target application bundle itself, since it’s meant to be shipped as part of the application. The easiest way to find it is to ctrl-click your application target in Xcode and select Show in Finder. Then in Finder, ctrl-click the application bundle and select Show Package Contents. Locate the folder with the .mlmodelc extension and copy it out along with all its contents.

Adding Core ML files into the Playground

Drag and drop the .swift stubs into the top-level Sources folder in the Playground and similarly the .mlmodelc folder into the top-level Resources folder. Initially these are grayed out in the Playground’s navigator tab but should become solid as soon as a file gets placed in there. Having the model placed into the top level folders would make it available to all playground pages.

Alternatively from the Finder, ctrl-click the .playground bundle and select Show Package Contents. Then create two folders there, one named Sources and the other named Resources. Copy  .swift files into Sources and the .mlmodelc folders into Resources.

Modifying the stub

By default the Swift auto-generated stub sources has internal protection levels. This means that all classes and their members are accessible only from sources compiled for the same module. Unfortunately ancillary classes in Swift Playgrounds — those defined in source files that are not part of a page — needs to have public accessibility to be usable from a page.

These stubs consists of three classes:

• A façade for the Core ML model object. This holds the MLModel object as well as prediction functions to make inferences from the model.
• An input object for the ML model, subclass of MLFeatureProvider. This describes the input of the ML model in terms of feature names and their respective types.
• An output object for the ML model’s results. This is also a subclass of MLFeatureProvider but instantiated by the ML model object’s façade class above as a result of prediction calls.

Just go in to the source file and change all three classes’ protection level to public. Make sure that you also do the same for those class’ members as well.

Note that in the Swift generated classes, the individual features of the input object gets generated as var with the internal protection level. Hence you probably only need to change init to public unless you need to mutate the input after making predictions.

However in the output object, feature fields are generated as let data members without any accessor. Thus you will need to make them public — don’t worry about setters since they’re let constants after all.

Sample Playground

I’ve constructed a bare-bones sample Swift playground that brought the pieces together. It brought over the ML model from Apple’s Mars Habitat Price Predictor into a playground environment. Download the playground from my Github account: MarsHabitatModelTest.

That’s all for now, take care!

---