From Grasshopper to QuickPreview in iOS: Making use of the USDZ file format, Part One
The idea
Part of my challenge as an architect+urbanist+designer and iOS Developer is being able to making something out of this crazy mix of visions, experiences and different fields. From the architect+urbanist+designer standpoint, I am always messing with parametric design, rapid prototyping and digital fabrication.
Grasshopper in Rhinoceros is a fantastic tool for this end. With it we are able to conceive incredible methods (called Definitions) that rule shape and form — and all that still making room for experimentation during the entire process. In case you are not acquainted with it, Grasshopper is a graphical algorithm editor completely integrated with Rhino’s 3-D modelling tools. It requires no knowledge of traditional programming or scripting, but still allows designers to build form generators. You basically use a graphical interface to define a bunch of 3D modelling instructions, tightly related with dynamic parameters determined by your idea. Changes in these parameters generate immediate response from the algorithm, creating new forms.
So, indeed, Grasshopper is a fantastic tool for designers. But the whole point of conceiving an entire composition method, instead of a static artefact, is allowing the final user to become active part of the design process — that is making them able to choose values compatible with their particular needs, experimenting with variety and personal taste. For that, Grasshopper simply doesn't offer a close enough to usual digital experience for the general consumer. From an iOS Developer standpoint, this insight lead to this guiding question:
What can be done — using iOS — to close the gap between Grasshopper Definitions and the consumers?
And to answer that, I had to consider my personal constraints for this project
As I am a beginner/intermediate at iOS development, it would be reaaally nice if the solution was simple to implement.
Explore AR as a form of visualisation and/or interaction with the design.
From that I got what would be the ideal flux:
1. < Inputs on iPhone >
2. < Generation of Artefacts inside Grasshopper >
3. < Output AR visualisation >
New USDZ file format
On WWDC 2018 we got to know about the new file format developed by Pixar, with the support of Apple and Adobe. USDZ simplifies the 3D model visualisation by combining crucial information for rendering a scene in one file. Geometry, textures, illumination, camera position — Pixar made everything bundled together, in a zero compression, unencrypted zip archive. The result is an archive with potential for quick but powerful visualisations. Apple's part was making iOS Quick Look Preview an incredible showcase for the file's capabilities — with no need for configuration we get to push our models into reality through AR.
With that in mind, my idea started getting into shape. So I began by verifying how easy it would be to work with such files. The main guidelines for them can be found below.
QuickLook Preview of AR models
Working with Quick Look is quite simple, as this tutorial show. With a few lines of code you are good to go. I really recommend this one, to get it working with AR. You can get some usdz samples to work with on Apple's showcase website.
Using Quick Look on items inside a UICollectionView, you have to define a variable from the class QLPreviewController and its DataSource and Delegate.
func collectionView(_ collectionView: UICollectionView, didSelectItemAt indexPath: IndexPath) {
thumbnailIndex = indexPath.item
let previewController = QLPreviewController()
previewController.dataSource = self
previewController.delegate = self
present(previewController, animated: true)
}
func numberOfPreviewItems(in controller: QLPreviewController) -> Int {
return 1
}
func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem {
let url = urlFromUsdzFile
return url as QLPreviewItem
}let url = Bundle.main.url(forResource: models[thumbnailIndex], withExtension: "usdz")!return url as QLPreviewItem}
But the fun part is creating your own USDZ files. Since Rhino couldn't export this new format, It was clear that I needed to convert the file to it. After doing some digging I found out that it was possible to make the conversion through command line using Xcode.
xcrun usdz_converter ~/Desktop/myFile.obj ~/Desktop/myFile.usdzBesides that, some very smart guy found out that it was possible to convert a obj file to usdz at runtime. And another candid soul converted the code to Swift.
After some struggle getting it to work — watch for proper way of writing, URLs and Paths — all I needed was to figure a way to send a .obj file from Rhinoceros/Grasshopper to the app.
BUT… on Grasshopper Side…
I took for granted, how hard would be to traffic from a software running on a Windows OS, to a mobile running iOS.
As I found out, communication between Grasshopper and other devices can be done using UDP Protocol.
UDP stands for User Datagram Protocol. It uses some IP Protocol, using a port number. This guy made an excellent analogy using a classroom for how it works, it goes like this:
So its a little bit like sending notes in class you pass the note(datagram) marked to the student at desk 5. You don’t know if the other students will pass it on or if they are too busy reading and the note ends up on the floor. Or someone could decide your message was too long and rip off the bottom part of the note and throw it on the floor.
To wrap it up: UDP doesn’t care about the size of the package, nor if the data is going to be received in any order, and not even if it is going to be received or not. But even after learning that, I didn’t give away my hopes, and investigated how could I establish an UDP session.
On the Swift side, things worked great. Using xBACP simple UDPClient, I could send and receive some Strings using a single view App. But on the Grasshopper side, it didn't go so well.
First, you are going to need some plug-ins to send and receive data using UDP. I tried two different plugins. gHowl allows to send/receive data through the network to any computer over UDP protocol, by using IP Address of the other computer, port to send to, data in a String format and type (ASCII, array of doubles or an OSC message). The other plug-in tested was Firefly, that works in a similar way to gHowl, but with fewer configuration. To make sure several packages were sent, It is recommended to use a timer to refresh the Grasshopper components continuously.
Even though the Clients (Swift, Firefly or gHowl) worked just fine in their own environments, no matter how much I tried, I simply could not exchange data between the devices.
My workaround
For now, I am going to leave the communication between Grasshopper and iOS for another moment. It seems like a better idea, learn how to mount proper USDZ Files using 3d models and publish them to a common, web based, medium. An AR portfolio one link away could surely be very handy.
But that comes next…
