Is USD The HTML Of The Metaverse?

I was able to attend the NVIDIA GPU Technology Conference (GTC) and focused on the extended reality (XR) track. If nothing else, it was clear that the attendees were enthusiastic about the use of Pixar’s Universal Scene Description (USD) format as an interchange format for the metaverse and XR in general.

There’s A Huge Need For A Common Format

Companies as diverse as Adobe, Apple, Autodesk, Lowe’s, Mercedes-Benz, Siemens, and of course NVIDIA and Pixar are looking to use USD. This is because USD is more than just a way to describe 3D objects. Pixar needed an effective way for a large number of collaborators to make changes simultaneously on different elements in the same scene at the same time. USD’s layers and composability give it a leg up compared to other formats, which encode just an exterior mesh or even a description of internals, texture, and the animation skeleton.

USD Has Had A Lot Of Refinement — And That’s A Problem

Pixar open sourced USD in 2016, but the company used it internally for years before then. Pixar has made a lot of design decisions that reflect its needs. Until recently, the primary users of USD have been in the entertainment industry — motion pictures and gaming. But the exciting advances will be coming from other verticals: medical, automotive, CAD, robotics, and retail. There hasn’t been the same level of refinement — and those use cases may conflict with the needs of the entertainment industry. I may not need or even want true-to-life physics in my summer blockbuster, but I certainly do if I’m using USD to describe the road boundaries for my self-driving car. As Tony Hemmelgarn, president and CEO of Siemens Digital Industries, said on his panel, “There’s a big difference between animation and simulation. We fully support USD, but there will be more to come. It’s beyond animation; we need to model the physics behind it.”

The USD Spec Is Already Big

One of the advantages of HTML when it first came out was its simplicity. Users couldn’t do too much, so it wasn’t hard to create something that looked similar to Netscape’s or Sun’s website. A beginner with USD, however, is not going to create a Turning Red or even a Luxo Jr. Pixar has released a number of tools to help get beginners up to speed — usdview is a helpful way to inspect USD source to see how it does what it does — but learning USD is still much like starting with no knowledge of web browsers and trying to understand Next.js running on your content delivery network (CDN). There’s a lot of material there and a lot of implicit knowledge that everyone who’s been in the industry “just knows.” Implementing all of USD in your development environment/simulation engine is even harder.

Beware CORBA!

USD models are complicated. Mason Sheffield, director of creative technology at Lowe’s Innovation Labs, talked about “product avatars,” the idea that all 3D objects would have behavior. His example was a light bulb that modeled reality — not only did you know its physical dimensions and color, but you could install it in a lamp in a virtual room, turn it on, and see how the bulb’s color temperature affected the rest of the room. If every physical object could have a real-time simulation (something Matthew Ball, CEO of Epyllion, suggested at the panel) then USD would need even more levels to puzzle out. Nobody wants USD to turn out like CORBA — the Common Object Request Broker Architecture standard of the late 1990s, which provided a way to pass objects for remote procedure calls. It became progressively more arcane with each release as new elements kept getting added to satisfy every requirement. Ultimately, CORBA lost out to simpler standards. USD must avoid the same fate.

Today, USD Is The Only Game In Town

Leveraging existing standards will help USD. In that respect, it hasn’t done itself any favors with the USDZ implementation, which requires 64-byte-aligned zip files. If you unzip and then rezip a .usdz, you’ve probably just corrupted it. In truth, though, there’s nothing better as an interchange format today. USD has done a number of things right; the file system agnosticism that’s been embraced by Adobe is a good start, as is its composability. Small, understandable tools that work on a single layer but preserve other layers will help. The USD Python libraries could give developers their first look at USD in a more familiar environment. USD must come up with a way to support hundreds of different verticals while making sure they don’t impinge on, or even know about, each other. Threading that needle will be challenging, but USD has the best chance of success yet.