MR

When you open a seemingly ordinary book, magical hologram creatures pop out. This seems like something augmented reality (AR) glasses were supposed to do years ago. There are also cases where something similar to this was actually provided. Years ago, Sony offered an AR Harry Potter book that worked with the PlayStation camera. Nintendo also used the Nintendo 3DS camera and 3D display to create an AR experience, offering unique cards and books using markers. New content from Jim Henson’s The Storyteller explores the idea that physical accessories like books may once again have a presence in future mixed reality (MR) headsets. is presenting.

AR version of Jim Henson’s “Storyteller”. The content is attractive, but will it become an opportunity for books to become AR accessories?
Credit: Scott Stein/CNET

This AR content was developed byFelix & Paul StudiosBut this is a new venture for the company, which makes immersive virtual reality (VR) entertainment. The company offers a series of 360-degree camera shots of the interior of the International Space Station, which perfectly capture reality and project it onto a VR headset. However, for “storytellers” who use AR, the mechanism is reversed. Rather than transporting the author to another location, Jim Henson’s The Storyteller: The Seven Ravens projects a small world of stories within a physical prop like a book. The visuals are attractive, and it’s like a moving miniature video game theater. The narration by Neil Gaiman isn’t bad either.

With this AR content, I felt as if I was looking into the book in my hand and seeing the world inside it. When you tilt the book or turn the pages, you can see a variety of things that go beyond the book’s boundaries. Sometimes characters jump out and stare at me from the window. In the ocean scene, if you tilt the book, water will spill out. Every time you turn a thick board-like page, a new AR scene begins. Also, when they thought they had reached the last page, they were instructed to turn the book upside down and turn the pages in the opposite direction. Following the instructions, I turned the page again and the story continued. With a book like this, it might be possible to continue the story indefinitely.

Tools that go beyond controllers

Future VR and AR headsets will likely eliminate the need for controllers. Hand tracking will become more standard thanks to devices like Meta’s upcoming Quest 3 (with a better depth-tracking sensor) and Apple’s Vision Pro in 2024. It should be. Once the controller-less future of MR arrives and we can reach out and touch things that don’t actually exist, we will see a wide variety of MR-enabled accessories and objects, expanding the use of AR to a wide range of haptic tools. I will go.

Keyboards and trackpads are the obvious next step. Apple’s Vision Pro and Meta’s Quest also support keyboards and trackpads, and some companies are already considering AR-enabled keyboards. What comes after that will probably be another accessory. It could be a book, or something that could be a small space where you can project AR.

The markers are reminiscent of many traditional toys and cards that work with AR-enabled smartphones, tablets, and even headsets, but the evolution of this technology could be just around the corner.

The content and book illusions in Jim Henson’s The Storyteller: The Seven Ravens work very effectively, cleverly using the book’s design to hide the major limitations of current AR glasses. . Headsets like the Magic Leap 2 have a limited field of view. In this case, the headset’s field of view is aligned with the edge of the book, a prop tailored for this content. Each time you turn a thick page with a marker code printed on it, a new chapter begins and appears to sink into the frame of the book (the book’s pages are dark black, increasing the transparency of 3D objects in AR) It also plays a role).

Storyteller content powered by Magic Leap was developed five years ago and was originally scheduled for release in 2020. However, the release was postponed due to the coronavirus pandemic and Magic Leap’s pivot from creative-focused experiments to a new strategy targeting enterprises. Ta. Now that the Storyteller content has been revived and debuted at the Venice Film Festival, Paul Raphael, co-founder of Felix & Paul Studios, has confirmed that the content will soon be turned into an actual book for headsets. We believe this will serve as a stepping stone to creating products. We also believe it will be available in products for MR devices such as Apple’s Vision Pro and Meta Quest 3.

Raphael believes that modern VR/MR headsets with pass-through capabilities and the ability to perform AR are ideal for such an idea. One reason for this is that unlike AR glasses, which are currently an extremely niche field, there are people who actually own VR/MR headsets.

“I think pass-through is the best option for this. It’s great in some ways, and it’s not so great in others. The user is aware of their surroundings and what they’re holding in their hands, but they’re not paying attention. It’s the content of the book,” Raphael said. “Even if what you’re seeing isn’t as realistic as a photo, you’ll get a pretty close experience with the Vision Pro. You’ll probably get a pretty realistic experience with the Quest 3 as well. These are the main features used in this content. I have no doubt that it will become a great device.”

Although the content of “Storyteller” has not yet been commercialized, the book’s ideas may become a reality. Mr. Raphael told the author after trying out the content, “If AR becomes commonplace and you can experience AR even with the glasses you wear, it will be possible to turn everything into AR. It should be,” he said. “When we set out to design this content, we were actually thinking about storytelling with objects in general. This book was our first attempt, and we’ve come a long way, but there’s more. There are all kinds of AR object concepts out there, and we want to make them a reality.”

This article is published by Red Ventures overseas.articleEdited for Japan by Asahi Interactive.