Virtual and augmented reality bring historical objects to life

For most people, the chance to walk through a re-creation of early 20th-century Chinatown in Los Angeles or page through a 15th-century Christian devotional book known as a Book of Hours is the stuff of fantasy. But faculty at the USC Dornsife College of Letters, Arts and Sciences aim to bring historical objects into people’s laps — sometimes literally — through innovations in virtual reality (VR) and augmented reality (AR).

“As faculty we want to conduct scholarly research, but not just for itself; we want to take that knowledge and make it as broadly and widely accessible as possible,” says Bill Deverell, professor of history, spatial sciences and environmental studies at USC Dornsife.

Deverell is collaborating with Professor of Cinematic Arts Scott Fisher and a team of scholars at the USC School of Cinematic Arts on the Chinatown project, which includes an AR program that lets Union Station visitors see what their immediate surroundings looked like in the early 20th century, before much of Chinatown was razed to make way for the train depot. Using archival materials, such as photographs and maps depicting the streetscapes, the team aims to create an app enabling users to look through their phones and see a model of the old neighborhood, streets, homes and shops.

The project, which also involves the Los Angeles County Metropolitan Transportation Authority and the Chinese Historical Society of Southern California, is still in the early stages, but Deverell says the team is making progress on filming, programming and research for the project.

“We want to have an opportunity to invite close to 100,000 people a day who move through Union Station to pause and contemplate the history of the space they’re moving through,” Deverell says. “Doing so has a commemorative effect by memorializing the people who were utterly disrupted by the demolition of their neighborhood.”

Objects in hand

Another AR app, created by Sean Fraga, assistant professor (teaching) of environmental studies and history at USC Dornsife, and built by a multidisciplinary USC team, brings pieces of the past into users’ homes.

Called Booksnake, the app allows users to select historical items, such as a 1930s street map of Hollywood, and view them through a phone or mobile device. Using the phone’s camera, the user can superimpose the object on a flat surface in their surroundings and fix it in place for closer inspection.

Fraga developed the idea for Booksnake while a Mellon postdoctoral fellow with USC Dornsife’s Humanities in a Digital World program, which collaborated with the USC Libraries’ Ahmanson Lab to advance the project.

The app’s name stems from an item that archival researchers often use called a “book snake,” a small, weighted string that holds down fragile materials so researchers can examine them. The app aims to “hold” virtual objects in place for the same purpose, Fraga says.

“Booksnake is using cutting-edge technology, in terms of cameras and sensors and the computation available in mobile devices, to enable something that is a very old, familiar practice of humanities research, which is laying out a source in front of you and really closely examining it, sitting with it,” he explains.

The app’s first version — which is currently in beta-testing — can display digitized materials from the Library of Congress. A grant from the National Endowment for the Humanities (NEH) enabled the team to add support for newspapers, books and magazines. Next, the team is building support to display items from the USC Digital Library.

Fraga plans to make Booksnake available for free to the public, but he feels the app will be especially useful in a classroom environment.

“College students are increasingly shifting away from having laptops and towards having phones and tablets. Booksnake is an experiment with using some of the technologies built into these devices to make them places for humanities research, teaching and learning,” he explains.

Immersed in history

Two projects led by USC Dornsife faculty are taking a more immersive approach to learning by using VR technology to place individuals in a long-ago context.

“VR brings together technology and the humanities. Through it we can examine lost experiences, such as what reading was like in past moments and past places,” says Lisa Pon, professor of art history.

Pon leads an interdisciplinary project, supported by a grant from the NEH and two Zumberge Research Coordination & Team Building grants, that builds on recent work in a USC Sidney Harman Academy for Polymathic Study Ahmanson Lab Collaboratory. The project enables an individual to enter one of the most famous rooms in the Vatican museums, Raphael’s Stanza della Segnatura, home to some of the artist’s most renowned works. Raphael’s patron, Pope Julius II, kept his own collection of books there nearly 500 years ago.

Pon explains that when a pope dies, his library is typically cleared to make way for his successor. Her project, which made its public debut in May at The Huntington Library, Art Museum, and Botanical Gardens, restores the space as it would have been during Julius’ tenure as pontiff.

Since many of the original books and objects relate to each other and the artwork in the room, it’s useful to present them as they were originally gathered so viewers can understand the relationships at play, she adds.

“The texts and the books speak to the paintings and the architecture, and vice versa, and that’s what we’re re-creating,” she says.

Pope Julius II intended that his private collection of books be kept in the Stanza della Segnatura, the first of the famous Stanze di Raffaello, or “Raphael Rooms,” and Raphael’s paintings there reflect that papal wish. Being able to stand in that room while virtually holding Julius’ books in hand also helps us better understand the paintings, Pon adds.

The project will see its worldwide launch at the Society for the Society for the History of Authorship, Readership and Publishing’s global online conference taking place June 26–30.

Putting books in their place

Experiencing the historical space in which a person read, thought and lived is also a key component of a VR project by Sabina Zonno, a scholar with USC Dornsife and The Huntington, and Lynn Dodd, professor of the practice of religion and spatial sciences at USC Dornsife.

Supported by two NEH grants, Zonno and Dodd have teamed with faculty members from USC Thornton School of Music and USC School of Cinematic Arts, and students from across the university. Their VR project enables viewers to page through a lavishly illustrated 15th-century Book of Hours from USC Libraries’ Special Collections while standing in a space that evokes the book’s original environment.

Currently, Zonno and Dodd are working to make the entire book accessible, adding more simulated pages, providing multiple translations of the book’s Latin prayers and creating interactive features that will enable users to gain a deeper understanding of the book’s story.

Using this first project as a template, they have also been creating virtual reality experiences featuring other ancient books. For example, users can access the oldest complete Hebrew Bible, produced in Old Cairo in the 11th century, and read its texts in Hebrew or translated into English while immersed in a virtual synagogue.

“These virtual reality experiences represent a new way to do research, teaching and outreach,” Zonno says. VR enables people to be in the places where these ancient artifacts were used and safely “handle” the original books without risk of damage.

These immersive and realistic experiences, say Zonno and Dodd, can make ancient books currently preserved in curation institutions around the globe virtually available to anyone who wants to learn about and appreciate the value of these assets of our cultural heritage.