Line 2 of the Beijing subway, which traces the path of the capital’s former city wall, was the first subway loop ever constructed in China. Nine stations are scattered along the line, their names taken from the wall’s Nine Old Gates. Regrettably, most of the gates have already disappeared: The only remaining examples are Zhengyangmen, to the south of Tiananmen Square, and Deshengmen, on the north side of the capital’s Second Ring Road.

Since January, however, augmented reality (AR) technology has been allowing people to travel back in time and appreciate the graceful beauty of the Nine Old Gates. Now, anyone changing lines at Xizhimen Station or on board the special AR train running along Line 2 is treated to a series of photographs of the gates themselves.

When people scan these pictures with the Baidu mobile app and turn on the AR function, all kinds of historic scenes showing the lives of ordinary Beijingers appear on their smartphones. The app also features spoken descriptions of the history of each gate, as well as literary allusions that have been made to the gates throughout history.

We created the tour of Beijing’s city wall here at Baidu’s AR lab. Its origins stem from the personal experience of my colleague Li Yingchao, whose adorable 5-year-old daughter loves to ride the subway. While riding along Line 2 one day, something piqued her curiosity: Why did so many stations end in men, the word for “gate”?

Li wanted to stimulate his daughter’s interests and give her more information. At the time, it so happened that he was working on Baidu’s AR project, and this is how he came up with the idea of restoring Beijing’s old city gates — at least in the virtual realm.

That was back in September 2016. At first, the project was only something Li did in his spare time, but it wasn’t long before it struck a chord with people at Baidu. Quite a few co-workers joined in, including me. This team was very young, and like Li, most of its members were “new Beijingers” who came from all over the country — that is to say, we weren’t particularly familiar with the “nine old gates.” For us, the project became a process of understanding our adopted city’s history.

The project was far from easy, however. We collected more than 500 old, grainy photographs of the gates, but the towers all looked so similar. Pinpointing which tower was shown in each photograph and when each image was taken would have been a challenge even for an expert in Chinese culture and history, so you can imagine how difficult it was for a team of engineers like us.

We needed to train machines to recognize blurry, low-resolution photographs that were more than a century old; to discover the subtle differences among these pictures; to perform this in spite of the overlapping effects of different angles, different levels of lighting, and the complex environment of Beijing’s subway; and then to bring all of this into our mobile app’s AR function. In short, it was quite the gauntlet of challenges.

To solve the technological issues, we at the AR lab worked with cutting-edge technologies like image recognition, 3-D perception and tracking, human-computer interaction, 3-D rendering, and the fusion of real and virtual worlds.

To showcase the technology we’re currently working on, we used Baidu’s Simultaneous Localization and Mapping (SLAM) to activate a triggering mechanism that occurs when the user arrives at Zhengyangmen. In simple terms, this technology enables a computer to use sensors — such as cameras, lasers, and gyroscopes — to estimate the user’s position in an unknown environment while simultaneously constructing a map of the surrounding area through which they can move. We worked hard to polish every single detail of the virtual tower, and looking back on our results, I think that, in the end, we succeeded.

Zhengyangmen is commonly known as “Qianmen,” which means “front gate.” It’s located at the southern axis of Old Beijing. The architecture of the city gate tower, arrow tower, and barbican formed a complete ancient defense system. Today, only the city gate tower and arrow tower remain. Not only was SLAM able to restore the appearance of Zhengyangmen in all its historic glory, but it also pushed back the limits of AR technology in rendering virtual figures and architectural angles, allowing people to witness breathtaking scenes of figures past and present, walking side by side in augmented reality.

Since hitting the market, our product has received a great deal of praise and media attention. I wonder if this is because, on some level, it provides emotional comfort. The disappearance of Beijing’s old city wall is a painful historic episode to the city’s residents. The government of the Republic of China demolished the barbican and sluice gate at Deshengmen in 1915 in order to build a railroad around the city, and razed the city gate tower in 1921.

After the People’s Republic was established in 1949, Beijing was earmarked to become a modern industrial city. Liang Sicheng, the deputy director of Beijing’s city planning committee, and Chen Zhanxiang, head of the planning department of the city’s construction bureau, drew up a plan for a new town located in the western suburbs, intending to preserve the old city. But this proposal was not approved, and Beijing’s city wall and city gates were gradually demolished through 1968.

Today, Zhengyangmen and the partial remains of Deshengmen’s arrow tower are all that are left of the Nine Old Gates. More and more Chinese people sympathize with Liang’s heartfelt words, uttered in the knowledge that his plan would never see the light of day: “Demolishing a tower from the city gate is like tearing a chunk of flesh from my body. Taking a brick from the wall is like peeling off a layer of my skin.”

Our fusion of history, culture, and innovation is a technological achievement with Chinese characteristics. Tech teams in more developed countries perhaps have less demand for AR recreations of historic sites, either because of strong preservation laws or because of a relative lack of usable historic remains. Yet the grave damage that the cultural artifacts of certain less developed countries, including China, have suffered mean that a market is there, even if others currently lack the technology to realize it in the short term.

For our tech team, this project serves as a warning as well as an innovation. It’s undeniable that we are wildly enthusiastic about technology and the pursuit of innovation; however, in the process of seeking cutting-edge technology, we must remind ourselves not to be blindly obsessed with progress — after all, this kind of hasty, shortsighted desire to “move forward” is what led to the disappearance of Beijing’s ancient city wall in the first place.

Technology should ultimately serve people’s needs. Using AR to restore Beijing’s nine city gates is just the beginning. In the future, we will use the strength of technology to awaken more of the city’s memories and forge an eye-opening link between history and culture.

Translator: Zachary Haluza; editors: Wu Haiyun and Matthew Walsh.

(Header image: A technician tests an augmented reality program near Zhengyangmen in Beijing, January 2017. Courtesy of Baidu)