NETEASE AR CAMERA - Bring Amazing AR Experiences to Real Lives
Current AR camera apps are mostly designed for selfies for which special effects on faces, mouths or head can be added. For us, instead of bringing virtual objects to the real world, we want people to come to a virtual environment that has strong relationship with the physical surroundings.
Involvement
Lead UX Designer
Lead Content Strategist
Team
My Role
As the lead user experience designer at AR Camera Team, I worked with Unity enginners, machine learning engineers and product mangers at NetEase to design the UX/UI of this AR product.
Zhiwen Qiu, Weiwei Li,
NetEase Engineering Team
Company
NetEase Inc.
Year
2017
Objectives
With this goal in mind, Netease Insight is therefore positioned not as a simple camera that users use merely for decorate effects, but as an AR browser that provides users with a thorough, totally immersive experience featuring their immediate surroundings. Similar to a web browser where various search results are displayed for users to choose, users can scroll through various AR scenarios as they open the app and freely select which experience they want to get a taste of. Once they download the scenario, they are then taken to a virtual environment where they see different effects featuring their surroundings and can interact with the physical setting as they move. The user really becomes the director of their own and his/her friends become actors and actresses.
Photographer
Director
People in camera
Actor
Technology
In order to achieve our goal, all virtual models and special effects, including sky, clouds and others are located on the detected plane with the help of ARKit. Virtual effects could merge seamlessly with the real environment powered by specifically designed filters. Additionally, we implement realistic rendering technology to present more immersive experiences.
ARKit
Face Recognition
Plane detection process
Four Scenes
1. Thunderstorm
Surrounded by overcasting sky, lightning flashes inside the grey clouds and some of them strike on the ground. When user taps the screen, the lightning will strike on that location.
The camera experience will be the major features of Netease Insight App as a provider of AR experiences, an AR browser. After user successfully installed the app, he can easily download the content package and start the camera experiences on the phone.
2. In the Ocean
The water effects completely change the normal environment to an under-ocean setting where schools of fish are swimming around. A large squid is moving around and when you touch the screen, it will move to the spot and burst out ink.
Taking inspirations from the nature, in this scenario we aspire to change a normal scene, for example, an office space, to an ocean setting, just like people are working under the water
3. Fruit Garden
The third scene is a fruit garden where different types of fruits are floating in the air statically. Each fruit will explode to a cloud of smoke and disappear afterwards when user taps each of them.
Inspired by Cerise Doucède's photographs, we thought why not let things that won't happen in real life happen in AR? Perhaps in the space? The movements of all the fruits are anti-gravity and the user -- the director -- thus has the full control of the floating fruit garden, and perhaps similar to color run, the fruit will explode to smoke on the detected faces in the screen.
Opening scene when the fruits are moving up
When user taps the fruit, one of the fruits will be triggered and then drops down like in the outer space and the person's nose and eyes will burst out smoke with the same color of that fruit.
5 types of fruits with different colors of smoke
4. Throwing Money
Inspired by cheesy money rain scenes in movie and some hand art, we want to create a scene where hands are coming out of the ground and reach down from top to throw money at people.
Similar to the other scenes, the clouds in which hands are hidden is designed to be on top of the screen. If the user scans the ground for some time, the detected plane will crack and hands will come out of it. When user taps the screen -- the universal interaction for the camera experience -- one of the hands hidden in the cloud will move down and start to throw money on that location.
Movement of hands throwing down money, repeating after completing a cycle of four steps
Initial movement of hands coming out of ground