Bridging the experience gap between AI agents, bots, and Bing search
I spent the summer of 2017 interning at Microsoft as a UX Design Intern under their Artificial Intelligent and Research group. One of the three projects I got to work on was a concept project thinking about what the Bing experience will be like a few years down the road with AI infused. This was a really exciting project as I had very little constraints and was able to take the project wherever I thought the future would be.
How do we bridge the experience gap between AI agents, bots, and Bing search?
Problem with Status Quo
Siloed – No communication or connection between traditional search and bots
Unaware – Bing is not very predictive and good at anticipating user needs
Non contextual – Search and bots aren’t aware of their users’ background, preferences, lifestyle, activities, etc.
Aware (location, user, history, etc.)
As this was a concept project, my design process was a bit different from usual. The process was still iterative but focused a lot more on my vision and what Microsoft has learned from their many AI experiments. I started off by looking at the AI products and experiments we had out in the public and examining users’ feedback to those products. Then after I had a good idea of what users liked and didn’t like about our different AI project, whether it’s a chatbot or entertainment app using computer vision, I started to brainstorm what the future of Bing with AI infused would look like. With the users’ feedback on other projects in mind, I was able to design knowing what frustrated users and what users wanted when it came to our next generation experiences.
Through a number of explorations, I was able to land on a temporary solution, shown below. As this was a visionary project, we really don’t know when an experience is “finished.” Our explorations could have gone in so many directions which could all be considered “solutions.” I am glad about what we ended up with given the time constraint. I believe the desktop and mobile experience is really pushing the envelope when it comes to the future of search, encompassing the core elements we set out to incorporate. The experience, shown below in user stories, is truly conversational, predictive, aware, and informed.
Desktop Experience - UN Research Project User Story
Alex, a high school student in NY, researches about the UN for a school project
This desktop experience specifically highlights a conversational and aware Bing. The search engine is able to understand the context of the conversation and continue to provide relevant answers even when Alex has moved on to a different screen. By using this chat model across our search result pages and non-Bing pages, we can allow users to ask all kinds of questions and receive high quality answers.
For example, on the search result pages users can ask questions like:
“Please summarize these articles for me”
Show me primary sources only
“Highlight articles from the New York Times”
And on a non-Bing websites, users can ask questions like:
“Who is this author?”
“Show me the section about UN Peacekeeping controversies”
“Is this source bias?”
For the desktop experience, we also experimented with a number of chat/search models shown below.
Mobile Experience - Sneaker Shopping User Story
Tiffany, a creative in Dallas, shops for sneakers
This mobile experience specifically highlights an aware and informed Bing. By knowing what Tiffany likes and her taste through shopping and search habits, Bing is able to provide her with the items it knows she’ll like. Different advance AR and AI technology is also shown in this experience making shopping easier for Tiffany.
Getting to intern at Microsoft was a great learning experience. I got to work on a total of three projects over the summer. While I was not able to show some of my other projects due to privacy concerns, this specific visionary project was extremely interesting. I was able to take full control of the directions I wanted to take it with minimum constrains. The openness of the project also allowed me to explore many aspects on how a search engine can help make life easier for users. This was my first time working within a big design team and I definitely loved being able to collaborate with other amazing designers. I learned the importance of storytelling and being able to communicate your design decisions to other collaborators and stakeholders within a greater organization.