Cocreator
Date
2022-2023
Role
Product design, Interaction design, Visual design
Description
Transforming Paint into a creator focused app.
Overview
Team
For the major Cocreator work I worked with a Principal designer for Paint and inbox apps. We worked together through iteration, exploration, and refinement of what would be Cocreator.
High level
Cocreator is an experience that allows users to create visually at a high level without an advanced skillset within Paint’s approachable experience.
Above: Screen recording of Paint app showcasing creator updates the current in experience
Objectives and Goals
- Improve Paint’s digital art creation experience by leveraging modern generative AI capabilities.
- Improve the perception of Paint to be seen as a more capable digital creation tool.
- Demonstrate value with Copilot + NPU enabled devices.
Challenge
Digital art is as big as it has ever been, and users are constantly seeking tools that enable them to create quickly and creatively without a steep learning curve. Traditional digital creation methods can be time-consuming and require a level of expertise that casual users might not possess. These perceived and experienced obstacles can prevent would be creators from accomplishing their creative goals.
Context
This track of work was part of a broader effort to improve Paint. It was also at a forefront of efforts to showcase generative AI capabilities in Windows. It was also an example of a feature built for Copliot+ PC’s the next generation of Windows devices built to run powerful local models.
Research & Discovery
Research came in multiple forms over the course of the project. Text based image generation was at the start of the work a very new pattern. So while normally I am able to leverage more competitive analysis and existing heuristics, for Cocreator it felt like we were very much learning as we go at a fast pace.
Magic Paint
One interesting flashpoint was a viral trend showing Stable Diffusion being used in combination with crude MS Paint drawings. Beyond being a technical demo it showed how Paint’s un-intimidating simplicity lent itself towards these new creators coming from a non-digital art background.
With these exciting developments and the release of the stable diffusion model as open source I was able to work with a design engineering colleague in a hackathon on a prototype derived from the concept. And with that “Magic Paint” was born.
Above: An early viral image to image demo using stable diffusion
Then Magic Paint demo ended up being very popular and ended up being shared with colleagues across windows. With an overall momentum building for AI at Microsoft I was able to explore further what these new experiences could look like in Paint.
Above: An early exploration of Magic Paint which would become Cocreator done during an internal Hackathon
Magic Paint had another breakthrough when generation speed began to improve, it soon became clear that “real-time”(continuous generation) was something that was going to be possible. With that I hacked together a demo to show a potential real-time experience for an internal audience.
User’s responded well to demos and prototypes of Paint updated with AI capabilities. Existing research had also shown that users were interested in doing creative more tasks in Paint.
Implimentation
There were two big stages for launching Cocreator. Stage one when Cocreator first launched it was text to image powered by Dalle. Stage two was when Cocreator was updated to be image to image and run on Copliot + devices. The text to image feature was then rebranded as Image Creator. There was friction but some insights that we were able to derive from this unorthadox release.
Style selection
Style selection was an area where the experience was able to go beyond the prompt box based experience. One thing that was an interesting facet of image generation was that describing an image style was useful for better and more consistent results and it was something that was particularly difficult for people to describe/write visual style. Given that I was a big fan of style selection menu as a means to simplify the process using a visual que without limiting the creative and potential simple potential of the prompt box.
Above: An early version of the style selection flyout control left and final design right
Initially we had a range of images representing the styles but for consistency and for better communicating the differences I made the Paint logos in different styles with stable diffusion and some post processing work for a point of extra delight. This control ended up being extremely successful as far as use with the majority of users generating with an image style selected.
Preview and selection
Above: Selection of wireframes made during explorations of panel experience for iterative cocreation
With that I felt that true active co-creation would require minimal active management. A single action to confirm and add to the canvas. While the user is creating what they are interested in on the canvas they should not have to worry about managing the panel.
Above: Cocreator panel preview for design September 2024 release
The final control put a higher emphasis on the preview above all other elements to allow passive review while Painting. The hover preview interaction would allow the user to quickly view the generation on the larger canvas without having to switch context.
Discoverability and user education
One area where we say a potential need and opportunity was the area of user education and discovery within Paint. Generative AI patterns are new to most users and there were a few aspects of the interaction that were useful to communicate. Also we had a desire to drive discovery of these new capabilities.
Above: Cocreator first run experience dialog
This took us towards implementing a first run experience. While there are many avenues for addressing our core goals a conventional model view aligned with our initial needs of discovery and education of the feature. For the FRE modal we new that we would have limited user bandwidth so we focused communicating the need for a prompt, the function of selecting a style, and the iterative process. Beyond the content we also wanted to use visuals to build excitement for the experience.
Conclusion
Impact
There was a lot more talk about Paint than in recent history but one aspect became clear. Paint Cocreator along with the Creator updates, helped build an improved impression of the app. Colleagues in the Surface team brought spoke about people using Cocreator and connecting with the Cocreator feature and giving feedback. Paint Cocreator was called out by CEO Satya Nadella during an interview with Wall Street Journal covering the launch of Copliot+ PCs. The functionality and Paint are featured prominently accross all windows social media channels.
Above: Select feedback from users on the new features
Paint Cocreator was called out by CEO Satya Nadella during an interview with Wall Street Journal covering the launch of Copliot+ PCs.
Above: CEO Satya Nadella talking about Paint Cocreator during an interview with Wall Street Journal
Final thoughts
Paint’s super power is how un-intimidating it is, this approachability allows it to be a tool for creativity for those who would otherwise have difficulty with an advanced creative tool. With Powerful AI capabilities what was once a more advanced workflow. Patterns for Generative AI tools have come a long way but are ultimately still very new. The extent to which people navigate these patterns needs to be better understood. As Cocreator has started to be delivered to larger audiences we are focusing on next steps for Cocreator to refine the experience, drive feature discoverability, and build stickiness.