Timeline
Jun-Jul 2023
Client
A Design-led approach to AI within Mixpanel.
Background
In 2023, Mixpanel launched Spark AI, which enables communication with customer data in plain English using generative AI. You can ask questions related to product, marketing, and revenue data, and Spark AI will generate the appropriate reports and visualizations. The purpose of this tool is to simplify analytics and make it accessible to anyone without technical expertise or struggling with complex SQL or tools. Say goodbye to data queries and complex tools and enjoy a hassle-free data analysis experience, right?
The Challenge
Mixpanel, known for its world-class product analytics, aimed to add AI to its lineup as the AI wave hit in 2023. Facing stiff competition, especially from Amplitude, we took on the challenge of rolling out our AI feature, Spark AI. Our goal was to keep pace with our rivals and lead the way in bringing generative AI into the analytics space.
My Scope
As the Product Design Manager, my primary responsibility was to orchestrate the work of multiple product designers to ensure that our team's output aligned with the company's business objectives. I provided guidance to the team on what to build and how to leverage new AI technologies effectively. Throughout the project, I emphasized the importance of prioritizing user needs and insights to ensure we delivered a product that met their expectations. Our goal was ambitious but clear: we aimed to introduce a conversational AI feature that would revolutionize how users interact with their data. By integrating Spark AI seamlessly into a customer's existing workflow, we can empower customers lacking data analysis expertise to explore their customer data in an efficient and intuitive manner.
Team time
Understanding AI and Ethical Considerations
I led working sessions to ensure that AI integration into our platform was done responsibly. I compiled various resources on AI models and ethical considerations to accomplish this. I gathered several Substacks, guides, and patterns into a Figjam file to facilitate learning and get everyone up to speed on this emerging technology.
Design Principles Workshop
Next, we organized workshops with Product and Engineering leads to establish the basic design principles for integrating AI: Trustworthy, Accessible, Augmentative, and Superhuman. We used a robust and value-driven framework to guide the project. This allowed us to establish a thematic perspective on AI throughout the platform, not just Spark. As designers, we applied humanities to a relatively new problem space.
Project Management
Then, I served in project management capacity by coaching a mid-level product designer named Matt to lead the initiative and maximize his potential for high-impact work. My responsibility was to provide support for Matt and allow him to focus on deep work by keeping the team and leadership updated regularly and delegating in-progress work to other designers. This helped Matt to concentrate on the critical work in Figma without any distractions.
Our Hypothesis
We believed that providing customers with a way to communicate with Mixpanel, like how they interact with customer support, would help them achieve an acceptable initial version of insights. If Spark AI delivers a decent first draft, it will become the preferred approach for constructing queries among our small and medium-sized businesses and mid-market customer segments. Could customers generate charts faster and more efficiently with Spark AI or traditional methods?
Early Access and User Research
I worked closely with my Product Management partner to oversee early access sign-ups, ensuring diverse user feedback. I was responsible for sourcing, recruiting, and conducting nine user research sessions to refine and optimize Spark AI, which resulted in significant insights for broader AI applications within Mixpanel. One of the key takeaways was the potential of AI as a data janitor, alerting customers to unused dashboards and recommending merging duplicate events and properties.
Outcomes
Design-Led Innovation
From conception through execution, Spark AI exemplified a design-led approach, emphasizing the importance of design in product development and strategic direction.
Impact on North-Star Metrics
The implementation of Spark AI positively impacted our North Star metric. It improved the quality and usefulness of generated reports, demonstrating the tangible impact of design-led initiatives on product and business metrics.
11%
Week over week implementation rate
9%
Increase in WoW Current User Retention Rate (CURR)
300+
# of reports viewed by 3+ users within 14 days