Summary
Shamp uses AI agents that simulate target users to run usability tests at scale. The system speeds up testing and produces structured findings so teams make confident decisions faster.
Development case study
Faster usability signals with AI personas
Project: Shamp
Link to web appRole: Design Engineer
Timeline: 2 months and 3 weeks
Problem
- Recruiting ideal test participants is slow and costly.
- Teams ship features without quick validation.
- Small product teams cannot run frequent moderated tests.
- Existing options rely on human panels that add time and bias.
Target users
- Product managers at startups and SMBs.
- UX researchers and designers.
- Product teams with limited budget for user testing.
Hypothesis
If teams can get fast, reliable usability signals from AI driven participant simulations then they will ship with fewer usability regressions and run more experiments.
Solution
Core idea
Create AI agents that mimic real user personas. Agents run tasks on prototypes or live builds. The system logs errors, timings, and behavioral traces. It outputs structured reports and suggested fixes.
Key capabilities
- Persona editor to define goals, skill level, and common behaviours.
- Task editor to define success criteria and steps.
- Parallel agent runner that executes tasks and records interactions.
- Results pane with task success, error types, time on task, and screenshots.
How it works
- Create a persona with goals and constraints.
- Define a task and success criteria.
- Launch multiple agents that interact with the product.
- Collect transcripts, events, timings, and screenshots.
- Aggregate results into a concise report with recommended fixes.
Shots
Tech stack
Frontend
- Next.js with TypeScript
- shadcn UI components and Tailwind CSS
- Zustand for state management
- React Hook Form for form handling
- Socket.io for real-time UI updates
Backend
- Node.js with TypeScript and Express.js
- MongoDB for persistence
- Vector database for agent memory
AI and infra
- Browser Use Agent for running tasks
- OpenAI for agent logic and test analysis
Expected impact and success metrics
- Time to first insight reduced from days to hours.
- Test throughput increased from 1 human test per week to 5 to 20 agent runs.
- Faster detection of common UX regressions before release.
- Adoption metric tracks percent of sprints using automated tests at least once.
Conclusion
Building Shamp solidified my belief that design and engineering are strongest when they live together. I owned the product experience end‑to‑end, from persona flows and task modeling to the component architecture and real data wiring, so I could iterate in one tight loop and validate ideas quickly.
What I learned
- Treat agents like real, imperfect users: design for flakiness and edge cases, not demos.
- Instrumentation matters as much as UI, good logs, timings and screenshots turn “it feels slow” into actionable fixes.
- Small, shippable steps beat big reveals. A thin, testable vertical slice created momentum and trust.
- Design tokens and accessible states upfront reduce rework, especially in data‑heavy surfaces like analysis views.
The result is a practical system that helps teams see usability risks earlier, with clearer evidence and faster decisions, and a workflow I would reuse on future design‑engineering projects.
See more
Increasing adoption of a unique feature
AllPeeP
Reducing procurement decision time
Sourcify
Want to work with me?
I'd love to hear from you!
Work with me