UX research has traditionally been dominated by A/B testing: incoming users are randomly shown one of two versions of a website or product for some period of time. This approach allows marketers to compare the performance of the two options and determine which performs better according to some metric, like conversion rate. Recently, however, multi-armed bandits (MABs) have emerged as a promising alternative to the traditional A/B testing paradigm. 

‍

While A/B testing effectively measures uncertainty and produces statistical data on which option is superior, it’s limited to comparing just two options over a fixed, often lengthy, testing period. On the other hand, MABs are not subject to these limitations. MABs continuously test several “arms”—like different variants of a signup button or several versions of copy text—while tracking their performance. As users visit the site, a higher proportion are served variants with higher success rates, while others may still be shown new or slightly lower-performing variants to gather data on their performance. While this provides a noticeable improvement over traditional A/B testing, it still requires a human to generate variants to test and may miss potentially better options. 

‍

Here’s where Coframe’s unique approach comes in. Rather than a static set of manually created variants, which would require constant monitoring to evaluate and update, Coframe leverages MABs along with the power of context and generative AI. 

‍

It continually tests a broad range of options while dropping low-performing arms and using generative AI to create new, potentially superior variants, increasing the likelihood of finding the optimal option. This method still quantifies uncertainty—so it answers the same questions as traditional A/B testing—but optimizes as it performs its experimentation. What’s more, in an era where personalization is king—think of your tailored Amazon or Netflix homepage—this approach can also take in user-specific contexts, like their demographics or prior interactions, to not only find the best variants but also tailor them to individual users. 

‍

So how does it work? It’s simple. Start by choosing a portion of your website you’d like to optimize—perhaps the copy text on your landing page or the UI on your demo page. Next, define a metric for Coframe to optimize, like clickthrough rate. Then, sit back and let Coframe work its magic. It’ll generate new arms for you and continually test them on your incoming traffic. As users interact with these variants, Coframe iterates the process, dropping and adding new arms based on what works and what doesn’t. In addition, it might perform segmentation, or partitioning traffic based on data such as demographics or geographic location. Then, we can look at what works best for particular user segments and also track the outcomes of the overall frame (the experiments on all segments). To evaluate how well this works, we might look at both experiment-level holdouts, which measure how much improvement specific experiments provide, and global holdouts, which look at how much improvement the aggregate of all experiments have provided since the beginning. 

‍

‍

‍

‍

But Coframe’s optimization approach extends far beyond just tweaking a piece of copy on your website. It can enhance many aspects of your online presence, from crafting engaging emails and messages to optimizing and fine-tuning your entire website, adjusting ad campaigns over time, and aligning SEO keywords. Managing all these elements manually can be overwhelming, especially for startups and small companies. There are so many moving parts involved in a company’s online presence that it’s simply impractical to rely solely on human effort to monitor, adjust, and modify them. 

‍

Generative AI has delivered on its promise to forever alter how we work—why not let it work for you? With Coframe, you can rely on AI to handle the heavy lifting, leaving you free to focus on more critical strategic aspects of your business.

‍