Traditional experimentation finds winners ~15% of the time. We, on the other hand, find winners 90%+ of the time. Here's how.

‍

UX research has traditionally been dominated by A/B testing: incoming users are randomly shown one of two versions of a website or product for some period of time. This approach allows marketers to compare the performance of the two options and determine which performs better according to some metric, like conversion rate. However, studies from experimentation platforms like Optimizely, VWO, and Convert.com have shown consistently that winners, or variations that out-perform the original, are only found ~15% of the time.

‍

There's another issue. While A/B testing effectively measures uncertainty and produces statistical data on which option is superior, it’s limited to comparing options over a fixed, often lengthy, testing period. During this time period, you're sending a lot of traffic to the losing variation(s), missing out on conversions. Fortunately - there's another popular optimization method that solves for this: multi-armed bandits (MABs). MABs continuously test several “arms”—like different variants of a signup button or several versions of copy text—while tracking their performance. As users visit the site, a higher proportion are served variants with higher success rates, while others may still be shown new or slightly lower-performing variants to gather data on their performance. While this provides a noticeable improvement over traditional A/B testing, it still requires a human to generate variants to test and may miss potentially better options. 

‍

We take this a step further. Here’s where Coframe’s approach differs.

‍

Rather than manually-created, static variants, Coframe combines the power of generative AI, business context, and our unique Optimizer algorithm to constantly optimize your website. 

‍

Coframe continually tests a broad range of options using our Optimizer algorithm;  dropping low-performing arms and using generative AI to recommend new, potentially superior variants, increasing the likelihood of finding the optimal option. Note that generative AI is not acting without permission or supervision on your website; you have full control to approve variations, but AI helps to accelerate the variation creation process significantly. You or your team can also add variants manually; we simply use AI to accelerate the process of creating variants. This method still quantifies uncertainty—so it answers the same questions as traditional A/B testing—but optimizes as it performs its experimentation. What’s more, in an era where personalization is king—think of your tailored Amazon or Netflix homepage—this approach can also take in user-specific contexts, like their demographics or prior interactions, to not only find the best variants but also tailor them to individual users. 

‍

How can you leverage this? It’s simple.

‍

Start by choosing a portion of your website you’d like to optimize—perhaps the copy text on your landing page or the UI on your demo page. Next, define a metric for Coframe to optimize, like clickthrough rate. Then, sit back and let Coframe work its magic. It’ll generate new arms for you and continually test them on your incoming traffic. As users interact with these variants, Coframe iterates the process, dropping and adding new arms based on what works and what doesn’t.

‍

In addition, it might perform segmentation, or partitioning traffic based on data such as demographics or geographic location. Then, we can look at what works best for particular user segments and also track the outcomes of the overall frame (the experiments on all segments). To evaluate how well this works, we might look at both experiment-level holdouts, which measure how much improvement specific experiments provide, and global holdouts, which look at how much improvement the aggregate of all experiments have provided since the beginning. 

‍

‍

‍

‍

But Coframe’s optimization approach extends far beyond just tweaking a piece of copy on your website. It can enhance many aspects of your online presence, from crafting engaging emails and messages to optimizing and fine-tuning your entire website, adjusting ad campaigns over time, and aligning SEO keywords. Managing all these elements manually can be overwhelming, especially for startups and small companies. There are so many moving parts involved in a company’s online presence that it’s simply impractical to rely solely on human effort to monitor, adjust, and modify them. 

‍

Generative AI has delivered on its promise to forever alter how we work—why not let it work for you? With Coframe, you can rely on AI to handle the heavy lifting, leaving you free to focus on more critical strategic aspects of your business.

‍