Maximizing A/B Testing: The Portfolio Strategy for Ecommerce Leaders

Most ecommerce brands treat A/B testing like a slot machine. They pull the lever on whatever test idea sounds promising that week, hope for a winner, and move on when it loses. There's no system behind what gets tested, no logic connecting one experiment to the next, and no compounding effect building over time.

The result is predictable: a few scattered wins that never add up to real growth, and a testing program that feels more like busywork than a revenue driver.

The problem isn't the tests themselves. It's the lack of a portfolio behind them. A single test, no matter how well-designed, can't carry a growth program. You need a mix of quick, low-risk experiments alongside bigger bets that can move your core metrics in ways that small tweaks never will.

That's the idea behind what we call the Portfolio Strategy, an approach to A/B testing that balances risk, speed, and scale across three distinct types of experiments. It's the framework we use to drive consistent, compounding growth for brands like Dr. Squatch, Hyperice, and Amika, and it works because it treats testing as an investment strategy rather than a guessing game.

Here's how it works, why it matters, and how you can apply it to your own optimization program.

The Foundation of the Portfolio Strategy

Before we explore the three types of tests that form the core of our Portfolio Strategy, it's important to understand the philosophy behind this approach.

The Portfolio Strategy is the idea that sustainable growth comes from a balanced, diversified approach to testing. Just as a financial portfolio balances high-risk, high-reward investments with more stable, consistent performers, our testing strategy aims to create a balance between quick wins, steady improvements, and potential breakthroughs.

This approach allows us to:

1. Mitigate risk: By not putting all our eggs in one basket, we ensure that a single failed test doesn't derail the entire optimization effort.
2. Maximize learning: Different types of tests yield different insights, giving us a more comprehensive understanding of user behavior and preferences.
3. Maintain momentum: Quick wins from iterative tests keep the team motivated while we work on more substantial changes.
4. Drive both short-term and long-term growth: We can achieve immediate improvements while also working towards larger, transformative changes.
   ‍

Now, let's examine the three types of A/B tests that form the backbone of our Portfolio Strategy.

Want to see the Portfolio Strategy in action? 

I've created a video that dives deep into how we've used this approach to drive massive growth for 8 and 9-figure ecommerce brands:

A/B Testing Strategies For Ecommerce Brands to Maximize Growth

In this 4-minute breakdown, I explain how we balance iterative, substantial, and disruptive tests to achieve both quick wins and long-term breakthroughs. You'll see real examples of how we've applied this strategy to generate millions in additional revenue for our clients.

Three Kinds of A/B Trials

1. Iterative Tests

Iterative tests are your daily bread and butter. They involve low risk, require minimal resources, and are quick to set up. We're talking about minor modifications to existing elements on your site. You might change the destination page of button, test a different headline, or adjust the placement of an existing element. 

While these are small changes, over time they can result in significant gains. The power of iterative tests lies in their compounding effect. A 2% improvement here, a 3% improvement there – these add up over time and across your entire funnel.

Key Characteristics of Iterative Tests:

• Low risk
• Quick to implement
• Require minimal resources
• Can be run frequently
• Results are often immediately actionable
  ‍

When to Use Iterative Tests:

• When you want to fine-tune existing elements
• To quickly test different variations of copy, design, or functionality
• When you need to gather data quickly to inform larger decisions
  ‍

SplitBase ran an A/B test on a client's website navigation, comparing visually highlighted "Shop Bundles" buttons against simple text links. Surprisingly, the text link-only variation gained 130% more clicks, challenging the assumption that visually prominent elements always perform better.

This test exemplifies the power of iterative testing. A simple change – switching from visual buttons to text links – led to a significant improvement in click-through rates. It's a reminder that sometimes, less is more, and that our assumptions about what works best can often be wrong.

An example of  an iterative test would be as follows:

Customer Quote Headlines:

SplitBase tested a new landing page for beauty brand Amika, using a customer quote as the headline. This approach proved to be a clear winner, likely because the language resonated with customers and added social proof to the landing page.

This test demonstrates how leveraging user-generated content in key areas can significantly impact conversion rates. It's a substantial change that goes beyond tweaking existing elements, potentially changing how visitors perceive and interact with the brand from their first touchpoint

2. Significant Tests

Significant tests demand more effort. They may require a fresh copy, design assistance, and coding. We're either significantly changing user behavior or introducing something new. It's like renovating a room rather than just rearranging the furniture.

These tests often involve changes to key pages or processes on your site. They might include redesigning entire sections on product pages, overhauling your side-cart, or introducing new features to your site.

Key Characteristics of Significant Tests:

• Moderate risk
• Require more resources and time to implement
• Often involve multiple elements or pages
• Results can have a substantial impact on key metrics
  ‍

When to Use Significant Tests:

• When iterative tests have plateaued in effectiveness
• To address known pain points in the user journey
• When introducing new features or functionality
• To align your site with evolving customer expectations or industry trends
  ‍

For example, the following is a significant test:

In-depth Product Information:

For a fashion client whose products included variations in material, SplitBase introduced a new product page design with a fabric info chart. This change increased conversion rates by 26.8% by improving the customer experience and removing barriers to purchase.

This example shows how providing more detailed, relevant information can significantly impact purchasing decisions. It's a substantial change that required careful design and implementation, but the payoff in terms of improved conversion rates was considerable.

‍

3. Disruptive Test

Here's where things start to get really interesting. Disruptive tests are high-risk, high-reward endeavors. They usually require significant changes to your site or business model and need extensive collaboration as well as substantial design effort. These tests might completely change how you approach e-commerce.

Disruptive tests often challenge fundamental assumptions about your business model, pricing, user experience, or value proposition. They're not for the faint of heart, but when successful, they can lead to exponential rather than incremental growth.

Key Characteristics of Disruptive Tests:

• High risk, high potential reward
• Require significant resources and buy-in from leadership
• Often involves fundamental changes to the business model or user experience
• Results can be transformative for the business
  ‍

When to Use Disruptive Tests:

• When you're looking for step-change growth
• To stay ahead of or create industry trends
• When current business models or user experiences are becoming outdated
• To differentiate yourself in a crowded market
  ‍

A prime example of a disruptive test is the introduction of a bundle builder. When added to Birthdate.co's website, it greatly improved conversions, average order value, and overall return on marketing spend for the personalized gift brand. 

This test fundamentally changed how customers interacted with the brand's products, allowing for a more personalized, engaging shopping experience. It's a disruptive change because it alters the entire product selection and purchasing process, potentially appealing to a whole new segment of customers or dramatically increasing the value proposition for existing customers.

However, it's critical to note that disruptive tests don't always yield positive results, and what works for one brand may not work for another. In another case, removing a bundle builder from a client's website showed no changes in key metrics. This highlights the importance of testing assumptions, even when they've proven successful elsewhere.

Real-World Application of the Portfolio Strategy

Now that we've explored each type of test in detail, let's look at how the Portfolio Strategy might play out in practice over time for an e-commerce brand.

Month 1-2: Iterative Testing Phase

• Week 1-2: Test different value propositions in product page headlines
• Week 3-4: Experiment with reorganizing links in the navigation bar
• Week 5-6: Try various image types (lifestyle vs. product-only) on category pages
• Week 7-8: Test different moving educational sections higher up on product pages
  ‍

Month 3-4: Significant Testing Phase

• Weeks 1-4: Redesign product pages to prominently feature user-generated content
• Weeks 5-8: Implement a section featuring new bundles on product pages
  ‍

Month 5-6: Disruptive Testing Phase

• Weeks 1-8: Develop and test a new pricing model, such as a subscription-first approach for certain product lines
  ‍

This timeline demonstrates how the Portfolio Strategy allows for continuous improvement (through iterative tests) while also working towards more substantial changes (significant tests) and potential breakthroughs (disruptive tests).

The Power of the Portfolio Strategy

The goal of the Portfolio Strategy isn't just about finding that one big win. It's about creating a systematic approach to improvement that compounds over time. Some tests will fail – that's part of the process. The key is to learn from each test, iterate, and keep pushing forward.

Here's why this strategy is so effective:

1. Balanced Risk: By running different types of tests simultaneously, you're not putting all your eggs in one basket. If a disruptive test fails, you still have the gains from your iterative and significant tests.
2. Continuous Improvement: Iterative tests ensure that you're always moving forward, even if in small steps. This continuous improvement can add up to substantial gains over time.
3. Learning at All Levels: Each type of test provides different insights. Iterative tests might tell you about user preferences for specific elements, while disruptive tests can reveal larger truths about your business model or value proposition.
4. Adaptability: The Portfolio Strategy allows you to adapt to changes in your business or market quickly. You can shift focus between different types of tests as needed.
5. Motivation and Buy-In: Quick wins from iterative tests can help maintain team motivation and stakeholder buy-in, which is crucial when working on longer-term significant or disruptive tests.
   ‍

Taking Action: Implementing the Portfolio Strategy in Your Business

Now that you understand the core elements of the Portfolio Strategy, it's time to put this knowledge into action. Here are some steps to help you get started:

1. Evaluate Your Current A/B Testing Efforts: Start by reviewing your existing A/B testing practices. Are you only focusing on minor tweaks, or have you explored more significant and disruptive tests? Understanding where you currently stand is essential for identifying gaps and opportunities.
2. Develop a Balanced Testing Plan: Based on your evaluation, develop a testing plan incorporating a mix of iterative, significant, and disruptive tests. Consider your business goals, available resources, and risk tolerance when planning your tests. Remember, the goal is to maintain a balance that drives both short-term wins and long-term growth.
3. Create a Testing Roadmap: Outline a timeline for your tests, similar to the example provided earlier. An A/B testing roadmap helps ensure that you're continuously running tests and gathering valuable data, while also allowing for flexibility to pivot when needed.
4. Invest in the Right Tools and Resources: Effective A/B testing requires the right tools and resources, from analytics platforms to skilled team members. Invest in tools that provide deep insights and support a wide range of testing types. Also, consider bringing on board experts in UX design, data analysis, and copywriting to maximize the impact of your tests.
5. Focus on Customer Insights: Use each test as an opportunity to learn more about your customers. What resonates with them? What are their pain points? How do they navigate your site? By focusing on customer insights, you can make more informed decisions that drive meaningful improvements.
6. Iterate and Optimize: The beauty of the Portfolio Strategy is its iterative nature. Each test result, whether a success or failure, provides valuable information. Use these insights to refine your approach, develop new hypotheses, and run subsequent tests. Over time, this iterative process will lead to more refined strategies and greater success.
7. Communicate Results and Learnings: Regularly share your test results and learnings with your team. Open communication helps maintain momentum, fosters a culture of continuous improvement, and encourages collaboration across departments.
8. Stay Agile and Adaptive: The digital landscape is ever-changing, and so are your customers' needs and expectations. Stay agile in your testing approach, be willing to adapt, and never be afraid to challenge assumptions or try something new. The Portfolio Strategy provides the flexibility needed to stay ahead of the curve.
9. Consider Professional Guidance: While the Portfolio Strategy can be implemented in-house, many brands benefit from expert guidance. SplitBase offers comprehensive conversion optimization services that can help you implement this strategy effectively, from analytics and customer research to A/B testing and implementation.
   ‍

The Portfolio Strategy we've implemented with 8 and 9-figure brands could be your key to steady improvements and breakthrough wins. By balancing iterative, significant, and disruptive tests, you create a robust, adaptable approach to conversion rate optimization that can drive substantial growth for your e-commerce brand.

Remember, the path to optimization is not always linear. There will be ups and downs, successes and failures. But with a well-structured testing strategy, each test – regardless of its outcome – brings you one step closer to understanding your customers and optimizing your e-commerce experience.

Work with SplitBase

The Portfolio Strategy we've implemented with 8 and 9-figure brands could be your key to steady improvements and breakthrough wins.

Ready to take your A/B testing to the next level? 

Learn how SplitBase can help drive significant growth for your ecommerce brand. 

Get a proposal

Frequently Asked Questions

What is the Portfolio Strategy for A/B testing?

The Portfolio Strategy is an A/B testing framework that balances three types of experiments: iterative (low-risk, quick wins), significant (moderate effort, meaningful impact), and disruptive (high-risk, high-reward). Instead of running random tests, you build a diversified testing program where each experiment type serves a different purpose, and the results compound over time.

How is the Portfolio Strategy different from standard A/B testing?

Standard A/B testing usually means picking whatever test idea sounds good this week and hoping it wins. The Portfolio Strategy adds structure and intentionality by categorizing tests by risk level and expected impact, then running a balanced mix across all three categories. This way, you always have quick wins building momentum while bigger experiments work toward step-change growth.

How do I decide the right mix of iterative, significant, and disruptive tests?

There's no universal ratio, but a good starting point for most brands is to weight the program toward iterative tests (roughly 50-60% of your testing volume), with significant tests making up 30-40%, and disruptive tests at around 10-20%. Your specific mix will depend on your traffic volume, risk tolerance, and where you are in your optimization maturity. Brands with less testing history should lean heavier on iterative tests to build a baseline of learnings before running disruptive experiments.

What happens when a disruptive test fails?

That's exactly why the Portfolio Strategy works. Because you're running iterative and significant tests alongside your disruptive bets, a single failure doesn't stall your entire program. The smaller tests keep producing gains and learnings while you analyze what went wrong with the bigger experiment. A failed disruptive test still teaches you something important about your customers' behavior or preferences, and that insight feeds your next round of hypotheses.

How much traffic do I need to run a Portfolio Strategy effectively?

You need enough traffic to run multiple tests either simultaneously or in quick succession. For most ecommerce brands, that means at least 50,000 to 100,000 unique monthly visitors to your site (or to the specific pages you're testing). If your traffic is lower, you can still apply the framework, but you'll need to run tests sequentially rather than in parallel, and you should focus more heavily on iterative tests that require smaller sample sizes to reach statistical significance.

Can I implement the Portfolio Strategy without a CRO agency?

Yes, but it requires discipline and the right skill sets. You'll need someone who can run analytics analysis and qualitative research to generate strong hypotheses, a designer and developer to build test variations, and someone with enough statistical literacy to interpret results correctly. Many in-house teams can handle iterative tests well but struggle with the design and development resources needed for significant and disruptive experiments.

‍