Ensure Checkout Performance with Ongoing Experimentation

Building the best checkout experience doesn’t end once it’s live on your site. From evolving shopper preferences to new technology, offering a best-in-class checkout requires constant testing and iteration. 

However, experimentation is no small task. For many retailers, whole teams are dedicated to deploying and testing new features and updates. And for others, developer resources may be allocated elsewhere so they don’t have the bandwidth to properly experiment. This leaves retailers with two options:

1. In-house experimentation: Requires dedicated developer resources, but can take months to reach statistical significance and even longer to implement changes universally.

2. No experimentation: Requires no developer resources, but means that retailers are either:

• Not updating their checkout experience and missing out on opportunities to optimize
• Making updates to their checkout without understanding the impact of these changes

At Bolt, we understand that ongoing experimentation is critical to building the gold-standard checkout experience. That’s why we take on the heavy lifting of experimentation. With Bolt, retailers can reallocate their developer resources to other meaningful tasks while ensuring they offer an up-to-date and highly performant checkout experience. 

Bolt offers a variety of experiments, including:

1. Individual retailer experiments: Retailers can not only A/B test new features on their site, but also test the impact of more nuanced changes like feature placement, copy changes, and user interfaces (UI) in checkout.

2. Cross-network experiments: In addition to experimenting on individual retailers, Bolt also runs experimentation across hundreds of brands in the Bolt Network. 

This combination of experimentation allows retailers to select what parts of the checkout experience they want to test individually. Plus, they benefit from cross-network experimentation and ensure any results that positively impact checkout are applied to their experience.

Available Experimentation 

Below is a list of all available experiments that retailers can currently choose to test on their sites. If you are a Bolt retailer and interested in running any of these experiments, please reach out to your Customer Success Manager. 

1. Full Page Checkout Modal

Learn if the current Bolt checkout modal or full page checkout modal performs better with your shoppers.

2. Gift Card Placement 

Test whether to display gift cards in the side cart or within the payment step of checkout.

3. Product Recommendations Placement

• Overview: Decide whether to showcase product recommendations before, during, or after checkout.
• Proof Point: 5.4% increase in average order values (AOV) for Spiceology when product recommendations were showcased during checkout. 
• Customer testmionial: “Not only does Bolt allow us to provide a fast, secure, and streamlined checkout experience to our shoppers, but it experiments and finds ways to make Spiceology’s checkout experience even better on our behalf. Bolt recently ran an A/B test to showcase product recommendations during checkout. As a result, we saw a 5.4% increase in average order values (AOV) when product recommendations were shown—which is a significant impact on our bottom line." - Katie Capka, Director of Ecommerce at Spiceology

4. Alternative Payment Methods in Checkout

Test how offering one (or multiple) APMs impact checkout conversion.

5. Product Page Checkout UI

Enable different checkout user interfaces (UI) for product page checkouts (i.e., checkouts initiated from the “Buy Now” button on a product page). Test the following UI:

• Expanded checkout modal that includes side cart order summary
• Simplified checkout modal that does not include a side cart order summary

Universal Updates

Below is a list of experiments that Bolt has already tested and found to positively impact the user experience. These experiments have been rolled out universally across all Bolt retailers.

1. Google Maps in Delivery Step

• Overview: Add a Google Map view to reduce drop-off at the delivery step of checkout to boost checkout conversion. This provides a visual image for shoppers to do an additional check and confirm if their address is accurate. Bolt experimented showing the Google Map view vs. no Google Map view and saw a lift in both checkout conversion and delivery step conversion. As a result, showcasing Google Maps in checkout was rolled out universally and is available across all Bolt retailers. 
• Proof Points for a leading beauty brand
  • 3.68% relative lift on checkout conversion
  • 2.04% relative lift on delivery step

2. Order Summary for Mobile

Bolt tested order summary placement for mobile devices including:

• Hidden in the top menu 
• Always available in the expanded footer
• As a result, always available in the expanded footer performed better and was rolled out across mobile experiences.

3. Loqate as Address Validator

Address auto-completion has a positive impact on conversion and address validation helps reduce potential support issues with incorrect addresses. Loqate is the market leader in address validation and recommendation services. Bolt is excited to now use Loqate as both our address recommendation and verification service. 

4. Skeleton Loader

Bolt now uses Skeleton Loaders, which are static placeholders for the information that is still missing, in Checkout. Skeleton loading is an industry standard for a faster and more streamlined user experience.

5. One-Time Password (OTP) Refresh

• Overview: One-time passwords (OTPs) let shoppers securely and quickly login to their accounts to enjoy a one-click checkout. Bolt recently updated our OTP screen to make it easier for shoppers to understand:
  • Why they are being asked for an OTP code
  • What action is required
  • How Bolt can assure they are logging into a secure account network
• Proof Points
  • 6.4% lift in login success rate 
  • 1.4% lift in conversion

6. Account Creation Checkbox

The account creation checkbox enables shoppers to seamlessly save their information to create an account so they can enjoy a one-click checkout the next time they come back. Bolt tested different copy variations for this checkbox:

• Provide bullets showcasing the value and benefits of creating a Bolt account
• Simple copy without bullets
• As a result, the version without bullets outlining account benefits to shoppers performed worse so we rolled out the update to include bullets in all account creation checkboxes across Bolt retailers. 
• Proof Point
  • Removing bullets led to a -9.5% decline on account registration.