Lens 3

Tool 15.

A/B‌ ‌testing:‌ ‌Product‌ ‌testing‌

This tool seeks to publicize what A / B testing is, how to calculate it and why it is key in the design of products and services. It presents a high level of complexity, and its management requires advanced knowledge of Power BI and knowledge of statistics.


Tool 12. Data analytics for my company, from lens 3, is an introduction to data analysis or data analytics


Scroll To Learn More

Tool 15.

A/B‌ ‌testing:‌ ‌Product‌ ‌testing‌

This guide explains what A / B testing is and why it is key when designing products and services with a gender perspective. The tool is explained on two levels: a first general level, with the aim of guiding product design teams to understand the relevance of this tool in the design phases; and a second specific level, with the purpose of guiding those who occupy technical positions in the step-by-step application of these tests. For the technical profile it is necessary to have knowledge of inferential statistics and advanced use of Power BI

Growing or consolidated companies that are designing new products or services are challenged to know which changes or innovations are the ones that have the best results for their customers. For this case, being able to implement A / B testing in the design process of those products or services can provide concrete evidence about what works best and for whom. There are women and girls who do not have access to various products and services and many companies are designing new value propositions in order to reduce gender gaps. In such a way that when introducing products and services to customer profiles who were not previously accessing them, it is key to be able to verify what works best in order to reach more people. 

Tool Benefits

Using A / B tests with the parameterization of relevant / interest variables such as sociodemographic variables is a key aspect if products or services are being designed with a gender perspective. This is because it allows to quickly know the level of response to the design or the way of communicating a product or service based on the sex and age of the clients.

A / B tests are particularly useful if the company designs products that are sold online or provides services on web platforms, since they are quick to implement and the benefits of carrying out this test are visible in the short term.

In terms of the gender approach, it is key to be able to design products or services that are tailored to the people we want to reach. For example, if we are designing a new product or service aimed at women and we want to verify which communicational message works best in terms of sales conversion, then performing an A / B test will allow us to test different messages and select the one that worked best with the profile of desired women. Another ideal situation to use these tests is when it is known that some products or services are not reaching the profile of women who are wanted to be considered. In this case, a test can be designed that compares the performance of different messages or characteristics in both men and women, and then compares which ones work better in each group. 

As postulated by the gender approach, people are not a homogeneous group. Women and men have different needs, perceptions and realities. For this reason, A / B tests allow the design of products or services that adapt to the various characteristics of both groups. 

What are A/B Tests?

It is known as A / B testing or A / B testing, the creation of two (or more) versions of a product or service to determine which of them best meets the objectives that have been set. It is widely used for the development of web pages and applications, where communication messages, page layout, the inclusion or not of certain elements can improve the achievement of objectives.

In an A / B test, content “A” is randomly shown to half of the users and content “B” to the other half, to then measure and analyze the results of each one of them based on certain variables, such as conversion, engagement and / or rebound.

In the web application development environment, A / B testing is commonly used when the objective is to:

Get insights about preferences

The first goal of A / B testing is to find out what users prefer. That is, what type of elements or messages make them more likely to click on an ad, become potential customers, make a purchase or stay on a website. In terms of the gender approach, it is used to verify if there are preferences based on the sex of the people for the different elements or messages to be piloted.

Increase conversion rate

A / B testing is one of the main tools of the CRO or Convertion Rate Optimization, which refers to a series of systematic improvements so that a content generates more leads, subscriptions, conversions, clicks, etc. In terms of the gender approach, the obstacles that prevent women and girls from having full access to the products and services offered can be identified. From this analysis, it is possible to compare what changes should be introduced to a sales, communication or positioning process to generate greater arrival or loyalty to the product / service among women.

Improve user experience

Finally, A / B tests allow us to perfect the content that we offer to those who visit us so that their user experience is more and more personalized and according to their expectations and needs. In terms of the gender approach, it is possible to compare what changes should be made to the value proposition to generate higher levels of satisfaction among women.

Example of the technical calculation of the A / B test

The following explains how the calculations are performed in order to verify the results of an A / B test. The video explains step by step how to do the A / B experiment calculation using Power BI. For this, it is recommended that those who carry out the technical exercise have advanced knowledge of Power BI and knowledge of inferential statistics. 

To simplify the explanation, the case of an advertising campaign via Facebook is used. You are going to promote a product from the catalog and, before launching the campaign, you want to pilot which content is the one that will generate the most conversions (clicks on the buy button). To do this, they design an A / B test, where they will show an advertisement for their product to a group of users and another advertisement to another group. The selection of which advertisement to show to each user is random, that is, each one has a 50% probability of seeing either of the two advertisements to be piloted.


Video Tutorial (in Spanish)

In the following links you can see a tutorial that explains step by step how to perform an A / B test using Power BI:

Did you know?

You can turn on subtitles on a Youtube video in a different language! Just click the CC button at the bottom of the video, then click the gear icon to chose your translation language.

Additional Resources:

Presentation about A / B testing:


AB Tasty. The AB testing guide


Optimizely. Otipedia – Optimization Glossary. A/B Testing:


Standard normal table:


Google Optimize


In case of A / B tests on a web page, Google has a free product called Optimize, which is part of the Marketing platform for companies. With Google Optimize, A / B tests can be designed without the need for advanced statistical knowledge as other tools do.

For example, if an organization has designed a shopping cart for its products on a website and would like to test which sales messages generate the most purchases, then it can test various messages using this tool. The main advantages of using this tool is that it allows you to configure the entire A / B test from one place and the results are generated automatically with the Google tool.

The statistical knowledge required to use the tool is minimal and is only required to analyze the results, since Google Optimize performs the calculations automatically. Also, there is no need to design and host two web pages in parallel. From the tool itself you can edit the site and Google takes care of showing the different versions randomly to the different users.

Deetken Impact

Suite 500 – 210 West Broadway
Vancouver, BC V5Y 3W2

+1 (604) 731-4424 ext. 110


Pro Mujer

1129 Northern Blvd #404
Manhasset, NY 11030

+1 (646) 626-7000


This toolbox is made possible by the support of the American People through the United States Agency for International Development (USAID) under the terms of Contract No. AID OAA-C-17-00090. The contents of this toolkit are the sole responsibility of Deetken Impact and Pro Mujer and do not necessarily reflect the views of USAID or the United States Government.