Example of Solving the Assessment Analytical Test #1

1. Write answers to the following questions:

List the systems from which you collected Internet metrics.

I collected metrics from Yandex.Metric, Google.Analytics, Liveinternet, Ahrefs, Serpstat, Google.Analytics, Topvisor, Roistat, Mailchimp, Unisender and other counters.

Write how you used the metrics and what conclusions you drew.

I used analytical data from all these counters with the goals:

  1. To understand from where visitors come to the site: from search engines, from advertising channels, referral links, banners, e-mail newsletters, etc.
  2. To understand which channel brings more traffic, CPA and CPL conversions, understand which channel brings leads at lower costs.
  3. On the heat map of clicks to understand what buttons, forms, banners site visitors better interact with, what is more clickable, and what is not, conducted experiments, ran A-B tests.
  4. On the heat map and analytical dashboards studied the path of customers, starting from its very first page, to the last, to understand their logic and in order to understand what can be improved in the design, so that the site was more convertible.
  5. In addition, the analysis of user behavior on the site was investigated in order to understand how the content strategy was working. For example, which pages better capture the focus of Blog readers and encourage them to go to the landing commercial pages.
  6.  In Yandex.Metrics I developed my own dashboard of metrics on several screens, which allowed me to understand at any hour of the day or night how healthy the sites are, to see anomalies at a glance.
  7. On Ahrefs, Serpstat, Topvisor, Megaindex, Keys.so and other services I tracked the positions of sites in search engines and analyzed competitors: what pages generate traffic, how many links they put and where, what promotion strategies they use.
  8. I collected information about the cost of clicks and phrases, CTR, CPM, etc. from advertising offices of Yandex.Direct, Google.Adwords, Direct Commander, Elama, Alytics, and calculated the profitability of spinning off advertising campaigns.

Write down what indicators should be used to evaluate paid traffic.

  • CPC (cost-per-click) – cost per click on our ads.
  • CPM (cost-per-mille) – the cost of 1,000 banner impressions of our ads.
  • CPA (cost-per-action) – cost of 1 target action on our advertising (for example, registration on the site).
  • CPL (cost-per-lead) – cost of 1 lead (interested client) on our advertising (i.e. the client not only registered, but also, for example, subscribed).
  • CR (conversion rate) – conversion percentage, which means how many people performed a targeted action (making a purchase, filling out a form, subscribing to a newsletter) in relation to all traffic on a given channel.
  • CTR (click-through-rate) – the percentage of clickability of our ads (the higher the CTR, the more attractive our banners are in terms of inducing people to click on them).
  • Bounce Rate – the percentage of bounce rate, i.e. people who entered the site and closed it in the first 15 seconds or looked at only 1 page and closed it.
  • Email Open Rate – the percentage of people who opened our emails in relation to the total number of emails sent.
  • LTV (Customer Lifetime Value) – the profit or revenue generated by a customer of our company for the whole time he/she is with us (subscription or purchase of services, goods).
  • CCR (Customer Churn Rate) – the churn rate of customers (for example, those who unsubscribe from our paid subscription).

Write down what indicators are used to evaluate e-mail newsletters.

As I mentioned above, this is the Email Open Rate. Also evaluated:

  • CTR – the percentage of those people who clicked on a button or banner in an email.
  • CR – conversion rate, i.e. the percentage of people who made a targeted action on our site to those who did not.
  • Unsubscribe Rate – the percentage of people who unsubscribe from our mailings. This is a very important indicator, which serves as a “bell” that either there is something wrong with the content in our emails, or with the advertising offer, or we are doing too aggressive mailing.
  • Subscriber Growth Rate – the percentage of growth in subscribers to our newsletters.
  • Open Time – the time of opening emails (allows you to evaluate from a marketing point of view when it is better to send a newsletter).
  • Signup Count – how many people filled out the signup form on the site after switching from email newsletters.

2. Show your knowledge when generating the report

Send a link to any report using Summary Tables in Google Spreadsheets.



As an example of an in-depth exploration of website metrics with visualizations.

3. Objective: What will be the ROMI of the advertising campaign?

What we know about the ad campaign:

  • Total impressions: 10,000.
  • CPM = 500 rubles.
  • CTR = 2%.
  • CR to order after going to the landing page = 3.5%.
  • Average order receipt = 1 000 rubles.

ROMI (Return on Marketing Investment) is an indicator that allows you to evaluate the return on investment in advertising.

The formula for calculating ROMI is as follows:

ROMI = (Revenue – Advertising Costs) / Advertising Costs

In our case (and with our data) it will look like this:

ROMI = (Revenue – (Number of clicks * Average cost per click)) / (Number of clicks * Average cost per click)

We don’t know the number of clicks, but it’s easy to calculate if you know the CTR: 10000*0.02 = 200.

We don’t know the cost of advertising, but it’s easy to calculate if you know the CPM: 500/1000*10000 = 5000, and the average cost per click is 5000/200 = 25.

We don’t know the number of orders, but it’s easy to calculate if you know CR: 200*0.035 = 7.

Accordingly, our revenue will be: 7*1000 = 7000 rubles.

Substituting the conditions above we get:

ROMI = (7000 – (200 * 25)) / (200 * 25) = 40%

Thus, we conclude that the advertising campaign was profitable and brought 4 rubles in profit for every 10 rubles invested.

4. Solve this problem

The goal of the product marketplace is to increase the share of regular customers and reduce marketing costs for their return, so we placed 2 variants of banners on our website:

  • Option A: “Install our mobile app in Google Play and get a 500 rub discount on your first order in it” (for Android users).
  • Option B: “Install our mobile application in App Store and get 800 rubles discount on your first order” (for iPhone users).

Both variants of banners were shown to users 3,000,000 times in the first month. Each user was given the opportunity to see the banner in one of the variants only in the first session and to click on the banner only once.

It is known that in the first month:

  • Option A yielded 25,130 clicks, 1,560 mobile app installations (assuming this is also the number of mobile app users in this cohort) and 345 orders in it.
  • Option B yielded 23,750 clicks, 1,345 mobile app installs (assuming this is also the number of mobile app users in this cohort) and 285 orders in it.
  • Customers who clicked on the banners and installed the app placed only one order each.
  • The average check (AOV) of iPhone customers is 10,875 rubles and the average check (AOV) of Android customers is 8,425 rubles.

In the second and third months, the conversion rate of returning users (from cohorts Option A and Option B) into orders remains stable, but their average check decreased by half compared to the first month.

In the second month, 24% of Android customers and 7% of iPhone customers returned to the app from the Option A and Option B cohorts who installed the app in the first month, and in the third month, 5% of Android customers and 2% of iPhone customers, respectively, returned to the app. Some of them ordered but not more than once in the second and third month.

The Marketplace Marketing team asks our analyst to provide digitally sound answers to the following questions:

1. Which banner option performed better in the first month? What metrics show this?

In terms of revenue, the iPhone banner performed better: AOV = 10875 vs. AOV = 8425.
In terms of clicks, installs, and orders, the Android banner performed better – 5.8% more clicks, 16% more installs, 21% more orders).

2. Did the hypothesis that the increased first-order discount for iPhone users would generate more sales based on the first month’s results hold true?

No, it didn’t. It brought in more revenue, but not sales.

3. Which cohort of customers who installed the app in the first month is more valuable to the marketplace based on the three month results – iPhone customers or Android customers?

Revenue from Google Play customers for 3 months:

  • Month 1: 345 * 8425 = 2906625 rubles.
  • Month 2: 83 * 4213 = 349679 rubles.
  • Month 3: 17 * 4213 = 71621 rubles.
  • Total: 3327925 rubles.

Revenue from App Store customers for 3 months:

  • Month 1: 285 * 10875 = 3099375 rubles.
  • Month 2: 21 * 5438 = 114198 rubles.
  • Month 3: 6 * 5438 = 32628 rubles.
  • Total: 3246201 rubles.

Thus, we can see that for 3 months the revenue from AppStore customers was 81724 rubles lower than from Google Play customers.

Consequently, in terms of generated revenue, the more valuable cohort for the marketplace is clients with Android devices. Although they initially brought in less revenue in the 1st month, but later they proved to be more loyal and solvent.

A successfully completed task is a Google Spreadsheet file with a calculation and a Google Docs file that substantiates your answers.