Weekly playbook for reading Shopify recommendation KPIs

What this playbook helps you do

You may be looking at recommendation KPIs like CTR and CVR but still be unsure: “Is this good or bad?” “How far can I trust this number?” This playbook turns those fuzzy questions into a weekly checklist you can actually run.

Here you will see, in a format you can start using immediately:
- What to look at once a week
- How to interpret CTR, CVR, and purchase contribution
- How to avoid putting too much weight on thin data

Premise: the 3 main axes of recommendation KPIs

When you evaluate recommendations weekly, it gets much easier if you separate your thinking along these three axes first.

• Impressions: how many times recommendation blocks were shown on users’ screens. This reflects the raw exposure of your recommendation placements.
• Click-through rate (CTR): among the times recommendations were shown, what percentage were clicked. This is your metric for how attractive the recommendations look.
• Conversion rate (CVR) and purchase contribution: how many purchases were generated from clicked recommendations. This shows how much your recommendations are actually contributing to revenue.

On a weekly basis, what matters more than the absolute numbers is the change against the previous week and the balance between impressions, clicks, and purchases. With that in mind, follow the steps below to review your dashboard.

Basic weekly check steps (CTR, CVR, purchase contribution)

• 1. Check overall sales trends in the Shopify admin
  Open Shopify admin → Analytics → Dashboard and quickly review sales, order count, and sessions for the target week. This gives you the base for separating “overall site trend” from “changes in recommendations only.” In weeks where everything is down, you will later need to distinguish whether the issue is recommendations or simply less traffic.
• 2. Open your recommendation app’s analytics
  Open the app you are using (for example: Apps → RecoBoost → Analytics) and set the date range to the last 7 days (or whatever weekly window you use). Choose a view where you can see, by widget, impressions, clicks, CTR, purchases, and revenue.
• 3. Sort widgets by impressions in descending order
  In the analytics view, sort the widget list by impressions, highest first. The reason: recommendation impact starts with the areas that get the most exposure. Begin with high‑exposure placements like cart page recommendations or product page bottom recommendations and work your way down.
• 4. Compare each widget’s CTR with the previous week
  Compare each widget’s CTR with last week or the average of the past few weeks.
  - CTR has dropped significantly: suspect changes in item ordering, stock status, copy, or widget placement.
  - CTR is flat but impressions have changed: suspect changes in traffic sources or page layout.
  If the change is tiny (less than 1 point, for example), it may just be noise. In that case, just jot it down and check whether the same pattern appears again next week.
• 5. Check that you have enough clicks, not just a nice CTR
  To avoid over‑interpreting, always check the click volume behind the rate. For example, if a widget with fewer than 10 clicks shows a big jump in CTR, it is likely just “a few extra clicks happened” and is not solid enough to base decisions on. For weekly decisions, focus on areas with a certain minimum volume, such as widgets that get at least a few dozen clicks every week.
• 6. Review CVR and purchase contribution together with click volume
  Check each widget’s purchases and revenue, and the CVR (purchases divided by clicks) derived from them. At this point:
  - Widgets with many clicks and stable CVR: these are your core slots; keep maintaining product variations and stock here.
  - Widgets with high CTR but low CVR: you are getting attention, but product content or pricing may be misaligned with user needs.
  - Widgets with high CVR but few clicks: there is room to improve visibility and exposure by adjusting placement.
  As with CTR, CVR can look artificially high when click volume is very low and a single purchase happened by chance, so always evaluate CVR together with click volume.
• 7. Write down just 2–3 “points of interest” each week
  Instead of tracking every number obsessively, narrow your “findings this week” down to 2–3 notes. For example:
  - Product page bottom recommendations: CTR down 30% week over week
  - Cart page recommendations: CTR flat but revenue slightly up
  - Home page recommendations: too few clicks to judge yet
  In the next section, you will decide what to tweak based on these notes.

What to watch by placement, and small action ideas

How you read recommendation numbers changes depending on which page they are on. Below are weekly focus points and small improvement ideas for three common placements.

• Product page recommendations (related items, frequently bought together)
  - KPIs to watch: CTR, clicks, CVR
  - What to focus on: because these target users who have already read the product details, CTR does not have to be very high. If CVR is stable, it means you are successfully reaching the people who actually want these items.
  - Small action examples:
  - CTR is low: move the recommendation block slightly higher on the page, or change the caption text (for example, from “Recommended items” to “Frequently bought together with this item”) and watch the numbers for 1–2 weeks.
• Cart page recommendations (upsell, cross‑sell)
  - KPIs to watch: CTR, purchase contribution
  - What to focus on: users here are right before purchase, so even if CTR is not very high, any purchase contribution is valuable. Check weekly whether you are seeing at least some steady purchases.
  - Small action examples:
  - CTR exists but purchase contribution is weak: remove products that are too expensive and adjust the recommendation settings to prioritize small add‑ons, consumables, or related accessories.
• Home page and collection page recommendations
  - KPIs to watch: impressions, CTR
  - What to focus on: many users here have not yet formed clear purchase intent, so purchase contribution alone will tend to look weak. First treat “are they being clicked at all?” as the key checkpoint.
  - Small action examples:
  - CTR is low: raise the recommendation block higher on the page so it appears earlier in the scroll, or reduce the number of displayed items to create a more “carefully curated” feel and improve visibility.

Checklist to avoid over‑interpreting the data

Recommendation analytics can look more meaningful than they really are because of how granular the numbers are. Here is a checklist to avoid common over‑interpretations when reviewing them weekly.

• 1. Check the order of magnitude of clicks and purchases before judging
  Even if CTR or CVR swings a lot, if clicks and purchases are still in single digits, you cannot yet call it a trend. Only use widgets with a reasonably steady volume of clicks and purchases each week as real decision material.
• 2. Do not draw conclusions from a single week’s change
  Campaigns, weather, sales events and the like can cause large one‑week swings. When you see a big change, look at at least 2–3 weeks of data before making “spec change level” decisions.
• 3. Separate overall site trends from recommendation changes
  If overall sessions and revenue are down, and recommendation CTR or purchase contribution is also down, that does not automatically mean recommendations are the problem. Always cross‑check with overall Shopify analytics to isolate whether there is a recommendation‑specific change.
• 4. Do not rely too heavily on a single widget
  If you depend on one placement only, its traffic fluctuations will swing your numbers dramatically. As part of your weekly review, roughly confirm that contribution is distributed across multiple placements such as product pages, cart page, and home/collection pages.
• 5. Question numbers even when they match your expectations
  If you are convinced “this placement must be strong,” you are likely to overlook oddities in the numbers. Each week, consciously scan the dashboard asking, “Is there anything that looks different from what I imagined?” This makes it easier to notice small changes.

Next steps if you use RecoBoost

If you are using RecoBoost, you can take the weekly checks above and turn them into concrete next steps like the following.

• Check performance by recommendation widget
  Go to Apps → RecoBoost → Analytics and review each widget’s impressions, CTR, and revenue in a single list. Each week, sort by impressions and note down your top three core widgets.
• Compare CTR and CVR by recommendation logic
  From RecoBoost’s Recommendation settings, review the logic assigned to each widget (for example, AI similar products, Recently viewed products) alongside the analytics for CTR and purchase contribution. If a logic shows high CTR but low CVR, run small tests switching to another logic.
• Test small placement tweaks on a limited number of widgets
  Instead of changing everything at once, pick just one widget each week to adjust its position or copy, then track the numbers over the next 1–2 weeks. Start with high‑traffic spots like product page recommendations so it is easier to see the effect.