Methodology

How verdicts get built

Every recommendation Shared Learnings produces is grounded in named sources, weighed against each other, and traceable back to where the evidence came from. This page describes the mechanism in plain terms. No black box, no editorial gut feel, no paid placement.

If you want the list of sources we draw from, see the Sources we monitor page. This page is about what we do with them.

Pipelines

Three pipelines, three roles

The engine runs three distinct content pipelines. Each pulls from a different mix of sources depending on the question being asked.

Settings Checker
Per-setting verdicts
You ask whether to enable or disable a specific ad-platform setting (Performance Max, broad match, auto-applied recommendations, attribution windows, etc.). The engine triangulates across the paid-media trade press, practitioner communities, and first-party tests submitted by the Shared Learnings community to produce a Recommended, Not Recommended, or Context-Dependent verdict with cited sources.
Insights
On-demand research
You ask a broader question (a topic, a tactic, a tradeoff). The engine returns a personalised answer tied to your account type, monthly spend, and the active business context. Same triangulation pattern as Settings Checker; the difference is the scope of the question and the level of personalisation.
Platform News
Daily ingestion
A daily refresh built primarily on the six official platform newsrooms, with the trade publications filling in where platform coverage of a category is sparse. Every article is classified by AI against a six-platform whitelist before insertion, so the feed reflects what the platforms themselves announced.
Evidence

What counts as evidence

Not every webpage carries the same weight. The engine classifies evidence into four broad categories and treats them differently when synthesising a verdict.

01
First-party tests from the Shared Learnings community
Practitioner-submitted A/B tests with stated platform, setting, hypothesis, result direction, magnitude, and (where available) a verifying screenshot. These are the highest-trust evidence class because they're dated, single-variable, and attached to a real account on a known platform. The submitter's own tests never influence their own verdict (your data informs the community, not yourself).
Highest weight when present
02
Official platform documentation and newsrooms
Authoritative on what the platform itself supports, deprecates, or rolls out. We treat these as the source of truth for platform behaviour questions ("does Performance Max support search exclusions?") but not for "is it worth using?" questions, because platforms are not impartial about their own products.
Authoritative for platform behaviour
03
Paid-media trade press
Established publications with editorial standards (Search Engine Land, Search Engine Journal, PPC Hero, Wordstream, Optmyzr, AdExchanger, Digiday, Adweek, Social Media Today, Marketing Dive, and others). Strong for industry-wide context, large-N benchmark studies, and post-rollout reality checks. Weaker for "what should I do tomorrow" because the audience is broad and the advice is generalised.
Strong for context, broad on specifics
04
Practitioner communities
Reddit subreddits where working practitioners discuss what's actually happening in their accounts right now (r/PPC, r/googleads, r/FacebookAds, r/PPCMarketing, and others). Weaker on rigour than first-party tests, stronger on currency than trade press. Useful for surfacing the lived reality of a platform change ("everyone seeing the same thing?") and for catching shifts before they hit the trade press.
Currency over rigour
Exclusions

What we deliberately don't count

The engine actively filters certain types of content out of the evidence stream. Inclusion would weaken every verdict the system produces.

Filtered out

Content that never reaches the synthesis layer

  • Undated tips-and-tricks posts. If we can't establish when the advice was written, it can't be weighted against newer evidence.
  • Evergreen guides and listicles. Useful as background reading, not as primary evidence for a verdict on a specific setting.
  • Content-farm and SEO-spam pages, including AI-generated thin content. The classifier rejects on source rather than content alone.
  • Opinion-without-data pieces. A practitioner saying "I think X is bad" without test data is weighted at zero for the verdict.
  • Paid placements, sponsorships, and advertorials. We don't accept them and we filter detected ones out.
  • Posts older than the platform's relevance window. Ad platforms change rules quarterly; a 2023 post about Google Ads bid strategies often describes a system that no longer exists.
  • The submitting user's own tests, when generating their own verdict. Self-influence would distort the signal back at them.
Synthesis

How a verdict gets to its category

After the engine collects the relevant evidence and weighs it against the classes above, the synthesis layer produces one of three outcomes. The category itself is the answer; the cited sources are the receipt.

Outcome A
Recommended
The weighted evidence agrees that enabling this setting is the right default for most accounts in your context. First-party tests, trade press, and practitioner discussion all point in the same direction. The verdict still names every source so you can disagree with the underlying data if you have a reason to.
Outcome B
Context-Dependent
The evidence agrees that the answer depends on something specific (account type, spend bracket, vertical, attribution model, campaign goal). The verdict names the dimension and what each side of it should do. This is the most common outcome on advertiser settings that exist because different account types genuinely need different defaults.
Outcome C
Not Recommended
The weighted evidence agrees that disabling or avoiding this setting is the right default. Often because the setting auto-applies platform changes the practitioner can't reverse, or because measurement loss outweighs the surface-level gains. The verdict cites the evidence that ties out to this conclusion.
Recency

Why old advice loses weight

Ad platforms change behaviour on a quarterly cadence. A piece of advice that was correct in 2024 is often wrong in 2026 because the platform feature it described doesn't exist anymore, or now defaults to the opposite behaviour. The engine treats time as a weighting input, not just a sort order.

In practice this means three things:

  • Older sources count less. A trade-press article from twelve months ago carries less weight in the synthesis than one from the last quarter, because the platform itself has likely changed since.
  • Platform announcements reset the clock. When the platform itself publishes a change to a feature, older third-party content about that feature is effectively demoted automatically.
  • Verdicts are generated fresh on demand. There is no cache of stale verdicts. Every check pulls the current evidence at the moment you ask.

If you want to see what the engine thinks today, run the check. If you want to see what it thought six months ago, you'll have to wait for someone to invent time travel.

Platform News

How the news feed stays accurate

The daily news pipeline pulls candidate articles from across the open web (prioritising the six official platform newsrooms, with the trade publications filling in where coverage is sparse) and runs every candidate through a Claude-based classifier before insertion. The classifier doesn't just reject low-quality articles; it rejects on three independent axes.

  • Scope. Is this article about one of the six tracked ad platforms (Google Ads, Meta Ads, TikTok Ads, Microsoft Advertising including LinkedIn, Snapchat Ads, Pinterest Ads)? Articles about adjacent topics (Google antitrust, YouTube creator economy, Meta earnings, Amazon DSP) get filtered out even when they look adjacent.
  • Source. Is this from a publisher we trust? Articles from content farms, low-quality aggregators, and AI-spam domains get rejected regardless of how reasonable the content looks.
  • Content. Is this displayable news? How-tos, opinion pieces, case studies, benchmark reports, recap posts, and predictions are useful background reading but they're not news; they get rejected by this axis.

A labelled fixture set runs through the classifier on every change to validate that all three rejection axes still hold at 100 percent recall. If a future classifier change drops below that threshold on any axis, the change doesn't ship.

Discipline

What we don't do

Worth saying out loud, because a methodology page that only describes the system's positive behaviour leaves all the meaningful guardrails invisible.

  • No editorial bias by hand. The verdicts are the output of a defined synthesis process. No one at Shared Learnings hand-edits a verdict to make a particular platform look better or worse than the evidence supports.
  • No paid placement. Platforms and vendors cannot pay to be recommended, cannot pay to suppress a Not Recommended verdict, and cannot pay to alter source weighting. This isn't a future commitment, it's a current one. If it ever changes, the change goes here first.
  • No self-influence loop. When you query a verdict for a setting you've personally submitted a test on, your own test is excluded from the aggregate. You can see the community's view of your contribution, not your own contribution echoed back at you.
  • No surfaced scoring math. The internal scoring that decides how much each piece of evidence counts is intentionally not displayed to users. This is a deliberate design choice. Surfacing it would tempt gaming, would force every UI to defend the formula, and would distract from the thing that actually matters: the cited sources behind the verdict. If you want to verify a verdict, read the sources; the score is an internal step.
  • No verdict caching. Verdicts are not pre-computed or pre-warmed. Every run is fresh against the current evidence. This is slower than caching would be, and we accept that tradeoff because a 2024 verdict served to a 2026 user is worse than a 30-second wait.
  • No fake confidence. When the evidence is genuinely thin or contradictory, the verdict says so. A Context-Dependent outcome on a setting that has only three first-party tests behind it is not the same as one with three hundred, and we don't pretend it is.
Start a 7-day free trial

See the methodology in action

Run a Settings Checker query and you'll see the verdict, the reasoning, and every source it pulled from. Card required for the trial; cancel before day 7 and you won't be charged.

Start 7-day free trial Card required · Full plan access for 7 days · Cancel any time