How to Verify a Performance Marketing Agency's ROAS and Case-Study Claims Before You Believe the Pitch

Key Takeaways

• Treat every number in a pitch as a marketing asset the agency built to sell you, not as evidence — a case study is designed to persuade, not to prove.
• A reported ROAS is usually inflated by attribution and blind to margin; ask how it was measured and whether it reconciles to the client's actual revenue and profit.
• Ask whether a claimed result was incremental — caused by the agency — or merely credited to it, and whether the account was already winning before the agency arrived.
• Verify who would actually run your account (rarely the senior people in the pitch) and whether the case-study client resembles your business, stage, and margins.
• Use references properly: ask past and current clients about real results, honesty, and what working with the agency is actually like — not just whether they were satisfied.
• The strongest signal is willingness to be verified — a great agency welcomes reconciliation and open references; a weak one deflects to logos, dashboards, and guarantees.

The Pitch Is Designed to Persuade You, Not to Prove Anything

When a performance marketing agency pitches you, it presents numbers that are meant to feel like proof: a 9x return on ad spend, a case study featuring a brand you recognize, a screenshot of a dashboard climbing up and to the right. The instinctive response is to be impressed and to treat these as evidence that the agency is good. But almost none of it is evidence in the sense you assume, because a pitch is a marketing asset — it was built by the agency, from data the agency selected, framed the way the agency chose, specifically to persuade you to sign. The agency that is best at producing a compelling pitch is demonstrating skill at pitching, which is a different skill from delivering results, and often an inversely correlated one: the best operators are frequently worse at slick presentation than the agencies that win on show. So the first move in verifying an agency's claims is to stop treating the pitch as proof and start treating it as a claim to be tested.

This is not cynicism; it is the appropriate posture for a high-stakes purchase where the seller controls the information. You are about to hand an agency a significant budget on the strength of what it tells you about its past results, and the agency knows exactly which numbers flatter it and how to frame them. A case study omits the clients that failed and the accounts that were already winning; a reported ROAS uses the measurement method that produces the biggest number; a famous logo implies a success the agency may not have created. None of this is necessarily dishonest — it is how marketing works — but it means the burden is on you to verify, because the pitch is optimized to make verification feel unnecessary. The agencies worth hiring understand this and welcome the scrutiny; the ones that need the pitch to go unquestioned are the ones you most need to question.

This guide gives you the tools to be a harder audience to fool: the reasons standard proof is weak by construction, the questions that separate a real result from a decorated one, the checks for incrementality and for who actually did the work, the proper use of references, and — most powerful of all — how to read an agency's willingness to be verified as the signal it is. The goal is not to assume every agency is lying; most are not. The goal is to be able to tell the difference between an agency whose impressive numbers reflect real, repeatable skill and one whose impressive numbers reflect skilled framing of ordinary or borrowed results — because that difference is invisible in the pitch and enormous in the outcome, and only verification reveals it.

Why a Reported ROAS Is Almost Never What It Seems

Start with the number agencies lean on most: return on ad spend. A reported ROAS is almost never the measure of real business impact you take it to be, for two structural reasons that apply regardless of the agency's honesty. First, attribution inflation: platform-reported ROAS credits the ad platform for conversions it merely touched, and because every channel claims the conversions it was near, the numbers across channels sum to far more than the actual sales — retargeting and branded search in particular take credit for demand that already existed and would have converted anyway. A 6x platform ROAS does not mean the ads generated six times their cost in incremental revenue; it means the platform, using its own generous attribution, credited itself that much. Second, margin-blindness: ROAS is revenue over spend, ignoring your costs, so a 4x ROAS is excellent at 60% margin and loss-making at 15% — the same number means opposite things for different businesses, and a ROAS quoted without margin context tells you nothing about profit.

So when an agency presents a ROAS, the number itself is nearly meaningless until you ask how it was measured and against what. The questions to ask are specific: is this platform-reported or measured against the client's actual revenue? Is it reconciled to the client's real financials — did the reported return actually show up in the bank — or is it the platform's self-credited figure? What was the margin, so we know whether that return was profitable? Over what period, and was the account already producing that return before the agency arrived? An agency that can answer these — that talks in terms of incremental revenue reconciled against the client's finance numbers, with margin context — is showing you it measures honestly. An agency that cannot, or that grows uncomfortable when you ask, is showing you that its impressive number lives only inside the platform's inflated, margin-blind attribution, where any competent account can look good.

Consider two agencies quoting the same 6x. The first, asked how it was measured, explains it is platform-reported and cannot readily tie it to the client's actual revenue or margin. The second explains it reconciled its reported results monthly against the client's booked revenue and contribution margin, and that on that basis the incremental, profitable return was more modest but real. The first number is bigger and worth less; the second is smaller and worth far more, because it reflects reality rather than the platform's self-flattery. If you take ROAS claims at face value, you will systematically prefer the agency that reports the most inflated number, which is precisely backwards — the honest agency looks worse in the pitch and is better in reality. Learning to ask how the number was measured is the single highest-leverage verification skill you can develop, because it inverts that trap.

Incrementality: Did the Agency Cause the Result, or Get Credited for It?

The deepest question behind any claimed result is causation: did the agency actually cause the improvement, or did it merely get credited for something that would have happened anyway? This is the difference between attribution (which channel got credit) and incrementality (what genuinely happened because of the activity that would not have happened otherwise), and it is where most impressive case studies quietly fall apart. An agency can show a case study where revenue grew during its tenure without that growth being caused by the agency — the brand may have been growing already, launched a hit product, benefited from seasonality or a competitor's stumble, or simply ridden rising demand. The revenue went up; the agency was present; the case study credits the agency. But presence during a result is not causation of it, and a case study almost never distinguishes the two, because distinguishing them would often deflate the claim.

So the verification question is whether the claimed result was incremental, and whether the account was already winning before the agency arrived. Ask directly: what was the account doing before you took it over, and how much of the improvement was genuinely caused by your work versus underlying growth the business already had? How do you distinguish incremental results from what would have happened anyway — do you use holdout tests, geo experiments, or other incrementality measurement, or do you rely on platform attribution? A famous logo in particular deserves this scrutiny, because a large, successful brand was likely succeeding before and after the agency, so the agency's presence on that account proves it was hired, not that it drove the success — the logo borrows the brand's credibility for the agency's pitch. An agency that took a struggling account and demonstrably turned it around, measured incrementally, has shown you real skill; an agency that was present while an already-winning account kept winning has shown you a logo.

This matters enormously for what you can expect, because you are not hiring the agency to be present during your success — you are hiring it to cause success it can repeat for you. An agency whose case studies are mostly already-winning brands and platform-attributed growth may have little demonstrated ability to actually move a result, which is exactly what you need. Ask yourself as you review each case study: is there real evidence the agency caused this outcome — a before-and-after on a struggling account, an incrementality measurement, a turnaround — or is this a successful brand whose success the agency is standing next to? If it is the latter, discount it heavily, however impressive the logo. The agencies worth hiring can point to results they demonstrably caused and can explain how they know they caused them; the ones that cannot are hoping you will confuse correlation with contribution.

Who Actually Did the Work — and Was the Client Like You?

Two further checks separate a meaningful case study from a misleading one: who actually did the work, and whether the client resembles your business. On the first, the senior, impressive people who present the pitch and appear in the case study are frequently not the people who would run your account day to day — agencies win business with their best people and service it with more junior ones, so the skill on display in the pitch may not be the skill applied to your account. Ask specifically: who worked on this case-study account, and who would work on mine? Will the people in this room be doing my work, or is this the sales team and the delivery team is someone I have not met? Insist on meeting the actual team that would run your account, and judge their thinking, because that is the capability you are buying — not the capability of the people hired to win you.

On the second, a result achieved for a business unlike yours may not transfer, so a case study is only relevant to the extent the client resembles you in the ways that matter: business model, stage, margins, sales cycle, and market. An agency that produced a great result for a large, established brand with fat margins and existing demand may have no demonstrated ability to do the same for an early-stage company with thin margins that still needs to prove its funnel — those are different problems requiring different skills. Ask whether the case-study client was at your stage, in a comparable model, with comparable economics, when the result was achieved. A relevant case study — same model, similar stage, comparable constraints, a result the agency demonstrably caused — is strong evidence; an impressive result for a business nothing like yours is a reason to ask whether the agency actually understands your situation, not a reason to be reassured.

The table below turns these into a scorecard you can apply to any case study or claim an agency offers, converting the vague impression of 'impressive' into specific, checkable questions. Run every claim through it, and you will quickly see which case studies hold up as evidence of transferable, agency-caused skill and which dissolve into inflated numbers, borrowed logos, mismatched clients, and work done by people you will never meet. The point is not to disqualify every agency — a good one will pass these checks comfortably — but to stop being persuaded by the ones that pass only the test of looking impressive.

Using References Properly — and Reading the Willingness to Be Verified

References are one of your best verification tools, but only if you use them properly rather than as a box to tick. A reference the agency offers is pre-selected to be positive, so the value is not in whether they are happy but in the specific, probing questions you ask them. Ask past and current clients about real business results — did the reported performance show up in your actual revenue and profit? Ask about honesty — did the agency tell you when something was not working, or only present good news? Ask what it is actually like to work with them — responsiveness, seniority of attention over time, how they handled problems and disappointing periods. Ask what they would change or what nearly made them leave. And try to speak to a former client, not only current ones, because a client who left will tell you why, which is often more informative than a current client's satisfaction. The goal is to get past 'they were great' to the texture of the actual relationship and whether the results were real.

But the most powerful signal of all is not any single answer — it is how the agency responds to the entire process of being verified. An agency that welcomes your scrutiny — that readily explains how it measures, offers to reconcile its claims against real numbers, provides references including former clients, introduces the actual delivery team, and is comfortable with fair contract terms around ownership and reconciliation — is demonstrating confidence that its results are real and repeatable, because only an agency with real results is comfortable being checked. An agency that deflects — that returns to logos and dashboards when you ask how numbers were measured, resists giving account access or reconciliation rights, is vague about who will do the work, or grows defensive under fair questions — is showing you that its impressive pitch has less underneath than it appears. You do not even need to catch a specific falsehood; the pattern of deflection versus openness is itself the evidence.

So run the whole verification as a test not only of the claims but of the agency's relationship to the truth, because that relationship is what you will be living with. The agencies worth hiring make verification easy because they have nothing to hide and would rather be judged on real impact than on a pitch; the ones to avoid make verification hard because the pitch is the best version of them you will ever see. Ask yourself, after the process: did this agency get more or less impressive the harder I looked — because a real operator becomes more credible under scrutiny as the substance shows through, while a pitch-driven agency becomes less credible as the deflections accumulate. Verify the ROAS by how it was measured, the case studies by incrementality and relevance, the team by who will actually do the work, and the whole agency by its willingness to be checked. Do that and you will hire on evidence rather than on show — which is the entire difference between an agency that transforms your growth and one that transforms your budget into its next case study. If you want an agency that welcomes reconciliation, open references, and scrutiny of every claim from the first conversation, that is exactly the standard our team is built to meet.