I've run managed detection and response (MDR) evaluations at three growth stages now, and the discovery calls all open the same way. Someone on my team asks how many security operations centers (SOCs) the provider runs, whether analysts are live 24/7, which threat feeds they use, and what percentage of MITRE ATT&CK they cover. Every vendor answers cleanly because every vendor has fielded those questions more than a hundred times. I used to leave feeling thorough, mostly with a transcript of the vendor's site. Those questions measure whether a vendor can talk about their product, not whether they'd catch anything in my environment.
Those questions feel rigorous because they produce numbers that fit neatly into a scoring matrix. They also come mostly from templates vendors wrote, and they measure things vendors are already prepared to answer. None of them predict what happens when an alert fires at 2 am on a Saturday, which is the only moment the contract actually gets tested.
In brief:
- The standard MDR discovery questions, including uptime service-level agreements (SLAs), analyst headcount, threat feeds, dashboards, and coverage percentages, get answered cleanly by every vendor because many buyer questionnaires mirror vendor-authored templates.
- MITRE warns against technique-count comparisons, and analyst evaluations compare features and strategy rather than run rigorous efficacy testing. The evidence base under feature questions is thinner than most scorecards assume.
- MDR SLAs typically govern notification timeliness and leave containment outcomes outside the metric. A provider can be green on every dashboard metric while an attacker stays active in the environment.
- Ask ownership questions that predict outcomes: who writes and tunes detections, and who can act without approval. Then require evidence that survives an audit.
What MDR discovery calls ask, and why it predicts nothing
Pull any published MDR request for proposal (RFP) or buyer's guide, and the same question categories appear, almost word for word. Here they are, stripped of the phrasing differences:
Uptime, headcount, threat feeds, dashboards, and coverage percentages
- How many SOCs do you operate, and can you provide analyst resumes for monitoring staff?
- Are your analysts live and monitoring 24/7?
- How many threat feeds does your detection pipeline use, and how fast do defenses update after a new exploit?
- What percentage of MITRE ATT&CK do you cover?
- What's your time to detect, understand, and contain, whether measured as 1-10-60 or published MTTR?
- What's your uptime SLA?
A 2025 county government RFP opens by asking how many SOCs the provider operates and requests sample resumes for monitoring staff. The AT&T MDR evaluator's guide asks whether analysts are available around the clock, how many threat feeds detection uses, and how fast defenses update after a new exploit.
The same logic drives 1-10-60 frameworks: one minute to detect, ten to understand, sixty to contain.
Every one of these six questions has a procurement-friendly answer. That's exactly why they dominate discovery calls, and exactly why they tell you so little, because they reward whatever can be counted before the provider has to prove how it works.
Why these questions feel rigorous and predict nothing
Take mean time to respond (MTTR) first. Analysis of large samples of public incident reports shows mean-based metrics are unreliable because incident duration is positively skewed: most incidents resolve fast while a long tail does not, so the average describes neither. Definitions drift too, and almost no two providers define MTTR the same way, so a 15-minute response can mean an analyst glanced at an alert or that a threat was contained. The MITRE managed services evaluation left SLAs out of its criteria entirely.
Coverage percentages fare worse. MITRE Engenuity's own guidance states that any analysis reflecting counts or ratios of reported techniques against a total is antithetical to the evaluation's purpose. A USENIX Security 2024 study found endpoint detection products hover at 48 to 55% technique coverage. Headcount is the same, because SOC analyst turnover runs 25 to 50% a year, making today's staff count a weak proxy for who works your queue in month eight.
Why do vendors keep getting asked the same things?
These questions aren't something buyers invent, they're inherited from RFP templates the vendors themselves wrote. That is why the same weak questions keep surviving from one procurement cycle to the next.
RFP templates and analyst checklists write the script
MDR vendors publish the RFP templates buyers use to evaluate MDR vendors. Field Effect, eSentire, and Kudelski Security each publish their own. The mechanics are plain: using a vendor's template introduces bias, because one vendor becomes the source of questions the buyer hands to its competitors. The eSentire template asks buyers to query analyst headcount, shift allocation, threat feeds, and signature update cadence, which are exactly the questions eSentire answers well.
The analyst layer reinforces this. Forrester's Josh Zelonis has written that its methodology compares features, strategy, and client satisfaction, with product exploit testing outside the analyst process. The frameworks are moving in the right direction, and outcome-driven response is the baseline. But the templates circulating in procurement inboxes predate that shift, and they're what buyers paste into RFPs.
Vendors train to answer the same questions, so the questions never change
Enterprise RFPs typically contain hundreds of yes/no feature requirements, and vendors train to answer yes, blurring current capability into roadmap or configuration-dependent possibility. That instinct extends past RFPs, to whatever is being measured, whether that's a checklist or an independent test.
Vendors optimize for whatever is being measured, whether that's an RFP checklist or an independent test, which is why MITRE had to add false-positive tracking after some vendors cranked sensitivity to alert on everything.
The loop closes on itself: vendors write templates, buyers ask them, vendors rehearse answers, analysts score what demos show, and buyers cite the ratings to justify the pick.
I've watched this loop from the inside. The last full MDR RFP I scored came back with three vendors at near-identical marks on headcount, feeds, SLAs, and coverage, and the scorecard gave me no way to separate them. What separated them showed up six months in, when one provider's escalations arrived with full investigation timelines and another's arrived as forwarded alerts needing context. Nothing in the 60 questions we asked would have surfaced that.
What feature questions can never reach
A real incident tests whether the provider can make and document decisions outside a yes/no template row. These parts of the service only surface when the provider has to decide something during an incident, which is why they deserve their own line in the evaluation.
Detection ownership
Most MDRs run shared detection libraries across their customer base. If your team has built custom Sigma rules or environment-specific security information and event management (SIEM) logic, those alerts often route back to your internal team, because the provider can't justify bespoke detection engineering per customer. Worse, some providers now claim ownership of detection content created during the engagement, so the tuning you fund can leave with the contract.
Response authority
MDR SLAs typically cover notification timeliness while resolution sits outside the contract. Target's 2013 breach is the canonical case: the Senate Commerce Committee's kill chain analysis found the monitoring stack triggered urgent alerts within minutes, the team reported them on schedule, and nobody acted for days while 40 million card records left the building. Notification without containment leaves the customer to act on a threat the provider already saw and chose not to stop.
Auditable evidence
Buyers now demand transparency into dismissals alongside detections, wanting to understand why threats were flagged and why non-threats were closed. A provider that can't reconstruct the reasoning behind a closed alert can't support a regulator's question about what you knew and when.
Feature questions never touch this, because ownership and authority only become visible under load, and evidence matters most when someone later asks you to reconstruct the decision.
The shift from feature questions to ownership questions
Make ownership the frame and ask who owns each part of the service. Stop inventorying features and test the operating model instead.
For MITRE ATT&CK coverage, ask who writes and tunes the detections that fire in your environment, and what you own at contract end. A coverage number describes a shared library tested against a taxonomy, while the ownership version surfaces whether your custom content gets investigated or handed back at renewal. Switching providers can mean starting the tuning over from zero, losing months of correlation rules and business context overnight.
For an MTTR SLA, ask what actions the provider can take without calling you first, and which response actions are preapproved. That question reveals whether you're buying response capability or notification forwarding. An SLA measures the clock, while this question measures the authority structure the clock runs inside.
Frequently asked questions about MDR evaluation criteria
What are the right MDR evaluation criteria?
Weight response authority and data ownership heaviest, because they're the ones you can't fix later. Run a 30-day proof of value in your actual environment and trigger a benign event to verify containment behavior rather than relying on the demo. Require metrics like mean time to detect (MTTD) and MTTR to carry contractual accountability if targets are missed, and apply the same standard to false-positive rates, treating published benchmarks as background only.
What should I ask in an MDR discovery call?
Ask who has the authority to act during an incident and which response actions are preapproved. Ask whether detection content, including tuning work, and the investigation history remain yours at contract end. Ask the provider to show the logic behind a closed false positive along with the percentage reduced, and ask how many different analysts have worked comparable accounts in the past year.
How do I compare MDR providers objectively?
Run the same proof of value against each finalist in your own environment, since analyst ratings compare features and strategy while your test measures efficacy. Ask each provider for a real incident walkthrough from detection through containment, because handoffs between separate teams in the retelling predict fragmentation during a real event. Treat ATT&CK alignment as useful with gap disclosure at the procedure level, not as a binary heatmap score.
What is the biggest mistake MDR buyers make?
In the evaluations I've run, the biggest mistake is undervaluing response. Buyers interrogate detection data sources and coverage while skipping the response authority and remediation capability the provider actually holds, which is the dimension measured in hours during an active ransomware event. Buyers also trust marketed MTTR over escalation-to-incident latency on genuine incidents, and leave exit terms unchecked until migration forces the question.