Vendor Risk, Fake Automation, and the Green Check Trap

A vendor questionnaire is not vendor risk management.

It’s a starting point. Sometimes a useful one. But if your process ends with a filled-out form and a SOC report in a folder, you’re not managing risk — you’re documenting optimism.

In this Get NIST-y, Jared and Mike use the Mythos supply chain mess to highlight a hard truth: your vendors’ vendors can absolutely become your problem. And right alongside that is a second issue creeping into a lot of compliance programs — tools that promise automation, but mostly deliver cleaner dashboards and more reassuring green check marks.

Want to hear all the details, use cases, and commentary? Stream the episode above and follow on Spotify!

The Checkbox Illusion

There’s a persistent belief that collecting the “right” documents equals due diligence.

SOC 2, SOC 3, security questionnaires — these are all inputs. None of them, on their own, are conclusions.

A SOC 3 in particular is often treated like a shortcut. It’s easy to share, easy to review, and easy to misunderstand. But on its own, it doesn’t tell you much about how risk actually shows up in your environment.

If your entire vendor review process can be summarized as “we got their report,” then what you have is a checkbox exercise, not a risk management program.

Incidents Matter — But Context Matters More

One of the most common questions is whether recent vulnerabilities or security incidents should change how you score a vendor.

Yes, they should. But not in a vacuum.

Every vendor will have vulnerabilities. Many will have incidents. The differentiator is how those issues are handled:

How quickly are they disclosed?
How clearly do they communicate impact?
Do they tell you whether you were affected?
Do they explain what changed afterward?

A “critical” vulnerability on paper does not automatically translate to critical risk in your environment. Exposure depends on how the service is used, how it’s configured, and what data is involved.

Blindly escalating risk scores based on headlines isn’t much better than ignoring them altogether. The real work is in the context.

The Vendors Worth Trusting

Silence is a signal.

When something goes wrong, strong vendors don’t disappear behind vague statements and delayed updates. They communicate early and specifically:

Who was affected
Where the risk exists
What actions were taken
What customers should do next

That level of transparency is part of the risk profile, not just good PR.

If you have to chase a vendor for clarity during an incident, that’s a data point worth factoring into your evaluation.

The Green Check Trap

Automation in compliance is useful. It can reduce manual effort, speed up evidence collection, and improve consistency.

But it has limits — and those limits matter.

Two issues come up repeatedly:

First, automated evidence collection depends on integrations and APIs, which are not always reliable. Data can fail to sync, controls can appear compliant when they’re not, and gaps can go unnoticed without human review.

Second, automation only captures what it’s designed to capture. If a control or system isn’t integrated, it effectively disappears from view.

That creates a subtle but dangerous psychological bias: if it’s not showing up in the platform, it’s easy to assume it doesn’t need attention.

Over time, teams start managing what the tool can see instead of what actually exists.

That’s how you end up with a dashboard full of green checks — and a risk posture that hasn’t meaningfully improved.

Automation Helps. It Doesn’t Own Risk.

Automated evidence collection can absolutely save time. It can make audits smoother and reduce repetitive work.

But it cannot:

Interpret context
Evaluate edge cases
Account for missing data
Replace human judgment

If your process treats automation as the source of truth instead of a supporting input, you might be obscuring risk rather than reducing it.

The Real Takeaway

Vendor risk management is not about collecting artifacts. It’s about understanding how risk actually flows through your vendors into your environment.

That means: