Working document · Questionnaire and evidence annex

A questionnaire records what a vendor tells you. The annex records what is visible without asking.

One half is attested, the other observed, and the gap between them is the finding. This pack is both halves plus the sheet that reconciles them: a vendor security questionnaire you can send under your own letterhead, an evidence annex stating what an outside-in view can independently corroborate for each control and what it cannot, a fillable scoring sheet, and guidance on banding your register before you assess a single supplier.

Written for whoever owns third-party risk and has to produce a defensible read on a supplier before a date somebody else set. Unbranded, yours to edit, and gated behind an email address rather than a call.

Get the questionnaire and annex The methodology behind it

Why a pair

A form is a self-assessment. It is the only source most programmes have.

Nearly every answer on a returned vendor questionnaire is true on the day it is written. That is not the problem. The problem is that it is the sole source in the file: written by the party being assessed, about itself, describing a moment that has already passed, and reviewed by somebody with nothing to set it against. Grading prose is not assurance, it is proofreading — and a reviewer who has only one account of an event cannot do anything with it except believe it or not.

A second, independent source changes the shape of the review, and for the externally observable controls one is available without asking anyone. Whether an administrative interface answers from the internet, whether the sign-in path in front of it presents a second factor, what a vendor’s systems actually resolve to, whether credentials attributed to its domains are circulating — these are visible from outside, from apex domains alone, on the day you want to know. Set beside the form, they turn a review into a reconciliation.

The pack is honest about the other half. Governance, change management, personnel screening, restore testing, contractual sub-processing — no amount of outside-in observation touches any of it, and the annex says so entry by entry rather than in a disclaimer at the end. That is what makes the corroborated rows worth anything: a pack that claims to see everything is discounted wholesale the first time somebody checks one line of it.

What is inside

Four parts, and the second one is the reason

Parts one and three exist in some form in every third-party programme. Part two is the half almost nobody writes down, and part four is the one that decides whether the other three are worth the effort on your register.

Contents of the External Vendor Risk Questionnaire and Outside-In Evidence Annex
Part	What it contains	What you do with it
One — the questionnaire	The control areas that actually matter for a vendor with an internet-facing estate: what they publish, who can reach the administrative side of it, how identities and credentials are handled, what happens to code and configuration, who they in turn depend on, and how you find out when any of it changes. Written to be sent as-is, under your own letterhead, with a short attestation variant for the tier that does not warrant the long form.	Send it. Yours to rename, cut and re-order, and written to go out under your letterhead rather than ours.
It is deliberately not a superset of every framework questionnaire in circulation. A form long enough to cover everything comes back late, comes back thin, or does not come back — and the questions that get answered carefully are the ones the sender can be seen to care about.
Two — the outside-in evidence annex	The other half, and the reason the pack exists. For each attested control, the annex states what an outside-in view can independently corroborate about it, what it can only partially speak to, and what it structurally cannot reach however long anyone looks. Each entry names the observation that would corroborate the answer, so the annex is a checklist somebody can actually work rather than a claim about coverage.	Fill it in before the form goes out. It tells you which questions are worth asking.
The entries stating what cannot be reached are the ones that make the rest usable. Governance, segregation of duties, change management, personnel screening, restore testing — none of these has an outside-in signature, and a pack implying otherwise is one question away from being unpicked in front of the people you least want to be unpicked in front of.
Three — the scoring sheet	A fillable sheet that takes the two halves row by row and resolves each one to a single state: the attestation and the observation agree, the answer stands on attestation alone, they disagree, or something is visible that no question on your form covers. Blank throughout — no pre-filled scores, no weighting we picked for you, and space for the date each observation was taken.	Reconcile. The disagreements are what the security schedule gets written from.
The state a row lands in matters more than any total. A score summarises; a contradiction with a date on it is something a negotiator can put in front of a counterparty, and it is the only output of the pack that changes a contract.
Four — banding and concentration	Guidance on sizing the programme before running it: how to band your register by vendor count, how to work out which of those vendors genuinely warrant continuous assessment rather than an annual form, and what to do when several vendors you treat as independent of one another turn out to sit on the same upstream.	Read it first. It decides how much of parts one to three is worth doing at all.
Most registers have a critical tier considerably smaller than their length suggests, and the concentration question tends to reshape that tier rather than confirm it. Both of those are worked in this part rather than asserted.

Contents of the External Vendor Risk Questionnaire and Outside-In Evidence Annex

One — the questionnaire

What it contains: The control areas that actually matter for a vendor with an internet-facing estate: what they publish, who can reach the administrative side of it, how identities and credentials are handled, what happens to code and configuration, who they in turn depend on, and how you find out when any of it changes. Written to be sent as-is, under your own letterhead, with a short attestation variant for the tier that does not warrant the long form.
What you do with it: Send it. Yours to rename, cut and re-order, and written to go out under your letterhead rather than ours.

It is deliberately not a superset of every framework questionnaire in circulation. A form long enough to cover everything comes back late, comes back thin, or does not come back — and the questions that get answered carefully are the ones the sender can be seen to care about.

Two — the outside-in evidence annex

What it contains: The other half, and the reason the pack exists. For each attested control, the annex states what an outside-in view can independently corroborate about it, what it can only partially speak to, and what it structurally cannot reach however long anyone looks. Each entry names the observation that would corroborate the answer, so the annex is a checklist somebody can actually work rather than a claim about coverage.
What you do with it: Fill it in before the form goes out. It tells you which questions are worth asking.

The entries stating what cannot be reached are the ones that make the rest usable. Governance, segregation of duties, change management, personnel screening, restore testing — none of these has an outside-in signature, and a pack implying otherwise is one question away from being unpicked in front of the people you least want to be unpicked in front of.

Three — the scoring sheet

What it contains: A fillable sheet that takes the two halves row by row and resolves each one to a single state: the attestation and the observation agree, the answer stands on attestation alone, they disagree, or something is visible that no question on your form covers. Blank throughout — no pre-filled scores, no weighting we picked for you, and space for the date each observation was taken.
What you do with it: Reconcile. The disagreements are what the security schedule gets written from.

The state a row lands in matters more than any total. A score summarises; a contradiction with a date on it is something a negotiator can put in front of a counterparty, and it is the only output of the pack that changes a contract.

Four — banding and concentration

What it contains: Guidance on sizing the programme before running it: how to band your register by vendor count, how to work out which of those vendors genuinely warrant continuous assessment rather than an annual form, and what to do when several vendors you treat as independent of one another turn out to sit on the same upstream.
What you do with it: Read it first. It decides how much of parts one to three is worth doing at all.

Most registers have a critical tier considerably smaller than their length suggests, and the concentration question tends to reshape that tier rather than confirm it. Both of those are worked in this part rather than asserted.

The download

Yours, unbranded, no call attached

The questionnaire and the short attestation variant, in a form you can rename, cut and send under your own letterhead.
The evidence annex — for each attested control, what an outside-in view corroborates, what it partially speaks to, and what it never reaches.
The fillable scoring sheet, blank: no pre-filled scores and no weighting chosen on your behalf.
The banding and concentration guidance, to run before any of the above.

The reasoning underneath the pack is published ungated on third-party risk management — the pack is that argument turned into something your team can run on Monday.

External Vendor Risk Questionnaire + Outside-In Evidence Annex

Questionnaire, evidence annex, fillable scoring sheet and banding guidance. We’ll email you a confirmation link — click it and the pack downloads, with no call in between.

Part four is the one to read first, and it is worth knowing roughly where your register lands before anyone follows up: the advice for a register one person can hold in their head and one spanning several business units is genuinely not the same advice.

The scoring sheet

Every row lands in one of five states, and only one of them is a finding

The sheet is a reconciliation, so what it produces is not a score per control but a verdict on the relationship between two answers. The vocabulary below is what the sheet is built around, published in full so the method can be judged before anyone hands over an address.

How a questionnaire row resolves once the attested half and the observed half are set beside each other
State	What it means	What follows
Corroborated	The vendor attested to a control and the outside-in view shows what that control would look like from outside. Two independent sources, one of which had no interest in the answer.	File it with the observation date beside the answer. It is the strongest row in the pack.
Attested only	The control is real and important and has no external signature at all. Nothing outside-in evidences a documented incident-response plan, a background-check policy or a restore test, and nothing ever will.	Leave it with the form, and ask for the artefact — the report, the policy, the test record.
Contradicted	What is observable disagrees with what was attested. Usually nobody lied: a policy with an exception nobody recorded, an inherited estate, a migration finished after the form was written.	This is the finding. Put it to the vendor with the observation and its date attached.
Observed, not asked	Something visible from outside that no question on your form covers. The annex finds these as a by-product, and they say more about the questionnaire than about the vendor.	Add the question. Then decide whether the answer changes the tier.
Examined, nothing found	The outside-in half was worked across everything the vendor exposes publicly and returned nothing attributable. Nothing was probed on the vendor’s side — this records an observation that came back empty, not a test that passed. Written down explicitly rather than quietly dropped from the sheet.	Keep it. Evidence that the surface was examined is what an assessor asks for.

Key

Both halves agree
No external signature — the form still carries it
The gap between the two. This is the finding
Your form has a hole in it
Recorded, not omitted

This is a different vocabulary from the one on third-party risk management, and the two should not be conflated. That one states what kind of evidence a ShadowMap vendor finding is — validated, observed but not validated, referencing you but not attributed to you, or reported as zero. This one states how a row of your questionnaire resolves. A single finding can carry a state from each.

How to run it

The annex goes first. Most programmes run this backwards.

The order below is not presentation. Banding decides how much of the work is worth doing, the annex decides which questions are worth asking, and nothing can be reconciled until both halves exist. Run it in a different order and the pack still produces paper — it just stops changing anything.

First

Band the register before you assess anything

Record how many vendors you hold, then how many of those would genuinely change something if they were compromised. Those are different questions and the second one is the programme. Part four walks the banding; everything after this step is sized by where you land.
Second

Fill in the annex first — before the form goes out

This is the order most programmes run backwards. Establishing the observable half first costs a vendor nothing, waits on nobody, and tells you which questions on the form are worth the counterparty’s attention. A question you can already answer yourself is a question you have spent for nothing.
Third

Send the questionnaire, weighted towards what you cannot see

The unobservable half is where an attestation carries weight an observation cannot: it has a signature on it, and observation cannot sign anything. Let the form spend its questions there, and let "we follow industry best practice" stop being a passing answer to something you can already see for yourself.
Fourth

Reconcile row by row, not in aggregate

Each row resolves to one of the five states above. Resist totalling it into a grade at this point — the total is for the board pack, and the row is for the negotiation. A grade tells somebody how worried to be; a contradiction with a date on it tells them what to write into a schedule.
Fifth

Write the schedule from the contradictions, and keep the dates

An observed exposure becomes a named remediation obligation. An undisclosed exception becomes a disclosure clause. A shared upstream becomes notice rights over a change of hosting. Keep every observation date: the question in a later review is what was true on the day you asked, and a pack that cannot answer that is a pack nobody can defend.

Part four, worked

How many vendors, and how many of them are the same vendor

Two questions decide the shape of a third-party programme before any assessment happens, and both are answered wrongly by default: the register is treated as the population, and vendors are treated as independent of one another because they invoice separately.

01 Banding

The register is not the programme

What the register says: A list, usually assembled by procurement for procurement reasons, where a payroll processor and a stationery supplier are the same kind of row. Its length is the number most often quoted upwards, and it is the least useful figure in the file.
What banding asks instead: Not how many vendors you have, but how many would change something if they were compromised — what data they hold, what access they have been granted on your estate, and what stops if they stop. The pack works this as a banding exercise rather than a scoring model, because a band is a decision you can make in a morning and a model is a project.
What it changes: The tier that warrants continuous assessment is usually a good deal smaller than the register suggests, and the rest move to a short attestation. That is not a reduction in rigour — it is spending the rigour where it changes an outcome, and it is what makes the long questionnaire survivable for the vendors that actually need one.

02 Concentration

The dependency nobody has a contract with

What the disclosure list shows: The sub-processors a vendor compiled, as at the date it was written, covering the arrangements that vendor holds directly. It is accurate on its own terms. What it cannot describe is the layer beneath its own suppliers, or a hosting migration finished the month after it was signed off.
What the resolution path shows: Which hosting, DNS, mail, certificate and authentication providers each vendor’s customer-facing systems actually resolve to — read the same way for every vendor, so the answers are comparable rather than reconciled across a set of differently worded files. The annex sets out how to record this per vendor and how to read it across the register.
What it changes: Concentration stops being an assumption inside a continuity plan and becomes something with a date and an owner: the same upstream serving several vendors you had been treating as alternates for one another. None of it is a vulnerability — it is architecture. The limit stays where the annex puts it: a technical dependency is not a contractual one, and only the vendor can tell you which of your processes is sitting on it.

Method and limits

What the annex is built on, and what it will never reach

The annex is only usable if somebody else can work out how each entry was arrived at. The exclusions below are the same weight as the method, not a disclaimer under it — they are the half a reader most needs before putting anything from the pack in front of a counterparty.

The outside-in half: how it is scoped, and what it deliberately is not As of August 2026

Scope is yours. You nominate the vendors and the apex domains they trade under, including the subsidiaries and acquisitions that make a register hard to keep straight. The pack does not tier your vendors for you and does not decide which of them are critical.
Everything in the annex is outside-in. Nothing is installed on your estate or a vendor’s, no credentials are used, and nothing is touched that is not already reachable from the public internet by anyone.
Attribution carries its own evidence. Each observation is tied to the vendor by something checkable rather than by name similarity — a vendor that disproves one attributed asset has, in practice, disproved the file.
Dates sit on the observation. Each entry records when something became observable, which is what measures when you could have known — not when the underlying event is claimed to have happened, and not when a vendor got round to mentioning it.
The sheet keeps what did not go well: the checks that ran and returned nothing, and the rows where your own form had no question to reconcile against.

Deliberately excluded

No testing of a vendor’s systems. Authorisation over somebody else’s estate is not yours to give and is not ours either. Continuous Automated Red-Teaming runs against your own estate where it is safe and authorised, never against a third party on your say-so — so every annex entry is an observation, never a test result.
No assurance over a vendor’s internal controls. Governance, segregation of duties, change management, personnel screening, restore testing and a vendor’s own resilience exercises have no outside-in signature. Attestations, audit reports and contractual audit rights carry that half, and the pack does not reduce how much of the work they are.
No compliance determination. Nothing in the pack establishes whether an obligation applies to you, whether a vendor arrangement is in scope of one, or whether any artefact satisfies it. Those belong to your compliance function and your own advisers.
No accuracy figure and no false-positive figure, here or anywhere on this site. That is policy rather than omission — we have not established one we would defend in front of a contract.
No claim that the pack replaces a questionnaire programme. It is built to make the form shorter and better aimed, which is a different and more defensible thing.

Before you hand over an email address

Fair questions about a gated document

Is the questionnaire actually usable as-is, or is it an advertisement?

It is a working document, written to be sent rather than admired. You rename it, cut the sections that do not apply, re-order the rest and send it under your own letterhead — there is no clause in it routing a vendor anywhere, and none of the questions are shaped to make any particular product look necessary. The evidence annex is written the same way: it names the observation that would corroborate each answer, not a product that produces it. The parts of the annex that say a control cannot be reached from outside are there for the same reason. A pack you cannot trust on its limits is a pack you cannot use on anything.

Do we need our vendors’ cooperation to fill in the annex?

No, and that is the point of the pairing — the annex is the half that does not wait on anyone. It works from apex domains you already hold in the procurement file, against what already answers from the public internet. What it does not do, and what the pack tells you not to do, is test a vendor’s systems: authorisation over somebody else’s estate is not yours to give, and it is not ours either. Entries in the annex are observations with dates on them, never test results, and the sheet is built to record them that way.

Does this replace our questionnaire programme?

No, and the pack argues against trying. A regulator asking whether a vendor has a documented incident-response plan is not asking a question anything outside-in can answer. What changes is the order of the work and what the form is spent on. You know the observable half before the form goes out, the form covers what only a signature can establish, and the reporting shifts from how many forms came back — a measure of your own activity — to what is observably true about the register, on a stated date.

What happens after we download it?

You get a confirmation link by email and the pack downloads from it. Someone from the team may follow up once, and it is worth telling us how large your register is when you do — the advice for a register you could read in an afternoon and one spanning several business units is genuinely different, and the follow-up is not worth either of our time if it is pitched at the wrong one. If you would rather see the observable half worked on a real estate before reading about it, the exposure snapshot runs it against a single apex domain and needs no call.

Where this gets sharper

The pack is a method. These are the pages it came from.

Third-Party Risk Management The argument underneath the annex: what a questionnaire structurally cannot tell you, how often each observable control actually changes, and the published vocabulary for what kind of evidence a vendor finding is. Vendor risk onboarding The same pairing under a deadline — the contract meeting is on a date somebody else set, and the column that matters is what each disagreement changes in the security schedule before signature. Security Ratings What a grade is for and what it is not. Useful for a threshold in a contract and a trend in a board pack; not the thing you negotiate with, which is the finding underneath it.

Or see the observable half worked for you, on one domain

The exposure snapshot applies the same outside-in method to a single apex domain and comes back as a written read of what is reachable from outside. Run it on your own estate first and you know what an annex entry looks like before you fill one in for a vendor. No call needed to get it.

Request an exposure snapshot See the methodology