Question 1

What is a pentest vendor evaluation scorecard?

Accepted Answer

A pentest vendor evaluation scorecard is a weighted decision rubric that turns a stack of penetration testing proposals into a defensible, comparable score. Buyers list the criteria that matter (technical capability, methodology fit, accreditations, tester experience, reporting and evidence quality, communication discipline, retest policy, security and data protection terms, and price), set a weight per criterion that adds up to 100, and score each candidate vendor against the same scale. The output is a numeric ranking with the underlying detail captured against each criterion, so the procurement record can show how the winner was chosen rather than relying on a hallway conversation.

Question 2

Why use a scorecard instead of comparing proposals informally?

Accepted Answer

Three reasons. First, regulators and auditors expect documentary evidence of vendor selection for material engagements: PCI DSS, SOC 2, ISO 27001, FedRAMP, and DORA all expect a defensible record of how the testing supplier was chosen, not a verbal account. Second, weighted scoring forces the buyer to set the weights before the responses arrive, which anchors the decision to the criteria rather than to whichever vendor presented most charismatically. Third, scorecards survive turnover. The next procurement cycle starts from the previous rubric rather than from scratch, and the scorecard can be reused for renewal reviews.

Question 3

How many criteria should a pentest vendor scorecard contain?

Accepted Answer

Eight to twelve criteria is the practical range. Fewer than eight collapses meaningful distinctions (a single quality criterion that bundles methodology, reporting, and tester experience hides where the vendors actually differ). More than twelve creates evaluation fatigue and the marginal weights become noise. The scorecard included on this page uses ten criteria covering technical capability, methodology fit, accreditations and qualifications, tester experience on the in-scope asset class, references on similar programmes, reporting and evidence quality, communication and SLA discipline, retest policy, security and data protection terms, and commercial value. That set has worked in the buyer-side procurement playbooks the SecPortal team has seen across CREST, CHECK, FedRAMP 3PAO, and standard mid-market RFPs.

Question 4

How should the criteria be weighted?

Accepted Answer

A defensible default is roughly 60% technical and quality criteria, 25% commercial and operating discipline, and 15% price. Specifically: 30% technical capability and methodology fit, 15% reporting and evidence quality, 10% accreditations and tester experience, 10% communication SLA and retest policy, 10% security and data protection terms, 10% references on similar programmes, and 15% commercial value. Heavier price weighting (over 25%) is a known indicator of programmes that select on cost and pay for it on quality. Heavier accreditation weighting (over 25%) tends to over-favour incumbent CREST or CHECK suppliers and miss capable boutiques. Adjust the defaults to fit your programme but document the rationale on the procurement record.

Question 5

What scoring scale should the scorecard use?

Accepted Answer

A 1 to 5 scale is the most defensible. 1 means the vendor does not meet the criterion at all, 3 means they meet the requirement adequately, and 5 means they exceed expectations with concrete evidence. Avoid binary pass/fail because it loses the gradient that drives the actual decision. Avoid 1 to 10 because evaluators cluster scores in the 6 to 8 range and the extra resolution is illusory. Document a brief written justification for each score so a successor evaluator can reconstruct the decision: "Scored 4 because the vendor proposed PTES Level 3 with explicit OWASP WSTG mapping, which matches our policy, but did not address ASVS Level 2 directly."

Question 6

How does the scorecard handle the price criterion fairly?

Accepted Answer

Treat price as a normalised score, not a raw input. The lowest priced compliant vendor scores 5 on price; the highest priced compliant vendor scores 1; intermediate prices score on a linear scale between them. This stops a single low bidder from dominating the price column while still rewarding commercial efficiency. Crucially, the price criterion is one weighted line item, not the deciding factor. A vendor that wins on price but loses on technical capability is the vendor you will regret next year. The scorecard makes that trade-off visible rather than implicit.

Question 7

How does the scorecard interact with an RFP?

Accepted Answer

The scorecard is the evaluation half of the RFP process. The RFP defines the questions; the scorecard defines how the answers convert into a ranking. The two are designed together: every criterion on the scorecard maps to one or more RFP questions, and every RFP question feeds into a criterion. If a criterion has no question behind it, vendors cannot evidence it; if a question has no criterion, the answer does not influence the decision and should be cut. The penetration testing RFP template on this site is built to produce answers the scorecard can score, including pricing model, methodology depth, accreditation evidence, tester resumes, references, retest policy, and security terms.

Question 8

Should incumbent vendors be scored with the same rubric?

Accepted Answer

Yes, but with one difference: incumbents must be scored against the same criteria as fresh candidates, but their delivery history (closed findings, retest velocity, SLA performance, communication discipline) is also scored, weighted into the references criterion. Treating an incumbent as automatically renewed without scoring is the most common procurement failure mode the SecPortal team sees during platform onboarding conversations. The scorecard makes the renewal a defensible decision: the incumbent either earns the renewal on the rubric or loses it on the rubric, and the record explains why.

Question 9

Who should score the proposals?

Accepted Answer

A small panel of three to five evaluators, each scoring independently, then reconciling. The panel should include at least one technical reviewer (security architect, lead engineer, or head of security), one procurement reviewer (commercial terms, contract law, payment), and one programme owner (accountable for the engagement outcome). Independent scoring catches bias: if all three give a vendor a 4 on technical capability, the score is robust; if scores diverge by 2 points, the panel discusses and converges on a documented rationale. The reconciled score with the dissent recorded is more defensible than a single combined number from a steering meeting.

Question 10

How does SecPortal help after the vendor is selected?

Accepted Answer

Once the procurement decision is locked in by the scorecard, SecPortal runs the engagement on the live record the chosen vendor and the buyer share. The Statement of Work, Rules of Engagement, scope, communication SLAs, and retest policy from the proposal become the engagement rules. Findings are logged with CVSS 3.1 vectors, SLA windows, and owners on day one. The branded client portal gives the buyer the same view the testers have, so the procurement promise (communication discipline, retest performance, evidence quality) is auditable rather than asserted. At renewal, the engagement record feeds the references column on the next scorecard cycle.

Rank	Vendor	Weighted total	Normalised (1 to 5)	Price (USD)
1	Vendor A	300	3.00	not set
2	Vendor B	300	3.00	not set

Criterion	Score	Weight	Contribution
Technical capability	3.0	18%	54
Methodology fit	3.0	12%	36
Reporting and evidence quality	3.0	15%	45
Accreditations and qualifications	3.0	7%	21
Tester experience on in-scope asset class	3.0	8%	24
References on similar programmes	3.0	8%	24
Communication and SLA discipline	3.0	7%	21
Retest policy	3.0	5%	15
Security and data protection terms	3.0	5%	15
Commercial value (price)	3.0	15%	45

Criterion	Score	Weight	Contribution
Technical capability	3.0	18%	54
Methodology fit	3.0	12%	36
Reporting and evidence quality	3.0	15%	45
Accreditations and qualifications	3.0	7%	21
Tester experience on in-scope asset class	3.0	8%	24
References on similar programmes	3.0	8%	24
Communication and SLA discipline	3.0	7%	21
Retest policy	3.0	5%	15
Security and data protection terms	3.0	5%	15
Commercial value (price)	3.0	15%	45

Rank	Vendor	Weighted total	Normalised (1 to 5)	Price (USD)
1	Vendor A	300	3.00	not set
2	Vendor B	300	3.00	not set

Criterion	Score	Weight	Contribution
Technical capability	3.0	18%	54
Methodology fit	3.0	12%	36
Reporting and evidence quality	3.0	15%	45
Accreditations and qualifications	3.0	7%	21
Tester experience on in-scope asset class	3.0	8%	24
References on similar programmes	3.0	8%	24
Communication and SLA discipline	3.0	7%	21
Retest policy	3.0	5%	15
Security and data protection terms	3.0	5%	15
Commercial value (price)	3.0	15%	45

Pentest Vendor Evaluation Scorecard
rank candidate testing firms with a weighted rubric

1. Set the criterion weights

2. Score each candidate vendor

3. Ranked result

Per-vendor breakdown

How to operationalise the scorecard

Related features

Orchestrate every security engagement from start to finish

Vulnerability management software that tracks every finding

Your brand. Your portal. Your clients love it.

Pentest client onboarding

Pentest project management

Pentest retainer management

Run the chosen vendor on the same record as the scorecard

1. Set the criterion weights

2. Score each candidate vendor

3. Ranked result

Per-vendor breakdown

How to operationalise the scorecard

Pentest Vendor Evaluation Scorecardrank candidate testing firms with a weighted rubric

1. Set the criterion weights

2. Score each candidate vendor

3. Ranked result

Per-vendor breakdown

How to operationalise the scorecard

Related features

Orchestrate every security engagement from start to finish

Vulnerability management software that tracks every finding

Your brand. Your portal. Your clients love it.

Pentest client onboarding

Pentest project management

Pentest retainer management

Run the chosen vendor on the same record as the scorecard

1. Set the criterion weights

2. Score each candidate vendor

3. Ranked result

Per-vendor breakdown

How to operationalise the scorecard

Pentest Vendor Evaluation Scorecard
rank candidate testing firms with a weighted rubric