Question 1

What is detection validation cycle economics?

Accepted Answer

Detection validation cycle economics is the analytical frame that prices the cost of validating that a security detection rule still works against the environment it is supposed to cover, across the full lifecycle of the rule rather than at one-off intervals. A detection rule is not a static asset. It is an operating record whose validity decays as the protected system changes, as the threat behaviour it watches for changes, and as the telemetry feeding it changes. The frame names five lifecycle states (drafted, deployed and unvalidated, validated and live, validated but drifting, retired) and prices the per-state cost shape on three axes: validation labour cost, false-positive carrying cost, and undetected-true-positive risk cost. The picture matters because the cost of running a detection programme is not the cost of writing the rules; it is the cost of keeping the rules valid against a moving target across a multi-year operating window.

Question 2

How is this different from breach and attack simulation economics?

Accepted Answer

Breach and attack simulation runs an attack technique against the protected environment and reads whether the detection layer catches it. BAS is one validation method inside the wider detection validation cycle. Detection validation cycle economics is the wider frame that prices the full lifecycle cost of a rule: drafting cost, deployment cost, validation cost, drift cost, and retirement cost. BAS sits inside the validation step of that cycle alongside purple-team campaigns, replayed historical attack traces, synthetic adversary emulation, peer review of rule logic, and live false-positive analysis. Programmes that treat BAS as the entire validation programme under-price the drafting, drift, and retirement phases of the cycle. Programmes that frame BAS as one validation method inside the wider cycle catch the full cost picture rather than only the per-test cost.

Question 3

What are the five detection validation lifecycle states?

Accepted Answer

Five states cover the bulk of the detection rule lifecycle in enterprise SOC and detection-engineering programmes. Drafted is the design state where the rule has been authored but not deployed to the detection plane. Deployed and unvalidated is the post-deployment state where the rule is live but no end-to-end validation has confirmed it triggers on the target behaviour. Validated and live is the steady-state where the rule has been validated against named attack behaviour and is operating inside agreed false-positive and true-positive thresholds. Validated but drifting is the regression state where the rule was once validated but environmental change, telemetry change, threat-behaviour change, or rule-logic change has degraded confidence in the validation. Retired is the end-of-life state where the rule has been removed from the active library and the validation record has been frozen as a historical artefact. Each state has its own actors, its own dominant cost component, and its own intervention path. Programmes that track only deployed-versus-not-deployed collapse the four operating states into one number and lose the cost distribution that drives most of the programme variance.

Question 4

Why does carrying cost vary by lifecycle state?

Accepted Answer

Carrying cost varies by state because the cost components compound differently in each state. Drafted rules carry design-labour cost (analyst time spent specifying the rule), peer-review cost (logic review before deployment), and shelf cost (rules drafted and never deployed sit as work-in-progress that ties up engineering inventory). Deployed-and-unvalidated rules carry false-positive-exposure cost (the rule may be generating alerts the SOC handles without anyone having confirmed the rule logic), silent-gap cost (the rule may be deployed but failing to trigger because of a telemetry pipeline change), and audit-defensibility cost (a deployed-but-unvalidated rule shows up at audit as a control claim without operating evidence). Validated-and-live rules carry the lowest steady-state cost but still consume operating labour for false-positive triage and quarterly health-check review. Validated-but-drifting rules carry the highest hidden cost because the validation record claims coverage that the live system may no longer produce. Retired rules carry archival cost only, but premature retirement transfers the cost to the gap the rule covered. Reading the five-state distribution surfaces where the cost is concentrated and which intervention lowers the right cost.

Question 5

What does the cost stack look like for drafted rules?

Accepted Answer

Drafted rules carry a cost stack dominated by design-labour cost, peer-review cost, and shelf cost. Design-labour cost is the analyst time spent specifying the rule logic, the telemetry sources it depends on, the alert payload it produces, and the response runbook it dispatches to. Peer-review cost is the senior-analyst time spent reviewing the logic before deployment; programmes that skip peer review absorb the cost downstream as false-positive incidents and as undetected-true-positive misses. Shelf cost is the carrying cost of drafted rules that never make it to deployment; this is engineering work-in-progress inventory that ties up capacity until the rule either ships or is retired pre-deployment. The dominant intervention is to surface drafted-state queue length with the named reason for non-deployment (peer-review block, telemetry-gap block, runbook-not-ready block) so the programme can read whether the design pipeline or the deployment pipeline is the binding constraint. Programmes that report only deployed-rule counts conceal the drafted-state inventory and the engineering capacity it absorbs.

Question 6

What does the cost stack look like for deployed and unvalidated rules?

Accepted Answer

Deployed-and-unvalidated rules carry the highest hidden cost in most programmes because the rule has been claimed as operating coverage without end-to-end validation. The cost stack is dominated by false-positive-exposure cost (the SOC may already be handling alerts the rule produces, accepting the false positives as operating noise without analysis of whether the underlying logic is sound), silent-gap cost (the rule may have been deployed against a telemetry pipeline that has since changed, leaving the rule live but no longer triggering on the target behaviour), and audit-defensibility cost (a deployed-but-unvalidated rule is a control claim without operating evidence; framework citations read this as a gap). The dominant intervention is to surface the deployed-and-unvalidated queue as a separate operational metric paired against a documented validation SLA, so the queue cannot persist indefinitely without management attention. Programmes that ship rules into production with no validation SLA produce a deployed-rule library that looks healthy on a dashboard while carrying unread risk in the gap between deployment and validation.

Question 7

What does the cost stack look like for validated and live rules?

Accepted Answer

Validated-and-live rules carry the lowest steady-state cost stack but the cost is not zero. The dominant components are false-positive triage cost (the per-week labour of triaging alerts the rule produces, separating the true positives from the false positives, and feeding tuning recommendations back to detection engineering), quarterly health-check cost (the per-quarter labour of reviewing the rule against current threat intelligence, current telemetry topology, and current environment baseline), and renewal-validation cost (the periodic re-execution of the validation method, whether BAS, purple-team, replayed historical trace, or synthetic adversary emulation, against the current environment). The dominant intervention is to set a per-rule renewal cadence based on the rule criticality and the rate of change in the protected system rather than treating all rules on a uniform cycle. Programmes that adopt risk-tiered renewal cadences spend the validation labour where the validation evidence matters and reduce the burden on lower-risk rules whose validation evidence ages more slowly.

Question 8

What does the cost stack look like for validated but drifting rules?

Accepted Answer

Validated-but-drifting rules carry a cost stack dominated by drift-detection cost, re-validation cost, and unread-risk cost. Drift-detection cost is the per-week labour of running drift triggers: change-event monitoring on the protected system, telemetry-pipeline change detection, threat-intelligence updates that invalidate the rule premise, and rule-logic change detection (whether the rule itself has been edited since the last validation). Re-validation cost is the cost of re-running the validation method once a drift trigger fires; if the drift trigger fires often, the re-validation cost can dominate the rule operating cost. Unread-risk cost is the cost of the rule continuing to claim coverage between the drift trigger firing and the re-validation completing; this cost compounds per day the drift queue grows. The dominant intervention is to define drift triggers explicitly per rule (which change events invalidate the validation evidence) and to fund re-validation capacity that matches the drift trigger rate rather than treating drift as exceptional. Programmes that treat the validation record as durable absorb the drift cost as a slow erosion of detection coverage that surfaces at the next audit cycle or at the next incident.

Question 9

What does the cost stack look like for retired rules?

Accepted Answer

Retired rules carry an archival cost stack dominated by record-retention cost and gap-transfer cost. Record-retention cost is the cost of keeping the historical validation record (the rule logic at retirement, the last validation result, the retirement rationale, the named approver) accessible across the audit window the framework citations require. Gap-transfer cost is the cost of any new gap the retirement opens; a rule retired without a replacement transfers the coverage cost to the gap the rule used to cover. The dominant intervention is to require a documented retirement rationale that names whether the rule is being retired because the underlying threat is no longer credible, because the rule was a duplicate, because the telemetry source is gone, or because a successor rule covers the same behaviour with higher fidelity. Programmes that retire rules silently produce a coverage map that drifts further from reality each quarter; programmes that record retirement rationales preserve the coverage chain across the multi-year audit window.

Question 10

What drift triggers should a detection programme watch?

Accepted Answer

Six drift triggers cover most of the regression cases that move a validated rule into the validated-but-drifting state. Telemetry pipeline change covers schema changes, log source replacement, agent upgrades, parsing changes, and indexing changes in the platform the rule reads from. Protected system change covers application releases, configuration drift, identity provider changes, network architecture changes, and platform migrations that change the behaviour the rule is watching. Threat behaviour change covers adversary tradecraft updates, new tooling, new techniques in MITRE ATT&CK revisions, and shifts in observed campaign behaviour from threat intelligence feeds. Rule logic change covers any edit to the rule body since the last validation, including condition changes, threshold changes, suppression edits, and severity changes. False positive rate change covers any sustained shift in the alert-to-true-positive ratio that suggests the rule is firing differently than the validation predicted. True positive miss covers any post-incident finding that the rule should have triggered but did not. The dominant intervention is to attach the relevant drift triggers to each rule at deployment time so re-validation is dispatched automatically when the trigger fires rather than waiting for the next scheduled review.

Question 11

What metrics survive audit scrutiny for detection validation cycles?

Accepted Answer

Seven paired metrics outperform deployed-rule-count-only reporting and survive audit scrutiny. Per-state queue length names how many rules sit in each lifecycle state and surfaces where work is concentrated. Per-state median resident time names how long rules sit in each state before transition. Per-state P90 resident time names the tail where the cost is concentrated. Validation freshness median names how recently the live rules were last validated against a documented method. Drift trigger fire rate names how often the rules in the live library cross a drift threshold between scheduled validations. Per-rule false positive rate names the precision of each live rule against the alert population it produces. Per-rule true positive verification rate names the recall of each live rule against the threat behaviour it claims to cover. The defensible programme reports the seven metrics per rule tier (critical, high, medium, low) paired against the framework citations the audit chain reads against. Programmes that report only deployed rule counts produce a number that grows over time independent of whether the coverage claims are operating, validated, and current.

Question 12

What framework citations expect detection validation evidence?

Accepted Answer

Several enterprise frameworks expect documented evidence that detection controls are operating, monitored, and periodically validated. NIST SP 800-53 Revision 5 SI-4 reads system monitoring with documented detection events; CA-7 reads continuous monitoring with documented evidence; CA-8 reads penetration testing including red-team exercises. NIST CSF 2.0 DE.AE reads adverse event analysis; DE.CM reads continuous monitoring across assets, telemetry, and the environment; RS.AN reads incident analysis using the detection record. ISO 27001:2022 Annex A 5.7 reads threat intelligence intake; A 8.16 reads monitoring activities with documented evidence; A 5.30 reads ICT readiness for business continuity which depends on detection coverage being operating. SOC 2 Trust Services Criteria CC7.2 reads identification of security events; CC7.3 reads evaluation of security events to determine impact and response. PCI DSS v4.0 Requirement 10.4 reads daily review of audit logs with documented evidence; Requirement 12.10 reads incident response readiness which depends on detection rules being operating and validated. CIS Controls v8.1 Control 8 reads audit log management; Control 13 reads network monitoring and defence. DORA Article 8 reads ICT business continuity and Article 17 reads ICT-related incident management which both depend on detection coverage being operating. The pattern is consistent: each framework expects detection rules to be operating and to have validation evidence pinned to them, not only to have been written and deployed.

Question 13

How does SecPortal support detection validation cycle reporting?

Accepted Answer

SecPortal pairs every validation outcome to a versioned engagement record where the rule reference, the validation method, the actor, the timestamp, and the supporting evidence sit together. Engagement management organises a validation cycle as an engagement record with named scope, named timeline, and named owner so the validation cadence is reproducible from the live record rather than reconstructed from ad-hoc artefacts. Findings management captures the validated rule outcome alongside CVSS 3.1 vector, severity band, owner, evidence, and remediation status; a missed-true-positive or false-positive issue from the validation cycle enters the same finding model as any other vulnerability finding. Bulk finding import accepts Nessus, Burp Suite, and CSV so validation outputs from external testing tools join the same lifecycle the workspace-native findings traverse. Retesting workflows hold the re-validation step as a first-class state and carry the closure evidence through to administrative closure. Continuous monitoring runs recurring detection-validation cycles on a documented cadence. Activity log captures every state change with named actor, timestamp, and entity reference and exports to CSV so per-state resident time is reproducible. Finding overrides (false positive, accepted risk, severity adjustment) sit as captured decisions on the finding so the validation picture reads against the actual operating record. Compliance tracking maps validation evidence to framework controls so audit fieldwork reads against the live record. The platform does not author detection rules, does not execute attack simulations against live systems, and does not ingest live SIEM telemetry. It does keep the validation cycle record on the same engagement and finding spine the wider security operating record uses, so the validation evidence is durable across reporting cycles and reproducible at audit week.

State	What it means	Responsible function
1. Drafted	Rule authored against a named threat behaviour but not yet deployed to the detection plane.	Detection engineer, senior analyst, or threat-research function.
2. Deployed and unvalidated	Rule live in production but no end-to-end validation has confirmed it triggers on the target behaviour.	Detection engineering, SOC operations, validation team.
3. Validated and live	Rule validated against named attack behaviour, operating inside agreed false-positive and true-positive thresholds.	SOC operations with detection engineering review cadence.
4. Validated but drifting	Rule was once validated but environment, telemetry, threat behaviour, or rule logic has changed since.	Detection engineering with named drift-trigger owner.
5. Retired	Rule removed from the active library with a documented rationale and a frozen validation record.	Detection engineering with programme owner sign-off.

Cost component	How it compounds	Intervention path
Design-labour cost	Analyst time on rule logic, telemetry sources, payload, response runbook.	Named drafting template; threat-behaviour scoping at intake; reusable detection patterns.
Peer-review cost	Senior-analyst time reviewing rule logic, false-positive risk, and response readiness.	Documented peer-review SLA; named reviewer rotation; review checklist.
Shelf cost	Drafted rules that sit indefinitely consume engineering inventory without producing coverage.	Drafted-state queue length metric; named non-deployment reason per rule; pre-deployment retirement option.

Cost component	How it compounds	Intervention path
False-positive-exposure cost	SOC accepts unvalidated rule alerts as operating noise; tuning recommendations never reach detection engineering.	Per-rule false-positive rate captured from day one; named tuning-feedback loop to detection engineering.
Silent-gap cost	Rule deployed against a telemetry pipeline that has since changed, leaving the rule live but failing to trigger.	End-to-end validation method (BAS, replayed trace, synthetic adversary) confirms trigger inside an SLA window.
Audit-defensibility cost	Deployed-but-unvalidated rules show up at audit as control claims without operating evidence.	Documented validation SLA; deployed-and-unvalidated queue length surfaced as a separate metric; named SOC owner.

Cost component	How it compounds	Intervention path
False-positive triage cost	SOC labour per week separating true positives from false positives and feeding tuning recommendations back.	Per-rule precision target; named tuning loop to detection engineering; suppression rules with documented rationale.
Quarterly health-check cost	Per-quarter review against current threat intelligence, current telemetry topology, current environment baseline.	Documented quarterly review checklist; named reviewer; cross-reference to threat intelligence feed updates.
Renewal-validation cost	Periodic re-execution of the validation method against the current environment to refresh evidence.	Risk-tiered renewal cadence (critical, high, medium, low); named validation method per tier; documented schedule.

Cost component	How it compounds	Intervention path
Drift-detection cost	Per-week labour running drift triggers across telemetry, protected system, threat behaviour, and rule logic.	Drift triggers attached per rule at deployment time; named owner for each trigger; automated dispatch on trigger fire.
Re-validation cost	Per-cycle cost of re-running the validation method once the trigger fires; can dominate operating cost if drift triggers fire often.	Re-validation capacity sized against drift trigger fire rate; tiered methods per rule tier; documented re-validation SLA.
Unread-risk cost	The rule continues to claim coverage between drift trigger fire and re-validation completion; risk compounds per day.	Drifting state surfaced as a separate metric; explicit risk-acceptance record for drift queues that exceed an agreed window.

Detection Validation Cycle Economics: Pricing the Cost of Keeping Rules Valid

Why deployed-rule count is the wrong headline number

The five detection validation lifecycle states

State 1: drafted

State 2: deployed and unvalidated

State 3: validated and live

State 4: validated but drifting

State 5: retired

The six drift triggers

1. Telemetry pipeline change

2. Protected system change

3. Threat behaviour change

4. Rule logic change

5. False positive rate change

6. True positive miss

Validation methods: matching the cycle to the rule

Breach and attack simulation

Purple-team campaign

Replayed historical trace

Synthetic adversary emulation

Live false-positive and true-positive analysis

Seven metrics that survive audit scrutiny

Framework citations: what each framework expects

For internal security teams

For security leadership and audit committees

How SecPortal supports detection validation cycle reporting

Conclusion

Frequently Asked Questions

Sources

Run detection validation cycles on the live engagement record

Cost component	How it compounds	Intervention path
Record-retention cost	Historical validation record stays accessible across the framework-required audit window.	Versioned archival of rule logic, last validation result, retirement rationale, and named approver on the engagement record.
Gap-transfer cost	A rule retired without a successor transfers the coverage cost to the gap the rule used to cover.	Successor rule reference at retirement; documented coverage-map update; SOC notification of the coverage transition.

Metric	What it reads	Why it matters
1. Per-state queue length	How many rules sit in each lifecycle state.	Surfaces where the work is concentrated across the lifecycle.
2. Per-state median resident time	How long rules sit in each state before transition.	Catches state-level bottlenecks that aggregate metrics conceal.
3. Per-state P90 resident time	The tail of the resident-time distribution per state.	Surfaces the outlier rules that dominate cost.
4. Validation freshness median	How recently the live rules were last validated against a documented method.	Direct read of whether validation evidence is current or stale.
5. Drift trigger fire rate	How often the rules in the live library cross a drift threshold between scheduled validations.	Surfaces whether the operating environment outruns the validation cycle.
6. Per-rule false positive rate	Precision of each live rule against the alert population it produces.	Operational sustainability signal; high false positive rates burn SOC capacity.
7. Per-rule true positive verification rate	Recall of each live rule against the threat behaviour it claims to cover.	Direct read of whether the rule is producing the coverage the validation predicted.

Framework	Relevant citations	What the citation expects
NIST SP 800-53 Rev 5	SI-4, CA-7, CA-8	System monitoring with documented detection events; continuous monitoring with evidence; penetration testing including red-team exercises.
NIST CSF 2.0	DE.AE, DE.CM, RS.AN	Adverse event analysis; continuous monitoring; incident analysis using the detection record.
ISO 27001:2022	A 5.7, A 8.16, A 5.30	Threat intelligence intake; monitoring activities with documented evidence; ICT readiness which depends on detection coverage operating.
SOC 2	CC7.2, CC7.3	Identification of security events; evaluation of security events to determine impact and response.
PCI DSS v4.0	Req 10.4, Req 12.10	Daily review of audit logs with documented evidence; incident response readiness which depends on detection rules operating and validated.
CIS Controls v8.1	Control 8, Control 13	Audit log management; network monitoring and defence.
DORA	Articles 8 and 17	ICT business continuity; ICT-related incident management; both depend on detection coverage operating.