There is a number that should make every AppSec team uncomfortable: the median time from public CVE disclosure to in-the-wild exploitation is around 12 days. That number has been shrinking. Some critical vulnerabilities in widely deployed libraries see working exploit code published within 72 hours of the CVE advisory going out. Automated scanners pick these up within hours. Threat actors running continuous reconnaissance are watching the same feeds your team is.
Now consider what happens during a typical quarterly pentest engagement. The scoping call happens in week one. The report lands six to eight weeks after kick-off. Remediation guidance gets triaged into the sprint backlog. Some findings get fixed in the next release cycle. Some get deferred. And meanwhile, the codebase that was tested in March has received forty to sixty pull requests by the time the report hits the security team's inbox in April.
The Scope Problem Is Structural, Not a Failure of Effort
Pentest teams work within a scope that was defined weeks before they started testing. That scope is a snapshot of your attack surface at the moment the statement of work was written. Every deployment between SOW signature and test execution is outside the scope that was actually probed. In a team shipping daily, that is a material gap.
This is not a criticism of the pentesters. A skilled red team working a two-week window can uncover sophisticated logic flaws, business-flow abuse paths, and chained exploits that no automated scanner would find. The problem is the cadence, not the craft. Testing quarterly is like taking your blood pressure once every three months and assuming you know your cardiovascular health.
When we were building the automation that underlies Strix, one of the clearest patterns we kept seeing was code that shipped between engagements introducing regression. A dependency upgrade fixes one CVE and pulls in a newer transitive dependency with a different CVE. A new API endpoint gets added without the same authorization checks applied to the existing ones. Neither of those changes shows up in a pentest that finished last month.
What Actually Happens to Findings in the 90-Day Window
The lifecycle of a typical pentest finding goes roughly like this. The finding is documented. It gets a CVSS score. It enters a remediation tracking spreadsheet or Jira epic. A severity-based SLA clock starts ticking. Critical findings might be targeted for 30-day remediation. High findings get 60 days. Medium findings get 90 days.
Here is the uncomfortable part: during those 90 days, three things are happening simultaneously. First, the vulnerability is still present in production while remediation is in progress. Second, any new CVE that appeared after the pentest closure has no remediation tracking at all because it was never in scope. Third, the exploit availability for existing findings is changing as the broader security community publishes proof-of-concept code.
A finding that was classified as High severity on day one because no public exploit existed can silently become a Critical finding by day 45 when a metasploit module ships. The CVSS score in your report has not changed. Your SLA clock has not reset. But the actual risk profile of that finding has shifted substantially.
The CVE-to-Exploit Window Is Getting Shorter
A few years ago, the window between CVE disclosure and reliable exploit availability was measured in weeks for most vulnerabilities. Teams could reasonably prioritize patching based on CVSS scores and assume they had time. That assumption has broken down across several dependency categories.
For vulnerabilities in widely deployed serialization libraries, web frameworks with large known install bases, and cloud SDKs, the exploit-availability timeline has compressed to days. Automated vulnerability scanners maintained by organized threat groups do continuous reconnaissance across internet-exposed endpoints, and they are tuned to detect specific version fingerprints. If your application banner, response headers, or error messages expose version information, those systems are noting it and queuing exploitation attempts.
The math here is simple and bad: if a CVE drops on a Tuesday and active exploitation starts on a Sunday, and your pentest was completed two weeks ago, you have a 90-day remediation SLA running against a 12-day exploitation window. Those numbers do not coexist safely.
Why Adding More Frequent Pentests Does Not Solve This
The intuitive response is to schedule pentests more frequently. Monthly instead of quarterly. The problem is that this approach runs into hard constraints on both the supply side and the demand side. Skilled pentesters are expensive and scheduling-intensive. An engagement requires scope definition, kick-off, testing windows, finding validation, and report production. Compressing this to monthly creates scheduling conflicts, reduces testing depth because each engagement gets less calendar time, and does nothing about the days between engagements.
More importantly, bi-monthly or monthly point-in-time tests still share the fundamental structural flaw: they test a specific snapshot of your attack surface, not the surface as it actually exists at the moment of risk. A dependency that gets upgraded on Monday and introduces a critical deserialization issue is not covered by the pentest that ran two weeks ago. It will not be covered by the one scheduled for three weeks from now either.
We are not saying manual pentests provide no value. A skilled human adversary working your application will find things that no automated system reaches on its own: business logic flaws, multi-step authorization bypasses that require understanding your application's intended workflow, subtle cryptographic implementation errors. Those require human judgment and contextual understanding that runs deeper than automated exploitation chains. The argument here is about what kind of problem you are solving with each approach.
What Continuous Testing Actually Catches
When we connect to a codebase and run the first baseline scan, we typically see a cluster of findings that the last manual pentest missed. Not always because the pentest team was insufficiently thorough. Often because those findings arrived after the pentest closed. A new endpoint that was added three weeks ago. A configuration change that was made without a security review because it looked like an infrastructure change, not an application change.
The more valuable thing continuous testing catches is the regression. A developer fixes a SQL injection vulnerability in the main query path. Two months later, a new developer adds a reporting feature that constructs a similar query using the same pattern. The fix was in place. The regression is new. A quarterly pentest that runs once between those two events catches neither.
Testing on every code push, or at minimum every deployment, changes the relationship between security and development from a periodic audit to a real-time feedback loop. A developer who sees a security finding surface in their pull request is looking at code they wrote yesterday. The mental context is intact. Remediation is fast. The defect does not compound into technical security debt over months.
The Organizational Pattern That Makes Quarterly Testing Persist
Despite these problems being widely understood in the AppSec community, quarterly testing persists partly because of how security budgets and compliance programs are structured. PCI DSS and SOC 2 specify penetration testing requirements with annual or periodic language. Compliance officers are comfortable with a pentest report that has a date on it. Annual or quarterly testing produces a clean artifact for the compliance file.
The gap between what compliance requires and what actual security requires is where most breach risk lives. Compliance testing tells auditors you ran a test. Continuous testing tells your security team what is actually exploitable right now. The former is a documentation exercise. The latter is operational security.
Teams that run both have the best posture: continuous automated testing for the detection cadence that matches attacker reconnaissance speed, and periodic manual engagements for the depth of coverage that requires human adversarial creativity. Treating them as competing options misses the point. They answer different questions on different timescales.