Daily · Article 01 · 4 min read
- Security
- Open source
How Cloudflare’s skill verifies an AI security warning
Open-source discovery · sources and weekly interest rechecked October 26
Summary
Cloudflare’s public security-review skill is a work manual for coding AI. A fictional invoice service shows what evidence is needed to check an AI’s suspicion before deciding what to fix.
An AI says it found a serious security bug. Should the owner immediately change the code? First establish which unauthorized data or action can actually be reached, and under which conditions.
Key facts
A skill is a work manual that gives an AI a particular task. Cloudflare, an internet infrastructure and security provider, publishes security-audit-skill to guide coding AI through finding, checking and recording security problems. Developer overview
One AI first identifies a suspicious path. A separate verifier checks whether a request reaches it, whether an ownership control blocks it, and what result is established. This follows our Agent Skills article with a concrete security-review application.
1. Ask whether another person’s data can be reached
An authorization boundary answers a familiar question: may this person see this record? Suspicious-looking code alone does not establish a leak if the actual request path checks ownership and blocks access.
Consider a fictional invoice service. User B asks for A’s invoice. The figure distinguishes denial, missing evidence and exposure; it is not a test against a real service.
Ownership is checked
A’s data is not shown to B
This request path is blockedActual settings and response are still unknown
It is not yet known whether another person’s data is visible
More evidence is neededAn isolated test lacks the ownership check
B can see A’s dummy invoice
A problem is established under these conditionsEven the final case establishes only that a dummy invoice was exposed in that test. It does not establish theft of every customer record or control of the server. The blocked case likewise refutes this path, not every possible vulnerability in the service.
The cat asks Q&A
The researcherIs finding code that seems to lack an ownership check insufficient?
Pocket NotebookCheck whether the request reaches it and whether an earlier control blocks it. Cloudflare’s guidance requires evidence of a real boundary failure. Review criteria
2. Separate discovery from verification
A person can select evidence that fits an initial guess, and an AI can repeat its first explanation. This workflow assigns a candidate to a separate verifier that rereads the evidence and attempts to disprove it. Verification procedure
In the fictional invoice case, a verifier checks requester identity, ownership enforcement and the returned result. Evidence of an earlier denial refutes the proposed path. If the execution environment is unavailable, the record retains the missing fact instead of inventing an outcome.
| Record | What a reader can assess |
|---|---|
| Source version and location | Which path was inspected, at which revision? |
| User, configuration and input conditions | Where does the conclusion apply? |
| Observation and remaining gaps | Was it demonstrated, or is evidence missing? |
- Record
Source version and location
- What a reader can assess
- Which path was inspected, at which revision?
- Record
User, configuration and input conditions
- What a reader can assess
- Where does the conclusion apply?
- Record
Observation and remaining gaps
- What a reader can assess
- Was it demonstrated, or is evidence missing?
The cat asks Q&A
- The researcher
Is agreement between two AIs enough?
Pocket NotebookRecheckable evidence matters more than agreement. Repeating a guess twice is not the same as independently comparing current source with an observation.
3. Keep three outcomes distinct
The report separates an established problem, a case needing more evidence, and a proposed problem ruled out by evidence. A case with a decisive missing fact does not receive a severity score. Report schema
| State | Meaning | Next step |
|---|---|---|
| Problem established | Evidence and observation establish the stated case | Fix and retest within that scope |
| More evidence needed | A decisive fact remains unresolved | Obtain the missing configuration or observation |
| Proposed problem ruled out | Evidence disproves the proposed problem | Keep the reason to avoid duplicate reporting |
- State
Problem established
- Meaning
- Evidence and observation establish the stated case
- Next step
- Fix and retest within that scope
- State
More evidence needed
- Meaning
- A decisive fact remains unresolved
- Next step
- Obtain the missing configuration or observation
- State
Proposed problem ruled out
- Meaning
- Evidence disproves the proposed problem
- Next step
- Keep the reason to avoid duplicate reporting
Saving defined fields in a file helps other programs read the result. Passing a format check does not make the content true, just as filling every field on a receipt does not establish that payment occurred.
The cat asks Q&A
- The researcher
Does “needs validation” mean no problem was found?
Pocket NotebookThe conclusion is still open. Read which fact is missing, then check that fact.
4. Prepare the environment before using it
The documented requirements include coding AI that can use tools and delegate work to other agents, a Node.js environment for running the checking programs, and an operating-system-enforced sandbox. A sandbox isolates the program under test from real accounts, files and services. Execution boundary
Reading source and executing it are separate operations. The instructions require controls including blocked external networking, limited writable paths and resource/time limits; if these cannot be enforced, execution is withheld and the finding remains unresolved. This article did not install the skill or run an audit.
The cat asks Q&A
- The researcher
Can I point it straight at a company’s production server?
Pocket NotebookEstablish ownership, authorization and an isolated test environment first. Ask the developer to define the source revision, scope, dummy data and observations the report must retain.
To close
Read the established scope, not just the bug label
The useful question is not how often an AI says “vulnerability.” It is whether the record establishes what data or authority was reached, who independently verified it and what remains unknown. That evidence separates a repair from an investigation still to be completed.
Publication and usage conditions
The background Cloudflare article was published on June 18. This is a practical follow-up, not a new launch. The license is MIT; the inspected Releases page had no formal release.
At 00:03 JST on October 26, the GitHub weekly page displayed 12,680 “stars this week” for the repository. That is interest over GitHub’s displayed weekly period, not an exact October 19–25 gain, adoption count or performance score.
Sources
- Developer overviewgithub.com
- Execution boundarygithub.com
- Verification proceduregithub.com
- Report schemagithub.com
- Cloudflare articleblog.cloudflare.com
- MITgithub.com
- Releases pagegithub.com
- GitHub weekly pagegithub.com