Everything gets reviewed twice. Does it need to?

The problem: everything got the full treatment

Anything shared outside INL had to clear the lab’s review system first. Every item got an export-control review and a classification review, no matter what was in it. A third review, which rated sensitivity in specific scientific subject areas from green to red, depended on how the submitter answered a set of yes/no questions.

In a lab-wide survey about the system, the most common complaint, by far, was how long reviews took.

Meanwhile reviewers were spending a lot of their time on items that never needed them.

Digging in: the input was broken

Submitters answered yes/no screening questions that were supposed to flag which reviews applied. In practice the answers were too unreliable to use, so export control and classification reviewed everything, just in case.

The third review shows how bad the answers were, because they failed both ways. Some items got flagged for a review they didn’t need. Other submitters answered no to every question to save time, which skipped reviews that should have happened. The first wasted reviewers’ time. The second was a security risk.

• I met with every reviewing team. In compliance work that’s not optional.
• I drew the rules. A flow chart of which conditions trigger which reviews, checked back with the teams.
• I sized it with the reviewers’ own notes. Reviewers left comments when a screening question had been answered wrong. From those, I estimated reliable answers would cut manual reviews by ~70–80%. I don’t have the exact counts anymore.

Constraints

• Regulated, sensitive content. A wrong routing decision isn’t a bug report. It’s a compliance problem.
• Nothing could leave the lab. The model was the lab’s call, not mine: a version of ChatGPT hosted on the lab’s own servers.
• Trust comes first. If reviewers don’t trust a tool, they double-check everything it does, and it saves nobody any time.

What I did

I went after the input, not the reviewers. I proposed the lab’s first AI review product: an AI agent that reads the uploaded document and answers the screening questions itself. It would be grounded in the lab’s internal classification guidelines, external export-control regulations and historical review data from the system. Its answers would feed routing, so each item would go only to the reviews it actually needs. Reviewers would keep the judgment calls and stop seeing items that never needed them.

I wrote the evaluation plan, and it’s deliberately slow. The plan has three phases:

1. Shadow. Run the AI alongside the existing process and compare its answers to what reviewers actually decided.
2. Rely. Use it for routing only once accuracy reaches 95% or better. That bar was really about the “no”s. A confident wrong “no” would let sensitive information skip a review it needed.
3. Maintain. Keep it current as guidelines and regulations change.

Tradeoff: a faster launch would have shown value sooner. I chose slow on purpose, because in compliance one bad miss can sink trust in the whole idea.

When it’s unsure, it answers “yes,” before the 95% gate and after it. “Yes” is the conservative answer: it sends the item to a person.

Tradeoff: all the savings come from correctly spotting the “no”s, so every unsure “yes” eats into them. I’d rather give up some savings than ask reviewers to trust a guess.

I also wrote the UI specs and the business case that got executives to back a multi-phase build.

What happened

The project was approved a few months before I left INL in December 2025. The build was just starting. It had not shipped when I left, so there are no measured results here, only projections:

• ~70–80% fewer manual reviews (projected). Estimated from reviewer comments flagging wrong screening answers.
• $1M+ annually (projected) in reviewer time. Method: sample reviewer salaries × average review time × how often unreliable flags caused an unnecessary review.
• Faster turnaround for submitters, which was the #1 complaint in the survey. Also projected, not measured.

From one feature to a department roadmap

The routing project got noticed. Through it, the head of the compliance department learned I knew my way around AI, and she asked me to build an AI roadmap for the whole department.

I met with the leadership of each team (Supply Chain Risk Management, Export Control and the Privacy Program), then wrote up where each one stood, what hurt and where AI could actually help.

One example from Privacy: the team was trying to go through every tool used at the lab and identify which ones held personally identifiable information (PII). It was largely manual and had made hardly any progress. The roadmap proposed training AI to spot PII using the same criteria the privacy team used themselves, then automatically adding each tool to their system along with how much PII it holds. The HR system, for example, would get flagged as holding more PII than the review system.

That’s a proposal, not a result. I don’t know what came of it after I left.

Looking back

If I were still there, I’d watch two things: how the shadow-phase answers compared to reviewer decisions, and how often low-confidence items landed on a human. If too many land there, the savings shrink. If almost none do, I’d want to know why the model is so sure of itself.