Skip to content
Lumen
Book a demo

Webinar45 min

Webinar: Passing a security review in 30 days

Two security leads walk through how they reviewed an AI agent vendor, and what they asked for.

Recording of the webinar Passing a security review in 30 days

In this 45-minute session, two security leads walked through how they reviewed an AI agent vendor from first questionnaire to signed approval, and how they kept the review to about 30 days. They covered the documents worth requesting, the red flags that slowed them down and how they ran a read-only pilot while the review was still open. This recap summarises what was covered, the main takeaways and the questions from the audience.

What the session covered

The speakers started with the problem most security teams face: a business team wants to try an agent this quarter, and the standard vendor review takes three months. Both had been in that position and both had approved a vendor within roughly a month without skipping steps.

The session moved through four parts:

  1. Scoping the review. How to decide which systems the agent will touch in the pilot and which it will not, so the review has a clear boundary.
  2. The document request. The short list of documents each of them asked for on the first day, and why.
  3. Red flags. The answers that made them slow down or walk away.
  4. The parallel pilot. How a four to six week pilot with read-only access ran alongside the review, and what evidence it produced.

Key takeaways

Ask for documents on day one, all at once. Both speakers sent a single request on the first day: the security questionnaire, the data processing agreement, the latest penetration test summary, a sample audit log export and the retention settings. Asking for everything at once removed weeks of back and forth.

Read-only access shortens the review. When every connection starts read-only, the review can focus on what the agent sees, which is a smaller question than what it can change. Write access was treated as a later, separate approval, granted one action at a time.

Test the log yourself. Each speaker asked for a test workspace, ran a few drafts through it and exported the audit log. If the log did not show what the agent read, who approved the action and when, the vendor did not pass.

Get the training answer in writing. A clear contractual statement that customer data is never used to train models was a requirement for both teams. Answers that relied on a web page or a toggle in settings were sent back.

“We stopped treating the pilot as a risk to the review. With read-only access, the pilot was the best evidence the review had.”

Elin Sørensen, Security Lead, Northpass

Red flags the speakers watch for

The speakers listed the answers that made them pause:

  • A vendor that needs broad write access before it can show any value
  • Default retention with no option to shorten it
  • An “autonomous” mode that lets actions run without a named person
  • Audit logs that can be viewed in the product but cannot be exported
  • User management without SSO, or without SCIM to remove leavers

None of these ended a review on their own, but each one added time, and three or more usually meant the vendor was not ready for a regulated team.

Questions from the audience

How did you handle teams that wanted to start before the review began? Marcus said his team agreed a written scope first: one workflow, read-only access to named systems and a fixed end date. That was enough for the business to start while the review ran.

Did you involve legal early? Both said yes. Legal saw the data processing agreement in the first week, so retention and training terms were settled before the pilot ended.

What did you ask for at the end of the pilot? The exported audit log covering the full pilot, a count of drafts approved, edited and rejected, and a list of the write permissions the team wanted next.

Would you approve write access for a whole system at once? Neither would. They approve individual actions, such as adding an internal note to a ticket, and review the log again after the first month.

Speakers

  • Elin Sørensen, Security Lead, Northpass. Elin leads security and compliance at Northpass and ran its first review of an AI agent vendor.
  • Marcus Hale, Head of Security Operations. Marcus runs the team that triages alerts and reviews access, and approved the pilot that put an agent into that queue.

The session was hosted by the Lumen team. The recording is available on request, along with the document checklist the speakers shared.