liveBook · Manning

Architecture review

As explained in AI Governance

An implementation review is a formal examination of an AI system design to verify that required legal, security, privacy, and operational controls are feasible and addressed.

An architecture review checks whether a proposed AI system design can meet organizational controls without introducing avoidable risks. Start before implementation, when storage, logging, vendor settings, prompts, workflows, and tests can still be changed cheaply. section 3.4

Bring in security and privacy engineering, adding ML Ops, data science, reliability, procurement, or vendor management when the design depends on their areas. Record findings, required fixes, accepted residual risks, owners, due dates, and evidence in the release-management system. Link the AI inventory to those records rather than using it as a substitute. section 3.4

Trace the system’s actual data and model flow, then turn findings into build controls:

  • Check what enters the system. Patient images may expose metadata or visible identifiers; strip EXIF metadata, randomize filenames, request tight crops, and reject or mask identifying content. section 3.4.3
  • Check every processing step. OCR could pass hidden image text to an LLM as an instruction; disable unnecessary OCR and constrain responses to approved recommendations. section 3.4.3
  • Check data handling after inference. Discard raw images and retain only approved limited log fields under the required retention rule. section 3.4.3
  • Check vendor and API choices against technical standards, including whether requests and responses are logged and whether retention and data-use guarantees are adequate. section 3.4

Derm-Cam

Reviewers traced photo upload, preprocessing, optional OCR, model call, patient wording, logs, and deletion. They found excess personal data in images and a prompt-injection path through OCR, leading to changes in image handling, logging, and model capabilities. section 3.4.3

Figure 3.6 Illustrative example of how image EXIF metadata might reveal sensitive information about the user.

Illustrates how image EXIF metadata can expose sensitive user information. section 3.4.3

Figure 3.8 Illustration of prompt injection through an image. Hidden text in the image could reveal system instructions or alter downstream model behavior.

Illustrates how hidden image text can inject instructions into a model workflow. section 3.4.3

How AI Governance covers it

How deep it goes
Traces a system flow, identifies privacy and security risks, and records fixes, residual risks, owners, deadlines, and evidence.
Examples
Uses Derm-Cam photo uploads to examine image identifiers, OCR-based prompt injection, vendor API controls, and post-inference deletion.
What you can do after
Apply it by tracing data flows and assigning fixes, launch requirements, residual risks, owners, deadlines, and evidence.
How it connects
Connects architecture review to privacy engineering, threat modeling, data retention, logging, OCR, prompt injection, and AI impact assessment.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.