Perspectives

Published guidance on AI governance now points to a small set of common references: the NIST AI Risk Management Framework (AI RMF 1.0), the ISO/IEC 42001:2023 standard for AI management systems, OMB memoranda M-25-21 and M-25-22 on federal agency AI use and acquisition, and guidance written for individual sectors. These documents set out what a governance program has to cover. The AI RMF, for example, calls for documented roles, responsibilities, and lines of communication (GOVERN 2.1). None of them can decide, for a particular organization, who holds which decision.

Those assignments are the harder part of the work. Who can approve a model for use in a critical system? Who has the authority to roll a model back when its performance drifts during operations? Which roles have to be defined before a capability moves from pilot to production? A framework people can follow answers each question in writing, naming the role that holds the decision and the point at which it escalates.

Decision-rights design for leadership teams is part of our AI governance practice.

AI features are reaching the people who run operational systems day to day faster than design practices for those operators have developed. Human factors research described the risk decades ago. Parasuraman and Riley (1997) separated misuse, meaning over-reliance on automation, from disuse, meaning neglect of automation that operators do not trust. Lee and See (2004) set the design goal as appropriate reliance, with trust calibrated to what the automation can actually do.

Calibrated trust therefore belongs in the system’s requirements, where it can be specified and tested before deployment. In practice that means showing confidence information and the reasons behind an output while the operator is still deciding, in a form the operator can act on. It also means designing a fallback to the non-AI workflow, so operators keep the skills to do the job when a model degrades or is unavailable.

An organization’s AI review process can make both a condition of deployment. Our AI governance practice designs that review process as part of framework implementation support.

AI training often starts with the tools, such as what a large language model is and how a model is fine-tuned. People who use AI output in operational work need that background, and they also need practice with their own tasks: checking a model’s output under time pressure, recognizing when its performance has degraded, and escalating when an output conflicts with what the task requires. The NIST AI RMF links training to the job in the same way. Its GOVERN 2.2 subcategory calls for AI risk management training that enables personnel and partners to perform their duties and responsibilities.

A curriculum built around the work looks different from tool-centric training. Modules are shorter and assessed with scenarios taken from the job, and follow-up sessions are built into sustainment so people keep practicing after the first course.

Our workforce enablement practice designs AI literacy courses for technical and operational teams.