Back

Establish, implement, and maintain human oversight over artificial intelligence systems.


CONTROL ID
15003
CONTROL TYPE
Behavior
CLASSIFICATION
Preventive

SUPPORTING AND SUPPORTED CONTROLS



This Control directly supports the implied Control(s):
  • Establish, implement, and maintain an artificial intelligence system., CC ID: 14943

This Control has the following implementation support Control(s):
  • Implement measures to enable personnel assigned to human oversight to intervene or interrupt the operation of the artificial intelligence system., CC ID: 15093
  • Implement measures to enable personnel assigned to human oversight to be aware of the possibility of automatically relying or over-relying on outputs to make decisions., CC ID: 15091
  • Implement measures to enable personnel assigned to human oversight to interpret output correctly., CC ID: 15089
  • Implement measures to enable personnel assigned to human oversight to decide to refrain from using the artificial intelligence system or override disregard, or reverse the output., CC ID: 15079


SELECTED AUTHORITY DOCUMENTS COMPLIED WITH



  • Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstandi… (Article 14 2., Proposal for a Regulation of The European Parliament and of The Council Laying Down Harmonized Rules On Artificial Intelligence (Artificial Intelligence Act) and Ameding Certain Union Legislative Acts)
  • identified and built, when technically feasible, into the high-risk AI system by the provider before it is placed on the market or put into service; (Article 14 3(a), Proposal for a Regulation of The European Parliament and of The Council Laying Down Harmonized Rules On Artificial Intelligence (Artificial Intelligence Act) and Ameding Certain Union Legislative Acts)
  • identified by the provider before placing the high-risk AI system on the market or putting it into service and that are appropriate to be implemented by the user. (Article 14 3(b), Proposal for a Regulation of The European Parliament and of The Council Laying Down Harmonized Rules On Artificial Intelligence (Artificial Intelligence Act) and Ameding Certain Union Legislative Acts)
  • The governing body should take responsibility for the use of AI, rather than attributing responsibility to the AI system itself. Members of the governing body are responsible for informing themselves about the possibilities and risks raised by using AI systems. Members of the governing body should b… (§ 4.3 ¶ 2, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • The use of AI can result in new obligations for the organization. These can be legal requirements or as a consequence of the adoption of voluntary codes of practice, whether directly within an AI system's automation of decision-making processes or indirectly through its use of data or other resource… (§ 5.1 ¶ 7, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • Address the scope of use. Ensure that the scope of automation is overseen by the governing body and implemented by appropriately authorized and skilled people (see 6.3). The governing body should ensure that the requisite authority, responsibility and accountability are maintained and that the conse… (§ 5.5 ¶ 1 Bullet 2, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • adequate human oversight is in place while using AI; (§ 6.2 ¶ 3 Bullet 3, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • The governing body should ensure that oversight arrangements for AI are established and are appropriate to the risks associated with the organization's use of AI. (§ 6.2 ¶ 1, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • The governing body should monitor the types of decision and output generated by automated systems and direct management to ensure that such systems are configured to operate within acceptable bounds by implementing appropriate controls. Such controls should provide the governing body with appropriat… (§ 6.3 ¶ 4, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • the extent to which the existing oversight processes, structures and controls address the specific aspects of AI; (§ 6.6.1 ¶ 4 Bullet 6, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • has the authority to make decisions (or knows to whom to request that a decision be made) and knows to whom to report back; (§ 6.2 ¶ 3 Bullet 4 Sub-bullet 3, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • A governing body can consider deploying AI in order to pursue specific opportunities that the organization has identified, including the organization's future growth or better achieving the organization's purpose and objectives. In such cases, the governing body needs to weigh those opportunities ag… (§ 5.4 ¶ 1, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • the use of AI to monitor other AI systems and the extra monitoring or alerting that can be required. (§ 6.6.2 ¶ 2 Bullet 5, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • Level of responsibility. Ensuring that the level of decision-making matches the authority granted and responsibility associated to the decision is a critical element of good governance. Defining the scope and impact of possible decisions and matching those to the levels of responsibility is necessar… (§ 6.3 ¶ 6 Bullet 2, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • For these reasons, the governing body should be explicit about its culture and values and have the appropriate governance mechanisms and policies to ensure such AI system behaviours can be monitored and corrected when needed. In some cases, the scope and impact of the AI system should be constrained… (§ 6.5 ¶ 3, ISO/IEC 38507:2022, Information technology — Governance of IT — Governance implications of the use of artificial intelligence by organizations)
  • Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the GOVERN function. (MAP 3.5, Artificial Intelligence Risk Management Framework, NIST AI 100-1)