The standard provides clear instructions for implementation at clause 6.1.4:
The organisation shall define a process for assessing the potential consequences for individuals or groups of individuals, or both, and societies that can result from the development, provision or use of AI systems.
To meet this requirement in our Artificial Intelligence Management System, we can draw guidance from two main sources:
- First, the standard itself, where Annex B provides implementation guidance for the controls in Annex A, though these are advisory rather than mandatory instructions from the regulator.
- Second, ISO 42005, the newly released standard that focuses specifically on AI system impact assessments.
Before Starting
When should you conduct your first Impact Assessment?
Impact assessments should be conducted before impacts occur. This means the first assessment should happen before your organisation even starts developing or using the AI system, and regular reviews should be scheduled throughout the system lifecycle.
Risk Assessment vs Impact Assessment
The impact assessment must be integrated into your broader risk management lifecycle. This is not only an ISO 42001 requirement, with risk being central to the standard, but also provides valuable strategic information for decision-making. For more detailed guidance on this integration, see ISO 42005, Annex B.2.
Who should contribute to informing the Impact Assessment
Impact assessments require collaboration across the entire organisation. Some sections can only be completed with support from technical teams, legal departments, compliance teams, and business stakeholders. This collaborative approach ensures comprehensive coverage of all potential impacts.
The Impact Assessment procedure
First of all, define the scope of the assessment, covering what the assessment will include, the triggers that initiate an assessment, what triggers assessment reviews, and the roles and responsibilities determining who does what in the process. These elements don’t appear in the final assessment document but must be defined in your procedures to establish the ‘who’ and ‘how’ of your AIMS Impact Assessment.
Step 1 – AI System Information
Provide a comprehensive description of the AI system within the assessment scope. This includes basic details such as the system name, life cycle stage, who prepared the assessment and who reviewed it. You’ll also need to include a system description explaining what it does and its main features, along with its purpose, intended uses and potential unintended uses.
Step 2 – Data information and quality
Data is crucial to AI systems. For each dataset used to train, test, and operate the system, provide detailed information including the name, owner, and access rights, as well as content, intended use, and origin of underlying data. You should also document known biases and any quality manipulations, plus the quality characteristics that make the dataset suitable for its purpose.
If you’re unsure about quality dimensions, The Government Data Quality Framework identifies six key areas: Completeness, Uniqueness, Consistency, Timeliness, Validity, and Accuracy.
Step 3 – Algorithm and model information
The technical team should provide detailed information about the algorithms used to develop the model, training parameters and methods, model selection processes, and performance monitoring metrics. They should also document the steps taken to address bias, data drift, and other potential harms.
Step 4 – Deployment environment
Consider the contextual factors surrounding the AI system. Start with geographic considerations, examining the legal, social, cultural, and economic constraints in deployment areas. You should also consider the temporal context, looking at how constraints might evolve over time and whether future deployment will become more challenging.
Step 5 – Relevant interested parties
Identify all stakeholders who will be impacted by the system deployment, including individuals, groups of individuals, and broader societies. Your impact assessment procedure should also determine if and how assessment outcomes will be shared with these identified parties.
Step 6 – Actual and potential impacts
For each stakeholder identified in Step 5, identify both positive and negative impacts using ISO 42005’s eight key perspectives: accountability, transparency, fairness and discrimination, privacy, reliability, safety, explainability, and environmental impact.
Step 7 – Measures to address harms and benefits
For each identified stakeholder, and considering the impacts from Step 6, document information on potential failure impacts and consequences of misuse or abuse. Include mitigation measures for identified harms and plans to maximise positive benefits.
Assessment Templates
Several impact assessment templates are available online, which organisations can use freely, provided they meet all ISO 42001 requirements. Alternatively, you can use the template we’ve prepared at Deviceology, which is built upon the framework provided by ISO 42005, ensuring full compliance with the standard’s requirements.
If you are unsure how to complete this template, don’t hesitate to contact us and we will be happy to help!
