AI Governance and the EU AI Act: A Technical Deep Dive

The European Union’s Artificial Intelligence Act (EU AI Act) entered into force on August 1, 2024, establishing the world’s first comprehensive, legally binding framework for AI systems. Unlike sector-specific regulations (GDPR for data, the Medical Device Regulation for medical AI), the EU AI Act applies horizontally across industries and use cases, classifying AI systems by risk level and imposing proportionate requirements.

For AI practitioners, researchers, and organizations building or deploying AI systems, understanding the Act is not optional — it will shape AI development practices globally, much as GDPR shaped data privacy practices.

The Risk-Based Framework

The EU AI Act’s central organizing principle is a risk pyramid that classifies AI systems into four categories:

1. Unacceptable Risk (Prohibited)

AI systems posing unacceptable societal risk are banned outright. Prohibited applications include:

• Social scoring by public authorities: AI systems that evaluate citizens’ trustworthiness based on behavior and assign them scores affecting access to services
• Real-time remote biometric identification in public spaces for law enforcement (with narrow exceptions for missing children, terrorism prevention)
• Cognitive behavioral manipulation exploiting subconscious vulnerabilities to alter behavior in harmful ways
• Emotion recognition in workplaces and educational institutions
• AI-based profiling from biometric data to infer sensitive attributes (race, political opinion, sexual orientation)
• Untargeted scraping of facial images from the internet or CCTV for facial recognition databases

These prohibitions apply as of February 2, 2025 (6 months after entry into force).

2. High Risk

High-risk AI systems are not prohibited but face substantial compliance obligations. They fall into two categories:

Annex I — Safety components of products already regulated by EU law:

• AI in machinery, medical devices, in vitro diagnostics, aviation, vehicles, and marine equipment

Annex III — Standalone high-risk applications including:

• Biometric identification and categorization
• Critical infrastructure management (power, water, transport)
• Educational access and assessment
• Employment screening, HR decision-making, task allocation
• Essential services (credit scoring, insurance, emergency services)
• Law enforcement (risk assessment, evidence evaluation, profiling)
• Migration, asylum, and border control management
• Administration of justice and legal proceedings

3. Limited Risk (Transparency Obligations)

AI systems with specific transparency risks must disclose their nature:

• Chatbots must inform users they are interacting with AI
• Deepfakes must be labeled as AI-generated
• AI-generated content (text, images, audio) must be marked

4. Minimal Risk

The vast majority of AI applications — spam filters, AI in video games, AI-powered search — fall into minimal risk and have no mandatory requirements under the Act (though voluntary codes of conduct are encouraged).

High-Risk AI: Compliance Requirements in Detail

High-risk AI systems face the most technically demanding requirements:

Risk Management System

Providers must establish, implement, document, and maintain a risk management system throughout the entire lifecycle:

Risk Management System Requirements:
├── Identification and analysis of reasonably foreseeable risks
├── Estimation and evaluation of risks arising from intended use
├── Evaluation of risks from reasonably foreseeable misuse
├── Adoption of risk mitigation measures
└── Residual risk assessment and documentation

This is not a one-time exercise — it must be a continuous process with updates when the AI system or deployment context changes.

Data Governance

Training, validation, and testing datasets must meet specific requirements (Article 10):

• Relevance, representativeness, and completeness for the intended purpose
• Freedom from errors and completeness
• Appropriate statistical properties including with regard to protected characteristics
• Data governance practices including examination for biases
• Processing of special categories of data only where strictly necessary with appropriate safeguards

Practically, this requires systematic dataset documentation, bias auditing, and documentation of data collection and labeling processes.

Technical Documentation

Providers must produce and maintain technical documentation (Annex IV) before placing a high-risk AI system on the market. Required documentation includes:

Automatic Logging

High-risk AI systems must be capable of automatic logging of events relevant to:

• Identifying situations that could result in risk
• Human oversight decisions
• System performance over time

Log retention requirements vary by use case — generally at least as long as the AI system is in use.

Transparency and User Information

Deployers (those using high-risk AI) must be provided with information enabling proper use:

• Clear intended purpose
• Performance characteristics including accuracy, robustness, cybersecurity
• Known limitations and risks
• Instructions for human oversight and override
• Technical specifications for deployment environment

Human Oversight Design

High-risk AI systems must be designed to allow human oversight (Article 14):

• Enable oversight by natural persons during operation
• Allow humans to monitor system performance
• Allow humans to intervene or interrupt
• Allow humans to override automated decisions
• System should be “understandable” by humans designated for oversight

This has profound implications for AI architecture — systems must be designed with oversight interfaces, not just accuracy optimization.

Accuracy, Robustness, and Cybersecurity

High-risk AI must achieve “appropriate levels of accuracy, robustness and cybersecurity” and perform consistently throughout lifecycle (Article 15). The Act specifies:

• Performance should be consistent, even when input is adversarial
• Feedback loops that affect performance must be identified and mitigated
• Technical means to measure the degree of confidence of results should be provided

General Purpose AI Models (GPAI)

A major addition to the final Act text addresses General Purpose AI (GPAI) models — foundation models used across many applications. All GPAI providers must:

1. Technical documentation: Maintain comprehensive technical documentation before release
2. Copyright compliance: Publish summaries of training data content under EU copyright law
3. Downstream use policies: Establish policies for downstream providers and comply with them

GPAI models with systemic risk (defined as training compute exceeding $10^{25}$ FLOPs) face additional requirements:

• Model evaluations: Adversarial testing, red teaming, model evaluation reports
• Incident reporting: Reporting serious incidents and malfunctions to the AI Office
• Cybersecurity measures: Against state-level adversaries and nation-state risks
• Energy efficiency: Reporting actual energy consumption

This covers frontier models from providers like OpenAI, Anthropic, Google, and Meta operating in the EU.

The AI Office

The Act establishes a new EU AI Office within the European Commission, responsible for:

• Supervising GPAI model providers
• Developing standards and evaluation methodologies
• Maintaining the EU AI database of high-risk AI systems
• Coordinating enforcement across Member States

Member States must establish National Competent Authorities (NCAs) for AI oversight and designate notified bodies to conduct conformity assessments of high-risk AI systems.

Conformity Assessment: CE Marking for AI

Before a high-risk AI system (not already covered by existing product regulations) can be placed on the EU market, it must undergo conformity assessment (Article 43):

Self-assessment pathway: For most Annex III high-risk AI systems, providers can conduct internal conformity assessment based on Article 9 (risk management), Article 10 (data governance), Article 11 (technical documentation), and Articles 12–15 (other requirements).

Third-party assessment pathway: Required for:

• Remote biometric identification systems
• High-risk AI systems not falling under harmonized standards

Successful conformity assessment results in CE marking and registration in the EU AI database.

Prohibited and High-Risk Classification: Practical Examples

AI System                              Classification    Why
─────────────────────────────────────────────────────────────────────
Credit scoring algorithm               High Risk        Financial services access
Resume screening tool                  High Risk        Employment decisions
Medical diagnosis AI                   High Risk        Healthcare (regulated)
ChatGPT-style assistant               GPAI + Limited   GPAI + must disclose AI nature
Spam filter                            Minimal Risk     No significant harm potential
Social media recommendation            Minimal Risk     No critical decision-making
Real-time public face recognition      Prohibited       Surveillance / social control
Emotion recognition in school          Prohibited       Educational setting restriction
AI-powered plagiarism detector         Minimal Risk     Non-binding evaluation
Court sentencing risk assessment       High Risk        Administration of justice

Technical Compliance Checklist for Developers

For teams building AI systems for EU deployment:

□ Determine if system qualifies as an AI system under the Act definition
□ Identify correct risk category (unacceptable / high / limited / minimal)
□ If high-risk: complete Annex IV technical documentation
□ Implement and document risk management system
□ Conduct data governance analysis (bias, representativeness)
□ Design and implement human oversight mechanisms
□ Implement automatic logging
□ Conduct conformity assessment (self or third-party)
□ Affix CE marking (if applicable)
□ Register in EU AI database (if applicable)
□ Establish post-market monitoring plan
□ Prepare incident reporting procedures
□ If GPAI: publish training data summary, establish use policy

Timeline and Penalties

Penalties for non-compliance:

• Prohibited practices: Up to €35 million or 7% of global annual turnover
• Other high-risk provisions: Up to €15 million or 3% of global annual turnover
• GPAI model obligations: Up to €15 million or 3% of global annual turnover
• Providing incorrect information: Up to €7.5 million or 1.5% of global annual turnover

Global Implications: The Brussels Effect

The EU AI Act is likely to have significant global impact through the “Brussels Effect” — the tendency for EU regulations to become de facto global standards because:

1. The EU market is large enough that compliance is commercially necessary
2. Building country-specific variants of AI systems is expensive
3. Many EU-compliant practices (like technical documentation and bias testing) represent good practice regardless of regulatory requirement

Similar to GDPR’s influence on global data privacy practices, the EU AI Act is already influencing AI governance conversations in the US (state-level), UK, Canada, Japan, Brazil, and international standard-setting bodies (ISO/IEC, IEEE).

For AI practitioners, the Act is both a compliance obligation and a framework for thoughtful AI development practice — its technical documentation, risk management, and human oversight requirements align closely with what responsible AI development looks like in practice.