Reclaiming transparency: contesting the logics of secrecy within the AI Act

Busuioc, Madalina; Curtin, Deirdre; LASMAR ALMADA, Marco Antonio

doi:10.1017/elo.2022.47

Download

Article (Scientific journals)

Reclaiming transparency: contesting the logics of secrecy within the AI Act

Busuioc, Madalina; Curtin, Deirdre; LASMAR ALMADA, Marco Antonio

2023 • In European Law Open, 2 (1), p. 79 - 105

Peer Reviewed verified by ORBi

Permalink
https://hdl.handle.net/10993/64101

DOI
10.1017/elo.2022.47

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

Busuioc et al. - 2023 - Reclaiming transparency contesting the logics of.pdf

Publisher postprint (430.45 kB)

Download

All documents in ORBilu are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

artificial intelligence; AI Act; explanation; Selected; (algorithmic) transparency; public accountability

Abstract :

[en] Transparency is widely acknowledged as a core value in the governance of artificial intelligence (AI) technologies. However, scholarship on AI technologies and their regulation often casts this need for transparency in terms of requirements for the explanation of algorithmic outputs and/or decisions produced with the involvement of opaque black-box AI systems. Our article argues that this discourse has re-interpreted and reshaped transparency in fundamental ways away from its original meaning. The target of transparency – in most cases, the provider of AI software – determines and shapes what is made visible to the outside world, and there is no external check on the validity and accuracy of such mediated accounts and explanations, opening transparency up for manipulation. Through a theoretically informed and critical analysis of the transparency provisions in the European Union’s AI Act proposal, the article shows that the substitution of transparency with mediated explanations faces important technical constraints, creates opportunities and incentives for both providers and public-sector users of AI systems to adopt opaque practices, and reinforces secrecy requirements that gag accountability in practice. An approach to transparency as disclosure thus becomes necessary, even if not sufficient in and of itself, to ensure the accountable development and use of AI technologies in the European Union. Transparency needs to be reclaimed as a core concept, accountability tailored and reinforced and the necessity for secrecy re-examined and cordoned off.

Disciplines :

European & international law
Public law

Author, co-author :

Busuioc, Madalina

Curtin, Deirdre

LASMAR ALMADA, Marco Antonio ; University of Luxembourg > Faculty of Law, Economics and Finance (FDEF) > Department of Law (DL)

External co-authors :

yes

Language :

English

Title :

Reclaiming transparency: contesting the logics of secrecy within the AI Act

Publication date :

2023

Journal title :

European Law Open

eISSN :

2752-6135

Publisher :

Cambridge University Press, Cambridge, Gb

Volume :

Issue :

Pages :

79 - 105

Peer reviewed :

Peer Reviewed verified by ORBi

Focus Area :

Law / European Law

Development Goals :

16. Peace, justice and strong institutions

Additional URL :

https://www.cambridge.org/core/journals/european-law-open/article/reclaiming-transparency-contesting-the-logics-of-secrecy-within-the-ai-act/01B90DB4D042204EED7C4EEF6EEBE7EA

Available on ORBilu :

since 05 February 2025

Statistics

Number of views

87 (1 by Unilu)

Number of downloads

31 (1 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

F Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information (Reprint Edition, Harvard University Press 2016).
2J Burrell, ‘How the Machine “Thinks”: Understanding Opacity in Machine Learning Algorithms’ 3 (2016) Big Data & Society 1.
M Busuioc, ‘Accountable Artificial Intelligence: Holding Algorithms to Account’ 81 (2021) Public Administration Review 825;
N Diakopoulos, Algorithmic Accountability Reporting: On the Investigation of Black Boxes (Tow Center for Digital Journalism 2014).
4Belastingdienst/Toeslagen: De verwerking van de nationaliteit van aanvragers kinderopvangtoeslag (Autoriteit Persoonsgegevens 2020) Research Report z2018-22445 accessed 1 May 2022;
Ongekend onrecht. Parlementaire ondervragingscommissie Kinderopvangtoeslag (Tweede Kamer, 2020) accessed 1 May 2022.
M Heikkilä, ‘A Dutch Algorithm Scandal Serves a Warning to Europe – The AI Act Won’t Save Us’ POLITICO (30 March 2022)
wont-save-us-2/> accessed 1 May 2022;
G Geiger, ‘How a Discriminatory Algorithm Wrongly Accused Thousands of Families of Fraud’ (Vice, 1 March 2021) accessed 1 May 2022.
6D Hadwick and S Lan, ‘Lessons to Be Learned from the Dutch Childcare Allowance Scandal: A Comparative Review of Algorithmic Governance by Tax Administrations in the Netherlands, France and Germany’ 13 (2021) World Tax Journal 6.
7On the broader debates about the impact of automation in administrative law, see, inter alia, DK Citron, ‘Technological Due Process’ 85 (2008) Washington University Law Review 1249;
R Calo and DK Citron, ‘The Automated Administrative State: A Crisis of Legitimacy’ 70 (2021) Emory Law Journal 797;
C Harlow and R Rawlings, ‘Proceduralism and Automation: Challenges to the Values of Administrative Law’ in E Fisher, J King and A Young (eds), The Foundations and Future of Public Law: Essays in Honour of Paul Craig (Oxford University Press 2020) 275–98;
J Cobbe, ‘Administrative Law and the Machines of Government: Judicial Review of Automated Public-Sector Decision-Making’ 39 (2019) Legal Studies 636;
D Curtin, ‘The EU Automated State Disassembled’ in E Fisher, J King and A Young (eds), The Foundations and Future of Public Law: Essays in Honour of Paul Craig (Oxford University Press 2020) 233–56;
S Ranchordás, ‘Empathy in the Digital Administrative State’ 71 (2022) Duke Law Journal 1341.
C Birchall, Radical Secrecy: The Ends of Transparency in Datafied America (University of Minnesota Press 2021) 4.
9Deirdre Curtin, ‘“Accountable Independence” of the European Central Bank: Seeing the Logics of Transparency’ 23 (2017) European Law Journal 28.
0I Koivisto, Thinking Inside the Box: The Promise and Boundaries of Transparency in Automated Decision-Making (European University Institute 2020) Working Paper AEL 2020/01 3. On this very point, see also Curtin (n 9); G Michener and K Bersch, ‘Identifying Transparency’ 18 (2013) Information Polity 233.
C Birchall, ‘Radical Transparency?’ 14 (2014) Cultural Studies ↔ Critical Methodologies 77.
A Buijze, ‘The Six Faces of Transparency’ 9 (2013) Utrecht Law Review 3.
M Finck, ‘Automated Decision-Making and Administrative Law’ in P Cane and Others (eds), The Oxford Handbook of Comparative Administrative Law (Oxford University Press 2020);
L Edwards and M Veale, ‘Slave to the Algorithm: Why a Right to an Explanation Is Probably Not the Remedy You Are Looking For’ 16 (2017) Duke Law & Technology Review 18;
T Wischmeyer, ‘Artificial Intelligence and Transparency: Opening the Black Box’ in T Wischmeyer and T Rademacher (eds), Regulating Artificial Intelligence (Springer International Publishing 2020);
M Brkan and G Bonnet, ‘Legal and Technical Feasibility of the GDPR’s Quest for Explanation of Algorithmic Decisions: Of Black Boxes, White Boxes and Fata Morganas’ 11 (2020) European Journal of Risk Regulation 18; M Fink and M Finck, ‘Reasoned A(I)Administration: Explanation Requirements in EU Law and the Automation of Public Administration’ 47 (2022) European Law Review 376.
8S Wachter, B Mittelstadt and C Russell, ‘Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR’ 31 (2018) Harvard Journal of Law & Technology 842.
19AI HLEG, Ethics Guidelines for Trustworthy AI (European Commission 2019); L Floridi and Others, ‘AI4People–An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations’ 28 (2018) Minds and Machines 689.
22See in general terms the blog with some overview, HCH Hofmann and P Leino-Sandberg, ‘An Agenda for Transparency in the EU’ (European Law Blog, 23 October 2019) accessed 13 June 2022.
D Curtin, ‘Second Order Secrecy and Europe’s Legality Mosaics’ 41 (2018) West European Politics 846.
V Abazi, Secrecy and Oversight in the EU: Law and Practices of Classified Information (Oxford University Press 2019).
5I Koivisto, The Anatomy of Transparency: The Concept and Its Multifarious Implications Working Paper EUI/MWP 2016/09.
6D Curtin and AJ Meijer, ‘Does Transparency Strengthen Legitimacy?’ 11 (2006) Information Polity 109.
S Bok, Secrets: On the Ethics of Concealment and Revelation (Oxford University Press 1984);
A Gutmann and DF Thompson, Democracy and Disagreement (Belknap Press 1998) ch 3;
O Pfersmann, ‘Norme de secret, normes secrètes et État moderne’ 26 (2006) Cités 115; DE Pozen, ‘Deep Secrecy’ 62 (2010) Stanford Law Review 257;
KL Scheppele, Legal Secrets: Equality and Efficiency in the Common Law (University of Chicago Press 1988).
9MGW den Boer, ‘Steamy Windows: Transparency and Openness in Justice and Home Affairs’ in V Deckmyn and I Thomson (eds), Openness and Transparency in the European Union (European Institute of Public Administration 1998) 105, emphasis added.
2I Koivisto, ‘Transparency in the Digital Environment’ 8 (2021) Critical Analysis of Law 1, 2.
G Porumbescu, A Meijer and S Grimmelikhuijsen, Government Transparency: State of the Art and New Perspectives, 1st edn (Cambridge University Press 2022) 11; Michener and Bersch (n 10).
C Hood and D Heald (eds), Transparency: The Key to Better Governance? (Oxford University Press 2006);
DE Pozen, ‘Transparency’s Ideological Drift’ 128 (2018) Yale Law Journal 100.
Porumbescu, Meijer and Grimmelikhuijsen (n 33) 11–12.
DE Pozen and M Schudson (eds), Troubling Transparency: The History and Future of Freedom of Information (Columbia University Press 2018).
C Grønbech-Jensen, ‘The Scandinavian Tradition of Open Government and the European Union: Problems of Compatibility?’ 5 (1998) Journal of European Public Policy 185.
V Mabillard and R Zumofen, ‘The Complex Relationship between Transparency and Accountability: A Synthesis and Contribution to Existing Frameworks’ 32 (2017) Public Policy and Administration 110.
A Meijer, ‘Transparency’ in M Bovens, RE Goodin and T Schillemans (eds), The Oxford Handbook of Public Accountability (Oxford University Press 2014) 507–24.
5M Fenster, ‘Transparency in Search of a Theory’ 18 (2015) European Journal of Social Theory 150, 150.
B Rittberger and KH Goetz, ‘Secrecy in Europe’ 41 (2018) West European Politics 825.
L Naudts, P Dewitte and J Ausloos, ‘Meaningful Transparency through Data Rights: A Multidimensional Analysis’ in E Kosta and R Leenes (eds), Research Handbook on EU Data Protection Law (Edward Elgar Publishing 2022) s 3.4.
5I Koivisto, The Transparency Paradox (Oxford University Press 2022).
59K de Fine Licht and J de Fine Licht, ‘Artificial Intelligence, Transparency, and Public Decision-Making’ 35 (2020) AI & Society 917, 922.
J Angwin and Others, ‘Machine Bias’ (ProPublica, 23 May 2016) accessed 1 May 2022.
5J Kroll and Others, ‘Accountable Algorithms’ 165 (2017) University of Pennsylvania Law Review 633, 633–4.
D Pavlovic, ‘Gaming Security by Obscurity’, in S Peisert and Others (eds), NSPW’11: Proceedings of the 2011 New Security Paradigms Workshop (Association for Computing Machinery 2011) 125–40;
B Biggio and F Roli, ‘Wild Patterns: Ten Years after the Rise of Adversarial Machine Learning’ 84 (2018) Pattern Recognition 317;
A Venturi and C Zanasi, ‘On the Feasibility of Adversarial Machine Learning in Malware and Network Intrusion Detection’ in 2021 IEEE 20th International Symposium on Network Computing and Applications (NCA) (2021).
7C Rudin, ‘Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead’ 1 (2019) Nature Machine Intelligence 206.
SM Lundberg and S-I Lee, ‘A Unified Approach to Interpreting Model Predictions’ in NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems December 2017, 4768–77; Wachter, Mittelstadt and Russell (n 18);
P Jonathon Phillips and Others, Four Principles of Explainable Artificial Intelligence (National Institute of Standards and Technology 2021) s 6.
S Barocas, AD Selbst and M Raghavan, ‘The Hidden Assumptions behind Counterfactual Explanations and Principal Reasons’ in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (ACM 2020);
H Lakkaraju and O Bastani, ‘“How Do I Fool You?”: Manipulating User Trust via Misleading Black Box Explanations’ in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (Association for Computing Machinery 2020);
S Bordt and Others, ‘Post-Hoc Explanations Fail to Achieve Their Purpose in Adversarial Contexts’ in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2022).
M Wieringa, ‘What to Account for When Accounting for Algorithms: A Systematic Literature Review on Algorithmic Accountability’ in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery 2020) 4–5.
G De Gregorio and P Dunn, ‘The European Risk-Based Approaches: Connecting Constitutional Dots in the Digital Age’ 59 (2022) Common Market Law Review 473.
N Smuha and Others, A Response to the European Commission’s Proposal for an Artificial Intelligent Act (LEADS Lab @University of Birmingham 2021);
L Edwards, Regulating AI in Europe: Four Problems and Four Solutions (Ada Lovelace Institute 2022) Expert Opinion accessed 31 March 2022.
B Kuzniacki and Others, ‘Towards EXplainable Artificial Intelligence (XAI) in Tax Law: The Need for a Minimum Legal Standard’ 14 (2022) World Tax Journal s 4.1.2.
D Curtin and M de Goede, ‘Bits, Bytes, Searches and Hits: Logging-in Accountability for EU Data-Led Security’ in D Curtin and M Catanzariti (eds), Data at the Boundaries of (European) Law (Oxford University Press 2022).
S Alon-Barkat and M Busuioc, ‘Human-AI Interactions in Public Sector Decision-Making: “Automation Bias” and “Selective Adherence” to Algorithmic Advice’ (2022) Journal of Public Administration, Research and Theory, .
A Bibal and Others, ‘Legal Requirements on Explainability in Machine Learning’ 29 (2021) Artificial Intelligence and Law 149, s 3.3.
H van Kolfschooten, ‘EU Regulation of Artificial Intelligence: Challenges for Patients’ Rights’ 59 (2022) Common Market Law Review 106; Smuha and others (n 85) s 4.3.1;
EDRi and Others, ‘An EU Artificial Intelligence Act for Fundamental Rights. A Civil Society Statement’ (30 November 2021) s 5 accessed 3 December 2021.
J Cobbe and J Singh, ‘Artificial Intelligence as a Service: Legal Responsibilities, Liabilities, and Policy Challenges’ 42 (2021) Computer Law & Security Review 105573.
C Cath and F Jansen, ‘Dutch Comfort: The Limits of AI Governance through Municipal Registers’ [forthcoming] Techné: Research in Philosophy and Technology.
K Foss-Solbrekk and AK Glenster, ‘The Intersection of Data Protection Rights and Trade Secret Privileges in “Algorithmic Transparency”’ in E Kosta and R Leenes (eds), Research Handbook on EU Data Protection Law (Edward Elgar Publishing 2022) 163–183.
Article 22 Europol Regulation (Regulation (EU) 2016/794 of the European Parliament and of the Council on the European Union Agency for Law Enforcement Cooperation (OJ L 135/53)).
M Almada and N Petit, The Primrose Path to AI Regulation: Combinatorial Troubles and Path Dependence in the AI Act (Governing Artificial Intelligence: Designing Legal and Regulatory Responses, Dublin, 3 June 2022).
160C Rudin, C Wang and B Coker, ‘The Age of Secrecy and Unfairness in Recidivism Prediction’ 2 (2020) Harvard Data Science Review accessed 1 May 2022.
2D Slack and Others, ‘Counterfactual Explanations Can Be Manipulated’, in Advances in Neural Information Processing Systems 34 (2021).
Algemene Rekenkamer, Understanding Algorithms (Netherlands Court of Audit 2021) 42 accessed 3 May 2022.
J Ryan and A Toner, Europe’s Enforcement Paralysis. ICCL’s 2021 Report on the Enforcement Capabilities of Data Protection Authorities (Irish Council for Civil Liberties 2021).
B Allen, ‘Source Code Isn’t’ in TS Mullaney et al. (eds), Your Computer Is on Fire (The MIT Press 2021) 273–95.
T Simonite, ‘What Really Happened When Google Ousted Timnit Gebru’ (Wired, 8 June 2021) accessed 3 May 2022.
K Peachey, ‘Post Office Scandal: What the Horizon Saga Is All About’ (BBC News, 23 April 2021) accessed 13 June 2021; Algemene Rekenkamer (n 165).
Algemene Rekenkamer, An Audit of Algorithms accessed 18 October 2022.
176See for instance, Heikkilä (n 5); Geiger (n 5); L Amoore, ‘Why “Ditch the Algorithm” Is the Future of Political Protest’ (The Guardian, 19 August 2020) accessed 20 December 2020;
V Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (First edition, St Martin’s Press 2018);
R Xenidis, ‘Tuning EU Equality Law to Algorithmic Discrimination: Three Pathways to Resilience’ 27 (2020) Maastricht Journal of European and Comparative Law 736.
178LM Ben Dor and C Coglianese, ‘Procurement as AI Governance’ 2 (2021) IEEE Transactions on Technology and Society 192;
J Raso, ‘AI and Administrative Law’ in F Martin-Bariteanu and T Scassa (eds), Artificial Intelligence and the Law in Canada (LexisNexis 2021) 181.
eg the provisions in the German Tax Code preventing disclosure of information on the system used for fiscal risk assessment (n 143).
C Rudin and B Ustun, ‘Optimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice’ 48 (2018) Interfaces 449;
J Dressel and H Farid, ‘The Accuracy, Fairness, and Limits of Predicting Recidivism’ 4 (2018) Science Advances https://doi.org/10.1126/sciadv.aao5580.
P Ohm, ‘Throttling Machine Learning’, in M Hildebrandt and K O’Hara (eds), Life and the Law in the Era of Data-Driven Agency (Edward Elgar 2020) 214–29.
‘Meta AI Is Sharing OPT-175B, the First 175-Billion-Parameter Language Model to Be Made Available to the Broader AI Research Community’ accessed 16 May 2022.
S Corbett-Davies and Others, ‘A Computer Program Used for Bail and Sentencing Decisions Was Labeled Biased against Blacks. It’s Actually Not That Clear.’ (Washington Post, 17 October 2016) accessed 6 June 2022.
K Zenner, ‘Documents and Timelines: The Artificial Intelligence Act (Part 3)’ (Digitizing Europe, 14 May 2022) accessed 13 June 2022.
E Govers and Others, The Citizen Is Not a Dataset. Vision on the Appropriate Use of Data and Algorithms by Public Sector Authorities (Nationale Ombudsman 2021).