Fairness, integrity, and privacy in a scalable blockchain-based federated learning system

[en] Federated machine learning (FL) allows to collectively train models on sensitive data as only the clients’ models and not their training data need to be shared. However, despite the attention that research on FL has drawn, the concept still lacks broad adoption in practice. One of the key reasons is the great challenge to implement FL systems that simultaneously achieve fairness, integrity, and privacy preservation for all participating clients. To contribute to solving this issue, our paper suggests a FL system that incorporates blockchain technology, local differential privacy, and zero-knowledge proofs. Our implementation of a proof-of-concept with multiple linear regressions illustrates that these state-of-the-art technologies can be combined to a FL system that aligns economic incentives, trust, and confidentiality requirements in a scalable and transparent system.

Disciplines :

Management information systems
Computer science

Author, co-author :

Rückel, Timon

SEDLMEIR, Johannes ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX

Hofmann, Peter

External co-authors :

yes

Language :

English

Title :

Fairness, integrity, and privacy in a scalable blockchain-based federated learning system

Publication date :

15 January 2022

Journal title :

Computer Networks

ISSN :

1389-1286

eISSN :

1872-7069

Publisher :

Elsevier, Amsterdam, Netherlands

Volume :

202

Peer reviewed :

Peer Reviewed verified by ORBi

Focus Area :

Security, Reliability and Trust

Available on ORBilu :

since 13 January 2023

Statistics

Number of views

51 (3 by Unilu)

Number of downloads

35 (1 by Unilu)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

WoS citations^™

Bibliography

Iansiti, Marco, Lakhani, Karim R., Competing in the age of AI: How machine intelligence changes the rules of business. Harv. Bus. Rev., 2020 URL https://hbr.org/2020/01/competing-in-the-age-of-ai.
LeCun, Yann, Bengio, Yoshua, Hinton, Geoffrey, Deep learning. Nature 521:7553 (2015), 436–444, 10.1038/nature14539.
Galakatos, Alex, Crotty, Andrew, Kraska, Tim, Distributed machine learning. Encyclopedia of Database Systems, 2018.
Kairouz, Peter, McMahan, H. Brendan, Avent, Brendan, Bellet, Aurélien, Bennis, Mehdi, Bhagoji, Arjun Nitin, Bonawitz, Keith, Charles, Zachary, Cormode, Graham, Cummings, Rachel, D'Oliveira, Rafael G.L., Rouayheb, Salim El, Evans, David, Gardner, Josh, Garrett, Zachary, Gascón, Adrià, Ghazi, Badih, Gibbons, Phillip B., Gruteser, Marco, Harchaoui, Zaïd, He, Chaoyang, He, Lie, Huo, Zhouyuan, Hutchinson, Ben, Hsu, Justin, Jaggi, Martin, Javidi, Tara, Joshi, Gauri, Khodak, Mikhail, Konecný, Jakub, Korolova, Aleksandra, Koushanfar, Farinaz, Koyejo, Sanmi, Lepoint, Tancrède, Liu, Yang, Mittal, Prateek, Mohri, Mehryar, Nock, Richard, Özgür, Ayfer, Pagh, Rasmus, Raykova, Mariana, Qi, Hang, Ramage, Daniel, Raskar, Ramesh, Song, Dawn, Song, Weikang, Stich, Sebastian U., Sun, Ziteng, Suresh, Ananda Theertha, Tramèr, Florian, Vepakomma, Praneeth, Wang, Jianyu, Xiong, Li, Xu, Zheng, Yang, Qiang, Yu, Felix X., Yu, Han, Zhao, Sen, Advances and open problems in federated learning. 2019 URL https://arxiv.org/abs/1912.04977.
Verbraeken, Joost, Wolting, Matthijs, Katzy, Jonathan, Kloppenburg, Jeroen, Verbelen, Tim, Rellermeyer, Jan S., A survey on distributed machine learning. 2019 URL http://arxiv.org/abs/1912.09789.
Li, Tian, Sahu, Anit Kumar, Talwalkar, Ameet, Smith, Virginia, Federated learning: Challenges, methods, and future directions. IEEE Signal Process. Mag. 37:3 (2020), 50–60, 10.1109/MSP.2020.2975749.
Yin, Xuefei, Zhu, Yanming, Hu, Jiankun, A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions. ACM Comput. Surv., 54(6), 2021, 10.1145/3460427.
Larson, David B., Magnus, David C., Lungren, Matthew P., Shah, Nigam H., Langlotz, Curtis P., Ethics of using and sharing clinical imaging data for artificial intelligence: A proposed framework. Radiology 295:3 (2020), 675–682, 10.1148/radiol.2020192536.
Yang, Qiang, Liu, Yang, Chen, Tianjian, Tong, Yongxin, Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol. (TIST), 10(2), 2019, 10.1145/3298981.
Hard, Andrew, Rao, Kanishka, Mathews, Rajiv, Beaufays, Françoise, Augenstein, Sean, Eichner, Hubert, Kiddon, Chloé, Ramage, Daniel, Federated learning for mobile keyboard prediction. 2018 URL http://arxiv.org/abs/1811.03604.
Elbir, Ahmet M., Soner, Burak, Coleri, Sinem, Federated learning in vehicular networks. 2020 URL https://arxiv.org/abs/2006.01412.
Aledhari, Mohammed, Razzak, Rehma, Parizi, Reza M., Saeed, Fahad, Federated learning: A survey on enabling technologies, protocols, and applications. IEEE Access 8 (2020), 140699–140725, 10.1109/ACCESS.2020.3013541.
Kaissis, Georgios A., Makowski, Marcus R., Rückert, Daniel, Braren, Rickmer F., Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2:6 (2020), 305–311, 10.1038/s42256-020-0186-10.
Yang, Qiang, Liu, Yang, Cheng, Yong, Kang, Yan, Chen, Tianjian, Yu, Han, Federated learning. Synth. Lect. Artif. Intell. Mach. Learn., 13(3), 2019, 10.2200/S00960ED2V01Y201910AIM043.
Hitaj, Briland, Ateniese, Giuseppe, Perez-Cruz, Fernando, Deep models under the GAN. Proceedings of the SIGSAC Conference on Computer and Communications Security, 2017, ACM, 603–618, 10.1145/3133956.3134012.
Melis, Luca, Song, Congzheng, Cristofaro, Emiliano De, Shmatikov, Vitaly, Exploiting unintended feature leakage in collaborative learning. Symposium on Security and Privacy, 2019, IEEE URL https://doi.org/10.1109/SP.2019.00029.
Nasr, Milad, Shokri, Reza, Houmansadr, Amir, Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. Symposium on Security and Privacy, 2019, IEEE, 739–753, 10.1109/SP.2019.00065.
Phong, Le Trieu, Aono, Yoshinori, Hayashi, Takuya, Wang, Lihua, Moriai, Shiho, Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13:5 (2018), 1333–1345, 10.1109/TIFS.2017.2787987.
Zhu, Ligeng, Han, Song, Deep leakage from gradients. Federated Learning, 2020, Springer, 17–31, 10.1007/978-3-030-63076-8_2.
Faltings, Boi, Radanovic, Goran, Game theory for data science: Eliciting truthful information. Synth. Lect. Artif. Intell. Mach. Learn., 11(2), 2017 URL https://doi.org/10.2200/S00788ED1V01Y201707AIM035.
Li, Li, Fan, Yuxi, Tse, Mike, Lin, Kuo-Yi, A review of applications in federated learning. Comput. Ind. Eng., 149, 2020, 10.1016/j.cie.2020.106854.
Kurtulmus, A. Besir, Daniel, Kenny, Trustless machine learning contracts; evaluating and exchanging machine learning models on the ethereum blockchain. 2018 URL https://arxiv.org/abs/1802.10185.
Mugunthan, Vaikkunth, Rahman, Ravi, Kagal, Lalana, BlockFLow: An accountable and privacy-preserving solution for federated learning. 2020 URL https://arxiv.org/abs/2007.03856.
Bokolo, Anthony Junior, Distributed ledger and decentralised technology adoption for smart digital transition in collaborative enterprise. Enterp. Inf. Syst., 2021, 10.1080/17517575.2021.1989494.
Nilsson, Adrian, Smith, Simon, Ulm, Gregor, Gustavsson, Emil, Jirstrand, Mats, A performance evaluation of federated learning algorithms. Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning, 2018, ACM, 10.1145/3286490.3286559.
Dwork, Cynthia, McSherry, Frank, Nissim, Kobbi, Smith, Adam, Calibrating noise to sensitivity in private data analysis. Theory of Cryptography, 2006, Springer, 265–284, 10.1007/11681878_14.
Dwork, Cynthia, Roth, Aaron, The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9:3–4 (2014), 211–407, 10.1561/0400000042.
Dwork, Cynthia, Smith, Adam, Differential privacy for statistics: What we know and what we want to learn. J. Priv. Confid., 1(2), 2010, 10.29012/jpc.v1i2.570.
Garrido, Gonzalo Munilla, Near, Joseph, Muhammad, Aitsam, He, Warren, Matzutt, Roman, Matthes, Florian, Do I get the privacy I need? Benchmarking utility in differential privacy libraries. 2021 URL https://arxiv.org/abs/2109.10789.
Dwork, Cynthia, Nissim, Kobbi, Privacy-preserving datamining on vertically partitioned databases. Advances in Cryptology, 2004, Springer, 528–544, 10.1007/978-3-540-28628-8_32.
Nguyên, Thông T, Xiao, Xiaokui, Yang, Yin, Hui, Siu Cheung, Shin, Hyejin, Shin, Junbum, Collecting and analyzing data from smart device users with local differential privacy. 2016 URL https://arxiv.org/abs/1606.05053.
Goldwasser, Shafi, Micali, Silvio, Rackoff, Charles, The knowledge complexity of interactive proof systems. SIAM J. Comput. 18:1 (1989), 186–208.
Fiat, Amos, Shamir, Adi, How to prove yourself: Practical solutions to identification and signature problems. Conference on the Theory and Application of Cryptographic Techniques, 1986, Springer, 186–194, 10.1007/3-540-47721-7_12.
Bitansky, Nir, Chiesa, Alessandro, Ishai, Yuval, Paneth, Omer, Ostrovsky, Rafail, Succinct non-interactive arguments via linear interactive proofs. Theory of Cryptography Conference, 2013, Springer, 315–333, 10.1007/978-3-642-36594-2_18.
Groth, Jens, Ostrovsky, Rafail, Sahai, Amit, Perfect non-interactive zero knowledge for NP. Annual International Conference on the Theory and Applications of Cryptographic Techniques, 2006, Springer, 339–358, 10.1007/11761679_21.
Ben-Sasson, Eli, Chiesa, Alessandro, Genkin, Daniel, Tromer, Eran, Virza, Madars, SNARKs for C: Verifying program executions succinctly and in zero knowledge. Annual Cryptology Conference, 2013, Springer, 90–108, 10.1007/978-3-642-40084-1_6.
Bünz, Benedikt, Bootle, Jonathan, Boneh, Dan, Poelstra, Andrew, Wuille, Pieter, Maxwell, Greg, Bulletproofs: Short proofs for confidential transactions and more. Symposium on Security and Privacy, 2018, IEEE, 315–334, 10.1109/SP.2018.00020.
Gennaro, Rosario, Gentry, Craig, Parno, Bryan, Raykova, Mariana, Quadratic span programs and succinct NIZKs without PCPs. Annual International Conference on the Theory and Applications of Cryptographic Techniques, 2013, Springer, 626–645, 10.1007/978-3-642-38348-9_37.
Ben-Sasson, Eli, Bentov, Iddo, Horesh, Yinon, Riabzev, Michael, Scalable zero knowledge with no trusted setup. Annual International Cryptology Conference, 2019, Springer, 701–732, 10.1007/978-3-030-26954-8_23.
Butijn, Bert-Jan, Tamburri, Damian A, Heuvel, Willem-Jan van den, Blockchains: A systematic multivocal literature review. ACM Comput. Surv., 53(3), 2020, 10.1145/3369052.
Xiao, Y., Zhang, N., Lou, W., Hou, Y.T., A survey of distributed consensus protocols for blockchain networks. IEEE Commun. Surv. Tutor. 22:2 (2020), 1432–1465, 10.1109/COMST.2020.2969706.
Nofer, Michael, Gomber, Peter, Hinz, Oliver, Schiereck, Dirk, Blockchain. Bus. Inf. Syst. Eng. 59:3 (2017), 183–187, 10.1007/s12599-017-0467-3.
Alt, Rainer, Electronic markets on blockchain markets. Electron. Mark. 30:2 (2020), 181–188, 10.1007/s12525-020-00428-1.
Fridgen, Gilbert, Radszuwill, Sven, Urbach, Nils, Utz, Lena, Cross-organizational workflow management using blockchain technology – Towards applicability, auditability, and automation. 51st Hawaii International Conference on System Sciences, 2018, 3507–3517, 10.24251/HICSS.2018.444.
Wüst, Karl, Gervais, Arthur, Do you need a blockchain?. Crypto Valley Conference on Blockchain Technology, 2018, IEEE, 45–54, 10.1109/CVCBT.2018.00011.
Nakamoto, Satoshi, A peer-to-peer electronic cash system. 2008 URL https://bitcoin.org/bitcoin.pdf.
Wohrer, Maximilian, Zdun, Uwe, Smart contracts: Security patterns in the ethereum ecosystem and solidity. International Workshop on Blockchain Oriented Software Engineering, 2018, IEEE, 10.1109/IWBOSE.2018.8327565.
Sedlmeir, Johannes, Buhl, Hans Ulrich, Fridgen, Gilbert, Keller, Robert, The energy consumption of blockchain technology: Beyond myth. Bus. Inf. Syst. Eng. 62:6 (2020), 599–608, 10.1007/s12599-020-00656-x.
Gudgeon, Lewis, Moreno-Sanchez, Pedro, Roos, Stefanie, McCorry, Patrick, Gervais, Arthur, SoK: Layer-two blockchain protocols. International Conference on Financial Cryptography and Data Security, 2020, Springer, 201–226, 10.1007/978-3-030-51280-4_12.
Zhang, Rui, Xue, Rui, Liu, Ling, Security and privacy on blockchain. ACM Comput. Surv., 52(3), 2019 URL https://doi.org/10.1145/3316481.
Garrido, Gonzalo Munilla, Sedlmeir, Johannes, Uludağ, Ömer, Alaoui, Ilias Soto, Luckow, Andre, Matthes, Florian, Revealing the landscape of privacy-enhancing technologies in the context of data markets for the IoT: A systematic literature review. 2021 URL https://arxiv.org/abs/2107.11905.
McMahan, Brendan, Moore, Eider, Ramage, Daniel, Hampson, Seth, y Arcas, Blaise Aguera, Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Vol. 54, 2017, PMLR, 1273–1282.
Lu, Y., Huang, X., Dai, Y., Maharjan, S., Zhang, Y., Differentially private asynchronous federated learning for mobile edge computing in urban informatics. IEEE Trans. Ind. Inf. 16:3 (2020), 2134–2143, 10.1109/TII.2019.2942179.
Domingo-Ferrer, Josep, Sánchez, David, Blanco-Justicia, Alberto, The limits of differential privacy (and its misuse in data release and machine learning). Commun. ACM 64:7 (2021), 33–35, 10.1145/3433638.
Khan, Latif U., Pandey, Shashi Raj, Tran, Nguyen H., Saad, Walid, Han, Zhu, Nguyen, Minh N.H., Hong, Choong Seon, Federated learning for edge networks: Resource optimization and incentive mechanism. IEEE Commun. Mag. 58:10 (2020), 88–93, 10.1109/MCOM.001.1900649.
Jaiman, Vikas, Pernice, Leonard, Urovi, Visara, User incentives for blockchain-based data sharing platforms. 2021 URL https://arxiv.org/abs/2110.11348.
Ramanan, P., Nakayama, K., BAFFLE: Blockchain based aggregator free federated learning. International Conference on Blockchain, 2020, IEEE, 72–81, 10.1109/Blockchain50366.2020.00017.
Toyoda, K., Zhang, A.N., Mechanism design for an incentive-aware blockchain-enabled federated learning platform. International Conference on Big Data, 2019, 395–403, 10.1109/BigData47090.2019.9006344.
Kang, J., Xiong, Z., Niyato, D., Xie, S., Zhang, J., Incentive mechanism for reliable federated learning: A joint optimization approach to combining reputation and contract theory. IEEE Internet Things J. 6:6 (2019), 10700–10714, 10.1109/JIOT.2019.2940820.
Sun, Jin, Wu, Ying, Wang, Shangping, Fu, Yixue, Chang, Xiao, A permissioned blockchain frame for secure federated learning. IEEE Commun. Lett., 2021, 10.1109/LCOMM.2021.3121297.
Luu, Loi, Teutsch, Jason, Kulkarni, Raghav, Saxena, Prateek, Demystifying incentives in the consensus computer. Proceedings of the 22nd SIGSAC Conference on Computer and Communications Security, 2015, ACM, 706–719, 10.1145/2810103.2813659.
Wu, Howard, Zheng, Wenting, Chiesa, Alessandro, Popa, Raluca Ada, Stoica, Ion, DIZK: A distributed zero knowledge proof system. 27th {USENIX} Security Symposium, 2018, USENIX, 675–692 URL https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-wu.pdf.
Feng, Boyuan, Qin, Lianke, Zhang, Zhenfei, Ding, Yufei, Chu, Shumo, ZEN: Efficient zero-knowledge proofs for neural networks. 2021 URL https://eprint.iacr.org/2021/087.
Zhang, Yupeng, Zero-knowledge proofs for machine learning. Proceedings of the Workshop on Privacy-Preserving Machine Learning in Practice, 2020, ACM, 10.1145/3411501.3418608.
Weng, Chenkai, Yang, Kang, Xie, Xiang, Katz, Jonathan, Wang, Xiao, Mystique: Efficient conversions for zero-knowledge proofs with applications to machine learning. 30th {USENIX} Security Symposium ({USENIX} Security 21), 2021, 501–518 URL https://www.usenix.org/system/files/sec21-weng.pdf.
Witten, Ian H., Frank, Eibe, Hall, Mark A., Pal, Christopher J., Data Mining. 2017, Elsevier URL https://doi.org/10.1016/c2015-0-02071-8.
Newman, M., How to determine accuracy of the output of a matrix inversion program. J. Res. Natl. Bur. Stand. B 78B:2 (1974), 65–68.
Mironov, Ilya, On significance of the least significant bits for differential privacy. Proceedings of the Conference on Computer and Communications Security, 2012, ACM, 650–661, 10.1145/2382196.2382264.
Balcer, Victor, Vadhan, Salil, Differential privacy on finite computers. 2019 URL https://arxiv.org/abs/1709.05396.
Canonne, Clément L., Kamath, Gautam, Steinke, Thomas, The discrete Gaussian for differential privacy. 2021 URL https://arxiv.org/abs/2004.00010.
How to Generate SNARK Parameters Securely. 2021, Wilcox, Zooko URL https://electriccoin.co/blog/snark-parameters/.
Grassi, Lorenzo, Khovratovich, Dmitry, Rechberger, Christian, Roy, Arnab, Schofnegger, Markus, Poseidon: A new hash function for zero-knowledge proof systems. 30th {USENIX} Security Symposium, 2021, USENIX URL https://www.usenix.org/system/files/sec21summer_grassi.pdf.
Albrecht, Martin, Grassi, Lorenzo, Rechberger, Christian, Roy, Arnab, Tiessen, Tyge, MiMC: Efficient encryption and cryptographic hashing with minimal multiplicative complexity. International Conference on the Theory and Application of Cryptology and Information Security, 2016, Springer, 191–219, 10.1007/978-3-662-53887-6_7.
Sedlmeir, Johannes, Ross, Philipp, Luckow, André, Lockl, Jannik, Miehle, Daniel, Fridgen, Gilbert, The DLPS: A framework for benchmarking blockchains. Proceedings of the 54th Hawaii International Conference on System Sciences, 2021, 6855–6864, 10.24251/HICSS.2021.822.
Network, Hermez, Open sourcing an ultra-fast zk prover: Rapidsnark. 2021 URL https://blog.hermez.io/open-sourcing-ultra-fast-zk-prover-rapidsnark/.
Bubeck, Sébastien, Convex optimization: Algorithms and complexity. 2014 URL https://arxiv.org/abs/1405.4980.
Gailly, Nicolas, Maller, Mary, Nitulescu, Anca, SnarkPack: Practical SNARK aggregation. 2021 URL https://eprint.iacr.org/2021/529.pdf.
Bowe, Sean, Chiesa, Alessandro, Green, Matthew, Miers, Ian, Mishra, Pratyush, Wu, Howard, ZEXE: Enabling decentralized private computation. Symposium on Security and Privacy, 2020, IEEE, 947–964, 10.1109/SP40000.2020.00050.
Chiesa, Alessandro, Ojha, Dev, Spooner, Nicholas, Fractal: Post-quantum and transparent recursive proofs from holography. Annual International Conference on the Theory and Applications of Cryptographic Techniques, 2020, Springer, 769–793, 10.1007/978-3-030-45721-1_27.
Gluchowski, Alex, World's first practical hardware for zero-knowledge proofs acceleration. 2020 URL https://medium.com/matter-labs/worlds-first-practical-hardware-for-zero-knowledge-proofs-acceleration-72bf974f8d6e.
Djamali, Alexander, Dossow, Patrick, Hinterstocker, Michael, Schellinger, Benjamin, Sedlmeir, Johannes, Völter, Fabiane, Willburger, Lukas, Asset logging in the energy sector: A scalable blockchain-based data platform. Energy Inform., 4(3), 2021, 10.1186/s42162-021-00183-3.
Shapley, Lloyd Stowell, A value for n-person games. Contributions to the Theory of Games (AM-28), Volume II, 1953, Princeton University Press, 307–318 chapter 17.
Guggenberger, Tobias, Lockl, Jannik, Röglinger, Maximilian, Schlatt, Vincent, Sedlmeir, Johannes, Stoetzer, Jens-Christian, Urbach, Nils, Völter, Fabiane, Emerging digital technologies to combat future crises: Learnings from COVID-19 to be prepared for the future. Int. J. Innov. Technol. Manage., 2021, 10.1142/S0219877021400022.
Singh, Sushil Kumar, Rathore, Shailendra, Park, Jong Hyuk, Blockiotintelligence: A blockchain-enabled intelligent IoT architecture with artificial intelligence. Future Gener. Comput. Syst. 110 (2020), 721–743.