[en] Distributed Ledger Technologies (DLT) promise to revolutionize business ecosystems by permitting secure transactions without intermediaries. A widely recognized challenge that inhibits the uptake of DLT is scalability and performance. Hence, quantifying key metrics such as throughput and latency is crucial for designing DLT-based infrastructures, applications, and ecosystems. However, current benchmarking frameworks for blockchains do not cover the whole benchmarking process; impeding transparent comparisons of different DLT networks. In this paper, we present the Distributed Ledger Performance Scan (DLPS), an open-source framework for end-to-end performance characterizations of blockchains, addressing the need to transparently and automatically evaluate the performance of highly customizable configurations. We describe our new framework and argue that it significantly improves existing DLT benchmarking solutions. To demonstrate the capabilities of the DLPS, we also summarize the main results obtained from a series of experiments that we have conducted with it, giving a first comprehensive comparison of essential scalability properties of several commonly used enterprise blockchains.
Disciplines :
Sciences informatiques
Auteur, co-auteur :
Sedlmeir, Johannes
Ross, Philipp
Luckow, André
Lockl, Jannik
Miehle, Daniel
FRIDGEN, Gilbert ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
Co-auteurs externes :
yes
Langue du document :
Anglais
Titre :
The DLPS: A New Framework for Benchmarking Blockchains
Date de publication/diffusion :
2021
Nom de la manifestation :
54th Hawaii International Conference on System Sciences
Date de la manifestation :
from 04-01-2021 to 08-01-2021
Manifestation à portée :
International
Titre de l'ouvrage principal :
Proceedings of the 54th Hawaii International Conference on System Sciences
M. Iansiti and K. R. Lakhani, “The truth about blockchain,” Harvard Business Review, vol. 95, no. 1, pp. 118-127, 2017.
S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” 2008. http://bitcoin.org/bitcoin. pdf.
V. Buterin et al., “A next-generation smart contract and decentralized application platform,” 2014. https://github.com/ethereum/wiki/wiki/White-Paper.
N. Szabo, “Formalizing and securing relationships on public networks,” First Monday, vol. 2, no. 9, 1997.
T. Jensen, J. Hedman, and S. Henningsson, “How TradeLens delivers business value with blockchain technology,” MIS Quarterly Executive, vol. 18, no. 4, 2019.
A. Rieger, F. Guggenmos, J. Lockl, G. Fridgen, and N. Urbach, “Building a blockchain application that complies with the EU general data protection regulation,” MIS Quarterly Executive, vol. 18, no. 4, 2019.
B.-J. Butijn, D. A. Tamburri, and W.-J. V. D. Heuvel, “Blockchains: A systematic multivocal literature review,” ACM Computing Surveys, vol. 53, no. 3, 2020.
J. Poon and T. Dryja, “The bitcoin lightning network: Scalable off-chain instant payments,” 2016. https://www.bitcoinlightning.com/wp-content/uploads/2018/03/ lightning-network-paper.pdf.
M. Vukolic, “The quest for scalable blockchain fabric: Proof-of-work vs. BFT replication,” in International workshop on open problems in network security, pp. 112-125, Springer, 2015.
A. Gervais, G. O. Karame, K. Wüst, V. Glykantzis, H. Ritzdorf, and S. Capkun, “On the security and performance of proof of work blockchains,” in Proceedings of the 2016 SIGSAC conference on computer and communications security, pp. 3-16, ACM, 2016.
J. Sedlmeir, H. U. Buhl, G. Fridgen, and R. Keller, “The energy consumption of blockchain technology: Beyond myth,” Business & Information Systems Engineering, vol. 62, no. 6, pp. 599-608, 2020.
D. Ongaro and J. Ousterhout, “In search of an understandable consensus algorithm,” in {USENIX} Annual Technical Conference, pp. 305-319, 2014.
L. Lamport, R. Shostak, and M. Pease, “The byzantine generals problem,” Transactions on Programming Languages and Systems, vol. 4, no. 3, pp. 382-401, 1982.
M. Castro, B. Liskov, et al., “Practical byzantine fault tolerance,” in OSDI, vol. 99, pp. 173-186, 1999.
R. Saltini, “Correctness analysis of IBFT,” 2019. https://arxiv.org/pdf/1901.07160.
P.-L. Aublin, S. B. Mokhtar, and V. Quéma, “Rbft: Redundant byzantine fault tolerance,” in 33rd International Conference on Distributed Computing Systems, pp. 297-306, IEEE, 2013.
S. De Angelis, L. Aniello, F. Lombardi, A. Margheri, and V. Sassone, “PBFT vs Proof-of-authority: applying the CAP theorem to permissioned blockchain,” 2017.
Z. Shi, H. Zhou, Y. Hu, S. Jayachander, C. de Laat, and Z. Zhao, “Operating permissioned blockchain in clouds: A performance study of hyperledger sawtooth,” in 18th International Symposium on Parallel and Distributed Computing, pp. 50-57, IEEE, 2019.
K. Christidis and M. Devetsikiotis, “Blockchains and smart contracts for the internet of things,” IEEE Access, vol. 4, 2016.
E. Androulaki et al., “Hyperledger Fabric: a distributed operating system for permissioned blockchains,” in Proceedings of the Thirteenth EuroSys Conference, p. 30, ACM, 2018.
A. Baliga, I. Subhod, P. Kamat, and S. Chatterjee, “Performance evaluation of the quorum blockchain platform,” 2018. http://arxiv.org/abs/1809. 03421.
T. T. A. Dinh, J. Wang, G. Chen, R. Liu, B. C. Ooi, and K. Tan, “Blockbench: A framework for analyzing private blockchains,” 2017. http://arxiv.org/abs/1703.04057.
C. Fan, S. Ghaemi, H. Khazaei, and P. Musilek, “Performance evaluation of blockchain systems: A systematic survey,” IEEE Access, vol. 8, pp. 126927-126950, 2020.
R. Jain, The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling. Wiley professional computing, Wiley, 1991.
J. Ousterhout, “Always measure one level deeper,” Communications of the ACM, vol. 61, no. 7, pp. 74-83, 2018.
H. Wickham et al., “Tidy data,” Journal of Statistical Software, vol. 59, no. 10, pp. 1-23, 2014.
J. Gray, Benchmark handbook: for database and transaction processing systems. Morgan Kaufmann Publishers Inc., 1992.
