evaluation; evaluation approaches; quality; quality characteristics; information model; data model
Abstract :
[en] Ensuring semantic consistency and data integrity of information exchanges between these interconnected systems requires robust information and data models. As new models are emerging, it is essential to evaluate them during the design process. However, model developers often lack clear methods or guidelines for evaluating their new information and data models. We conduct a narrative literature review of academic publications to understand the current state. We generate a simple visual illustration mapping our focus in the context of information and data model evaluation with existing approaches to assist model developers in identifying suitable methods. Our findings highlight two main approaches for information models, while also identifying a gap in evaluation approaches for data models. Future work could focus on designing a combined approach for information and data models to provide a structured guidance for model developers.
Research center :
Interdisciplinary Centre for Security, Reliability and Trust (SnT) > FINATRAX - Digital Financial Services and Cross-organizational Digital Transformations
Disciplines :
Computer science Management information systems
Author, co-author :
VAN STIPHOUDT, Christine ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
POTENCIANO MENCI, Sergio ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
FRIDGEN, Gilbert ; University of Luxembourg > Interdisciplinary Centre for Security, Reliability and Trust (SNT) > FINATRAX
External co-authors :
no
Language :
English
Title :
Ex-Ante Evaluation Approaches Within the Design Process of Information and Data Models
Publication date :
30 June 2025
Event name :
15th International Symposium on Business Modeling and Software Design
Event place :
Milan, Italy
Event date :
01.07.2025 - 03-07.2025
Audience :
International
Main work title :
Business Modeling and Software Design
Editor :
Shishkov, Boris
Publisher :
Springer Nature AG, Cham, Switzerland
Pages :
220-229
Peer reviewed :
Peer reviewed
Focus Area :
Security, Reliability and Trust
Development Goals :
9. Industry, innovation and infrastructure
FnR Project :
FNR13342933 - DFS - Paypal-fnr Pearl Chair In Digital Financial Services, 2019 (01/01/2020-31/12/2024) - Gilbert Fridgen
Name of the research project :
R-AGR-3740 - SynErgie II - FRIDGEN Gilbert
Funders :
FNR - Luxembourg National Research Fund Federal Ministry of Education and Research Bonn Office
Funding number :
13342933
Funding text :
This work has been supported by the Kopernikus-project “SynErgie” by the German Federal Ministry of Education and Research (BMBF) and by the Luxembourg National Research Fund (FNR) and PayPal, PEARL grant reference 3342933/Gilbert Fridgen. The authors gratefully would like to acknowledge the project supervision by the project management organization Projektträger Jülich (PtJ).
Villar, J., Bessa, R., Matos, M.: Flexibility products and markets: literature review. Electr. Power Syst. Res. 154, 329–340 (2018). https://doi.org/10.1016/j.epsr.2017. 09.005
Kadry, S.: On the evolution of information systems. In: Systems Theory: Perspectives, Applications and Developments, pp. 197–208 (2014)
EEBUS, EEBus-Empowering the digitalisation of Energy transition (2025). https://www.eebus.org/. Accessed 24 Apr 2025
Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S.: A design science research methodology for information systems research. J. Manag. Inf. Syst. 24(3), 45–77 (2007)
Williams, M., and Williams, J.: A framework facilitating ex-ante evaluation of information systems. In: AMCIS 2004 Proceedings (2004)
Moody, D.L.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data Knowl. Eng. 55(3), 243–276 (2005). https://doi.org/10.1016/j.datak.2004.12.005
Pras, A., Schoenwaelder, J.: On the difference between information models and data models. Technical report RFC3444. RFC Editor (2003). https://doi.org/10. 17487/rfc3444
Olivé, A.: Conceptual Modeling of Information Systems. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-39390-0
Krogstie, J.: Model-Based Development and Evolution of Information Systems: A Quality Approach. Springer, New York (2012)
Stuckenholz, A.: Basiswissen Energieinformatik: Ein Lehr-und Arbeitsbuch für Studierende und Anwender [Basic knowledge of energy informatics: A textbook and workbook for students and users]. Springer Fachmedien Wiesbaden, Wiesbaden (2020). https://doi.org/10.1007/978-3-658-31809-3
Lee, Y.-T.T.: Information modeling: from design to implementation. IEEE Trans. Robot. Autom. (1999)
Rahimifard, S., Newman, S.: A methodology to develop EXPRESS data models, vol. 9 (1996). https://doi.org/10.1080/095119296131814
Green, B.N., Johnson, C.D., Adams, A.: Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J. Chiropr. Med. 5(3), 101–117 (2006). https://doi.org/10.1016/S0899-3467(07)60142-6
Paré, G., Trudel, M.-C., Jaana, M., Kitsiou, S.: Synthesizing information systems knowledge: a typology of literature reviews. Inf. Manage. 52(2), 183–199 (2015). https://doi.org/10.1016/j.im.2014.08.008
Snyder, H.: Literature review as a research methodology: an overview and guidelines. J. Bus. Res. 104, 333–339 (2019). https://doi.org/10.1016/j.jbusres.2019.07. 039
Callahan, J.L.: Writing literature reviews: a reprise and update. Hum. Resour. Dev. Rev. 13(3), 271–275 (2014). https://doi.org/10.1177/1534484314536705
Moody, D.L., Shanks, G.G.: Improving the quality of data models: empirical validation of a quality management framework. Inf. Syst. 28(6), 619–650 (2003). https://doi.org/10.1016/S0306-4379(02)00043-1
Levitin, A., Redman, T.: Quality dimensions of a conceptual view. Inf. Process. Manage. 31(1), 81–88 (1995). https://doi.org/10.1016/0306-4573(95)80008-H
Nelson, H.J., Poels, G., Genero, M., Piattini, M.: A conceptual modeling quality framework. Softw. Qual. J. 20(1), 201–228 (2012). https://doi.org/10.1007/s11219-011-9136-9
Krogstie, J.: Quality of conceptual data models. In: International Conference on Informatics and Semiotics in Organisations (2013)
Wand, Y., Weber, R.: An ontological model of an information system. IEEE Trans. Softw. Eng. 16(11), 1282–1292 (1990). https://doi.org/10.1109/32.60316
Moody, D.L.: Measuring the quality of data models: an empirical evaluation of the use of quality metrics in practice. In: ECIS 2003 Proceedings (2003)
Shanks, G., Darke, P.: Quality in conceptual modelling: linking theory and practice. In: Pacific Asia Conference on Information Systems (PACIS) (1997)
Moody, D.L., Shanks, G.G.: What makes a good data model? Evaluating the quality of entity relationship models. In: Loucopoulos, P. (ed.) ER 1994. LNCS, vol. 881, pp. 94–111. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-58786-1 75
Krogstie, J., Lindland, O.I., Sindre, G.: Towards a deeper understanding of quality in requirements engineering. In: Iivari, J., Lyytinen, K., Rossi, M. (eds.) CAiSE 1995. LNCS, vol. 932, pp. 82–95. Springer, Heidelberg (1995). https://doi.org/10. 1007/3-540-59498-1 239
Taentzer, G., Kesper, A., Matoni, M.: How to define the quality of data and software models? A data quality perspective (2024). https://doi.org/10.18420/MODELLIERUNG2024-WS-010
Helskyaho, H., Ruotsalainen, L., Männistö, T.: Defining data model quality metrics for data vault 2.0 model evaluation. Inventions 9(1), 21 (2024). https://doi.org/10.3390/inventions9010021
