References of "Sonnleitner, Philipp 50003122"
     in
Bookmark and Share    
Peer Reviewed
See detailGeschlechtsunterschiede in Lesen und Mathematik: Ergebnisse aus 69 Ländern.
Brunner, Martin; Keller, Ulrich UL; Sonnleitner, Philipp UL et al

Scientific Conference (2011)

Detailed reference viewed: 78 (0 UL)
Peer Reviewed
See detailThe Genetics Lab: An Innovative Tool for Assessment of Intelligence by Mean of Complex Problem Solving
Hazotte, Cyril; Mayer, Hélène; Djaghloul, Younes et al

Scientific Conference (2011)

Detailed reference viewed: 87 (0 UL)
Peer Reviewed
See detailThe Genetics Lab - A new Computer-Based Problem Solving Scenario to assess Intelligence
Sonnleitner, Philipp UL; Brunner, Martin; Keller, Ulrich UL et al

Scientific Conference (2011)

Assessments of intelligence by means of paper-pencil tests faced several critiques that point to their lack of face validity, insufficient coverage of the definition of intelligence, their sensitivity to ... [more ▼]

Assessments of intelligence by means of paper-pencil tests faced several critiques that point to their lack of face validity, insufficient coverage of the definition of intelligence, their sensitivity to the emotional state of the test taker, and the danger of getting outdated. The present paper discusses to what extent these limitations can be overcome by computer-based problem solving scenarios–so-called microworlds. Generally speaking, microworlds are supposed to be highly accepted by test takers, to provide process measures by directly tracing problem solving behavior, and to realize game-like characteristics that may increase test motivation and reduce test anxiety. To capitalize on these potential advantages, we developed the microworld Genetics Lab that was completed by a large, heterogeneous sample of more than 600 Luxembourgish students. Performance scores were derived for students’ problem solving strategies as well as their mental problem representations–important cognitive data which are not accessible with typical paper-pencil tests. Analyses of the psychometric characteristics of the Genetics Lab empirically underscored the construct validity for the derived performance scores. For example, process oriented measures of strategy use were found to possess discriminant validity with respect to grades. Further, acceptance and induced test anxiety of the Genetics lab was explored relative to a paper-pencil measure of intelligence. Our results show that the Genetics Lab is a reliable and valid assessment instrument and emphasize the benefits of using microworlds for assessing intelligence. [less ▲]

Detailed reference viewed: 103 (1 UL)
Peer Reviewed
See detailAssessing Intelligence for Education in the 21st Century: The Benefits of Microworlds
Sonnleitner, Philipp UL; Brunner, Martin; Keller, Ulrich UL et al

Scientific Conference (2011)

Computer-based problem solving scenarios—so-called microworlds—are contemporary educational assessment instruments of intelligence that offer several benefits compared to traditional paper-pencil tests ... [more ▼]

Computer-based problem solving scenarios—so-called microworlds—are contemporary educational assessment instruments of intelligence that offer several benefits compared to traditional paper-pencil tests. This involves tracking of students’ mental representations of the problems as well as their problem solving strategies by means of behavioral data which provides key information for educational interventions. Moreover, microworlds realize game-like characteristics that may increase test motivation and reduce test anxiety. In the present study, the Genetics Lab, a newly developed microworld, was completed by a representative sample of more than 800 Luxembourgish students. Students chose among three different languages (German, French and English) in which the problem content of the Genetics Lab was presented. The present paper analyzes the psychometric properties of the various performance scores derived for the Genetics Lab with respect to their relations to school grades, and measurement invariance across gender, chosen test language, and migration background. Moreover, a direct comparison with traditional measures of intelligence demonstrated construct validity of the performance scores of the Genetics Lab. In sum, the results obtained for the Genetics Lab show the benefits of behavioral data obtained for computer-based problem-solving scenarios and support the notion of microworlds to be a valuable measure of intelligence. [less ▲]

Detailed reference viewed: 151 (0 UL)
Full Text
Peer Reviewed
See detailDesigning the test booklets for Rasch model calibration in a large-scale assessment with reference to numerous moderator variables and several ability dimensions
Kubinger, Klaus; Hohensinn, Christine; Hofer, Sandra et al

in Educational Research & Evaluation (2011), 17(6), 483-495

In large-scale assessments, it usually does not occur that every item of the applicable item pool is administered to every examinee. Within item response theory (IRT), in particular the Rasch model (1960 ... [more ▼]

In large-scale assessments, it usually does not occur that every item of the applicable item pool is administered to every examinee. Within item response theory (IRT), in particular the Rasch model (1960), this is not really a problem because item calibration works nevertheless. The different test booklets only need to be conceptualized according to a connected incomplete block design. Yet, connectedness of such a design should best be fulfilled severalfold, since deletion of some items in the course of the item pool’s IRT calibration may become necessary. The real challenge, however, is to meet constraints determined by numerous moderator variables such as different response formats and several topics of content – all the more so, if several ability dimensions are under consideration, the testing duration is strongly limited or individual scoring and feedback is an issue. In this article, we offer a report of how to deal with the resulting problems. Experience is based on the governmental project of the Austrian Educational Standards (Kubinger et al., 2007). [less ▲]

Detailed reference viewed: 69 (1 UL)
See detailChallenges of modern Computer Based Assessment: Usability, Scoring, and “Digital Natives”
Sonnleitner, Philipp UL; Brunner, Martin; Keller, Ulrich UL et al

Scientific Conference (2011)

In recent years, computer based assessment has undergone substantive change. Test developers as well as test users have become aware of the fact that computers can do more than administrate traditional ... [more ▼]

In recent years, computer based assessment has undergone substantive change. Test developers as well as test users have become aware of the fact that computers can do more than administrate traditional (paper-pencil) item formats like multiple-choice. More complex computer based item types allow tracking test takers’ mental representations of the problem or even their problem solving strategies by means of behavioral data. An example for such a modern item type are microworlds – dynamically changing problem solving scenarios with which the test taker has to interact. However, with the advent of complex item types, new challenges arise. First, usability is at stake - test takers do not intuitively know what to do or how to interact with complex tasks. Second, a massive load of data is produced and it gets difficult for the test developer to decide on relevant scores. Third, today’s students - so-called “digital natives”- grew up with computers and therefore set high quality standards for software applications. They may quickly loose trust and interest in tests with old fashioned design, cumbersome handling or even malfunctioning software. On basis of the Genetics Lab – a microworld developed to assess general mental ability – these challenges of modern computer based assessment are discussed. Three consecutive small scale studies were carried out to investigate usability issues, validate scoring algorithms and to ensure acceptance among students. The results demonstrate the importance of considering usability during the test development process, particularly with regard to scoring. The modification of conventional test development procedures for modern computer based assessment is suggested. Moreover, possibilities to satisfy even a critical target population are presented. [less ▲]

Detailed reference viewed: 52 (3 UL)
Peer Reviewed
See detailWandel im Bildungswesen, zunehmende Komplexität und Web 2.0: Sind traditionelle Intelligenztests noch "up to date"?
Sonnleitner, Philipp UL; Brunner, Martin; Keller, Ulrich UL et al

Poster (2010)

Grundlegender Wandel im schulischen Bereich führt zur berechtigten Frage, inwiefern nicht Innovationen in der Intelligenzdiagnostik nötig wären, um die hohe prädiktive Validität und dadurch wesentliche ... [more ▼]

Grundlegender Wandel im schulischen Bereich führt zur berechtigten Frage, inwiefern nicht Innovationen in der Intelligenzdiagnostik nötig wären, um die hohe prädiktive Validität und dadurch wesentliche Bedeutung bei Schulplatzierungsentscheidungen weiterhin zu sichern. Die stärkere Betonung fächerübergreifender Kompetenzen, eine vermehrte Einbindung des Computers in Unterricht und Alltag sowie zunehmende Komplexität in der Arbeitswelt führen zu neuen Herausforderungen und beleben alte Kritik wieder. In diesem Kontext wurden immer wieder computerbasierte Problemlöseszenarien als vielversprechender Ansatz genannt, Schwächen traditioneller Intelligenztests zu überwinden. Vorteile werden, neben dem dynamischen Testformat und einer vollständigeren Abdeckung des Intelligenzbegriffes, unter anderem auch in erhöhter face-validity gesehen. Während bisherige Studien den Schwerpunkt auf das empirische Verhältnis zwischen Leistungsmaßen der Szenarien und traditionellen Intelligenzmaßen setzten, versucht dieser Beitrag, bewusst die Diskussion um theoretische Aspekte zu ergänzen. Einer Analyse der Anforderungen die an eine aktuelle Intelligenzdiagnostik zu stellen sind, folgt ein systematischer Vergleich traditioneller Intelligenztestformate mit computerbasierten Problemlöseszenarien. Konkrete Beispiele sowie aktuelle Daten zur Pilotierung eines derartigen Szenarios zur Intelligenzdiagnostik ergänzen die Diskussion. [less ▲]

Detailed reference viewed: 133 (2 UL)
See detailInstrumente zur informellen Kompetenzmessung (IKM)
Sonnleitner, Philipp UL; Kubinger, K.D.; Schleicher, Eva

Presentation (2009)

Detailed reference viewed: 47 (0 UL)
Peer Reviewed
See detailVersuch eines Konstruktionsrationals zum Leseverständnis: Entwicklung und Evaluation auf Basis kognitiver Modelle
Sonnleitner, Philipp UL

Scientific Conference (2009)

Due to inconclusive findings concerning the components responsible for the difficulty of reading comprehension items, this paper attempts to set up an item-generating system using hypothesis-driven ... [more ▼]

Due to inconclusive findings concerning the components responsible for the difficulty of reading comprehension items, this paper attempts to set up an item-generating system using hypothesis-driven modeling of item complexity applying Fischer’s (1973) linear logistic test model (LLTM) to a German reading comprehension test. This approach guarantees an evaluation of the postulated item-generating system; moreover construct validity of the administered test is investigated. Previous findings in this field are considered; additionally, some text features are introduced to this debate and their impact on item difficulty is discussed. Results once more show a strong influence of formal components (e.g. the number of presented response options in a multiple-choice-format), but also indicate how this effect can be minimized. [less ▲]

Detailed reference viewed: 31 (1 UL)
Peer Reviewed
See detailDoes the position of correct answers in multiple-choice items influence item difficulty?
Sonnleitner, Philipp UL; Hohensinn, Christine; Kubinger, K.D. et al

Scientific Conference (2009)

Due to economical reasons, the multiple-choice response format is often the preferred response format for test construction, especially in large-scale-assessments which claim for quick processing on the ... [more ▼]

Due to economical reasons, the multiple-choice response format is often the preferred response format for test construction, especially in large-scale-assessments which claim for quick processing on the examinees side and automated scoring on test makers side. As a disadvantage, research on effects of answer-key balancing and various “response sets” indicates an unfavourable influence of the position of correct answers on item difficulty. To examine this effect, two different multiple-choice response formats (“1 out of 6” and “2 out of 5”) were administered in 3 versions each (correct answers at extreme position, middle position or random) in the course of a large-scale competence assessment in reading comprehension. Furthermore, to investigate age-dependent effects this design was applied to 4th and 8th grade students and finally analyzed using the LLTM (Fischer, 1973). Results give useful hints for further test developments, especially in the field of Large-Scale-Assessments. [less ▲]

Detailed reference viewed: 71 (1 UL)
Peer Reviewed
See detailDas Wiener Self-Assessment Psychologie mit seinen Verfahren der experimentalpsychologischen Verhaltensdiagnostik zur Messung studienfachübergreifender Soft Skills
Sonnleitner, Philipp UL; Kubinger, K. D.; Frebort, Martina

in Rudinger, G.; Hörsch, K. (Eds.) Self-Assessment an Hochschulen: Von der Studienfachwahl zur Profilbildung (2009)

Detailed reference viewed: 110 (0 UL)
Peer Reviewed
See detailUsing the LLTM to determine an item-generating system for reading comprehension
Sonnleitner, Philipp UL

Poster (2008)

Due to inconclusive findings concerning the components responsible for the difficulty of reading comprehension items, this paper attempts to set up an item-generating system using hypothesis-driven ... [more ▼]

Due to inconclusive findings concerning the components responsible for the difficulty of reading comprehension items, this paper attempts to set up an item-generating system using hypothesis-driven modeling of item complexity applying Fischer’s (1973) linear logistic test model (LLTM) to a German reading comprehension test. This approach guarantees an evaluation of the postulated item-generating system; moreover construct validity of the administered test is investigated. Previous findings in this field are considered; additionally, some text features are introduced to this debate and their impact on item difficulty is discussed. Results once more show a strong influence of formal components (e.g. the number of presented response options in a multiple-choice-format), but also indicate how this effect can be minimized. [less ▲]

Detailed reference viewed: 22 (0 UL)
Peer Reviewed
See detailMeasuring the effect of multiple choice response format by some IRT models
Hohensinn, Christine; Kubinger, K.D.; Holocher-Ertl, Stefana et al

Scientific Conference (2008)

Detailed reference viewed: 28 (0 UL)
Full Text
Peer Reviewed
See detailLarge Scale Assessment at the Austrian Educational Standards: a Review
Kubinger, K. D.; Frebort, Martina; Khorramdel, Lale et al

in Testing International (2008), 19

Detailed reference viewed: 82 (0 UL)
Full Text
Peer Reviewed
See detailUsing the LLTM to evaluate an item-generating system for reading comprehension
Sonnleitner, Philipp UL

in Psychology Science Quarterly (2008), 50(3), 345-362

Due to inconclusive findings concerning the components responsible for the difficulty of reading comprehension items, this paper attempts to set up an item-generating system using hypothesis-driven ... [more ▼]

Due to inconclusive findings concerning the components responsible for the difficulty of reading comprehension items, this paper attempts to set up an item-generating system using hypothesis-driven modeling of item complexity applying Fischer’s (1973) linear logistic test model (LLTM) to a German reading comprehension test. This approach guarantees an evaluation of the postulated item-generating system; moreover construct validity of the administered test is investigated. Previous findings in this field are considered; additionally, some text features are introduced to this debate and their impact on item difficulty is discussed. Results once more show a strong influence of formal components (e.g. the number of presented response options in a multiple-choice-format), but also indicate how this effect can be minimized. [less ▲]

Detailed reference viewed: 158 (1 UL)
See detailTesttheoretische Analyse des Lese-Kompetenztests (4. Schulstufe) gemäß Bildungsstandards (Phase 1: 2006)
Hohensinn, Christine; Reif, Manuel; Gruber, Kathrin et al

Report (2007)

Detailed reference viewed: 19 (0 UL)