Home»Research»DiaKom»DiaKom Phase 2

Doctoral Program
Diagnostic Competencies of Teachers: Diagnostic judgment processes as information processing and the importance of person and situation characteristics.

  • Objectives: Improve the state of research, develop trainings of diagnostic competence.

  • Research questions: Diagnostic judgement processes and the significance of personal and situational characteristics

  • Partners: University of Education Freiburg and Heidelberg University of Education

  • Speakers: Prof. Dr. Timo Leuders, Jun.-Prof. Dr. Katharina Loibl, Prof. Dr. Tobias Dörfler

  • Funding amount: 12 secondment positions + 1 junior professorship for 3 years

  • Funding period: 08/2020 – 07/2023 2nd funding phase

  • Funding: Ministry of Science, Research and the Arts Baden-Württemberg

  • Project type: Doctoral Program

Description

Among the many activities that teachers carry out on a daily basis, diagnostic activities, i.e. any form of gathering information about the state of learning or the learning process in order to prepare pedagogical decisions, are considered to be key. The totality of knowledge, skills, motivations and beliefs of teachers which enable them to do this are summarised under the term diagnostic competence . The great importance of this facet of professional competence is contrasted by an unsatisfactory state of research, which is characterized by a number of research traditions that have so far been poorly interconnected. A first phase of the doctoral program DiaKom was dedicated to the clarification of influences, structures and ways to develop diagnostic competence.

The second phase of the doctoral program aims to generate explanatory knowledge in twelve sub-projects for the emergence of diagnostic judgements in typical diagnostic situations and for relevant diagnostic objects in various teaching subjects. From the research point of view, it contributes to the description of diagnostic judgements on the basis of cognitive modelling. From a practice perspective, the findings provide the basis for developing effective designs for promoting diagnostic competencies in teacher education. The sub-projects began in August 2020. Eight sub-projects are being carried out at University of Education Freiburg and four sub-projects at Heidelberg University of Education .

Subprojects

SP 1: Linking pedagogical content knowledge and logical reasoning when setting up and testing diagnostic hypotheses on student errors

To the overview of all subprojects

Adaptive and individualized instruction is based on accurate identification of faulty student conceptions (especially unsustainable conceptions) by teachers. Such misconceptions can be diagnosed on the basis of incorrectly solved tasks. The diagnosis requires not only the perception of task characteristics and error patterns, but also the application of pedagogical content knowledge about misconceptions as the cause of such error patterns. In the field of decimal fraction comparison, the most common misconceptions and the resulting error patterns are well researched. However, when working on tasks, different misconcepts can lead to the same error pattern. The challenge for the diagnosing teachers is to select further tasks for students in such a way that a clear attribution of different error patterns to one error concept as the cause can be made. It can be assumed that teachers may be subject to confirmation bias, i.e. they choose tasks that confirm their initial hypothesis instead of examining a possible refutation (Norman & Eva, 2010; Wason, 1968). Therefore, in addition to pedagogical content knowledge (about possible misconceptions about the diagnostic object), they also need strategic knowledge for appropriate diagnostic reasoning. In experimental studies, the influence of pedagogical content knowledge on the interpretation of error patterns (inference from a solution to possible causes), the influence of strategic knowledge on task selection (to narrow down the cause of the error), and the linking of these two knowledge facets are investigated in the area of comparing decimal numbers.

SP 2: Integrated use of pedagogical content knowledge and language education knowledge in the interpretation of learning-relevant features in mathematics tasks and student solutions

To the overview of all subprojects

Teachers assess the demands placed on learners and their expressions in subject lessons on the basis of their pedagogical content knowledge. Considering the importance of linguistic features of teaching-learning situations, the language education perspective must also be taken into account in the assessment. While a wide range of findings are already available on language features relevant to learning, knowledge about teachers’ judgment processes in this area has not yet been systematically investigated and is largely based on experience from in-service training contexts. With regard to the teaching and learning of mathematics, the linking of the two areas “linguistic structures” and “mathematization” is of particular importance: While on the level of surface features the two dimensions language and mathematics can rather be considered separately, in the case of deep features the two areas are closely related for diagnostic judgements as well as for didactic decisions: Teachers have to recognize, for example, which sentence constructions or which prepositions can be centrally relevant or aggravating for learning, taking into account the required mathematization or the basic concept to be activated (Wilhelm 2016). In addition to content and linguistic knowledge, they also need knowledge of pedagogical content knowledge and language education knowledge. It can be assumed that a close linking of the specific pedagogical content knowledge and language education knowledge enables a higher diagnostic accuracy. In the present study, therefore, different representations of knowledge are constructed (pedagogical content knowledge and language education knowledge either separate or integrated) and their use in the interpretation of tasks and solution attempts (in the form of vignettes) is investigated. It is assumed that in the case of verbal short answers of diagnosing teachers, the integrated form of knowledge also leads to a more profound and interconnected assessment of task difficulties.

SP 3: Integration of content knowledge and pedagogical content knowledge in the perception and interpretation of students’ ideas about substances and their properties.

To the overview of all subprojects

The ability of teachers to diagnose student (mis)conceptions as learning prerequisites of students in chemistry classes and to be able to assess their influence on teaching success contributes to a high degree to the quality of adaptive teaching. There is agreement that above all pedagogical content knowledge (PCK), spefically the “knowledge of student conceptions” (KOSC), but also the content knowledge (CK) of a teacher have an important influence on the quality of the diagnostic judgement. However, it is unclear whether and at which points the knowledge domains become effective in the process of the genesis of a diagnostic judgement. According to the DiaCoM framework (Loibl et al., submitted 2019), we assume that the genesis of a diagnostic judgment about student beliefs is influenced by information processing (a cue leads to a relevance assessment and this leads to the diagnostic judgment) through specific knowledge (CK and PCK-KOSC): It is investigated whether the knowledge domains CK and PCK-KOSC enable the information processing in the first place (mediation) or whether this knowledge merely changes the information processing (moderation). For this purpose, different models of information processing are postulated theoretically and modeled and compared on the basis of empirical data. This is expected to provide clues about the way in which the CK and PCK knowledge domains are integrated into information processing.

SP 4: Integrating or weighting the use of information in the interpretation of student products and utterances in elementary school mathematics.

To the overview of all subprojects

Teachers’ classroom-based diagnostic activities are characterised by a high degree of complexity; the situational information (cue) is often very rich. Cues have to be perceived and it has to be decided whether they are relevant or irrelevant for a diagnostic judgement. Previous findings indicate that novices do not distinguish between relevant and irrelevant information, but use all information for their judgements – in doing so, they resort to so-called information-integrating strategies (cf. Böhmer et al., 2017). As teachers (prospective and in-service) move toward diagnostic expertise, they should increasingly be able to identify and use relevant information (i.e., with high cue diagnosticity). The aim of this subproject is to examine whether novices do in fact predominantly resort to information-integrating strategies and make unfiltered use of irrelevant information in addition to relevant information, or whether information processing and the quality of diagnostic judgements can be influenced by knowledge about the relevance of information. The focus is on the subject richness of task-based instructional settings in elementary school which are represented by authentic vignettes. The relevance of the situation features is defined based on the existing research literature and validated in interviews with pedagogical content experts. The appropriate use of these features (cue utilization) is determined by varying the information provided and by process data (using the Mouselab method).

SP 5: Influence of pedagogical content knowledge and educational-psychological knowledge on the perception and interpretation of competing features in student utterances on the topic of acids and bases.

To the overview of all subprojects

Being able to adequately assess students’ learning prerequisites is a central task of the teaching profession. For diagnostic decision-making processes, it can be assumed that evidence-based theories and models form a basis for diagnostic decisions, as this supports the recognition of features relevant to the decision, substantiates the interpretation of the situation and thus justifies the diagnostic decision. Although there are theoretical assumptions and empirical evidence on how different areas of theoretical knowledge affect this information processing, further analysis of such processes in different diagnostic situations is needed to gain a deeper understanding. In this study, decision-making situations are generated in which a variety of information is provided. It is assumed that a diagnosing teacher perceives, interprets and uses specific information for decision-making depending on the theoretical knowledge base. Against this background, the question will be explored whether knowledge of a particular psychological or didactic theory influences information processing in such a way that different (diagnostically relevant) features are perceived and processed and decisions are changed as a result. This will be investigated in diagnostic judgments of student utterances on the topic of acids and bases. The findings obtained in this subproject contribute to a closer link between the understanding of knowledge-based processes in professional action and research on diagnostic decisions. Insights can also be expected into the way in which different bodies of knowledge contribute to decision-making in the presence of multiple decision options.

SP 6: Interpretation of student utterances on the selection of appropriate tasks on fractions depending on time pressure and situated prior knowledge.

To the overview of all subprojects

Teachers’ diagnostic competence includes the ability to identify misconceptions in student utterances and adaptively select tasks that can help overcome those misconceptions. The present project focuses on such situations and aims at the systematic analysis of the influence of situation characteristics (with/without time pressure) on diagnostic judgement processes of persons with different personal characteristics (pedagogical content knowledge, professional experience). The judgment processes are described in terms of the DiaCoM framework as information processing in which the information to be processed results from a) situational cues and b) the teacher’s knowledge structure. The cues are student utterances that indicate a particular misconception in the area of fractions, as well as more or less relevant features of tasks on fractions. With regard to the teacher’s knowledge structure, pedagogical content knowledge about misconceptions and task characteristics is particularly important. According to Brunswik’s lens model (1955) and the DiaCoM framework (Loibl et al., 2019), a teacher’s judgment regarding the most appropriate task in a specific diagnostic situation is assumed to depend on which of the presented information she perceives and how she uses her pedagogical content knowledge to process that information. Information processing is expected to be influenced both by the availability of episodic experiences (e.g., Leinhardt & Greeno, 1986; Putnam & Borko, 2000) and by time pressure (dual-process theories, e.g., Kahneman, 2000, cf. Rieu, Loibl, Leuders & Herppich, in press, for judgment processes in difficulty estimation with and without time pressure). To document the judgment processes, not only the selection of tasks but also protocols for thinking aloud are recorded and eye movements are tracked.

SP 7: Influence of the information richness of statistical tasks on the decision about their suitability depending on pedagogical content knowledge and content knowledge

To the overview of all subprojects

While statistical preconceptions of students in statistical, i.e., uncertainty-driven, decision-making situations are now considered well researched, there are still many unanswered questions about teachers’ diagnosis of students’ statistical competencies. This project will use the DiaCoM framework (Loibl et al., 2019) to investigate the ways in which teachers assessing decision-making tasks that are typical of statistics instruction, incorporate the richness of the information presented in the task into their judgments. To investigate this question, statistical decision tasks are varied experimentally with respect to the richness of information, and influences on diagnostic judgment and the judgment process are examined. For this purpose, assessments and justifications of prospective teachers are collected via vignette-based (closed and open) test formats. In addition, transcripts of guided interviews will be used to examine the underlying rationales and their elaboration as a function of the available wealth of information.

SP 8: Influence of biology and religion teachers’ own professional attitudes on learners’ perceptions of cosmogenesis, evolution and creation.

To the overview of all subprojects

Accurate diagnoses of students’ attitudes and beliefs about the origins of the world and life are a necessary precondition for initiating successful teaching and learning processes. While many studies have surveyed the attitudes of both students and teachers, little is known about how teachers themselves arrive at appropriate judgments about students’ attitudes. It seems that teachers in general are able to form accurate judgments. However, there are considerable differences between teachers in terms of accurate assessments. The study aims to elucidate the information processing (perception and processing of cues) involved in diagnostic judgments of students’ attitudes and beliefs about cosmogenesis and the origin and development of life, and to assess the influence of personal characteristics of the judging teachers on diagnostic performance. The results contribute to the targeted training of the prerequisite-rich facet of diagnostic thinking that is the focus of this study.

SP 9: Influence of subjective theories about the relationship between autism and academic performance on the perception and interpretation of autistic students’ learning.

To the overview of all subprojects

In the project, subjective theories about autism of (prospective) primary school teachers and their influence on the diagnosis of the academic performance of autistic pupils are investigated. Based on the DiaCoM framework (Loibl et al., 2020), subjective theories are the central person characteristics, while the academic performances to be diagnosed are the situation characteristics. The diagnostic thinking of (prospective) primary school teachers is influenced by their subjective theories about autism. We assume that diagnostic judgment of school performance is biased due to subjective theories about the relationship between autistic symptoms and aspects of academic performance. Diagnostic behavior is examined by means of processes (stimulation of subjective theories by information about symptoms of autistic students) and by means of products (judgment of academic performance).

SP 10: Stress as a cause for the reduction of cognitive capacity in the perception and interpretation of demands in tasks and reading strategies of learners.

To the overview of all subprojects

Teachers’ content knowledge and pedagogical content knowledge of difficulty-generating task features in reading tests and reading strategies play a crucial role in assessing and promoting their students’ reading competence. However, the process of diagnosing in real teaching-learning situations often happens under the influence of stress. To date, such bias factors have either not been taken into account in studies or have been eliminated where possible. Considering confounding factors is crucial to assessing the validity of previous research findings and quantifying the extent of the influence of such biasing factors. To this end, the present subproject investigates the effect of stress as a state component on judgment accuracy on task difficulty and on the adequacy of reading strategies. We expect that content knowledge and pedagogical content knowledge can mitigate the effects of stress. In addition, it is likely that this data will allow conclusions to be drawn about the processes involved in reaching a diagnostic decision.

SP 11: Influence of mindsets on the certainty and accuracy of diagnostic hypotheses about mathematical operational ideas when interpreting student solutions.

To the overview of all subprojects

Diagnostic judgments about learners are usually associated with uncertainty (because, for example, multiple causes for an error are possible and student behavior is not always consistent). Such uncertain judgments are to be understood as hypotheses that are supported or weakened by further information. Theoretically, such a multi-step diagnostic judgement can therefore be seen as a reduction of uncertainty through data-based inference: Based on new available data (e.g. further student solutions) the uncertain diagnostic hypotheses are updated. This process can be described using models of Bayesian inference (cf. Griffiths, Kemp & Tenenbaum, 2008). While Bayesian models describe ideal inferences in the presence of uncertainty, human inferences are affected by contextual conditions (e.g., Krolak-Schwerdt et al., 2012). For example, mindset theory (Gollwitzer, 2012; Gollwitzer & Keller, 2016) predicts that uncertainty is more strongly considered in so-called deliberate mindsets than in implemental mindsets. The deliberate mindset is characterized by weighing multiple decision aspects before making a decision. An implemental mindset, on the other hand, focuses on goal-oriented, rapid problem solving after a decision to act has been made. In the project, prospective teachers are placed in a deliberate or implemental mindset prior to the diagnostic situation. Subsequently, the (prospective) teachers make decisions on the basis of learner products (with systematically varied cue stimuli) about their latent characteristics (e.g. misconception, knowledge level etc.). The (non-verbalized) uncertainty of the decision is captured by having the subjects locate their decision in a geometric space (where the vertices are to be interpreted as high certainty, the spaces in between as uncertainty, see work program). We expect to gain insights into how well information processing in diagnostic judgments can be modeled as uncertainty reduction in terms of Bayesian inference and to what extent the deviation of human judgments from an ideal Bayesian judgment can be explained by the mindsets of the judging teacher.

SP 12: Interpretation of different requirements in tasks on price formation in dependence on an induced focus of attention on pedagogical content knowledge

To the overview of all subprojects

A central topic of economics education is the concept of price formation under competition. Tasks on price formation, as they are usually used, vary from a scientific point of view especially with regard to the addressed aspects of the concept of price formation as well as from a pedagogical point of view by the pedagogical approaches to the concept with their advantages and disadvantages. However, for the assessment of the quality of a pricing task from an economics educational point of view, the scientific or pedagogical quality by themselves are not sufficient; it is also crucial that both aspects are appropriately linked, i.e. that the selected pedagogical approach is not only suitable in principle, but also supports the learning process at the crucial points for the development of a scientifically appropriate concept. In order to arrive at an adequate interpretation of the task quality from an economics education perspective, pedagogical content knowledge must therefore be used in the sense of a genuine integration of (scientifically correct) content knowledge and pedagogical knowledge. Accordingly, the project investigates to what extent specific content knowledge and pedagogical knowledge can be integrated spontaneously or through an attentional focus in order to perceive the deep structure of the tasks in the sense of pedagogical content knowledge, and to interpret their quality accordingly. For this purpose, the perception and interpretation of the task quality of test persons with specific pedagogical content knowledge and pedagogical knowledge without or with an attentional focus (experimental groups F&D and F&D+) will be compared with the judgment processes of test persons with specific pedagogical content knowledge (comparison group FD), within the framework of an experimental design.

Funding organization

zurück zur Übersicht