Repository logo

Browsing by Author "Nowlan, Nuket"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Higher-order thinking skills assessment in 3D virtual learning environments using motifs and expert data
    (2023) Qorbani, Sam; Nowlan, Nuket; Arya, Ali; Abdinejad, Maryam
    The research reported in this paper addresses the problem of assessing higher-order thinking skills, such as reflective and creative thinking, within the context of virtual learning environments. Assessment of these skills requires process-based observations and evaluation, as the output-based methods have been found to be insufficient. Virtual learning environments offer a wealth of data on the process, which makes them good candidates for process-based evaluation, but the existing assessment methods in these environments have shortcomings, such as reliance on large data sets, inability to offer specific feedback on actions, and the lack of consideration for how actions are integrated into bigger tasks. Demonstrating and confirming the ability of three-dimensional virtual learning environments to work with process metrics for assessment, we propose and evaluate the use of motifs as an assessment tool. Motifs are short and meaningful combination of metrics. Combining time-ordered motifs with a similarity analysis between expert and learner data, our proposed approach can potentially offer feedback on specific actions that the learner takes, as opposed to single output-based feedback. It can do so without the use of large training datasets due to reliance on expert data and similarity analysis. Through a user study, we found out that such a motif-based approach can be effective in the assessment of higher-order thinking skills while addressing the identified shortcomings of previous work. We also address the limited research on similarity-based analysis methods, compare their effectiveness, and show that utilizing different similarity measures for different tasks may be a more effective approach. Our proposed method facilitates and encourages the involvement of instructors and course designers through the definition of motifs and expert problem-solving paths.
  • Thumbnail Image
    Item
    ScienceVR: a virtual reality framework for STEM education, simulation and assessment
    (2021) Qorbani, Sam; Arya, Ali; Nowlan, Nuket; Abdinejad, Maryam
    This paper addresses the use of Virtual Reality (VR) in Science, Technology, Engineering, and Math (STEM) education. There are limited studies investigating the proper design and effectiveness of VR in STEM education, and current VR frameworks and applications lack explicit links to the established learning theories and assessment mechanisms to evaluate learning outcomes. We present ScienceVR, an educational virtual reality design framework, illustrated through a science laboratory prototype, to bridge some of the gaps identified in the design and development of a VR environment for learning. We established design guidelines and implemented an in-app data collection system to measure users’ learning, performance, and task completion rate. Our evaluation using ANOVA and other non-parametric methods with 36 participants in three groups: immersive VR (IVR), desktop VR(DVR), and 2D indicated improved usability and learning outcomes for the IVR group. Task completion rate in the IVR group was higher (68% compared to DVR with 50%). For memorability, the IVR condition performed better than DVR while for learnability, IVR&DVR performed significantly better than 2D. IVR group has performed better and faster with more accuracy compared to the DVR group in completing the tasks.