Keywords
scoring consistency
arts education
inter-rater reliability
rubric-based assessment
process assessment
assessment fairness
Abstract
Assessment in arts education often relies on teacher judgment, which may cause large score differences when shared criteria are absent. This study tested whether a structured rubric could improve scoring consistency while keeping room for professional judgment. The sample included 60 arts teachers and 900 student portfolios from visual arts, design, creative practice, and interdisciplinary arts courses. Each portfolio was scored by three teachers under open and rubric-based conditions, producing 5,400 rating records. The criteria included originality, technical improvement, experimentation, reflective depth, process records, final presentation, and overall achievement. Intraclass correlation coefficients, generalizability theory, mixed-effects models, variance analysis, and teacher interviews were used to assess score agreement and identify the main sources of variation. The mean score changed only slightly, from 74.8 under open judgment to 73.9 under rubric-based assessment. However, inter-rater reliability increased from 0.58 to 0.81, and the mean score range among three teachers fell from 12.4 to 7.1 points. Teacher-related variance decreased from 17.6% to 7.2%, while variance linked to portfolio quality increased from 43.2% to 59.1%. Agreement was highest for process records, technical improvement, and final presentation. It remained lower for originality and reflective depth. Interviews showed that teachers valued clear criteria but still needed freedom to judge complex artistic choices. The results show that structured rubrics can improve score consistency and fairness without removing professional judgment. These findings may support rubric design, teacher training, and portfolio assessment in schools, universities, and community arts programs.
References
Huang, J., Yin, J., Xu, T., & Yang, J. (2026). Quality Assessment and User Adoption Prediction for Enterprise-Level Financial Reporting Data Products. Available at SSRN 7179118.
Schmid, D. (2025). Assessment as a pedagogical tool and research method for programmatic evaluation. In The Intellect Handbook of Dance Education Research (pp. 232-250). Intellect.
Lin, T., Spivack, Z., & Chen, A. (2026). Monitoring Teaching Quality and Validating Educational Equity in Public Art Education. Available at SSRN 6512778.
Theerapan, S., Chuangprakhon, S., Chaiyason, N., & Chen, W. (2026). Rater fairness and assessment literacy in preservice music teacher education. Music Education Research, 1-13.
Zheng, J., & Makar, M. (2022). Causally motivated multi-shortcut identification and removal. Advances in Neural Information Processing Systems, 35, 12800-12812.
Yao, G., Zheng, J., Wang, Z., Zhang, W., Han, R., Zhao, C., ... & Liu, R. (2026, March). V-pruner: A fast and globally-informed token pruning framework for vision transformer. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 40, pp. 34396-34404).
Ferreira, P. (2026). Portfolio Assessment as Reflective Practice: Documenting Metacognitive Growth in Language Teacher Education. International Journal of Changes in Education.
Wu, J., Zhang, J., Wu, D., & Peng, Y. (2026). Visual Transformation Mechanisms for Integrating Digital Cultural Heritage Resources into Junior High School Art Curricula.
Zhang, G., Xu, Y., Zhang, M., Wang, S., & Lin, N. (2021). The brain network in support of social semantic accumulation. Social cognitive and affective neuroscience, 16(4), 393-405.
Mallik, R., Das, R., Das, S., Smarandache, F., & Kumar, J. R. (2026). AI-Driven Teacher Education: A Neutrosophic Analysis. Infinite Study.
Zähl, P. M., Schulte, L., Eimler, S. C., Eggert, M., & Wolf, M. R. (2026). A Machine Learning Approach to Benchmarking Social Debt. IEEE Access.
Du, Y. (2025). Research on Digital Quality Traceability System for Temperature-Controlled Supply Chain of Foreign Trade Wine Driven by Blockchain and IoT. Business and Social Sciences Proceedings, 4, 57-65.
Zhang, Z., Gao, Y., & Tong, Y. (2026). CausalML4CX: A Causal Inference and Machine Learning Framework for Customer Experience Optimization with Application in Banking.
Belyaev, D. A., Skripkin, I. N., & Rochnyak, E. V. (2026). The Discipline of Philosophy in the Modern Higher Education System: A Comparative Analysis of the Current State of Affairs. Перспективы науки и образования, (3 (81)), 175-189.
Qi, C., & Qiao, X. (2026). Using AI to Monitor AI: Automated Operations Through Log-Driven Intelligence. Available at SSRN 6795840.
Uchida, S., & Negishi, M. (2025). Assigning CEFR-J levels to English learners’ writing: An approach using lexical metrics and generative AI. Research Methods in Applied Linguistics, 4(2), 100199.
Zhang, Y., Wang, J., & Gu, W. (2026, February). Research on the Construction and Effectiveness of a Computer-Aided Systematic Training Framework for Wind Farm O&M Quality Amidst High Personnel Turnover. In Proceedings of the 2nd International Conference on Digital Management and Information Technology (pp. 647-651).
Huang, R. (2026). Less Is More: When Data Augmentation Hurts Small CNN Generalization. Journal of Computer and Communications, 14(7), 1-11.
Prichard, R., Peet, J., El Haddad, M., Chen, Y., & Lin, F. (2025). Assessment moderation in higher education: Guiding practice with evidence-an integrative review. Nurse education today, 146, 106512.
Wu, J., Zhang, J., Wu, D., & Peng, Y. (2026). Visual Transformation Mechanisms for Integrating Digital Cultural Heritage Resources into Junior High School Art Curricula.
Huang, J., Xu, T., Yang, J., & Yin, J. (2026). A Graph Neural Network Approach for Financial Data Validation and Risk Identification in Large-Scale Intercompany Transactions. Available at SSRN 7175058.
Val, S., & Quintas, A. (2025). Key performance indicators for optimizing academic performance and course design in online educational platforms. Cogent Education, 12(1), 2529420.
Xu, T., Zhang, J., & Zhu, W. (2026). Reproducible Modeling Pipelines and Cross-Window Stability in Subprime Auto Loan Credit Risk Assessment. Available at SSRN 6893861.
Xiong, W., & Zeng, Y. (2026). Multi-Layer Coupled Diagnosis of Energy Efficiency Degradation in Complex Building MEP Systems and Identification of Optimization Boundaries. Available at SSRN 7121782.
Yoon, J., & Lee, K. (2026). Development of self-observation assessment rubrics to improve teacher competency in creative gifted education. Gifted Education International, 42(1), 169-182.
Hong, Z., Chen, H., & Xu, T. (2026). Real-Time Capacity Risk Forecasting for Low-Latency Financial Trading Cloud Platforms. Available at SSRN 7118180.
Baracco, B., Escribano, N., Da Silva, D., Belliard, V., Ceballos, L., & Fuentes, V. (2025). Factors influencing the agreement between teachers and students in the assessment of preclinical endodontics using a rubric. BMC Medical Education, 25(1), 626.
Li, J., Zhang, Y., & Gu, Y. (2026). Chain-Hash Audit Framework with Trusted Anchoring for Tamper-Evident Evidence Lifecycle Management.
Asma, B. (2026). Comparing Teacher and Artificial Intelligence Scoring in Writing Assessment: A Generalizability Theory Analysis. Language Teaching Research, 13621688261463602.