Default header image

Exploring whether incorporating the stoplight system improves inter-observer reliability between individual assessors

By Jiayi Cai

In recent years, the use of animal-based measures has been emphasized as they reflect an animal’s response to its environment, compared to resource- or management-based indicators. Due to its straightforwardness, the stoplight system has become popular, over the past few years, as a way of assessing animal-based measures. Our study aimed to determine whether incorporating the stoplight system would improve inter-observer reliability between individual assessors. An audit was done at the Elora Dairy Research Center by 4 graduate student auditors, with around 12 hours of welfare assessment training, and an expert welfare auditor. In total, 30 cows were observed for body condition score (BCS), hock lesions, lameness, and injured tails. The stoplight system was used for each measure with green indicating acceptable, yellow as requiring intervention, and red as unacceptable. To determine inter-observer reliability, results from each animal-based measure were compared among students and an expert, as well as compared to a mock audit performed whilst students only had roughly 4 hours of training. Results suggest high inter-observer reliability in BSC and injured tail assessment. However, lameness and hock lesion scoring proved to be more challenging and might require additional training. For future projects, we suggest that the training be more intensive, such as exposure to free stall farm environments, to ensure more cohesive results in the evaluations. These results can be used to improve the training of animal welfare auditors along with promoting the inclusion of the stoplight system to facilitate ease of training and to obtain better inter-observer reliability.