It is a natural tendency for humans to favour or prefer information that has been provided by automated or computer-based systems. This tendency is referred to as automation bias and is based on social psychology research that has found positivity bias occurs in environments where human-automation interaction occurs and that automated decisions are rated more positively than neutral (Wikipedia, 2018). This bias has a flow-on affect as humans “accept” the automated decisions and tend to disregard or not search for contradictory information (Bekier, 2018).
This trait is a concern for systems designers as it implies that system users will “trust” and have the tendency to “over rely” on automated systems. This overreliance may also have further implications in that de-skilling may occur in environments where the overreliance is embedded into the work culture – and in the event of a system failure or dubious output, the operators may not have the necessary skills to generate or obtain the required information they need (due to previously over-relying on the automated systems).
While not easy to initially identify, automation bias (and subsequent de-skilling) is a primary concern for all implementations of automated systems. Salas & Maurino (2010) theorise that automation bias and de-skilling may have contributed to the 1995 American Airlines Flight 965 accident outside of Cali, Columbia. In this accident, the flight crew relied on the automated Flight Management System (FMS) for navigation however this particular FMS wasn’t able to recognise the proximity to terrain which resulted in the crew not being able to determine whether they had passed a particular waypoint or not. Automation bias may have played a role in this incident as the crew were reliant on the FMS. Subsequently, the erosion of skills (de-skilling) is highlighted in this incident as the location of the aircraft could’ve been easily calculated had the pilots been better skilled in the traditional methods of reading flight charts.
References
Bekier, M., 2018. Human Factors in ATC , Sydney, Australia: UNSW.
Salas, E. &. Maurino. D., 2010. Human Factors in Aviation, 2nd ed. London, England: Academic Press.
Wikipedia, 2018. Automation bias. [Online]
Available at: https://en.wikipedia.org/wiki/Automation_bias
[Accessed 10 June 2018].
