As intelligent autonomous agents and multi-agents systems applications become more pervasive, it becomes increasingly important to understand the risks associated with using these systems. Incorrect or inappropriate agent behaviour can have harmful effects including financial cost, loss of data, and injury to humans or systems.
Thus, security and safety are two central issues when developing and deploying such systems. We refer to a multiagent system’s security as the ability of the system to deal with threats that are intentionally caused by other intelligent agents and/or systems, and the system’s safety as its ability to deal with any other threats to its goals.

In complex and rich environments, such as multiagent system environments, it is often necessary to involve the agents of the system in achieving some of these design goals, by making the goals explicit for the agent itself. For example, the agent must be aware of user-specified safety conditions if it is going to avoid violating them. This often means that an agent needs to be able to identify, assess, and mitigate many of the risks it faces. This is particularly true when the agent is going to be deployed in dangerous environments without immediate user input; for example, command of a spacecraft where communication with mission control involves considerable delays. Moreover, agents often integrate such activities as deliberately planning to achieve their goals, dynamically reacting to obstacles and opportunities, communicating with other agents to share information and coordinate actions, and learning from and/or adapting to their environments. Because agents are often situated in dynamic environments, these activities are often time-sensitive. These aspects of agents make the process of developing, verifying, and validating safe and secure multiagent systems more difficult than for conventional software systems. Hence, new and different techniques and perspectives are required to assist with the development and deployment of such systems.

This page is the responsibility of the author and does not necessarily represent the views, policies or opinions of The University of Melbourne.