Tatiana V. Guy Miroslav Kárný - Decision Making: Uncertainty, Imperfection, Deliberation and Scalability

Machine Learning Research Group, University of Oxford, Oxford, UK

In many scenarios, decisions must be made by combining information from a number of different agents, be they people, sensors or computer systems. These agents may possess useful analytical skills that cannot easily be replicated, or they may have access to complementary information. For example, the fields of crowdsourcing and citizen science often employ human annotators to classify a data set, since people have sophisticated pattern-recognition and reasoning skills and the ability to learn new tasks given simple, natural language instructions. A large number of annotators can be used to compensate for the limited time that each person can dedicate to the labelling task, and for the use of non-expert and potentially un-trusted individuals. Agents may also provide diverse observations for applications such as situation awareness, where information can be obtained from mobile sensors, cameras and human reporters to build an overview of events in a particular scenario. By obtaining and aggregating information from a pool of decision-making agents, we can form a combined decision, such as a classification or an action, taking advantage of the wealth of existing skills, knowledge and abilities of the decision-making agents.

The canonical situation we consider in this chapter is depicted in Fig. , showing a crowd of agents making decisions about a set of objects , which can be data points, text documents, images, locations in space and time, or other items about which a decision is required. The right-hand side of the diagram shows an agent that combines decisions from the crowd, then exerts weak control to influence the assignment of agents to objects, represented in the diagram by connecting arrows. Weak control consists of suggestions and rewards for completing tasks that meet the weak controllers goals, and is used in situations where the controller cannot force agents to complete particular tasks. Such a situation occurs when working with human agents, who may choose whether to accept or reject tasks, but may be directed toward completing tasks that are informative to the combiner.

Our previous work [] focused on principled, Bayesian methods for aggregating responses from multiple decision-making agents, and inferring agent reliability. In this chapter we consider the complete system for selecting informative agents, assigning them to specific tasks and combining their responses. Both the choice of task and the suitability of an agents skills for that particular task affect the utility of the information we can obtain. By deploying agents effectively, we can minimise the number of responses required to confidently learn a set of target decisions. This allows us to analyse larger data sets, reduce the time taken or decrease costs such as payments required by workers in a crowdsourcing system. We therefore propose an information-theoretic approach, intelligent tasking , to estimate approximately-optimal task assignments, which can exploit additional descriptive information obtained through computational analysis of the objects or environment of interest. For settings in which the agents are paid to undertake tasks, we introduce an automated method for selecting a cohort of agents (workers) to complete informative tasks, hiring new members of the cohort and identifying those members whose services are no longer needed. The results demonstrate clear advantages over more simplistic approaches, but also indicate opportunities for future work, for example to automate agent training and motivate human analysts.

This chapter begins by looking at related work on information aggregation systems and whether they account for these issues. A case study is then introduced for a crowdsourcing system in which it is important to select and deploy agents efficiently. In this scenario, we wish to classify a large data set given a small subset of unreliable, crowdsourced labels. To do so, we extract features from the objects and use the crowdsourced subset of labels to learn how the features relate to the target classes. To handle the unreliability of the crowdsourced labels, we propose extending a Bayesian approach to decision aggregation, namely Dynamic Independent Bayesian Classifier Combination ( DynIBCC ) [], to augment discrete agent decisions with continuous object features in the range . This extension is demonstrated with the crowdsourcing case study, attaining strong performance with limited data. We then introduce an intelligent tasking framework for optimising the deployment of agents, balancing the cost of each task with a desire to maximise information gain. This framework naturally negotiates the need to explore and exploit the agents skills. The approach is used to develop the Hiring and Firing algorithm, which addresses the need to select both tasks and agents in a unified manner, and shows promising results in our experiments. The final section of this chapter discusses opportunities for extending intelligent tasking by considering delayed rewards, including those obtained through training and motivation of human agents.

In many existing systems, there is no attempt to select agents to perform particular tasks based on ability or diversity of skills. In Citizen Science applications, such as Galaxy Zoo [] using a partially-observable Markov Decision process (POMDP), but this choice is not tailored to the individual agents, which are not modelled in their approach.

Related work on crowdsourcing has considered the problem of selecting trustworthy workers. Web-based crowdsourcing platforms such as Amazon Mechanical Turk ( AMT )], who demonstrate how to reject poor performers from a pool of workers by thresholding a changing reliability value.