Citation Context

...tion for our study is to ensure the availability of enough training images for each object detectors. We therefore focus our attention on obtaining the objects from popular image datasets such as ESP =-=[31]-=-, LabelMe [30], ImageNet [5] and the Flickr online photo sharing community. After ranking the objects according to their frequencies in each of these datasets, we take the intersection set of the most...

Citation Context

...rge jobs are broken into many small tasks that are then outsourced directly to individual workers via public solicitation. Workers sometimes work for free, motivated either out of intrinsic enjoyment =-=[3]-=- or some form of social reward [6]; however, successful examples of volunteer crowd sourcing have proven difficult to replicate, in part because arbitrary tasks tend not to be intrinsically enjoyable,...

Citation Context

...fect the validity of experiment results? HUMAN COMPUTATION IN HCI SYSTEMS Human computing systems [2,19] involve using human thinking to drive computational systems. For example, Games With A Purpose =-=[3]-=- have used anonymous crowds of gamers on the web to label images, identify music, and perform other human intelligence tasks to support a variety of projects with low operational costs. Although scala...

Citation Context

... structure to Wikipedia ([46] ), but they depend on voluntary compliance. An interesting approach is to combine data entry with a social system that structures the behavior. For example, Luis von Ahn =-=[47]-=- has created games in which people are rewarded for teaching the computer things such as what to label an image. The data structure is fairly simple: an entire image, or a well-defined region of the i...

Citation Context

...cuss some lessons about using MTurk as a programming platform. RELATED WORK One challenge in writing human computation algorithms is motivating humans to do work. One approach is Games With a Purpose =-=[1]-=-, where humans perform useful computation as a byproduct of playing computer games. Usergenerated content websites such as Wikipedia use human computation to generate content, and this content alongw...

Citation Context

...l. Quinn and Bederson give a good overview of distributed human computation systems [16]. Individual systems have also been studied and explored in academic literature, including Games with a Purpose =-=[1]-=-, Wikipedia [9] [15], and Mechanical Turk [7] [8] [12] [13] [14]. Human Computation Algorithms Many human computation systems are embarrassingly parallel, where tasks to not depend on each other. Huma...

Citation Context

...Human computation is the idea of using human effort to perform tasks that computers cannot yet perform, usually in an enjoyable manner. The first human computation game, or Game With A Purpose (GWAP) =-=[13]-=-, called the ESP Game [14], follows a specific mechanism: two players are given the same object (in this case, an image) that they are asked to describe, and the descriptions upon which the players ag...

Citation Context

...erceived promise of crowdsourcing,1 would-be users face a serious practical challenge: they need to attract a crowd. Crowdsourcing projects have used a variety of inducements, including entertainment =-=[20]-=-, information [1, 13], the chance to be altruistic and attention from others [12]. Until recently, it was extremely difficult to offer a crowd money, but with the advent of online labor markets like o...

Citation Context

... goal is to identify a hidden state of the world by collecting multiple assessments from human workers. Examples of consensus tasks include games with a purpose (e.g., image labeling in the ESP game) =-=[13]-=-, paid crowdsourcing systems (e.g., product categorization in Mechanical Turk) [6], and citizen science projects (e.g., efforts to classify birds or celestial objects). Consensus efforts can be subtas...

Citation Context

...l approaches. This paper now proceeds with a discussion of related work. Then we present our model of iterative and parallel processes, and apply each process to three problem domains, discussing the results of each in turn. We end with a generalized discussion of our results, as well as proposals for future work. 2. RELATED WORK Human computation in general has recently received a lot of attention. Quinn and Bederson give a good overview of distributed human computation systems [14]. Individual systems have also been studied and explored in academic literature, including Games with a Purpose [1] [2] [3], Wikipedia [5] [9], and Mechanical Turk [7] [8] [13] [15] [16]. Researchers have also tried to break down and categorize human computation, including Quinn and Bederson [14]. Their aggregation dimension is most applicable to this paper. It asks how work is coordinated and combined to achieve a final result. Malone et al. [12] also break down collective intelligence systems along several dimensions. They make a distinction between workers working dependently and independently, which roughly maps to our iterative and parallel processes. They also distinguish between creation and decisio...