User studies are important for many aspects of the design process and involve techniques ranging from informal surveys to rigorous laboratory studies. However, the costs involved in engaging users often requires practitioners to trade off between sample size, time requirements, and monetary costs. Micro-task markets, such as Amazon’s Mechanical Turk, offer a potential paradigm for engaging a large number of users for low time and monetary costs. Here we investigate the utility of a micro-task market for collecting user measurements, and discuss design considerations for developing remote micro user evaluation tasks. Although micro-task markets have great potential for rapidly collecting user measurements at low costs, we found that special care is needed in formulating tasks in order to harness the capabilities of the approach.

Wikipedia, a wiki-based encyclopedia, has become one of the most successful experiments in collaborative knowledge building on the Internet. As Wikipedia continues to grow, the potential for conflict and the need for coordination increase as well. This article examines the growth of such non-direct work and describes the development of tools to characterize conflict and coordination costs in Wikipedia. The results may inform the design of new collaborative knowledge systems. Author Keywords Wikipedia, wiki, collaboration, conflict, user model, Web-based interaction, visualization. ACM Classification Keywords

Wikipedia’s success is often attributed to involving large numbers of contributors who improve the accuracy, completeness and clarity of articles while reducing bias. However, because of the high coordination needed to collaboratively write an article, increasing the number of contributors is costly. We examined how the number of editors in Wikipedia and the coordination methods they use affect article quality. We distinguish between explicit coordination, in which editors plan the article through communication, and implicit coordination, in which a subset of editors set direction by doing the majority of the work. Adding more editors to an article improved article quality only when they used appropriate coordination techniques and was harmful when they did not. Implicit coordination through concentrating the work was more helpful when many editors contributed, but explicit coordination through communication was not. Both types of coordination improved quality more when an article was in a formative stage. These results demonstrate the critical importance of coordination in effectively harnessing the “wisdom of the crowd ” in online production environments.

In this article we argue that social visualization can motivate contributors to social production projects, such as Wikipedia and open source development. As evidence, we present CodeSaw, a social visualization of open source software development that we studied with real open source communities. CodeSaw mines open source archives to visualize group dynamics that currently lie buried in textual databases. Furthermore, CodeSaw becomes an active social space itself by supporting comments directly inside the visualization. To demonstrate CodeSaw, we apply it to a popular open source project, showing how the visualization reveals group dynamics and individual roles. The article concludes by presenting evidence that CodeSaw, and social visualization more generally, can motivate contributors to social production projects if the visualization leaves the laboratory and makes it to the community visualized.

question and answer (Q&A), market mechanisms, incentives, motivation, social relationship theory, information overload, spam, interruption, human attention. Requesting and sharing information through computer-mediated technology is an integral part of our lives in this information age. However, when deciding whether or not and how to engage in information exchanges, parties involved often have different needs and constraints. In addition, they are often unaware of each others ’ needs and constraints. Such asymmetry in motivation and information leads to suboptimal allocation of attention and time and contributes to the growing problems of information overload, costly interruptions and missed opportunities. A potential solution is to employ market mechanisms to support information exchange. Markets are institutions that allow individuals to trade goods and services efficiently. Applying markets to information exchange, askers can use pricing to signal the importance of the information exchange and compensate the answerers for their time. Answerers can use pricing mechanisms to filter incoming requests, reducing interruption costs and information overload. This dissertation studies the strengths and weaknesses of using economic markets for interpersonal information exchange. Are there costs in incorporating markets into our everyday information exchanges?