Amazon to Shut Down Mechanical Turk, the Crowdsourcing Platform Behind ImageNet

Amazon has announced that its crowdsourcing platform Mechanical Turk will completely cease operations on September 30 this year. The official statement is restrained, merely saying that 'as part of our continuous evaluation of our products and services, we have decided to discontinue MTurk.' This decision brings an end to a platform that has operated for 21 years.

MTurk launched in 2005, initially addressing tasks that are 'difficult for computers but easy for humans,' by breaking tasks into HITs (Human Intelligence Tasks) for global parallel processing. Bezos once called it 'artificial artificial intelligence,' a phrase that proved prophetic.

Around 2006, when Fei-Fei Li was teaching at Princeton, she decided to build a large-scale visual dataset, ImageNet. At the time, mainstream datasets had only a few thousand to tens of thousands of images, but ImageNet needed to process over 160 million candidate images, making human filtering a bottleneck.

Li's team turned to MTurk, breaking down image filtering into micro-tasks distributed to global Workers. A 2009 paper showed the dataset had already collected 5,247 concepts and 3.2 million organized images. The team later noted that 49,000 MTurk workers from 167 countries participated.

Without MTurk, the millions of manual annotations for ImageNet would have been nearly impossible. This inference is based on the platform's scale and task volume, though lacking direct causal evidence, the logical chain is clear.

The ImageNet challenge began in 2010, and in 2012, Hinton, along with his students Alex Krizhevsky and Ilya Sutskever, used a deep convolutional neural network to reduce the Top-5 error rate to approximately 15%, significantly outperforming traditional methods.

This victory propelled deep learning in the visual domain, followed by VGG, GoogLeNet, and ResNet, expanding deep learning to speech, NLP, and eventually the era of large models. Fei-Fei Li, Hinton, and Ilya, among others, solidified their positions as a result.

CNBC, citing Krista Pawloski from Turkopticon, reported that MTurk has been declining in recent years, with reduced investment from Amazon and new platforms diverting workers and clients. But a more direct reason is AI's own capability improvement; tasks that once required human effort can now be easily handled by multimodal models.

Industry demand has shifted toward expert-level data: platforms like Scale AI, Mercor, and Prolific have risen, offering specialized training and evaluation data. MTurk's model of 'anyone can take it, pennies per task' is becoming outdated.

A 2023 study by Swiss scholars found that up to 46% of surveyed MTurk workers used AI to complete text tasks. Clients pay for human judgment, but workers use AI as a substitute, creating a darkly humorous situation.

MTurk's name originates from the 18th-century European Mechanical Turk, a chess-playing machine that appeared automated but hid a human chess master inside. Amazon brought this anecdote to the internet, where programs appear to handle tasks automatically but are actually completed by humans.

Now AI can perform these tasks itself, and the humans who once made AI look smart are stepping aside. MTurk's fate is a microcosm of AI development over the past 20 years: humans used crowdsourced data to feed AI, and AI ultimately replaced those human jobs.