How Amazon Used Oreos and Dog Toys to Develop an Army of Robots to Grab What You Buy

Amazon's e-commerce business is undergoing a significant transformation, with the company investing billions of dollars in robots to make its warehouses more efficient and profitable. But how did this massive initiative begin? The answer lies in a quirky competition that started over a decade ago, when Amazon launched the Amazon Picking Challenge.

In 2015, the company called on university engineering teams to design robots for a common warehouse task: grabbing products from a shelf and putting them in a box. The challenge was sparked by Amazon's desire to solve the problem of how to motivate researchers to focus on this specific area of robotics AI. By opening up the competition, Amazon hoped to attract top talent and accelerate progress in the field.

The first Amazon Picking Challenge took place over two days in Seattle, with more than 25 teams from prestigious universities like MIT, Duke, Rutgers, and Georgia Tech participating. The contestants had just 20 minutes to pick 12 target items from a typical shelf on a Kiva Systems warehouse pod and put them into containers.

The products used in the challenge were surprisingly diverse, ranging from Oreos to dog toys. This may seem unusual, but it was designed to test the robots' ability to handle various shapes, sizes, and materials. The researchers wanted to see which approach would be most successful in real-world scenarios.

A Human Advantage: Warehouse Workers Outperform Machines

The main finding from the first Amazon Picking Challenge was that human warehouse workers were significantly better at picking products than machines. While humans could complete the task at a rate of around 400 sorts per hour with minimal errors, the best robot achieved a rate of just 30 sorts per hour with a failure rate of 16%.

This discovery was unexpected but insightful. It highlighted the limitations of current robotics technology and showed that there was still room for improvement in the field.

The Evolution of the Challenge

Over the next year, the competition evolved to include more complex tasks. The Amazon Robotics Challenge continued to attract top talent and accelerated progress in the field. Research on robotic manipulation exploded from 2016 through 2018, with many institutions publishing their results and insights.

The challenge also inspired the creation of graduate-level classes related to Amazon's challenge, which are still producing experts with valuable practical applied knowledge in robotics.

Real-World Applications

Today, Amazon's robotic arms, such as Robin and Sparrow, are being used to pick packages from conveyor belts and place them on other mobile robots called Pegasus. These machines use computer vision and AI to identify which item is which and how to grasp it.

Sparrow, in particular, has made waves in the industry with its ability to handle individual items rather than packages. It can pick more than 200 million items from containers and place them in totes, revolutionizing the way Amazon fulfills orders.

Financial Implications

Wall Street is taking notice of Amazon's warehouse robots, too. Morgan Stanley recently estimated that these machines could save the company as much as $10 billion a year. The big story is that we're just getting started, according to Tye Brady, the chief technologist at Amazon Robotics.

The Future of Warehouse Automation

Amazon's investment in robots and automation is poised to transform the warehouse industry. As the company continues to innovate and improve its systems, we can expect to see significant changes in the way orders are fulfilled and products are delivered.