How Generative AI is Transforming the Rollout and Performance of Amazon’s Robots

Amazon has introduced a handful of robots in its warehouses that the e-commerce giant says will improve efficiency and reduce employee injuries.

Two robotic arms named Robin and Cardinal can lift packages that weigh up to 50 pounds. A third, called Sparrow, picks up items from bins and puts them in other containers.

Proteus, an autonomous mobile robot that operates on the floor, can move carts around a warehouse. The bipedal, humanoid robot Digit is being tested to help move empty totes with its hands. And there’s also Sequoia, a containerized storage system that can present totes to employees in a way that allows them to avoid stretching or squatting to grab inventory.

Amazon says Robin is currently being used in dozens of warehouses. The others are in a testing stage or haven’t been rolled out widely. But the company says it’s already seeing benefits, such as reducing the time it takes to fulfill orders and helping employees avoid repetitive tasks. However, automation also carries drawbacks for workers, who would have to be retrained for new positions if the robots made their roles obsolete.

In October, Amazon held an event at a Nashville, Tennessee, warehouse where the company had integrated some of the robots. The Associated Press spoke with Julie Mitchell, the director of Amazon’s robotic sortation technologies, about where the company hopes to go from here. The conversation has been edited for length and clarity.

Q: When you’re working on robotics, how long does it typically take to roll out new technology?

A: This journey that we’ve been on has taken a couple of years. Luckily for us, we’ve been at this for over a decade. So we have a lot of core technology that we can build on top of. We started these particular robots – Cardinal and Proteus – in this building in November 2022. We came in and began playing around with what it would look like to pack and move a production order. Less than two years later, we are at scale and shipping 70% of the items in this building through that robotics system.

Q: So, two years?

A: We talk about “build, test and scale” and that’s about a two-year cycle for us right now.

Q: It’s challenging to build robots that can physically grab products. How does Amazon work through that?

A: As you can probably imagine, we have so many items, so it’s an exceptional challenge. We rely on data and putting our first prototype in a real building, where we expose it to all the things we need it to do. Then we drive down all the reasons that it fails. We give it a lot of sample sizes in a very short period of time. For example, a couple of years ago, we launched our Robin robotics arm – a package manipulation robot – and we’re at 3 billion picks. So the ability to launch into our network, rapidly collect data, scale and iterate has enabled us to go fast.

The challenge itself can be boiled down to three simple things: you need to perceive the scene, plan your motion and then execute. Today, those are three different parts of our system. Artificial intelligence is going to help us change all of that, and it’s going to be more outcome-driven, like asking it to pick up a bottle of water. We’re on the verge, so that’s why I’m personally excited to be here at the onset of generative AI and use it to dramatically improve the performance of our robotics.

Q: How do you think about the impact of automation on Amazon’s workforce as you’re developing the technology?

A: With the technology we’ve deployed here, we’re creating new roles for individuals that can acquire new skills to fulfill those roles. And these new skills are not something that is too difficult to achieve. You don’t need an engineering degree, Ph.D. or any really technical skills to support our robotics systems. We designed the systems so they’re easy to service and train on the job to be a reliability maintenance engineer.

We are working backwards from the idea that we want to employ more skilled labor. These opportunities are obviously higher paid than the entry level jobs in our buildings. And partnering with MIT has helped us understand what matters most to our team as we’re deploying these technologies across our network.

Q: Are you experiencing any challenges as you introduce these robots in your warehouses?

A: Not in the adoption. We’re integrating it. But these are complex systems and this is the real world, so things go wrong. For example, we had bad weather due to the storms in the Southeast. When I look at the robotics systems data, I can tell the weather is bad outside because that dramatically affects how the ship dock works.

When trucks don’t arrive on time or when they can’t leave, you see bottlenecks in the building in strange ways. Containers build up, we have to put them in different places, and then humans need to recover them. So communication between what our robotics system is doing and what we need employees in the building to do to recover is important. It’s a collaboration of automation and humans to deal with real-world problems. It’s not a matter of having robotics take over but making it one system of humans and robotics working together to accomplish the goal of shipping the product.