Eric Colson on Humans and Machines Working Together

Session Description: Stitch Fix uses AI to select clothes for its customers, but human stylists do fine-tuning. This created 3400 stylist jobs.
The transcript can be found below, with a PDF version available here.

Humans and Machines Working Together

(Transcription by RA Fisher Ink)

Kirkpatrick: Eric, tell us what Stitch Fix has discovered. It’s hard to say your company’s name too many times.
Colson: Yes, understood. All right. Well, thank you, David. So there seems to be an increasing amount of concerns that machines are going to take over the world and leave us humans without jobs. Well, this talk is not about the competition between machines and humans for work. Rather, I’m going to tell you about a very different potential outcome.
All right, let’s start by acknowledging that machines and humans have different abilities. Now, you all appear to be mostly human.
[LAUGHTER]
So to illustrate this point, I’m going to ask you to perform a couple of tasks. A prompt is going to appear on the screens and then you’re going to raise your hand when you have the answer. Again, this first one, by the way, no using phones or devices. This is a test of your human abilities. Add up all the multiples of three.
[LAUGHTER]
Come on, hands, please. When you have the answer, please raise your hand. How about for a free drink? What if you can use pencil and paper? Okay. Consider yourself off the hook for this task. You can all do it. Let’s start there. You can all do it. You all remember how, if you can add up the digits, you can tell if it’s divisible by three, then you have to hold that number in your memory while you do the next and then add it. We don’t love to do this because it takes effort. Doing math in our head takes a lot of effort. But you all can do it. Now, a machine, of course, is going to do this flawlessly and in milliseconds.
All right, let’s move on to task number two. Raise your hand when you have the answer and if you want to use your device on this, please have at it. Go ahead. Is the person to your left paying attention?
[LAUGHTER]
Okay, we have some hands. Okay, and actually, I don’t even need to know the answer. I just wanted to see if you can do it. Yes, this doesn’t take much effort at all. And did anybody think to ask their device? Asking Siri wouldn’t have been very helpful, right? That’s because this task requires access to ambient information. It also requires your ability to relate to other humans. You had to assess body language and social norms. It’s actually a lot of information to process. Far more than what’s on the table on the left there. It’s much more information, yet us humans can do these tasks rather effortlessly.
Okay, so they have different abilities, machines and humans. So if you have a task that requires processing a lot of structured data and doing a lot of rote calculations, you’re going to be better off going with a machine processor for that. But if you had a different task, one that needed to leverage ambient information or the ability to relate to other humans, then you’re going to be better off going with this other processor, a human.
Now, there are tasks out there that require processing of both types of information. And for such tasks, you’re going to be better off leveraging both resources together. That is, a task that needs both types of information processed. One that uses machines and humans is going to be better than one that uses either one alone. So what we’re looking for is to combine them, so we can get their additive abilities. So we’re looking to add their contributions and perhaps any synergies. The last term on the right is their synergies, or interactions.
Okay, for that statement to be true, we just need to make two assumptions. Number one is, machines and humans both have to have non-zero contributions to offer, and number two is that those contributions have to be different. They can’t be doing the same thing. So it turns out there’s lots of examples out there of this.
Consider the game of chess. It was long held as perhaps the ultimate demonstration of human thinking abilities. But in 1997, the renowned chess champion Garry Kasparov lost to Deep Blue, a supercomputer built by IBM. Now, this was just a matter of time. It was inevitable. The human mind is just no match for the brute force methods and the fast calculation speeds of a supercomputer.
But after that day, something new emerged. Something called freestyle chess. This is where the chess player can use a device of his or her own, and it’s usually a much more modest device than a supercomputer. And it turns out that the two working together, human and a modest device, can team up to beat the supercomputer. Kasparov, himself, engaged in freestyle chess and he said that by ceding the rote calculations over to machine, that he felt freed up to focus on the more creative aspects of the game. So the two can combine to achieve higher levels of performance than either one alone.
And this pairing of humans and machines has commercial applications as well. I happen to work for a company that has a lot of needs for machines and humans to combine their abilities. How many have heard of Stitch Fix? Good. It’s getting pretty mainstream. In fact, the last time I was here in New York was in November where we did our IPO, so we’re no longer a small company. We’re now a public company.
Now, we have a number of cases where we need to combine humans and machines. Oh, and by the way, if you don’t know about Stitch Fix—there was a few people that didn’t raise their hands—it’s an e-commerce company. You can buy clothes from us, but there’s no shopping. The service takes care of that for you. Okay, we have lots of examples where we need to combine machines with humans. So consider the styling function. This is the part of the service that actually picks out the clothes for you. To do that, we need to leverage both lots and lots of structured data as well as some other types of information.
So for example, figuring out what size somebody is, is really an empirical task. This is because there’s no standards out there for apparel sizing. Size can vary tremendously by brand. So what we have to do is digest millions and millions of past shipments to figure out how to place each person and each piece of merchandise on that spectrum, and that way we can get much more precise. Now, because that entails millions of calculations on past shipments, we cede that task over to the machines. That’s their job. But there’s other things to consider when picking out clothes for someone.
Some of our clients write in notes, such as this one. She says, “I need something to wear to my ex-boyfriend’s wedding.”
[LAUGHTER]
Okay, that seems important. This is a relevant bit of context. And so we’re going to need a process here that has the ability to relate to other humans. And so that part will be handled by our human stylist. They will know exactly what that means to that client. And so we need the two combined.
Now, we’re also combining machines with humans in fashion design. What we do here is, we take machines and we train them on algorithms to go recombine existing elements from past styles. So what we might take is a silhouette from one style like this, and we’ll combine it with the lace stitching on the shoulders of a second one, and then we might take a floral pattern from a third and recombine those elements to create something new that has never existed before, yet the data is telling us that this is a good bet, that this has a high probability of success.
Now, there is literally billions and billions of combinations to try and each one requires a bunch of calculations, so this a task that we cede to the machines to do. But once they finish all processing the billions and billions of possible combinations, they turn over their results—only a few of the results, those with the best chance of success—they turn those over to human designers who merely need to react to them. They are presented with combinations that they would have never found on their own and then they use their creative faculties to adjust things slightly. So in this case, they took that style and they decided to adjust the floral pattern a little bit, making it a little less symmetrical, a little more blotchy, and then you get the final product which is the thing on the right. We call this program, appropriately, hybrid designs. A hybrid between machines and humans. And they have produced some of our best, most successful styles ever.
And finally, we are also combining machines and humans in our merchandise buying. Of course, we need to keep our inventory full of the things that are currently in fashion. And to do that, we employ machine algorithms to watch and learn what things are working well and what to go buy more of. But we also need to be continually on the lookout for the next thing. Fashion evolves very quickly. I hope we’re not going here, but we do need to watch out for the next thing because fashion evolves very quickly. And for this, this is more of a social task and one that is done almost unconsciously, say, by attending a fashion show or seeing who wore what to the Oscars. For this task, we give it over to our human buyers, who merely need to digest their surroundings.
So specialization. In each of those cases, we had a lot of structured information to process, but we also had ambient information in the situational context that we wanted to take into account. And so we divided up the work onto these two very specialized processors. All the rote calculations on structured data went over to the machines because they were purpose-built to do this with their switches and logic gates. Our human brains, however, were built to digest our surroundings, to relate to other humans, and to react to things. All this is handled here, in the limbic system. This is one of the older parts of the brains and is, therefore, much more developed. This is why we can do these things rather effortlessly.
Analytic thinking, however, is done up here, in the neocortex. This is a newer part of the brain and less developed. That’s why it takes us effort to do math in our heads and, frankly, why we’re not very good at it. But by using the two combined, we can get the best of each. Now, specialization by the division of labor is actually a great source of economic growth. Yet, in this case, it worries us a little bit because we’re giving away some of the work to a machine. But we need to think about what is it we’re giving up, and what is it we’re doing more of.
Each of our employees, aforementioned groups, they tell us that by ceding the rote tasks to machines that they feel freed up to focus on the more creative aspects of the game. For the stylist, it gives him more time to relate to their fellow human clients, the buyers have more time to digest their surroundings, and the designers have more time to improvise on the aesthetics.
So this not only—in each of those cases, they were doing less of what they didn’t like and more of what they did love. And so it not only provides for much more fulfilling roles, but it also brings out the best of what we humans do naturally. So in essence, you can argue that we are making people better humans.