Facebook’s Adam Mosseri

Algorithms and Values at Facebook
The information that over 1.5 billion people see every day is determined by the algorithms embedded in Facebook's news feed. How do those algorithms work and what values do they embody? Adam Mosseri oversees the news feed.
 
Kirkpatrick: Adam Mosseri’s been running the Facebook newsfeed for two years, and I think you can all agree that if we’re talking about values and modern technology, there’s hardly any modern technology that’s more determinative of all of our lives right now than the Facebook newsfeed, which for 1.55 billion people is one of, if not their main source of information about the world around them, to use Facebook phrase.  I just wanted to start, Adam, by asking you to just talk about, how does the newsfeed work? We can talk about algorithms, dig down into them, but what’s the basic way you describe what it is?
Mosseri: What a newsfeed is?
Kirkpatrick: Yes.
Mosseri: Essentially what we do at Facebook, especially on newsfeed, is try to connect people with the stories that matter to them. So, everything we do is with that goal in mind. And so, what we do is, every time you open up the app, you refresh, or you pull the refresh, or you switch tabs and you come back to newsfeed, we try to take a look at all of the content that we could show you, based on who you friended, what publications you followed, et cetera, and then try to figure out what it is that you find interesting, based on, do you tend to like content from that person in the past? Do you tend to like content of that media type in the past? Et cetera. So, an example might be, I have a sister, who is a furniture design student at RISD, and she posts photos of her funky looking furniture all the time, and I like those photos, every time I see them.
And newsfeed notices that, so it infers that I am interested in my sister, and tries to show me my sisters stuff higher up. And it infers that I am interested in photos, so it tries to show photos more often, et cetera. So we’re always learning, and trying to better figure out what people are interested in.
Kirkpatrick: Well, let’s go straight to the values question. Because, people often do wonder what decides what I see and what I don’t see? And when you’re making those decisions—before we get to the actual kind of mechanics of it, which I want to talk about a little bit—what do you think of as the values that do guide your decision making?
Mosseri: So, when people talk about ranking algorithms, they often talk about them as these third parties with their own agency that kind of go around and make their own decisions. But, it’s not how they work really. Essentially, what ranking algorithm’s always trying to do is make a recommendation to you. So, before I go to Facebook, think about other services. Amazon is trying to figure out what shoes you might want to buy. Netflix is probably trying to figure out what TV shows or movies you might want to watch. Facebook, we’re trying to figure out what story you might be interested in seeing at that moment. And that’s where values come in, right? What do we define as interesting? How do we figure out what relevance is? And we really have three primary values on newsfeed. First and foremost, is to connect you with your friends and family. That’s the value proposition that our company was built on, that’s what people they tell us they want to see most. Then, to inform people about the world around them. That could be hard news, but that could also be, just learning about the fact that there’s Canadian elections right now, which John Oliver posted about, or that—actually recently I had a friend ask about a plumber. I have a group, we call it Homies, but essentially we post questions about houses. And someone asked, “Does anyone have a plumber?” and another friend recommended it. So, the second is to inform you about the world around you—
Kirkpatrick: Which is a phrase that Mark Zuckerberg used way back
Mosseri: Yes. And we’ve been talking about that for a long time. And lastly, is fun content, entertaining content, content that helps you pass the time. But, we think about our values in that order.
Kirkpatrick: So, fun is an explicit part of what you’re thinking about when you’re making decisions about what people would see.  
Mosseri: Yes. I think that people often use Facebook while they’re waiting in line to buy lunch, or maybe, someone in the back here is not that interested in what I’m saying and maybe they’re checking it right now, that’s cool too. In those moments, I think amusing content or fun content, can be really valuable. But again, we think about, first and foremost, to connect you with your friends and family. And then, informing. Because, that’s the order in which people ask for things, so that’s the order in which we try to best invest our time.
Kirkpatrick: So, there’s algorithms that basically follow those guiding principles, that all sort of work together, to determine what you finally see. Now tell me this, because I know that everyone—well, the average Facebook user has how many friends, at this point?
Mosseri: A few hundred.
Kirkpatrick: A few hundred. So, the average user would be eligible to see roughly how many posts and items?
Mosseri: Roughly 2,000 or so.
Kirkpatrick: In a day?
Mosseri: Yes.
Kirkpatrick: And how many do they typically actually get presented with.
Mosseri: A few hundred.
Kirkpatrick: A few hundred, so you’re basically filtering down to maybe 10 or 20 percent of what would be eligible to be seen.
Mosseri: Yes.
Kirkpatrick: So how do you do that?
Mosseri: Well, there’s more and more information out there, right? There’s more and more that we can consume, there’s more and more that we want to consume. And there’s only a finite amount of time every day, right? And so, what we try to do is figure out what you’re the most interested in, to make the best out of your time.
Kirkpatrick: Right. How do you do that?
Mosseri: Well, we—[LAUGHS]
Mosseri: I don’t want to say the same thing again. Well, I think that, what we do is we try to predict things. Are you going to like this? Are you going to comment on it? Are you going to share it? Et cetera. And we get better at predicting, and better at figuring out, “Oh, Adam likes Ana’s content,” that’s my sister, “let’s show her content higher up on the newsfeed.” But I think the meaningful question is, how do we evaluate success? How do we know that things are going well?
Kirkpatrick: Right.  
Mosseri: So, there’s really two buckets. One is what people do. So, do you like more? Do you comment more? Do you spend more time on Facebook? Do you share more? These sort of behaviors. But the other is what people say. So, one thing we have is something we call the Feed Quality Panel. Have any of you guys ever heard of the Feed Quality Panel? Nobody’s heard of the Feed—
Kirkpatrick: I hadn’t even heard of it, until I spoke to him on the phone.  
Mosseri: So, essentially what the Feed Quality Panel is, is about a thousand people, right now in the US, Indonesia and Brazil. And they spend four hours a day, rating 60 stories.
Kirkpatrick: Do you pay them for that?
Mosseri: We do, yes. We actually do lightweight surveys internationally, everywhere, that are not paid, but we have actually professional raters for—
Kirkpatrick: And there’s a total of how many?
Mosseri: Roughly a thousand.
Kirkpatrick: But, that number’s going up.
Mosseri: Yes. And, imagine you spend a couple hours this morning, ordering stories from the one you found the most interesting to the one that you find the least interesting. We can then compare your order to the order that we would have put stories in. And that difference is our opportunity. That difference we try to minimize over time. So, when we launch changes to how ranking works, we look at, are people liking more things? Are they sharing more things? But we also look at, are we getting closer to the order that people tell us they want the stories in, that the people tell us they find most meaningful.
Kirkpatrick: One of the things you said on the phone was that the single most important thing that determines what you see is who your friends are.
Mosseri: Yes.
Kirkpatrick: Talk about that and why that’s so important.
Mosseri: Well, when you first sign up, you have a completely blank slate for newsfeed. There’s nothing in there. And so, over time, you friend people, you follow maybe sports teams you’re interested in, celebrities, public voices, publications, news, et cetera. And so, the content that’s in your newsfeed is only eligible because you followed a page or a person. Actually, not everyone knows this, but every single post from every one of your friends is in your newsfeed. If you keep scrolling. Most people don’t read their entire newsfeed, so they miss things. That’s not true for what we call page content, business content, because many businesses post many hundreds of things a day and feeds would just get totally overrun.
Kirkpatrick: But, from actual people, you put it all in there.
Mosseri: It’s all in there.
Kirkpatrick: Yes, that is interesting. So, you talked about also, this—you evaluate all these things and you do it based on certain set algorithms, but you apply that algorithm to each individual. So in effect, everyone has their own individuated set of controls for what they see. Is that a fair way to put it?
Mosseri: Yes. Essentially, what we value is consistent, but how we execute or how we apply the algorithms is very personalized. So, a simple example would be, we look at how long you read a story for. So, if you read two stories, one for four minutes, and one for four seconds, we’re going to assume that the one that you read for much longer is more interesting. But, four minutes for you might be a really long time, and four minutes four minutes for you might be a really short time, because maybe you read everything end to end, that’d be pretty cool. And so, what we think of as a long time or a short time is totally personal, it’s totally independent, it’s different for every single individual. That’s true for most of our features.
Kirkpatrick: But you said that some of it is curated. What do you mean by that? Is that going back to the Quality Panel, or is that something else?
Mosseri: What I mean by curation is, we do our best to figure out what people find interesting, but we know that we don’t always get it right, we know we make mistakes, which is why it’s really important for us to build controls for people to curate the experience when we do. So we invest a lot in building and educating about controls for newsfeed. So, unfollow, you can unfollow anybody. So, actually have any of you unfollowed anybody?
You’ve got to unfollow people, it makes a huge, huge difference.
Kirkpatrick: Does it really? Good. Because I just went through the other day and did it, and nobody noticed.
Mosseri: That’s the point, nobody knows—so, I probably shouldn’t talk about unfollowing people on stage, because then people know.
Kirkpatrick: Well, if it’s important for people to do it, you should talk about it
Mosseri: I’m trying to think of a safe example.  I’m a Steelers fan, I’m from New York, but I’m a Steelers fan, it’s a long story, I can explain later. But, we weren’t doing that well a couple years ago and it was just sort of depressing me every Sunday, so I unfollowed the Steelers. They don’t know about it, which is good, and then that content is out of my feed and then I have a more happy sort of experience. But we also do things about the inverse too, so, See First is newer, so I’d actually be surprised if most of the room has heard of See First. Has anybody used See First? We’ve got one over here? Just one. All right, See First is good. What See First allows you to do is pick someone you want to have show up at the top of the newsfeed, every single time they post. I have a brother, he’s in a band, he posts videos and images of his gigs and so I’ve See First-ed him. So, every time he posts, it shows up at the top. And this blend, which we talked about, of this automated ranking, which is trying to figure out what is interesting to you, and then manual curation, where you can actually correct the system when it’s getting it wrong, or tell the system more of what you want, “No. I always want to see my brother’s stuff and I never want to see stuff from the Steelers.” I think makes for a very meaningful experience.
Kirkpatrick: So, the curation is by the user who is seeing the newsfeed. I see. And then, the algorithms work on top of that, right.
Mosseri: Exactly.
Kirkpatrick: Now, I know you—this is one interesting thing that many people may not know—that you have nothing to do with ad algorithms whatsoever, right?
Mosseri: True.
Kirkpatrick: All you do—if I might say, and tell me if I say this wrong—is, you set aside certain slots, that are predetermined by some other algorithm probably, about how many times a given average user can see an ad, and then the ads that are placed in there are determined by other algorithms that are from the ads team. Is that roughly right?
Mosseri: Yes. Two different teams, two different systems, that then integrate on the front end.
Kirkpatrick: So, you don’t think about ads at all, except that you have a certain number of slots you have to reserve, is that the way it works?
Mosseri: Essentially. What we focus on is trying to figure out what people find interesting, and connecting them to that content, better and better and better. Ads does something that’s pretty parallel, they try to figure out what ads you’re interested in. They also have to consider what the advertiser is also interested in, so that they’re trying to sell shoes to guys in their 30s in San Francisco, I’ll show up as an option. But these teams are independent, the ranking systems are relatively independent, and then we integrate on the front end.
Kirkpatrick: Certainly in the business community, there’s tons of frustration that people don’t know how they can figure out what things will be seen by their followers, but we’ll leave that aside. But, I think even in the developed countries, even among individuals, people get confused and frustrated—they feel like Facebook has too much control, I’m sure you hear that a lot. How do you respond to that mindset, that people—“I just wish that they didn’t decide so much what I see!” I mean, is there any way around that? What do you think about that?
Mosseri: I think we have a huge responsibility. Because a billion people use Facebook every given day, they load newsfeed a bunch of times per person. And so, we have a responsibility to, as best we can, connect them with the stories that they find meaningful. And we need to get better at it. We’re not bad at it, we could be better at it, which is why we’re working so hard on it. In terms of frustration, or in terms of curation, I think that what I would probably do—and I do this a lot actually—is walk them through how the system works and then what tools they have to improve the system if it’s not working for them. I think it’s really important that everybody who uses Facebook, whether you’re—you guys maybe, hopefully use Facebook a little bit, which is good—so people, or—
Kirkpatrick: I don’t think we need to ask.
Mosseri: Actually I have a couple cousins who just like refuse, which is good.
Kirkpatrick: Oh, there’re probably a few non-Facebook users—
Mosseri: Yes. Does anybody not use? I’m going to call you out.
Kirkpatrick: How many people here don’t use Facebook, at all?
Mosseri: Nice, there we go. See?
Kirkpatrick: Okay, one, two, three, four—oh! About 15, 20 people. Woah, that’s higher than an average outing. Okay, go on.
Mosseri: That’s work for us, which is good.
Kirkpatrick: Hey, growth opportunity.
Mosseri: Right. [LAUGHS] I didn’t think about that when I came here. Whether it’s people who use Facebook, or might use Facebook in the future, or publications, it’s super important they understand how the systems work.
Kirkpatrick: Yes. Which has never been explained much at all in the past, so it’s nice that you’re here. But why is it so important?
Mosseri: Because, I think that if it’s going to be something you use—and right now, I think the last stats we posted publically were, people use it, on average, over 40 minutes a day, in the US.
Kirkpatrick: How many millions in the US? Like two hundred—
Mosseri: Probably 150 million-ish? Give or take a few tens of millions…
Kirkpatrick: 150 million people use for 40 minutes a day. Go on.
Mosseri: It’s about a billion people daily, and in the US I think it’s about about 150 million monthly.
Kirkpatrick: Is 40 minutes a day roughly a global average, too?
Mosseri: Yes. And so, if you’re going to use it for that long, you should understand how it works, so that you can get the most out of it. You can decide who you want to follow and who you want to unfollow. You can understand why you’re seeing more of this and less of that. So that you can then go in and correct it if it’s getting it wrong. So, we look at education as a kind of broad issue, so speaking like this is great. But what’s even more impactful is actually doing education in the product itself. So letting somebody know when they hide a story that they can also unfollow that person, in that moment. Knowing when to ask and when to educate is actually something we spend a lot of time thinking about.
Kirkpatrick: I’m going to get to the audience, but I asked on Facebook today what some people would ask. Robert Scoble, who has opinions about a lot of things asked, “Why can’t we have keyword based filtering?” Meaning, he wanted to be able to get rid of any post that mentioned Donald Trump, for instance.
Kirkpatrick: Is that sort of thing possible, coming, an idea you’ve considered?
Mosseri: The Donald Trump filter is not an issue I consider—
Kirkpatrick: Not a Donald Trump filter but, to be able to say, “I don’t want to see anything that has such and such a word or phrase.”
Mosseri: Yes. I think that, the important thing with controls is that they’re easy to understand and they’re powerful. Because, if they’re not easy to understand and powerful, they’re not going to get adopted, and then what’s the point of building them? I get a lot of, actually more obscure requests. One I’ve gotten more than once is, “Can I stop seeing baby photos from high school friends I don’t talk to anymore?” Which is like a very convoluted thing. But, like, I can empathize with, like, I’ve had that experience.
Kirkpatrick: Well have that experience, pretty much.
Mosseri: But, if we build all of those, I think that the concern is that the controls would get too complicated and less approachable. And they might be good for a few people who are very technical but I want to make sure that our controls have broad appeal. That my mom can use them and understand them, for that, I think they have to be simple.
Kirkpatrick: That’s interesting. Because another person said, “Why don’t they give users more control over what the algorithm prioritizes?” and mentioned there used to be sliders that you could slide around that made various things come and go in larger quantities or lesser quantities.
Mosseri: Yes, that was a—
Kirkpatrick: That was a long time ago.
Mosseri: Yes, that might even have predated me.
Kirkpatrick: Yes, I think it was probably a few years ago. Then maybe not. But I remember that.
Mosseri: I think the other important thing about controls is that they can deliver on the promise. So if you say, “I never want to see anything from my kooky aunt again.” Then we will take it out of your newsfeed and that’s great. If you say, “I always want to see my wife’s stuff first,” if you say that, then we’ll always show that first. We can deliver on those promises. If you say, “I want to see a little less photos, and a little bit more from Bob,” it’s very hard to deliver on that promise in a way that’s meaningful. And then people actually end up getting frustrated there as well. So, it’s a balance though. And we’re not done. I actually think one theme for us going forward is how can we develop more powerful controls? How can we better ask people what they find interesting? So that we can improve the experience.
Kirkpatrick: We’re going to go to the audience, in a second. Do you consider Twitter a rival? And here’s why I ask, I use both a lot, and I set it up so that my tweets go into Facebook. And I noticed, for a long time, that when I tweeted and then it went into Facebook, it got almost no distribution. It seems now, that Twitter tweets that I do, go into Facebook, and get more distribution. Is that a change that you would consciously have made? And, just how do you think about Twitter as a partner, ally, enemy, whatever?
Mosseri: First for the second part, and then I’ll do it in reverse order if that’s all right? Certainly not a change we would have made explicitly to decide to promote or demote any content coming in any other third party
Kirkpatrick: Really? You don’t do that?
Mosseri: No, because again, it would be at odds with the mission, right? So, whether it’s how often we show ads, or whether it’s how does partner content do in this system? We believe that if we continue to make it more meaningful by showing you content that you find more interesting than we did the day before, you’re going to spend more time with us and that’s going to be better for everyone in the long run.
Kirkpatrick: Wait so, you’re saying that it’s solely a matter of how people respond to what goes into my feed, once they see it the first time. You don’t have rules about this kind of content we show people less or that kind of content we show people more.
Mosseri: No, because it’s a complicated thing but feed composition, right? How much we show of what? Is a byproduct of trying to optimize for each individual. So if you don’t like videos, we probably should never show you videos. If I don’t want to follow any friends, if I don’t want any friends or family on Facebook, I should be able to do that, and I actually just follow publications. We believe that if we build a system that allows those types of use cases, then we’ll have broader appeal and that people will use our services more in the long run.
Kirkpatrick: And you still can say more of this or less of this on—don’t you still have those drop downs?
Mosseri: I don’t think so.
Kirkpatrick: They’ve gone away, okay. So who has questions or comments? Okay, please. Identify yourself.
Audience1: Hi, Elin Elkehag,  I’m actually wondering if you can do too good of a job?  Because, I’m running a company, I’m building a hardware startup. And I was using so much time on Facebook, on my phone. It’s like procrastination crack, so I deleted my Facebook app on my phone, because I don’t have time. And actually, I know a lot of my friends have started to do the same. Because it’s hard to build a startup and it just takes too much time. Do you see that that is a trend? Or, is it something that you worry about?
Mosseri: Do we worry about people spending too much time on Facebook?
Kirkpatrick: That’s not a bad question.
Mosseri: No, it’s a good question.
Elin: No, but deleting the app.
Kirkpatrick: Well, deleting the app is another matter. But if you worry about spending too much time, a lot of people worry about that for themselves, there’s no doubt.
Mosseri: We worry about sentiment. We ask surveys all the time about how people feel about their experiences. Are they deriving value from the experiences? Are they being connected with their friends and family? Being informed? Et cetera. And if we see any shifts there, by market, or by device type, or et cetera, we certainly worry about it. But it’s important to remember that people use Facebook very, very differently in different parts of the world, right? So, Brazil last year in the World Cup, it became—we saw a huge swell of activity around sharing about soccer, or what they would call football. In some parts of the world, where there’s less free press, Facebook newsfeed becomes a very important source of uncensored news. Myanmar is a good example, as of late. Here, it’s more about connecting with your friends and family—
Kirkpatrick: Romania probably is another one. Go on, keep going.
Mosseri: Yes. And then, so it’s different in different places but we would worry about that if we saw that becoming a global trend.
Kirkpatrick: Okay, who else has a comment or question? Please. Identify yourself.
Berkowitz: Hi, my name is Avi Berkowitz.
Mosseri: It’s a nice wide room.
Berkowitz: I’m a Rabbi at Chabad but I also consult with companies and nonprofits on impact. So Facebook is something that is a part of the lives of a billion people of every community and society. You built a neutral platform and you keep talking about how everyone can control their feed and remain neutral. So you can either be a positive platform or evil networking. And it’s used by ISIS and the worst characters in the world to destroy lives. At the same time, think about—I can tell you five words of the negative, cyberbullying, body image, divorce—a lot of negatives. A lot of positives also. We saw what happened when Nepal’s earthquake happened, you raised more money than the United States, $10 million by the US government, $12 million by Facebook. That was because you guys put a banner on our newsfeed. I feel that Facebook needs to not only be a neutral platform of connecting the world, you need to tell us how you are using your platform to be a positive platform. Thank you.
Mosseri: Sure. I think that there’s a bunch of ways you can think about what negative experiences we might unintentionally facilitate and what positive we do. For in terms of anything that has anything to do with violence, we take that very, very seriously. We’ve got what we call Community Content Standards and anything that celebrates violence or is trying to organize violence, et cetera, that content we take off of Facebook. The way that works actually is people report it, and they report it really, really often. We then manually review it and we remove it. In terms of things like natural disasters and et cetera, what we’re trying to do there is build tools to allow people to facilitate those positive experiences. And so we’re doing more and more there. I do think it’s an area where we can do more and we’re looking to more. So, we are open to ideas.
Kirkpatrick: Ok, we’re running out of time. Actually, we ran out of time a long time ago, but go ahead. Over here, identify yourself.
Mosseri: I don’t mind.
Reynolds: Thank you. Josh Reynolds with Quantifind. My question is this, you talked about some very smart ways that Facebook helps us stay connected with and prioritize the things we already know about, the things we care about. But, what I’m curious about is this, where does finding new things, where does serendipitous discovery, where does broadening our perspectives come into play? The reason I ask is this, if we just continue to look closer and closer and closer at the things we don’t already know, our blind spots are going to grow exponentially. When we type in a search term into Google it’s because we think we have a question in mind. But in the old day, when my parents grew up, they had someone on a screen delivering news that they would never have seen otherwise. They had new things brought to them.
Kirkpatrick: Yes, it’s the filter bubble question.
Reynolds: Exactly, so my question is, Facebook, what role do you think you play in enhancing human curiosity and bringing us closer to things that we don’t already know about? What responsibility do you have around that?
Kirkpatrick: Good. Thank you for that question.
Mosseri: I think a big responsibly. I think, one, I’ll talk about what happens today, and two, I’ll talk about what I think we can do in the future. On what happens today, there’s really three types of content on Facebook. There’s content from friends, there’s content from publications, and then there’s hybrid content, which you’re seeing not because you followed—let’s say—let me think of an interesting example. Do you guys know the page IFL? This is like a benign example. It’s I F’ing Love Science. It’s an amazing page about pretty cool science stuff. I learned about them on Facebook because someone posted a link to an IFL video. And this happens a lot, it’s actually a growing percentage of the content on Facebook. And that allowed me to—one, it was a cool video, it was about sleep science and about how you might sleep less and think you’re fine, but you just get used to it, if you just sleep more it’s better for everybody. Which I used as an excuse to go to bed earlier and get up later. But I learned about that on Facebook and I ended up following that page and now I get updates from IFL all the time. So, it happens, in a significant way on the platform already. That said, I think there’s more and more we can do to help people discover content that they don’t even know about yet. Or that they do but they just don’t happen to be connected to yet. My mom actually on my last birthday she said, “I hear you work on newsfeed, I’ve got a complaint.” This happens a lot if you work at Facebook. And I was like, “Okay, what’s going on?” And she’s like, “You never show me any content from ‘The New York Times.’” And I try to explain how ranking works, and she wasn’t having it. So I took out her phone and I opened up ‘The New York Times’ on Facebook and she’s not even following them.
Mosseri: And so, at first it was like, “This is on you, Mom.”
Don’t ever say that to your mom. And then I realized, it’s not really, she actually sees links to ‘The New York Times’ all the time, because her friends share them, and she just assumed she was following the page. So we need to do better connecting my mom to ‘The New York Times,’ but also helping people discover things based on their current interests that they might not know about yet. And so we’ve got some ideas that we’re currently working on in that space.
Kirkpatrick: And like you said before, helping people understand how the whole thing works really helps, and you’ve got to explain it a lot because it’s complicated.
Mosseri: Yes, that’s the foundation on which everything is built.
Kirkpatrick: One final question.
Kirkpatrick: But, do you feel—do you walk around feeling, “My God, I have so much responsibility, 1.5 billion people see this thing that I have to—” Do you feel like the power of what you do day-to-day, does that weigh on you?
Mosseri: I feel like it’s a lot of responsibility. I tend to think day-to-day about two things. One, how can I help my team get done what they need to get done this month? But two is how can I make sure that we’re thinking about these long-term issues, like serendipitous content, or like safety on Facebook, et cetera. And I think my role is to balance those things, is to help the team manage the week-to-week grind. And it’s intense and it’s stressful, which is good, but also to think about those longer things. And as long as I’m doing that, that energizes me and helps me deal with the fact that I do think it’s a very serious responsibility.
Kirkpatrick: Well, cool. Thank you so much, Adam, for coming and talking.