Magisto’s A.I. Helps Anyone Produce Polished Video

Magisto wants to do for video what Instagram did for photos—provide intuitive tools to edit and enhance them and make them easy to share. Founded in Israel in 2009 by two experts in computer vision and artificial intelligence, Magisto enables a user to simply select photos and videos on their smartphone, choose a visual theme, and automatically create a sophisticated edited product in minutes. There’s a lot of computer science on the back end making that possible. Magisto launched in January 2012 at the Consumer Electronics Show, won an app competition there, and now has 38 million registered users worldwide, up from 3 million last year. With 35 employees, the company has offices in Tel Aviv, New York, and San Francisco. Techonomy’s David Kirkpatrick sat down with Magisto CEO Oren Boiman for a wide-ranging talk about video, social media, and how people want to express themselves. Boiman, like so many Israeli entrepreneurs, began in high-tech while in the military, where he worked in intelligence. The following are edited highlights. Download a PDF of the full interview.
How did Magisto get started?
When my daughter was born, it was the first time I actually did real video, not just in a lab. You have a newborn and you want to capture their life experience. I bought a Sony Handycam in 2005 and I was shooting for two months. I thought, “I’m curating the family experience, it’s going to look great.” Then we wanted to have a party and share our young family’s experience. I looked at the videos and they were just horrible. We had to do something. Me and my wife—both engineers—said, “Let’s do video editing. It will probably take us an hour or so.” We spent two weeks slaving to get to five minutes we were proud enough to show. On the one hand I’m doing my Ph.D. in video analysis. On the other hand I cannot do anything to help with our home videos. But then again, when my mother saw the final video, she cried.
So after all, it worked.
But because of that experience I understood the solution needs to be as simple as pressing a button.
So you took the expertise you developed doing your PhD and began to apply it in this context.
My PhD is about one mechanism which can address many, many problems in computer vision. But when it comes to video editing, you have to deal with freeform consumer video and photos. You cannot do just face detection or face recognition. You have to have a general understanding of what is in a video, who are the important characters, what are the important scenes, when do they interact with each other, the speech parts, the audio. You need to create something that moves people. That’s what video editing is about.  But even one video has so many pixels you need to analyze, so people were doubtful whether we can do it. There was nothing that analyzes videos and photos so deeply at that scale when we started.
So you do really do deep analysis of the imagery?
You have to. Without getting to the level of understanding characters, scenes, interactions, actions, speech, you cannot get to storytelling, because this is what stories are built from.
What’s the long term vision for Magisto?
There are endless ways to improve our platform. For instance, we introduced different editing styles. So when the user chooses “travel,” or “cute,”  we don’t change just the looks. We use different algorithms and choose different parts of the video. So if I want it to feel sentimental, we might take dialog between the main characters. But if I’m going to do fast paced music video editing with the same footage, I’ll choose completely different parts. It’s not about what’s important, but what’s important to tell the story in a certain way. What we’re doing is injecting emotion.
So everything you do is about video.
Yes. When it comes to mobile, video is the best form of consumption. Video as a medium is like a thousand times more powerful than photos and text. Video has another layer of emotion, another layer of story. Today’s social media is very shallow. I mean, we use smileys to try to put some emotion there, but it’s not there. But when people see things in video, they really laugh. With Magisto if you want to make it cute, we’ll make it cute. You want to make it sentimental, we’ll make it sentimental. You want to make it funny, let’s make it funny.
Where is the technology headed?
We’re trying to reimagine the next stage of photography. We want Magisto to be your 24/7 video editor, to put Magisto in the background to always be there for you. So if you capture things with photos and videos on your phone’s camera, Magisto will surprise you with a produced movie. You don’t even have to open the app if you don’t want to. We announced a technology with Qualcomm at CES, called CamCrew. It’s a camera crew inside your phone. We take some of the storytelling brains of Magisto and run it in real time, so now you have not just an editor, but you suddenly get a 24/7 photographer and director to help you get better photos. When you’re shooting video or photos, you get notified by Magisto, “Here’s some people you might want to highlight.”
[Boiman picks up his phone.] So let’s highlight Josh [who is sitting at the table]. When I move the camera, Magisto knows that and will start to track him, resetting the focus and exposure, instead of me obsessing on the camera controls. Okay, smile for a second. It reframed the picture, saw you smiling, and it’s taken it for you. This is starting to do the thinking for you. If I’m starting to cut off your head or something, Magisto will tell you to put it in the middle. It helps the average photographer start to capture like a pro. When you press “done,” Magisto produces the movie in minutes.
You have an ambitious agenda.
Disruptive technologies often start from the root, not with high-end users. The first lousy digital cameras were used by consumers. They were not used by prosumers or by professionals. So that’s exactly what’s happening here. We cannot yet address the high end needs of professional video editors and brands, but this is exactly what consumers want. We will just build in more and more sophistication, and add more and more tools. It has to happen this way.
Techonomy has created a series of videos from events around the world using Magisto technology. Click on the links below to watch.
The Vibe at Davos
At Europe’s DLD: Innovation, Anxiety, and Inspiration
A Techonomist’s View of CES
Techonomy at Mobile World Congress, Barcelona