The Misty robot is a great and affordable open source robotics platform. A unique robot that encourages hacking and extensions! Join us for an interview with the inventor - Ian Bernstein

Interview with Elad Inbar of RobotLAB and Ian Bernstein from Misty 

Elad: Hello everyone. Welcome back to our channel. Today we are with Ian from Misty, the inventor of Sphero, the aero robotic that we all love, which turned to be BB8 and BB9 and many other kool robots, and also the inventor of Misty a robot. And the robot is the heart. Thank you, Ian for joining us today.

Ian: Thanks.

Elad: And today we're going to talk mostly about the Misty and what this new kool robot has to offer for the education research community. "Hey Misty". We have a lot of like interesting in this product, especially from the research community because it's an open platform. It's really a robust and and you know, it looks like very, very open. But I would love to hear from you a little bit, you know, more about the background where Misty came from. Why this form factor, right? Why this name even? So if you can share a little bit about that, it'll be cool.

Ian: Yeah, I mean, the short story is it really came Adam Wilson, my cofounder at Spiro and I were out in LA in 2014 that's when he first heard about BBA, started working on that product. We had already started to go in a little bit to education with Sphero but we were just sitting, having a beer, actually had a brewery. They were working out there and started thinking about, you know, big ideas that we could maybe implement and then thought, why can't we make, you know, we just started working on BBA.

Why can't we make a real BBA or like a real RTDT, like a real robot and not, it doesn't have to look like BBA or look like RTB too, but the idea of a robot in our lives helping us with different tasks. And that was sort of the inspiration of this more advanced robot, this robot that could actually start to be useful things.

And we just thought a lot about like what things this robot can do. If it were, you know, we're making the product to do like document or a very specific task, and ended up landing on the platform idea as a way of utilizing the community, like thinking about phones and computers and it wasn't my phone, right? Most of the apps on my phone aren't written by Apple, Google if your Android, they're written by third parties who were experts in different use cases.

So we wanted to create this city as a platform. We wanted a robot that (was listening for her name she didn't beat next to me). We wanted a robot that wasn't scary that people would be accept in your workplace, your home, like not too big that it would be hard to develop for understanding that there is a lot of applications still for a smaller robot. And then to say that the set of capabilities that would allow it to actually do real things, voices interactions, navigation application, movement expressions with the personalities who can connect with people and provide all these sort of toolbox to developers, educators like whoever it is.

Elad: I'm starting to see as well, you know, factor.

Ian: It's kind of like the short story of like how we got to Misty.

Elad: Gotcha. I noticed that everyone with Misty referred to "It as a her". What's the reason behind this choice?

Ian: It can be whatever you want. There's a lot of the inspiration behind Misty, like as far as the design. They for a long time have like a required movie lists for people on my team, which is like all kinds of AI and like her and that's mocking. But also like Wally, chappy, short circuit, Johnny five. So a lot of inspiration for the design came from movies like like Wally with Vive, but like Baymax short circuit sort of that like factories and stuff. The important for like people in its work, like when we go to different events, you take Misty out in public with people to be over and like want to interact.

Elad: Yes. Yeah. We got, we got one of the early units in our office for testing and I experienced it myself. I mean everyone was like cool, what's that, how do you interact with it? And everyone started touching either and talking to her.

Ian: To your question. Like why, why is it a her like we just, we felt that like, like more of a feminine robot then it doesn't have, you know, people didn't really make, somebody could make Missy at him as well. It's not the more of like the feminine side would do more virtual, that's one reason.

Elad: Cool. so, you know, we you know, focus on the education and the research market and you know, being an open platform where you encourage people to develop for it. It's not a toy, right? You don't have a button to push. And you know, that's about it. I believe in grains a lot of flexibility to the space. Can you talk a little bit about, you know, the, the programming interface or no, how you see that you know, what can you do with this robot once you get it out of the box?

Ian: Yeah. We want like, we want to allow people to be as creative as possible around the product and in order to do that, if we wanted to make sure that it was super modifiable from the software perspective, like all the low level data you get out of sensors and stuff is available for people. Beause we never know how they might want to use that data. We might have attracted to a certain layer that we think makes sense that somebody might have an application where that broad data is important at all levels. And then even like the hardware side very modifiable from a hardware perspective as falling, just sort of standardized screws.

So you could even take Misty apart. You'd assembler for the most part, like most pieces are very easy to take apart a back together. We have a backpack. So Mitsy has this backpack on so if the back pops off the Camino version of this existing Arduino shields, so you can hook in other sensors or control motors. So we had some students, local university created a three D printed arm. Once screwed, the arm pops off, so they created their own arm who really saw the CAD files for Misty for all the shell.

So you can create attachments like they did. The arm can reach out and grab things and that was controlled from the backpack. We've done things like our chariot, it's like a four foot tall base that Misty sits on top and the chair is controlled through the backpack. For education. I mean it's, I think it's really cool that Misty is also like a real thing. Like it's not, it's not a toy.

Elad:  Yeah.

Ian: So if a student is working with, with Misty the things that are creating the actually like real things, like they might start with some fun things that maybe aren't like, it's not gonna like solve a real problem or something, but but they can extend that to actually solve real problems for elderly people or therapies or all kinds of different stuff, which I think is really cool. It's not like, you know, you're not like this kid students working with toy.

Elad: Yeah, yeah, absolutely. Absolutely. And so you mentioned that you can modify the hardware. So unlike, you know, I would say traditional or other robots that, if you open the shells and void the warranty or you're saying you actually encourage that you want them to hack it away.

Ian: I think it's crazy. Like I was on just like two days ago, it was on, Ifixit.com website by like parts, they'll start to find a part for like a piss for remote's broken. I was looking to see if they had it. And they, the thing popped up when I went to the website, it said is apparently like laws that are, might be passed that to prevent you from like repairing things. Like asking for my email, like to sign up for like thing like to support repairability and products. Just crazy.

Elad: Like, yeah, actually Apple got hit by that cause their battery thing. There is in California, I think the case they had the right to repair act or something like that.

Ian: Exactly. Yeah. So Misty like super expandable, lots of different attachment points. Like the backpack arms, it's just one screw. You take off really solid model. And the models that are, we also released the cad of the shells of actual Misty, but we also released the bunch of parts that could be 3D printed because it'd be 3D printed parts of it.

Like some parts like when you 3D print them, they expand or contract a little bit. So like your model might be perfectly sized, but we need 3D print, it doesn't fit. We released a bunch of leg blanks for a bunch of different patent attachment plates on Misty that are already resized. So like a hundred cheaper 3D printer, they come out well.

Elad: Yeah. Cool. Yeah. So this is on the, on the hardware side, one of the software side. How can you program which languages to support?

Ian: So, so Misty Misty connects to your wifi network and basically once she's on her network, just like every other device that's on your network, she has an IP address. And with that IP address, you can send what are called breasts commands, so you can send commands to this IP address. And what you send to that IP address can come from anything. So you can write in Python or or C Sharp or like whatever you want.

Elad: JavaScript

Ian: Yeah, yeah. So it's, it's basically any language out there that they can send commands to like an IP address or run a restful command. So that's sending from an external device. So if your program is running on your computer or your Chromebook or whatever it is, it could be anything on Misty, if you want to run a skill autonomously, Navaudits JavaScript or C sharp.

So those are the two languages that we support on Misty.

Elad: Got you. And we have some web tools. So basically you just go to SDK.mistyrobotics.com, there's a little box in the upper corner of where you put in your IP address and those tools you to play around with Misty. So right out of the box, like if you bought a brand new Misty, put her on your tablet, turn her on. You use an iPhone or Android app to connect her to your network. The app shows you with the IP address you put punched into that website and then that's it.

So from there you can play around with some of the different functionalities you can drive Misty around. You can train her on your face, like face recognition, create a map and navigate within that map. And you can play sounds. You can create different graphics for eyes or face. Less stuff around the personality side, so we've created a pack of sounds and a pack of eyes for different expressions, can also obviously create your own. And then you can create programs just in notepad or whatever on your computer. And on that same website you can just drag those files into running program. We spent a lot of time on our documentation cause that's very, very important for an educator. Developer documentation is, yeah.

Elad: Yeah. You know, if is that it's a belief I have to say and typically manufactures   don't follow up on the condition, they just run forward and later we'll do documentation but it never happens.

Ian: No. And it takes us, I mean, we have like a full time technical records as well and our staff is just really good at writing clear documentation. You know, having somebody that's really dedicated to that as important too.

Elad: Cool. So what's your saying basically, if I'm running something outside of Misty, I can use practically any language out there Java, C sharp or whatever, you know, even Ross or whatnot.

Ian: Yeah, I was going to say, like, I didn't say Ross. That's a very important one for education universities as well. So you don't run Ross on board on the Misty right now, but if you can run it as a record on your PC or whatever you're running your Ross on.

Elad: Yeah. So basically anything that you know, supports you know, basic PCP commands you know, and implement the protocol that you put in place, you know, can can work. There's no limit or no, you know, DLLs or whatnot to, to install, which is awesome. That's, that's a very fresh approach to programming. And yeah, it's really good. And running on the robot what you said that you support JavaScript and C Sharp, which hints that the operating system is windows based.

Ian: Yes. We have two operating systems running in Misty. So there's actually two separate processes. There's one processor that's more powerful, much more powerful that does all our slam mapping navigation like computer vision processing. And that's running Android. But our main processor that's running like user programs, if you say your JavaScript or C Sharp program to Misty that's on a separate processor that's running windows 10 IOT.

Unless you're in the C sharp side, you can get pretty, pretty deep into the hardware and mechanics of Misty on the JavaScript side. Like you wouldn't even know like what operating system is really running underneath cause you're basically just sending JavaScript commands. The reason we did that is what Misty to basically be accessible to any developer. And JavaScript is one of the most use known languages out there. There's lots of documentation on JavaScript. So you might have somebody, maybe a student who's built a website. Now they can program the verb for Missy.

Elad: Yes. Or hook their website too. That's even better.

Ian: Oh yeah.

Elad: Okay. Cool. What would you say is the most exciting feature for someone that doesn't know the robot from what you've see so far?

Ian: For me, it's the ability to connect Misty with all the other services online. So sort of that ability to connect to third party pies as well. So you have this physical robot in your space, but then, but then you connect it to Google's dialogue flow API APIs to do natural language processing. More like one of the first skills that I wrote for my Misty while back last year was I just wanted a hurted Bisley "Show Me The Weather" in kind of an acute way.

So in Photoshop I'd created like 20 different items for like smelling and windy, like all different kinds of stuff, and I just looked online for like weather API, and literally in like five minutes just using our own documentation to figure out how to do it was able to pull like current weather for where I'm at in Boulder, Colorado. And then based on the current weather, like display that via her eyes. So basically, taking the power of this physical robot in your space, but also connected to all the crazy API that you can find out in the world. I think it's really cool to me.

Elad: Yeah, absolutely. So I would say since the extendability that's, yeah, you can extend it as much as you want, goes on harder and softer.

Ian: Yeah.

Elad: That's a great feature. Are there any pre-requisites or if I'm totally new to robotics or you know, to programming in general, what do you need to know before starting to work with Misty?

Ian: There's a lot of different sort of having the avenues that you can go with Misty. So we, like, we have some people that have worked on like the Blockly port to program Misty just visually. Not all the commands are supported and it's sort of in the community.

Elad: But technically no experience, that's what you're saying, right?

Ian: Yeah. Up to, you know, some professors like the ones that do you in Denver they created the 3D printed arm, like they were more like creating physical attachment. So they had sort of a small multidisciplinary team, software engineers, mechanical people students that were working together to create this arm. You can go down the whole avenue of like socially assistive robot and studying interactions with kids personality and character design so that you know, that's using more of the eyes and the sounds and like, and maybe face recognition on Misty using that set of features to study that.

Of course there's all the mapping and navigation so you could have other types of interactions you're looking into. So there's like, there's just so many different sort of paths you can come at it. So you know, an educator that's maybe thinking about Misty you know, either just look online or even better like hit us up. We'd love to sort of have a conversation with you to talk about sort of like what your goals are that you think you might, you know, when you're trying to accomplish. And then we can sort of you know, hopefully not create the whole thing from scratch.

Because there's lots of people that have already created different things for Missy, so we can sort of connect you with somebody that has their direction. A lot of the skills that people have written are already like open source. So pointing to their libraries or that to sort of satisfying that goal.

Elad: Yeah. That's awesome. And I noticed that you have a pretty active community as well. So yeah, if it's not been answered by Misty, I meant the team that uses Misty to get them from the community, will be able to, to help really, really in a welcoming this in this word. So what's the, I would say, you know, typical age group that you know, can start working with Misty? If not by age group, what's the level that you expect people to have?

Ian: Yeah, I mean that's, that's a good question. It also kind of depends on what they're doing, right. Because there's a couple of customers of ours who are in the socially assistive robot space therapy using Misty as a device, teach social skills and things like that to kids with special needs. So their students can be quite young. I'm trying to remember.

Elad: I mean it's like they're not programming.

Ian: There were five, right. They're just, they're sort of interacting with it. Right. and, and Misty is that, is the teacher in that case if you're to be like how you have to be, to sort of actually sit down and start programming Misty. If you leave out like maybe the open source, like Blockly stuff, if you're more sitting down actually write code it's probably, it's probably 10, 12 and up.

But I would say the sweet spot for Misty is really older, high school kids into university and college. And beyond. Right? Yeah. I mean if you're, if you're just like doing block based programming you know, there's a lot of these devices like Sphero, stuff like that. But you're seeing this city is sort of that next step. Like you've maybe done some basic stuff, played around with some of the other robots out there and now you want to get a little bit more advanced or do something real. And that's where Misty comes in. So it's probably more like, you know, 16, 17, 18 definitely in university and college.

Elad: Yeah. So the one thing graduated from the sphero level of robots, then Misty is a great next step.

Ian: Exactly.

Elad: Gotcha. Nice. Well what's in the pipeline, what's coming up for Misty? What's the future, at each versions or features that you see coming up?

Ian: Yeah, so we just released a couple of different versions. So we, we have a lower end version of Misty basic that has basically we took out the structure sensor and Misty and Basically this lowered the cost. But that's useful for somebody that can, wants to do like voice interactions, maybe working on therapies and stuff. They don't need to map and navigate a space and it's more accessible for them. And then we released a pro version.

There's basically one of the processors inside. Misty has a lot more memory and that allows her to map and navigate a much larger space. And then the standard version is the version. It's the same as the version that we launched with last year. So we're working with more customers now. Some of those customers and moving further along developing their solutions. There's always new skills that are coming out into the community that you can, you can build off of starting from scratch. You can take an existing skill that somebody has posted and create something even cooler.

And then just from our end and on the backend side, just improving different functionalities of the functionality that's in there, like making, making mapping better and making voice interactions better, releasing new I commands to be able to do work match features, like abstracting some of that stuff out so that what took a hundred lines of code. Now I can take two lines of code. So we'll continue to do that. And then we're obviously, like on our side, we're building some of our own example code templates to build off of, to make it easier and always releasing the more of those as well.

Elad: Got you. That's great. Yeah, I can't wait to see what's coming up for her. Yeah, I mean, we had wanted to test that one of the first, I think robots up came off the production line and we're very, very impressed and I know that the customer that we already shipped to are very, very happy with that. So, more is welcoming and always better.

Ian: Yeah. Great.

Elad: Is there anything else you would like to share with our viewers here?

Ian: We'll just love for you guys to check out our community. If you go to our website, Mistyrobotics.com as a community link other than the menu bar, share ideas, things you're thinking about, obviously we would love for you to have a Misty, and be sharing skills and things like that with the community. But definitely check it out. And if you have any questions you know, it's up.

Elad: Yeah. And we are here to help as well. Great. Thank you. Yeah. I appreciate your time. Stay safe out there in this virus crazy thats been going on out there. And and thank you everyone for watching. Bye Misty.