This episode of Inform & Connect, an American Foundation for the Blind podcast, features Marty Schultz, co-founder and president of ObjectiveEd, an education technology company that provides digital curricula for children with disabilities.
ObjectiveEd’s mission is to provide students with disabilities, and the parents and professionals dedicated to their education, digital curriculum and integrated performance monitoring and reporting tools that will help the students achieve their best educational outcomes. As an example of their work, early on in the coronavirus pandemic, ObjectiveEd developed Braille AI Tutor – an innovative system to enable students to improve their braille literacy through a combination of speech recognition and engaging games. The system was specifically geared to facilitate learning braille from home or in a distance learning environment.
The conversation will center around the importance of education for blind students and the push for inclusion in the classroom.
“Marty's passion for exceptional educational outcomes for children with disabilities is infectious,” said Melody Goodspeed, AFB Major Gifts Specialist. “I look forward to delving into his current projects as well as his work in accessible gaming!”
Melody: Thank you so much, John. Yes, we are so excited. Education is really near and dear to my heart and I'm really excited to have our guests here today. We have not really talked about education at the school level and all the fun things, and the technology for children that are blind or vision impaired.
Melody: We have such a special guest with us today that's going to walk us through his adventure and who's just quite frankly, visionary. He's just a visionary. That's the best thing I can say. Where I have with us co-founder of Objective Ed and dear friend of the American Foundation for the Blind, Marty Schultz. Marty, it's so exciting to have you here today.
Marty: Well, thank you so much for inviting me. I'm looking forward to our conversation, so this will be great.
Melody: I know I'm really looking forward to it too. So let's go ahead and jump on it. Marty, can you tell us a little bit about Objective Ed and yourself?
Marty: Sure. So Objective Ed is about two and a half years old. I'll kind of get into the backstory a little later on. But our mission is to provide a software and a system to improve the educational outcome for children with disabilities, starting with kids who are blind or visually impaired.
Marty: So we started this back in around 2018 and raised some private money for the company. Then based on the suggestions of a lot of different TBIs and O and Ms, started building out a set of curriculum where the students could play games as they develop different types of skills.
Marty: As an example of a game, we have a game called Barnyard where a student is asked to drag in an animal based on either left, right, or swapped directions or compass directions. They do that and they're able to practice these types of simple orientation, mobility skills. Then what that ends up doing is that at their next lesson with their teacher, they're able to move more quickly through the lessons.
Marty: What we noticed was that a lot of kids, they have their sessions with their teachers and then they fail to practice at home, and because the practice is boring, they don't do anything. We said, well, if we build out some iPad or iPhone based games to let them practice and have them learn, then they'd actually move ahead. So then the teacher does not have to repeat the lesson the next time she [inaudible 00:02:26]... at the next session with the child.
Marty: We tested it out a bunch with one or two games. It was amazingly successful. I think there was one teacher in Missouri that was using our system in the last quarter of last year. And she said her student made more progress in those three months than they did all the prior year. So you take the idea that kids like playing video games and they have these skills that are part of the expanded core curriculum that they need to learn. And by combining the two, we're able to teach those skills.
Marty: Now we're not just games in Objective Ed, we're really a matter of skill based learning. So the way the system actually starts working is the teacher, using a web-based dashboard, assigns what skills she wants the student to work on that week. And these could be skills in [inaudible 00:03:17] technology, O and M, braille related skills, there's a wide variety. So she'll pick a set of skills on the web dashboard. And then the students will use the games that are associated with those skills. And the teacher could set the kids at the very beginning of the skill, like, well [inaudible 00:03:34], with the Barnyard game as an example, left or right. Or could move into very advanced things like all the different clock directions from 12, one, all the way, two, three, four up to 12 again.
Marty: And what they have to do when they're playing this game, it'll say there is a penguin at three o'clock to your finger. And then you move your finger along screen heading to the right until you get to the penguin. And then it'll say, "Okay, now drag the penguin to the North fence or something like that." And then we can kind of drag it towards the top of the screen. So the kid plays a game on the iPad and the iPhone, the teacher configures [inaudible 00:04:16] web dashboard. And then since they're not at the same location, the teachers at home or at school or the kid's at home playing these games, the next time a teacher logs in, she can actually see the progress that this child has made. And she'd say, "Okay, the kid is doing real well on simplified directions. Let's move to advanced [inaudible 00:04:34] directions."
Marty: She can see charts up there. Matter of fact, she can take a lot of the information we'd show up on a web dashboard and copy that into the child's IEP progress reports. So we've kind of understand the whole special ed process from the assigning of the skill, the playing of the game to build that skill, to the reporting of the skill, to seed that back into the IEP, kind of in this closed loop process. And teachers tell us this is a really good way for children to build out these skills. So in essence, Objective Ed is providing reinforcement learning for students anywhere from pre-K on up through transition age.
Melody: That is so awesome. Because people don't really think about children that are blind or vision impaired or anybody in that situation playing video games, like that's just... And you've brought it to the education. That is so... That's really incredible. When you... Speaking of the teacher being and she logs on, I understand COVID must've really put some challenges to a lot... to all the children really that... and to the education world. Can you kind of tell us how you navigated that?
Marty: Yeah. So when we were actually just starting to pilot our software on system with the school districts across the world in January and February of last year. We spent about a year and a half building all of this out and then COVID hit. And clearly that was a big problem and everything we were reading was indicating that students with disabilities were really not able to access any of their remote or distance learning. We thought, "Well, what should we do to help that situation?" Well, Objective Ed system, as a reinforcement system could really be used in that way. So we thought, "Wouldn't it be cool if with the teacher who can't go into the classroom, she's back at home, could assign lessons for her students and the students who can't go into the classroom could play these. So at least the child doesn't regress over the next four to five months, but they could make progress."
Marty: And there was one woman, a teacher out of, I think, Auburn, Washington. And we have this testimonial actually up on our website and she was previously a VRT, mainly working with adults. And the local school district contacted and said, "Would you like to work with kids? We'll be working four or five, six, seven years old." And this was like in February. She said, "Okay." Then COVID hits. And she thought, "Oh, well I guess I'm not going to be doing that." But she had heard about Objective Ed. She downloads it to the tested out. At the same time, this is when we announced to the world that any school district could use our system for free and in that spring 2020 semester. So she downloads Objective Ed. Starts playing with it and realizes, this will help me teach the kids even though I can't see them face to face because mostly the kids have an iPad or an iPhone at home.
Marty: Well she starts contacting the parents and she's talking to one parent and that parent was completely burned out because here she was at home running, in essence, a one room school house with three different kids. One of the kids was visually impaired. And [Beth Sutton 00:07:54] was telling the mom, "Hey, don't worry. Your kid can play these games without your help." And the mom said, "Well, what do you mean?" Well, she said, "Well, the game will teach assistant technology gestures. It'll teach them orientation mobility to your student... to your child. And all you have to do is give them the iPad and they're completely on their own. All the games are fully accessible and we'll be able to work together to make sure the child will make progress." And Beth said, the relief on this woman's face was palpable because now she had something for this other child to keep making progress, not really regress over what is now the past almost a year. That experience Beth Sutton had is the same experience we've heard from school districts all over the world.
Marty: There was, I think the Los Angeles County school system brought us in and a lot of the teachers started using it and realize that this is great. I mean, it's the second largest school system in the United States. And they were able to use it to really help them moving the kids ahead. And we heard this time and time again. So in the end, I think we've heard from about 3,000 different school and teachers. They're all using it for the extended school year in August. Many of them came back and actually started subscribing to the system, starting in September to use it with their kids because they found out not only did it help with the hybrid and distance learning, but it met its original goal, which was to be reinforcement learning through games between sessions with their [inaudible 00:09:25] teacher. So we're really happy.
Melody: Yeah. That's exciting. I mean, in providing that, especially in these times, it's just it's really crazy. And I really thank you so much for sharing that story with us because we've all had to pivot, right? We've pivoted many multiple dances this year. One of the things that I wanted to talk to you about, you do so much in that space. But can be talk about some of the grants and projects in particular with braille.
Marty: Sure. So we've got a couple of different things in braille, but the first thing we did was we were able to take a sheet of braille, put it on an iPad and then build some technology so the iPad knows what's on that sheet of braille. And we call that one Braille Sheets. And for that project, we won the national braille [inaudible 00:10:10], Louis Braille touch of genius award. And a lot of teachers are using it sometimes with the APH patterns curriculum, sometimes with their own things where they can create their own sheet braille story, put it on the iPad, the child plays a game around that braille story and then improves it.
Marty: So for example, let's say you have a girl 10, 11 years old, visually impaired, and she loves horses and she's getting really bored practicing her braille reading. And she's early on in learning some of the contractions like AR. So the teacher writes a story about a day in the life of a horse and she prints it out on the braille [inaudible 00:10:54] sheet. She puts it on. She gives it the sheet of braille to the girl with the iPad. The girl first reads through the story once to learn about the horse and the day of the life of the horse.
Marty: And then as part of braille sheets, we let the teacher create a bunch of questions. And one of the questions the teacher for this girl writes is where does the horse sleep at night? So the girl reads through the story a second time, gets the word barn in it's contracted form and double taps the screen and wins points. But when we did that and when we got the AFB... The touch of genius award, it was given to us at the CSUN conference out in LA a couple of years ago. And I was sitting down with someone from Microsoft and [inaudible 00:11:39]. And the Microsoft person said how come you don't apply for the Microsoft art AI for accessibility grant.
Marty: And I said, "Well, what is that?" She said, "Well, if you can think of a way to use AI as part of the tools you're creating, then you might want a grant us to actually build that out." So Kirk and I and the Microsoft person started brainstorming about what to build. So I happened to ask her when you were young, how did you learn braille and how did you practice? And she said, "Well, I'd usually have an instructor next to me, as I was reading the sheet of braille or the braille display. And they would verify that what I'm speaking as I read is the same thing that was on the braille display." I said, "Well, we can use speech recognition to do that."
Marty: So we started brainstorming a little longer. And then we came out with what's called Braille AI tutor where it sends a sentence at a time to the braille display. The child reads, orally, that sentence that's on the braille display, we pick up that speech that they spoke, convert it back into text using Microsoft speech recognition technology. Compare the original sentence to the text sentence, if it's the same, that means a child decoded and read correctly. And if it's different, we know what word they got wrong and can take remedial actions. That led us to be then going after another grant where we were talking with the AFB, as well as the National Commission for the Blind and Florida Commission for Blind Services and entities like that.
Marty: And they said, "One of the big problems we have is how do we get kids who are transition age to acquire all the skills they need to get a job. They don't want to practice. They only have these sessions. There are not enough teachers to teach the kids." And we started bouncing ideas about, and we thought, "Why don't we use the concept of interactive fiction to help do this?" So interactive fiction is like a choose your own adventure game, where the computer will say something. And then it'll ask you a bunch of questions... or it'll ask you one question about how you want to respond to that situation. And then the story takes another path. And this continues with a narration, question, answer, narration, question, answer. And you go through. And what we created was a five chapter interactive fiction game for students to practice their preemployment transition skills.
Marty: And we won a grant from Midler, which I know AFB is associated with them, to build out chapter one, which is hearing that there's a job available. Chapter two is putting together your resume and applying for the job. Chapter three is going for a phone interview. Chapter four is having a face-to-face interview. And then chapter five is your first day on the job. And we're actually going to be... We're just about done on that grant with the story writing. And the next step is to test it with a bunch of visually impaired adults to make sure they can get from one end of the store to the other and they like it and it's fun. And the next step will be testing with a dozen or two teens. All of them are in preemployment transition training, just for them to be able to use this, to see if they can then try out different strategies over to self-advocacy, and problem solving, and dealing with difficult issues.
Marty: And the cool part of interactive fiction is it lets somebody experiment with it with results without any penalty of failure. From that, we've been working with a couple of other researchers, Karen Wolf and Sharon Sachs, to apply that same concept of interactive fiction to teaching social skills to kids anywhere from like third grade up through high school. And we have that grant coming in. Hopefully we'll get that grant as well. So we're doing a lot of things. And then Microsoft invited us back because we did such a good job building Braille AI Tutor to come up with another grant that uses AI. And we're looking at things how we can help kids, either who are low vision or who are blind, improve their reading. And we're talking to book Bookshare as well in what we can build there.
Melody: Marty, this is so excellent. I'm just... I love how you've thought about the child... The kindergarten all the way up to... You've been through every step all the way up to thinking about employment, which is so huge. But one thing I want to point out to the audience, this has gone by so fast and we're already at the time for question and answer, but I just want everyone to know, Marty, can you tell us what your experience with with blindness? Did you know anyone prior to your journey into this world?
Marty: No, actually this was... I just stumbled into this. So I was kind of [inaudible 00:16:20] at time and the full backstory that happened was my daughter, who is 12 at the time, she's not visually impaired, she was working on her birthday wishlist. And everyday she [inaudible 00:16:32] the month of her birthday, she would write up a new birthday wishlist, rip it up the next day and write out a new one. She was doing this day after day of day. And I thought, "Well, there should be an app for that." So I figured I could either create the app and then focus [inaudible 00:16:43] her other preteen friends, or make it a STEM learning opportunity for the kids at her school. Contacted the head of school and said, "I want to run an afterschool club for six weeks. I'll meet with the kids three times a week for an hour together. We'll design the app and then I'll build it in the evenings."
Marty: So we have to get fingerprinted and background checked and all that. And we run the app club for about six weeks, finished the app, this birthday wishlist app. At the end, the head of school comes back to me and said, "Marty, can you run the app club again? The kids loved it. The parents loved it. Also, if you drop your daughter off in the morning, can you teach them middle schoolers how to program?" So I said, "Sure, I'll be an engaged parent." When we met with the app club a second time the kids said, "We don't want to build some stupid app. We want to build a game." I said, "Well, if I'm going to put my programming time into building a game. It has to be really different than any of the games in the app store. So go off and think of something unique."
Marty: They come back two weeks late and every idea they had was a clone of something else in the app store. So I'm simply holding an iPad in my hand and twirling it around and thinking, "Well, why don't we do something that doesn't need the screen? Because every in the app store uses a screen. And I happened to be moving in almost like I was moving a steering wheel. I said, "Well, why don't we create a driving game for blind people?" So the kids look at me like, "What do you mean?" I said, "Well, they probably would drive with their ears." [inaudible 00:18:04] That's how they navigate.
Marty: So they didn't really get it. So I took one of the girls, I said, "You stand in the middle of the room and act like a cow." So she goes in the middle of the room and starts mooing. Then I took one of the boys and said, "Put on a blindfold. I want you to walk up to the cow, not touch the cow and get to the other side of the room." And he does that. And I said, "Well, that's probably how blind people get around." So we designed the game where you're driving, steering the car with your iPad by twisting the steering wheel, the iPad, left and right. And then wearing headphones, if the music gets too loud on your left... If you're moving too far to the left fence, that means it's because the music getting louder in your left ear. If it's getting too loud in your right ear, it's because you're too close to the right fence. So you try to stay centered, keep the music centered in your head. And then you aim for noisy prizes like popping popcorn and avoid obstacles, like barking dogs.
Marty: When we built out the 60 level game, it was an absolute hit. I mean, in fact, I first tested it at the Miami Lighthouse of the Blind. And one of the students there had said, "Is the screen dark when you're playing?" I said, "Yeah." He said, "Well, you better put something on the screen for sighted people who think their iPad is broken." We took his advice. We put some things on the screen for sighted people when they play it. And then I tested it a few weeks later in Fort Lauderdale at the lighthouse. And there was one girl who had been blind since birth and she was playing the game. And after about 45 minutes, I said, "Well, what do you think?" And she said, "I'm so going to beat the butt of my sighted friends."
Marty: So we figured it would probably have a hit there. It jumped to the top of the accessibility list in the Apple app store. I start hearing from blind people all over the world asking me to create more and more games. And over the period of about five years, I ended up creating about 80 accessible audio games, anywhere from card games like UNO or Crazy Eights and Rummy and Hearts to sports games like basketball and bowling and baseball to action games like Flappy Bird or Color Crush, or games like Wheel of Fortune and Jeopardy. So an innumerable number of games in the last...
Marty: In like 2017 and '18, I started hearing from a lot of teachers telling us they were using the games to teach orientation mobility skills for the kids. That's when I started to get to know a lot of the teachers. And that's when we saw, "Hey, there's an opportunity out there for us to make a real difference with these kids," because visually impaired kids, by the time they're in school, they're already significantly behind their peers because they can not model their parents or their siblings. And if we can produce games that can help them solve that then when we're helping out. So, that kind of led right into Objective Ed.
Melody: Wow, Marty, that is that incredible. Just you're incredible. And I'm very in awe right now, so exciting. And I'm sure our audience is too. And I thank you so much for being here and we'll get to how people can find out more about you towards the end, but I'm sure that we have, John, I'm sure we have some questions for Marty.
John: We do.
Melody: I just, I can't... I'm going to go do some games soon here in a minute.
John: Go fire up the games. Thank you for that. Melody. Marty, thank you for being here. So first question, in changing up your focus on serving blind or low vision students specifically during COVID, will any of the way you approach curriculum change moving forward?
Marty: Not really because our whole idea is that kids, if they're engaged in the material, they will use whatever it is that play the game and then build the skill. They might not realize they're building the skill by playing the game, but they do. And I think we're going to see more of a shift in education in general in teachers will do in class what they're good at, which is helping kids move forward. But a lot of the bulk work might be done on a computer in a very engaging way. I think gamification is going to move through a lot of education. I think hybrid concepts will move through education.
Marty: And we're really an implementation of that, which is hybrid and reinforcement learning. So really kids can have the curriculum tailored to their benefit. One of the difference between a student who is receiving special education and one who is in general ed, is that in special education, you can really tailor the curriculum to the needs of the child and computers make that very easy to do. And we're doing that with all our technologies. I think that actual individual educational approach we'll move into general ed in the next five years to ten years because computers make that realistic.
Melody: That's incredible.
John: Here's another, how do you see technology opening the door to inclusion?
Marty: I think technology can make things... Make available for people with disabilities, much more effective. I think that the invention of the smartphone has drastically changed the lives of people with vision impairments in that they can optionally ask for help from somebody, but there is so much technology built into a smartphone now between bus schedule to getting to the bus or calling an Uber or having GPS or any innumerable things [inaudible 00:23:28]. Just an amazing number of things that previously required assistance from people, now, people can be a lot more self-sufficient and I see that trend continuing.
John: Note to developers make it accessible out of the box, right?
Melody: Yes.
Marty: It's amazing. That if you don't build in inclusion and accessibility from the ground up, it's going to... and it's a built on, it's not going to be accepted within the community of people who want to use it. All of our apps, whether they were blind [inaudible 00:24:00] games, apps, or the Objective Ed apps are built with the audience in mind, knowing that people who don't have a vision impairment or don't have other impairments can still use it because it's equally accessible to all populations. People who have an impairment and people who don't. But if you don't design [inaudible 00:24:23] way, you can't simply add it later on.
John: Like at the ground floor. Marty, what has been your most exciting project, most exciting to you anyway?
Marty: I liked the opportunity to work with really smart people who were both, within the community and who research the community, to come up with innovative ideas that we can find the funding for it to pull it off. The transition adventure, in that choose your own adventure game, the employment skills, we have such a great team of people working on it. Our research is out of University of Mississippi who really understand what's needed. I have teachers who teach this. I have Brian and Kim Charleston who are excellent advocates and have been, are at the top of their fields. But the chance to work with all these amazing people, and then combining their efforts with our software engineers and our [inaudible 00:25:25], to be able to do something that's life changing for people and for kids. It's just so much fun.
John: Thank you for that.
Melody: That's really awesome because you're talking about not only like the story, you talked about the little girl playing the game for 45 minutes and being able to have a conversation with her sighted peers. I mean, right? That's kind of the name of the game.
Marty: Well, one of the things that is a side story is we actually won Objective Ed. I went to one of the AER conferences and just ask teachers if they were familiar with blindfold games and do they use it with any of their visually impaired students? And I learned three things at that meeting. The first was that the games are great. We [inaudible 00:26:08] with the students, which we would expect. Secondly, that the games can teach orientation mobility skills. But the third thing that struck me that I never thought of was, they said, all your games are audio meetings. So since our visually impaired students are excellent at audio games, it moves the students from being a follower into a leader because they have to explain to their sighted peers how to play these games and how to excel at these games. So it was a new social environment for these students. And I thought that was pretty neat.
Melody: That is incredible. It's like flipping the switch. And that is so true. We've been talking about that a lot lately in conversations is having that leadership role, especially with our blind leaders program. But I also love the fact that you brought up at the very beginning that you're at the CSUN conference with our president and CEO and having these conversations and just the camaraderie and partnerships that really do occur to make these things happen because we just can't do any everything on our own. Right? And I just love that. And we're so glad to have you here today. Do we have any more questions, John?
John: I think we have time for one more. I do have one more sitting here. In case it wasn't already clear about how entrepreneurial Marty is, marty, you authored a book, the book, No Investors? No Problem!: A Serial Bootstrapper's Playbook for Breakthrough Success on a Shoestring Budget. Tell us a little bit about your book.
Melody: Yes, please do.
Marty: Well for while I was going to do mentoring of young entrepreneurs and start speaking on that, this is before we kicked off Objective Ed. And I was starting to hit the talk show circuit. So I was speaking, basically doing a lot of speaking. People said, you should write a book on your stories. So I've worked with somebody to put together the book. I've done about six or seven ventures over the course of my career. And every one of them, I've kind of bootstrapped to get going and learn all the different guerrilla tactics and what's necessary to kick off a company.
Marty: And basically everything I learned was by trying everything else that failed until I finally got something that worked. And in each company, you try something, you try a dozen different things. It's like throwing spaghetti on the ceiling. Some of it will stick in some won't. Most won't and you proceed with that and you continue. And I wrote that book with someone and we published it about six months, almost a year ago now. And if anyone's interested in getting a copy just to have them email me and I'll send them a copy on Kindle. It's... You can listen to it through Kindle voice speech.
Melody: What is your email? Well, is that [inaudible 00:28:51] How can people contact you, Marty?
Marty: It's easiest marty@objectiveed.com.
Melody: Nice. And we didn't because they just go... If they wanted to check out all of your fun things and what you're doing, where could they... Your website.
Marty: But yeah, they can visit objectiveed.com or they can visit blindfoldgames.org, which has the link to objectiveed.com.
Melody: Awesome. Well, thank you so much. I would like to go read that book. It sounds like a really good one. And thank you so much for being here with us today, Marty. Really appreciate it. And we also really do appreciate your support. We are in our Centennial year here at AFB and we have kicked it off. We have an amazing event coming up. It's this on February 11th. So tomorrow everybody. It's 7:00. If you want to check it out, just go to afb.org/100 and sign up. You can get your recipe from Christina Ha, get a good playlist from Marcus Robbits, and some wine because we all need that now. And make your own very much beautiful Valentine's day night. So thank you guys so much for being here with us today. And I just want to say thank you to Marty and Objective Ed for advertising that for us as well. And thank you, Marty, for sharing this and we look forward to moving forward and keeping this partnership going. Thank you so much for everything you do from behalf of AFB. And I know teachers and parents across the world.
Marty: Oh, thank you. And again, thanks for the opportunity to speak to your listeners.
Melody: Thank you. This has been so much fun. I hope everyone has a great Wednesday. Thank you so much.
Marty: Bye-bye.
Melody: Bye.
