ed: hello, everyone. good evening. i'd like to welcomeyou to the - (laughter) the mit club of california. tonight, we have an excellent,distinguished speaker, but before we introducehim, i hope you have enjoyed the sandwiches and cookies. we learned to make those ourselves by watching one ofsal's videos. (laughter)
just kidding, they'refrom subway. (laughter) let me read a bio for sal. sal is a social entrepreneur whohas become an internet sensation and inspiration for his fun and highly effectiveeducational videos on youtube. every month, more than 100,000 students, both young and old, across the globe, use his over 1,300 free videosto learn about arithmetics, finance, chemistry,(unintelligible) calculus.
he founded the khan academy,which was recently covered on cnn, pbs news, npr, san franciscochronicle, and nbc nightline. his mission is nothingshort of creating a free and world class virtual schoolfor anyone in the world. he will speak about hisexperience starting the academy and massively transformationalchanges in learning and teaching. (unintelligible) in thenext five to ten years. sal graduated from mit in 1998 with computer science and math degrees
and received his mba fromharvard business school. he left his day job as ahedge fund analyst in 2009 to dedicated full time tohis non-profit academy. we're delighted and honored to havethis distinguished speaker tonight, an emerging giant in theonline education space, a world-class educator,a dynamic communicator, and a selfless good-doer,salman khan. (applause) sal: can you all hear me? i'd prefer to just becloser to you all, than ...
can you hear me in the back? okay, cool. just like ed said, i'm sal,the founder and faculty of the khan academy. (laughter) this is - i really want to make this - i think this will be a fun conversation. i don't want this to be me at thepowerpoint for the whole hour. i really want you all tointerrupt me or raise your hand,
ask me questions, and we cantake this wherever we want to go, because there's really a couple ofdifferent conversations we can have and maybe we can have them all. how many of you all havewatched a khan academy video? okay, that's a pretty good cover. just so you know what it is, and actually, maybe to get a sense, of the growth. ed said 100,000 and when hefirst asked me it was 100,000, now it's 200,000 unique students amonth and it's only been two months.
it is the fastest growingopen course effort. now, it's actually passed up - i'm not - well, i'm a little competitive. it passed up mit recently,in terms of (laughter) being the most used. it's getting about 60 or70,000 video views a day. it has closed to 16million views on youtube. one of the things we cantalk about is maybe why. it might be something about thecontent, how it's delivered and whatnot.
we can talk about all of that. this is the site, just for thoseof you all who have never seen it. we can also debate myuser interface skills for just putting all 1,400links (laughter) on one page. those are the videos and ifyou click on a random video, if you want a review of(unintelligible) you can click there. we can let it load up or whatever. that's the khan academy. what i want to do - (unintelligible)
is there sound? the sound isn't any different thanwhat you're hearing right now. (laughter) you see this and youhear my voice and you don't see me. this'll give you a flavorfor the form factor. that could be a conversation we could have about why this is an interestingform factor and why i did it and all of that type of thing. what i'm going to focus on,because i think most people are familiar with the video library.
it has this reach, it's gotten popular, it's growing 10 or 15%a month at this point. i want to talk about the bigger vision. ed has talked about this idea ofstarting a virtual school for the world and obviously, videos are a start. they're the lecture part of a school, but a real school's got tobe more than just video. you have to have someinteraction with other people, you have to have practice,you have to have assessment,
you have to have dataand all of these things. that's what i want to talk aboutand i'm generally open to ideas, because this is a work in progress. some of it is starting to be built. hopefully over the next few monthsit'll be built in a much bigger way. some volunteers have come outof the woodwork very recently to supercharge some of these efforts. it's exciting times. let me get the presentation up.
a good number of you all saw the videos. how many of you all saw theweb application on the site? okay, that's a muchlower penetration rate. just so you all know alittle bit of the history, and i'm happy to go intothis as much as you want, but i started doingthis remotely tutoring, because i was in boston, mycousins were in new orleans, and she was having troublewith units and i said, "hey, after your school and mywork let's get together virtually.
"we'll use yahoo doodle,it'll be a shared notepad "that we can each see whateach other is writing." i got her a little wacom pentablet, i had one for myself. we'd just do conference callsand that's how i started. then she started doing really well, then i started tutoring herbrothers, other cousins. then a couple of things happened and i usually go straight to the videos, because that's what it's known for,but being a computer science geek,
i didn't even think ofdoing the videos at first. i said, "hey, you know what,i'm going to write soft - i would give them these little lectures and i would just point themto random websites, like, "hey, there's a cool appleton that website over there. "why don't you try it out and tellme tomorrow whether you understood, "whether you're able todo those exercises, or usethat website over there," i just got really frustrated, thispiecemeal website here, website there, so i said, "i'm going towrite a couple of javascript,
"little things that'llgenerate problems for them." it's not just ten problems and itmay come up with some random numbers and give them as manyproblems as they need. i made a bunch of these modules. i'll show you what the modules look like. this is like a multiplying decimals module and we can go into as muchdepth on these as you want. i started making these littlejavascript modules for them and it was starting really whereshe was, at units and basic algebra
and adding and subtractingnegative numbers, and basic linearequations and then i said, "gee, you know, i don't trust - i'd say, "hey, nadia,did you do the problems? "how many did you do?" and she's like, "oh yeah, i didthem (unintelligible) (laughter) i was like, "i'm going to puta database behind this thing." i put a database behindit so i could keep track of when she did the problem,how long was it taking her,
how many times was she doing, how frequently was she gettingone right or one wrong. then i started making them up. on that first slide right there,each of these circles is a module, and literally, the first module - this was a very organic process. it was really just me workingwith my cousins and iterating, saying it would be nice to have the data. then, i started makingmodules here one off
and i would at firstjust point them to it. then after a while, i was like,there's some kind of dependency here, because nadia got that, butshe needs help with that, so to fill in that gap,let me make another module. then, after a while, i got tiredof having to assign the modules, so i said, "well, it shouldautomatically assign the modules, "because once she getsten in a row in this, "she's ready for this, this, and this." so, i created this internaldependency in the little app
to do this and i said, "well,this is kind of a neat graph." this is what i was using justto keep track of the modules, i should just expose this to the students, so they know what's going on. this is the app and this startedwell before i made the first video, but it's kind of like thesecret of the khan academy, so you're getting a little tidbit. but i think it's also thefuture on a lot of levels. so this right here, this isliterally level one addition,
so it's literally 1+1. if you go to the website,there is literally a video on 1+1 and at first, it's almosta joke that i made the video on adding 1+1, but thewhole idea behind it, besides being cute, isto give people the notion that no matter how basic yourskills, you can go to this site and you will find somethingthat fits your need and the idea that you can, whenyou see 1+1 on the same page as vector calculus, you're like,
"gee, i just have to step through thatthing and i can get to vector calculus. "there's not some mysterious - (laughter) "and it's the same dude who's goingto take me through the - (laughter) "my kindergarten teacher isteaching me vector calculus." (laughter) you get ten in a row and - once this was made, thenit started to make - this became what a lot ofpeople in education talk about, differentiated learning,with data and all of that, but it made me pause and say,"this is working for my cousins,"
but there was actuallyone piece that was missing and that was the - i still had to do - when i first started it, i thought - it randomly generated these problems and picked random numbers hereand then when you clicked hint, it would show you the exactsteps for that problem. i was like, "oh, that's all youneed to learn how to do a problem. "you just need the exactsteps for that exact problem.
"this'll work for my cousins." but it didn't. it would help them if they kind ofknew how to multiply decimals already, but it didn't get them over that hurdle. i still had to do that one-on-onesession with them remotely. that became hard to scale. literally, another buddyfrom mit, we were at dinner and he's like, "why don'tyou make youtube videos? "the thing that you do with your cousins,
"why don't you justrecord that on youtube?" i was completely dismissive of the idea. i was like, "oh, that's sosilly, i have this software. "it generates problemsdynamically, (laughter) "it's got hints, what's ayoutube video going to do?" he's like, "yeah, you're right,it's probably a stupid idea." (laughter) i went home and i was like, "well, maybe there's something to it." literally, when i kept recordingvideos with my cousins,
i would do the tenth lecture onnegative numbers with a new cousin, i was like, "yeah, maybe ayoutube video will make sense." i made the first youtube videoon greatest common divisor and this and that and then - i've said this in talksbefore, the immediate feedback i got from my cousins was that, "you are better on youtubethan you are live." (laughter) the point, i think, that hewas trying to communicate is it had none of the pressure ofa traditional tutoring session.
"did you get this?" andyou're staring at the person. they're like, "yeah,maybe i did," (laughter) and they're giving you bad information,because they didn't get it. they feel bad pausing or repeating me, but they can do it all theywant with a youtube video, or they can go to wikipedia, or theymight be afraid of even admitting that they don't know how toborrow or regroup or whatever, but now they can actually reviewit in these one off videos. then other people startedwatching it, as well, and said,
"hey, there's somethingthere," and on top of that, i incorporated with this,because that was the whole point in the first place, to have if you'venever seen this concept before, you at least have thisoutlet, these youtube videos that you could somehow access, butbefore i go into the depths of this, i want to - once this was built, then i hadthis, "this seems to be working, "but let's compare it to what goeson right now in the school system." if you think, the whole conversationabout education right now,
it's all about the student-teacher ratios, how can we make theinner city, poor schools look a lot like the fancy, richschools, give people computers, but no one is fundamentallyquestioning the model. if you really thinkabout the model itself, i don't want to taintyour opinion before - it's bizarre. you have this situation,you go to the classroom and you're all in a room together -
we'll talk about what you could doif you're all in a room together, but there's no interaction. it's just this broadcastlecture, especially once you get to the university level, but evenin the classroom in middle school. there might be a littleone question here or there, but fundamentally, it's abroadcast lecture to the room. the teacher is pretty muchguaranteed to lose half the students, or bore half the students,and at some point, they're going to switch the whole time.
you do that, then the student hasto go home, do their homework. they do it in a complete vacuum. they have a textbook, every other problem there's a solution inthe back, you don't know why you got it right or wrong, anything. you don't have anyone to help you. if you're lucky you might get a tutor or a parent might be able to help you. then you go to the class the next day
and there might be sometype of review of homework or something, but really no real feedback and the cycle repeats itself. you get another lecture,then you go back home, you do homework in avacuum, and this whole time you don't know what you're doing at all. the teacher has no data, nothing. that occurs for a couple of days and then, bam, you have an assessment.
the assessment, some studentsget 50%, 80%, 70%, 90%. the good students get 95%, next concept. even if you fail anexam, the class moves on. you don't know half of thesubject, half of the material in that previous concept andthe class is going to move on. you might say that failing student's bad and the a students are all right, but they still didn'tknow 5% of the concept. maybe it was a careless mistake,but maybe they didn't know
what happens when you takesomething to the zeroth power. maybe they didn't know exactlywhat a fractional exponent was, that one problem on the examwith the fractional exponent. that a student, the one who'sgetting the positive label, when they go to calculusand they see that, what are they going to do? they're not going to get 95%,they're going to get 90%, because they're going to missthat and another concept. that's not even to mention about the 50%.
you have this bizarre - the first time you getassessment is when it's time to move to the next concept. in no way is it actionable. if you really think about the process, it's just this huge filtering mechanism. it's just this processwhere every step of the way we're just knockingmore and more people off and the lucky people in thisroom, we were the people
who happened to be luckyenough that we were assessed and filtered at the rightplace at the right time and that led to us feeling reallysmart and going on. (laughter) that's the way it works. it makes you really question - we're all lucky. (laughter) that's the big takeaway. what's happening here, whati think is happening here, is that whole cycle, it'sall happening instantaneously
with every interaction. the instruction, if you've never seen multiplying decimals before inyour life, you click "watch video", it pumps in from youtube,and you can imagine there's one or two videos right now,but you eventually have ten videos, you're tracking whether theperson watched the video, you can eventually trackwhether they paused or repeated, where they watched, and i'lltalk more about that later on, how you can perfect the content.
they get the instruction there,then they can do the practice, so you're giving them practiceand you get immediate feedback. did i get it right or wrong? if you don't know how to do it, youget the exact steps for that problem and then you're also getting assessment, because every time you do a problem, it's tracking everything about that. when you did it, how long it took you, how many times did you click "hint",
and then the paradigm here, and iliterally did this just on a whim, i was just like, "how do i know ifsomeone really knows something?" it's not 90%, or it's not 95%, just keepdoing it until you get ten in a row. ten in a row should be the paradigm, because then that will enforcea certain level of discipline. people say, "oh isn't that - no, when you take the sat, you can't make those little careless mistakes. you should enforce acertain level of precision.
i just threw out ten in a row. it doesn't have to be ten in arow, it could be five in a row, or whatever, depending on the concept, but if you think about what'shappening when you enforce it is you don't allow that95% student to move on while not knowing that 5% and thenonly get lost in the next topic. that's the general idea. this right here is the - this is just another example of a module.
this is slope of a line. if you don't know how to do, just do"hint" and it draws things for you. it's just another indication of - all of this is live. you can go right now tokhanexercises.appspot.com and you can try out all ofthis stuff and i encourage you if you have friends or family members. there was even a guy at thehedge fund i worked with who was going to take the gmatand he was summa cum laude
from yale and he's like,"what should i use for math?" "you should use this." he was like, "hey, but thisis starting me at 1+1." i said, "just do it." two weeks later, he was like,"that was good." (laughter) it's amazing how long many peoplehaven't multiplied decimals. (laughter) that's the self-pacedstudent view of the world. i still haven't given youthe whole virtual school. right now we're talking about videos
and then we've kind of automatedthe self-directed learning and learning at your ownpace and getting assessed, but then you have to think abouthow can you incorporate that with the human experience. you're coming into a roomtogether to essentially learn how can you use this technologyto hyper charge that? actually, before i even talk about this, there's actually a very simpleflipping of the classroom that i think should occur tomorrow.
you don't need any of this software,because the videos already exist and some teachers - it wasn't my idea - a teacher in london said that he is now - if you think, it's actually a no-brainer. i don't understand why thisdoesn't happen worldwide tomorrow. he is now assigning thelectures for homework and he's having the studentsdo homework in the classroom. there's no other technology necessary. think about what happens there.
right now, we all gather atseven in the morning in 10250 and we're all with ourpeers and (unintelligible) we have to sit passively andjust get this broadcast lecture and then we go off on our own and we do these problem sets in a vacuum. if you flip the model andthe lecture, as we said, is one size fits all, imight be lost or bored, i might have gaps from the lastlecture that i didn't understand, but i still have toget this exact lecture.
you flip the model, you geton-demand video at home, when you're ready forit, when you're awake. i got a letter from a studentabout they did actually have a good calculus teacher, buthe had two rows in front of him this really cute girl and everyday he's just trying to figure out how he can ask her out and he missedtwo months of calculus. (laughter) this was his only - (laughter) i say that's a story of anotherperson, but - (laughter)
we all had similar stories. (laughter) we go home and we do theseproblem sets in a complete vacuum. we have no one to helpus, but if you flip it, you can watch things at yourown pace, fill in your gaps, do whatever you need and thenwhen you're in the classroom, you have 30 or 40 or 400of your peers to help you and you have the professorwho can walk around and in real time, seehow people are doing, two weeks before theassessment, two weeks before
everyone gets that labelplaced on their head that they're a b student. if you really think about it,honestly, and i'm not this touchy - these people who areanti-grades and that's like, oh, that's touchy feely. i'm anti-grades becausei think it's ridiculous. it's not about - it's because if you're ab student, you shouldn't - everyone should be forcedto be an a student.
it's on that concept. it's silly, you'll meet many, many people, and i think this room is an exception, but you'll meet many, many people who went through algebra, algebratwo, geometry, pre-calculus, trigonometry, calculus, andwhen they see an algebra problem in front of their face, they don't see it. they have never had masteryof anything in between and this whole exercise ofgoing from one hoop to the next
and being an a- student the whole way, it's just an exercise infiltering, it's not an exercise in actually making peoplehave tangible skills that they can dosomething productive with. what i just described, flipping the model, that's if you have no technology, but if you have the technology,what you can do is teach - there's a bunch of studentnames here that you can't see, i've blanked them out, but this isa report that a teacher could have
and this is actually a real report,this isn't a photoshop version. this was at a summer camp where they used what i showed you inthe last couple slides as their math curriculum,where every student was able to work at their own pace andthe teacher walks in every day. each column here is one of those modules that we saw on that firstslide, one of those concepts. each row here is a student. green means the student alreadygot ten in a row on that concept.
they did it, they're cool. purple means that they'reworking on it right now, that that's their frontierof learning at the moment. everything seems to be going okay. red means that students done 50 problems. sometimes you can think of the heuristic. that student's done 50 problems, 95% of students who have done 50 problems in that module usually move on.
this student has not. some type of intervention should occur, some type of human intervention. a teacher could look at this andeither they could eyeball it. they could say, "hey,look, everyone's doing fine "but these two kids needhelp in level two division. "let everyone else work at their own pace, " let me just takethese two students aside "and do a very intimate,focused intervention
"on just those two students." even better, you can pairthese two students up with these two students initially and see if that gets them over the hurdle. you can actually capture thedata before and after that. i'm not there yet, but if the software is actually scheduling the interactions, you can actually seewhat happens before those and start to rank teachers andstudents and all of that type of thing.
slowly work up to thelevel of the teacher. what i think should happen andhope that this catalyzes it is we go back to a one-room schoolhouse. we go back to room full ofpeople, from kindergartners all the way to anybody. senior citizens who wantto come back and learn and they're all working at their own pace and if you have trouble with aconcept, even if you don't know, this will know if you havingtrouble with a concept.
it will pair you with somebody elseand it'll know about that interaction, it'll know how good of aninstructor that person was before and after, you'llrate them, whatever else. then, you can keep escalatingup and up the ladder until eventually you canget by the paid teacher. you can actually have a reality where - obviously there's a huge aspect of actually teaching people for retention. this isn't just a way toget economics and scale.
this whole mechanism, i'm a ninth grader and i'm teaching a sixthgrader negative numbers. that, by itself, is ahugely powerful tool. i'm learning things betterthan i ever learned before just doing these videos. you can imagine if you're - and this whole level of maturity. i think, actually, ifyou hang out with people older and younger than yourself,you actually act more mature
for a whole set of reasons. then, all of a sudden, you have amodel, where everyone is learning, the classroom is actuallyan interactive experience, it's no longer a passiveexperience, everyone is operating exactly at the level that they need to, and it actually scales. you can actually make it economic. you could actually have - everyone's obsessed withstudent-teacher ratios,
but student-teacher ratios assume a model that if there's one of me and ten of you, then we can have a conversation. if there's one of me and 20 of you, we're going to haveless of a conversation. if there's one of me and 100 of you, we're not going to have a conversation. that's true, to some degree, but here, everyone works at their own pace.
even if there's 100students and one paid tutor, because you have thisinteraction, i'm willing to bet that you're going tohave a better interaction with that one paid teacher whocan now be paid 100 grand a year, instead of what teachers get paid. you're going to have a betterinteraction in this world with a 100:1 student-teacherratio, than you'd probably get in even the fanciest privateschools that there are right now. this right here, this isliterally the summer program
had asked about this andi was curious myself. this was literally me justdoing sequel queries in the data every night from the summer camp. it's not like a rigorous study,but to me, it speaks to the power of every student working at their own pace and throws into question, thetraditional notions of assessment, which we, i think, sense are(unintelligible) to begin with. what this chart is doing,this axis right here is the days in the summer camp.
it was a six-week programand this is just a count of the modules completed. this black line right here, that'sthe class mean, the class average. on day seven in this classof 30 or so students, the average student hascompleted 23 modules. that's what that tells us. you can actually even seethat they all started at 1+1. it was actually an interesting idea, because we startedanother group not at 1+1
and some subset of thestudents always got stuck. they just never moved on, eventhough we started them at fifth grade and they were seventh graders,some subset always got stuck. we saw when we started everyoneat 1+1, no one got stuck, which tells you that thereare some smart people who have some gaps and itnever gets addressed otherwise. you can have the bestalgebra teacher on the planet and you can have an innatelyvery intelligent kid, but if he or she just is a littlefoggy on multiplying decimals,
there's no hope, they're justgoing to talk past each other and it's just going to be avery frustrating experience. when we started everyone at1+1, and you see it here. it's a very steep curve,this is the review. this was the kids just getting tenin row, ten in row, ten in a row. then it starts to level out a bit. this is, you can seethe learning happening. the interesting thing, at least to me, the interesting thing aboutthis is the green lines
are standard deviationabove and below that mean. you can plot all the students here,but this student in particular, on day five, if you did apre-assessment for the class, if you did an assessment righthere, he only completed 12 modules. more than one standarddeviation below the mean. if you were in the traditional,if you were tracking students, you'd say, "oh, she's a remedial student, "she's going to hold the class back, "or we have to hold the restof the class back for her.
"she doesn't belong in here," all that and you would put a biglabel on her forehead. "you're never going to be anengineer," and all of these things and actually it does happenaround sixth or seventh grade. that is when these labels, thesecareers get ruled out for students. because she was able to spend a little - i was curious, i was like,"what happened to her?" and i could go into the data and icould see exactly what she was doing. she was spending moretime than everyone else
on adding and subtractingnegative numbers. just to understand thischart right here, each bar, this is the first problem she did. red is wrong, blue is right, andthe height is how long it took her. this is just from thedata of the interaction. you can see she got acouple of questions wrong, then she maybe guessed one right, then you have this big long timeshe actually watched the video here. i looked at the video log.
she watched the video here, thenshe got a couple more right, but then she still got a few wrong. once again, in a traditional classroom, she would have been said, "oh, atthis point you are an a student. "you got nine of ten right,or you got 19/20 right "and you're an a student,"and force her to keep going. a couple more right, one wrong,then finally she got ten in a row. you can actually see thatshe also got faster on it. there's a whole other element of speed,
which does matter onsome standardized tests. once she got throughthat, once she was able to just remediate herself and really build a strongfoundation, she just raced ahead. this was just in six weeks. she went from being the secondto worst student in the class, if you want to measure that way,to the second to third best. all the students are on the chart here. in just six weeks.
it makes you question, ifyou did an assessment here, and i show that becauseshe literally raced through level three linearequations, which is like 2x+7=-3x-4 and she just thought it was intuitiveafter doing maybe level two. she just raced. if you saw that student,you'd say, "oh, she's gifted." (laughter) it makes you really question. we were just the lucky kids thatgot assessed at the right time and got all positive feedback andwe were assessed at that point,
but there's probably a lot ofpeople who might've been able to eat our lunch eventually,who got kicked off. this, i show this, thisis data from youtube. i think everything i talked about before is how do you make surea student learns properly and how do you make sure thattheir foundation is built and how do you use that data to optimize the actual physicalinteractions, but i show this to show that there's another dimensionof perfecting the actual content.
this is data that youtube gives me. they call this audience attention. what they do is theymeasure the drop off rate off of a youtube video relative to the average youtubevideo of that length. let's say the averageyoutube video at minute three has lost 40% of its audience. that's just a natural drop off rate. at minute three, if i'velost 40% of my audience,
i'll be hugging the line right there. if i've only lost 20% of myaudience, i'll be above average. that's what they're measuringthere, but you can imagine - actually, i've seen a few of myvideos where this curve does this. it just drops off. i can go at that exactminute in the video and see, oh, i said orthogonality andi never defined orthogonality. i can either re-record the video,i could put a little bubble there, where you can get more information.
you're getting real data thatallows you to actually iterate and perfect on what you're producing and you can imagine twodimensions, this is attention. you could also have efficacy, if youcouple it with the actual exercise and you keep iterating on ituntil you have a gem of a video and you can even havemultiple people's videos and see what's betterfor different people. this all exists in netflix. there's no reason why itshouldn't exist in education.
this right here is the big picture. i'm talking about a virtualschool for the world. this is what i think it's goingto be and i'm open to any ideas or any nuance parts of it, butyou have your video library. that's the low-hanging fruit, it'sthe first line of instruction, you can pause it and repeat it. one thing, i spoke to a journalof higher education recently and sometimes i'm very diplomatic,i don't rock too many boats, but i was in a quirky mood and i hadn'teaten for a long time. (laughter)
people who know my eatinghabits, i often don't eat for long periods of time,but i just went off. i was like, "i challenge you,"because this is a journal read by professors and educators. i was like, "why does a 300 personlecture even exist anymore?" in what way, just tell meone way that that is better than on-demand video. i'm happy for any of you all to - it's this broadcastlecture to a room of people
in a certain amount of timethat's 90 minutes long, people have gaps, and theprofessor sees blank faces. it's just this all around - i think the only reason it exists is because people associate itwith the college experience. people expect when i pay 30or $40,000, i should get this. that's the only, in mymind, why it exists. with on-demand video youcan pause, repeat it. the producer can get data onwhat's working, what's not.
you can get your best professorsto produce the best content and keep iterating it,making it better and better. people can fill in their gaps. that's what the video library's all about. then you have the web applications. you saw one of them, butthere's actually another one. there's some gentlemenin the back of the room who started working on it. there's a whole suite of webapplications you can build
for allowing people tolearn at their own pace. what you saw was for math. we're working on something. i'd be happy to talk aboutit if you're all curious for more fact-based knowledge. learning american history orgeography, things like that. you can even have webapplications for peer assessment, for creative writing, all sorts of things. when you actually compose thingsfor other people to review,
it becomes a whole otherlevel, whole other game than when you're writingsomething for a teacher to mark up and say you're a b+and then throw it away. and then on top of thatyou use all of that data to actually facilitatepeer-to-peer interactions. i hinted that in the classroom,but there's no reason why it just has to belimited to the classroom. if i'm having troublewith quantum mechanics and the best instructor forquantum mechanics is in mexico city
and we're both free at4:00 pm eastern time, no reason that we can'tset up an interaction with a pen tablet, the way i usedto do it with my cousin nadia and have an interaction, evenbetter record that interaction, so that other people inthe future can use it. i rate them based on how they were, they can even rate me onhow i was as a student and the system knows what i didbefore and after that interaction. it actually knows whether itwas a statistically significant
intervention in some way. this is the big picture and youcan just keep getting the data and iterating and iterating and iteratingit to make it a better solution. that's the master plan. (laughter) any questions? yeah. audience member: i knowyou've got this initial focus of k through 12 and math and whatnot. audience member: sorry, wecan't hear the question.
audience member: okay,one of the things you said really struck me as aneducator, which is your comment about everyone should beforced to be an a student. if you think about this isa (unintelligible) exercise with a lot of unwilling institutionstrying to get them to change. it seems to me that thisidea, just this one idea would be very powerful to takeinto professional certification, say nursing degrees, whereyou're not going to be a nurse until you achieve this levelof competency on the exam.
take a profession, where this idea that everybody doeshave to be an a student in order to have a career andget them to buy into this model and maybe get the resourcesfrom that sort of problem or that sort of fieldto test these ideas out and to show how they work or theyfind implementation or whatever. sal: i think it exists inprofessional development, because people say,"look, if you're a nurse "and you don't know this,people are going to die,"
so they do enforce thatwith doctors, nurses, and - that exists, but it isn't - we could do studies, but ithink it's almost intuitive that if you go back to - someone that has real mastery of algebra is going to be way morepowerful than someone who has b- student level of understanding of algebra through differential equations. they're never going to beable to apply any of it,
as opposed to - i've worked at - when i wasworking at a hedge fund, there was once a cfo who said - i was like, "what's yourincremental gross margin?" he's like, "oh, we don't divulge that." i was like, "you just gave it to me. "you told me your grossprofit in two periods "and how many units you soldin two periods (laughter) "that's your -
everyone in the room was like, "oh my god, "this is some kind ofrocket science," (laughter) that is algebra one, that isalgebra one, and no one in the room. there are mbas, there were phdsin the room, very smart people, but no one could recognizeand that is a failure and these are the successfulpeople in our education system. people need to - if we had real mastery of algebra,the world would be a different place. all the calculus andall of that other stuff
would just be for fun, but it's not - i don't even think it has to be tested. if people really understood algebra, that's going to be way better than people halfway understanding algebra and then moving on to other things. yeah? audience member: a question aboutthe peer-to-peer interaction. do you see that happening now, actually?
if you have 16 million views onyoutube, i imagine there's comments. is there comment moderation happening? sal: you all can go home and look. some of the debates oreven some of the commentary on the youtube videosare like what teachers would dream of having in their classroom. someone says, "hey,this was a great video, "but i don't quite get whatsal did at minute 2:34," and there's a little link inyoutube you can go right back
to that little point in the video. then other people willcomment on it and say, "oh, you just have towatch that other video "and he explains that in depth," or,"here's another page you can watch," or they'll actually justexplain it right there. this is just on a simple message board. it definitely already existsand it's literally just - i get at least five emails a dayasking for a better community to tap into this and i've justbeen waiting to do it right.
it's there and peoplelove to help each other and they just don't have agood way of doing it right now. audience member: i have children inthe first grade and fourth grade. what would you suggest ido with them with this? sal: first through fourth? audience member: first and fourth. sal: this is not scientific. i've tried out the appwith different aged kids. the 1+1, it is 1+1 but it doesn'thave dancing bears. (laughter)
i hope to eventually have dancing - everyone likes dancing(laughter) a kindergartner. i found kindergarten - i've seen some precocious secondor third graders get into it. they don't need thedancing bears and the music your fourth grader, i think,could be ready for this. the videos i've seen veryyoung kids like them. they like the - i don't know what they like about it.
they like the blackbackground and the colors. they respond to the videos. i've seen three or four yearolds, it holds their attention. i don't know if they'reretaining anything. (laughter) you might want to start with thevideos for your younger student, but the fourth grader, ithink, is ready for the app. audience member: the younger kidgoes straight to the youtube. sal: yeah, i think so, and youmight want to watch it with them. some people have told me thatthey watch a couple of videos
at night with their kid and it's inplace of a bedtime story. (laughter) why not? audience member: are yourvideos in your system a little too big for one laptop per child? audience member: what's the question? sal: the question is yourvideo and your system i guess there's a couple of - in terms of ... audience member: gettingany of that content to them,
because right now i think they'rejust doing textbook format, right? sal: yeah, so thequestion is, is it too big for getting the contentinto one laptop per child? the easy answer is no. when i first saw one laptop per child, this is before khan academywas really well thought out, i was like, "what are they going todo on that one laptop per child?" there's examples of peopleusing it for lights and things as opposed to for a computer.
i think it could be usedtomorrow on one laptop per child. audience member: but dothey have enough drive space and things like that? a lot of them don't havenetwork connectivity. sal: the raw videos, they'regoing to be on itunes u soon, so i just talked to the applepeople and i handed them the disk that had all the rawvideos and it was 40 gigs. i think if you took theflash versions, it's 14 gigs. i don't know how big thehard drives are, though,
but soon they're going to beable to store all of the - audience member: it's actuallysomething you should look into, because that's a big deal, right? sal: i would love to. if they called me up, i'dsay, "yeah." (laughter) i'd love to, i'd love to. we can have another talk about - i'm talking to a couple of foundations. one laptop per child would beawesome if there's the resources,
but you don't necessarilyneed one laptop per child. get some $200 refurbishedpc, put it in the village. now, all of a sudden, you canhave asynchronous learning, which is what these village kids need, because they have totake off for the harvest, they have a whole spectrum. some of them can't read, some of them have pretty reasonable proficiency. that $200 refurbished pc,instead of being one per student,
you have 20 studentsper day working on it, each half an hour a day,doing this type of thing, and then as a foundation, forthe first time in the history of philanthropic giving,you actually have real data on what's happening in realtime on the actual utilization on your assets, where studentsare, how they're performing and in terms of the actualcost, it's a $200 pc that'll last four years,so it's $50 a year and you're educating 20 students.
that's $2.50 per studentper year for an education, which is competitive withanywhere in the third world. it's cheaper than pencil and paper. i think that's what should be the model, but if they get a laptopthat's even better. oh, in the back, yeah. audience member: i think you said you do all the video and lectures yourself. how long does it take youto put together a module?
there's some topic areas surely youdon't know all of it. (laughter) sal: a video, if it's a work - there's probably people emailingme right now some algebra. i think i may really be doingsome kid's homework for them. those type of things whereit's just an algebra problem, someone emails it to me, i haven'tdone that exact problem before, let me just do it, that'sthe ten minutes on the video, plus five minutes to upload,it's a very small amount of time. if you saw that first screen, i'vestarted doing world history videos.
those take a lot of - audience member: whatdo you mean by a lot? sal: when i started onthe french revolution, i spent a week doing nothing butreading on the french revolution. i literally just took time off and isteeped myself in the french revolution. i think we've all found thiswhen we were going to school is that when you're inthe midst of a class, you really are an expert forthat short period of time until the exam you are an expert,but then it all goes away. (laughter)
for that short periodof time, you can talk to any academic anywhere in the world and you know all of the detailsand the nuances and everything. about a week of studyingand it's fun studying. it's not for an exam, it's for meand i get to make youtube videos. i'll do that and then i'llstart making the videos and then i can probably producetwo or three videos a day. once i have the big picture andi understand the major pieces and i have a sense of -
if you watch those videos, icut and paste a bunch of images and maps from wikipediaand i just explain them. i'm a big map person, so i do that. in the back? audience member: is the softwarethat you use to do things like draw perfectly straightlines or doing a ratio something you can buy offthe shelf (unintelligible) or did you design that all yourself? sal: no, some people haveoffered to pay for it,
but i didn't build any software. actually, all i use is i startedoff using microsoft paint. i made the backgroundblack and i used this thing called screen video recorder,too, which is this shareware, freeware, or $20 screen capture software. that's what i started the first500 videos were done like that. then someone just sentme a camtasia license. they're like, "this is great,we need to (unintelligible). "you need a better screen capture."
so i started using camtasia andthen i used smooth draw, too, which is pressure sensitive,so it actually looks likei'm writing with a pen, as opposed to microsoft paint,but you could get set up for $200 at home with exactly what i have. it's all found stuff. there's probably better stuff out there. what i'm using right now, ican't draw a straight line. with microsoft paint i could,because it had the line tool. audience member: in line with youridea of this as entertainment,
so replacing bedtime stories, ithink there's a niche for retirees. i know any number of peoplewho, first thing in the morning, do three crosswords and a sudoku. sal: exactly. audience member: (unintelligible) sal: (unintelligible) my mother in law(unintelligible) (laughter) i have gotten letters from retirees saying that they started doingit instead of sudoku
and other than keeping them sharp, they're not connectingwith their grandkids. they're becoming the grandkids' tutor and this is a whole level of connection that they couldn't havebefore, so there is something. not the retirees, there's alot of homeschooling parents - i think there's a lot ofhomeschoolers who are doing great until about algebra and thenthey're like, "what do i do next?" this is helping them out.
there's a lot of people who surprise me. there's a lot of militarypeople who are actually my age who are going back to college. if you think about it,if you're in your mid-30s and you want to go back to college, and the first thing you're goingto see when you go to college is college algebra and (unintelligible) there is nothing out there for you. you're helpless and this isthe only thing that's out there
and they're eternally thankfulthat they could re-learn arithmetic and algebraand all these things that they haven't seen for 20 years. it's surprising. audience member: i have an eighth grader and my question is, would yousay that the videos sometimes could replace going back to thetextbook when they get stuck, or - sal: yeah, so the question and this - audience member: because the textbooks -
sal: oh, don't get me startedabout textbooks. (laughter) no, i will get started about textbooks. the question was, "ihave an eighth grader. "do you think the videoscan replace the textbook?" let's think about what a textbook is. there's two things in a textbook. there's something that claimsto be some form of explanation, two pages of text on something,and then it's exercises. there's 50 problems (unintelligible).
i guarantee you on-demandvideo, whether it's khan academy or anybody, is going tobe better than that prose or whatever you want to call it,that's trying to explain the concept and it's usually explainedmore for peer-review, more for the othertextbook authors saying, "oh yes, he did itrigorously, that's excellent," as opposed for theeighth grader who wants - sal: exactly, i think there'sa small subset of people who could actually read a textbookand that's what a lot of us
would probably get wherewe got, because we could actually consume thatinformation, but most people - on-demand video is better than that and from an exercise point of view, the textbook company doesn't knowwho's doing what exercise when, who's watching what, what'sactually being effective. they're not getting real-time data. that stuff i showed - you can argue what themodules should look like,
but something like that isgoing to be infinitely better, especially if you can tweak itand get data on what's working, what are people using. for all we know, textbook publishers, half of those problems arenever touched by anybody. they're in there andkids are getting hernias carrying those problemsaround and it's a - the answer's yes, i think that textbooksneed to go, on a lot of levels. audience member: so for world history,
what kind of problems do you put out there so you can make your assessment? sal: the question is for world history what kind of problemscan you put out there? what i showed you, those modules,that's good for something that can be templatized, that you can map. you can even do some physics problemslike that, the core problems. you could do some types ofchemistry, maybe genetics, but what about thecauses of the civil war.
audience member: not just that,but analysis of complex issues. sal: yeah, there's two levelsof things like history. one is the, "what didmillard fillmore do?" there's that level andthen there's the level of, "let's talk about the impactsof slavery today," or something. the core stuff, andthat's what i'm working on with those two gentlemenin the back there. literally i met them,(unintelligible) we met last week, but what we're working on is a -
we're working on - if we were studying fora factual based thing, we would get together in a big study group and write flash cards for eachother and then quiz each other on each other's questions. what we're working on is a world - for american history, putthe whole curriculum there and then if you say, "i'mgoing to write a question "on the antebellum period."
click antebellum, you write a question, you can embed a video,image, whatever you want. once you feel good about it,you put the correct choice, incorrect choices, explanation. once you feel good aboutit, you publish it. if i'm the first personto see your question, i'll say, "hey, warning, thisis an untouched question," and all of that stuff,but you should try it out. i do the question, i rate it onquality, difficulty, and importance
and then you start buildingthis library of questions with metadata aroundit and you can imagine that you're collecting data,people who knew this question didn't know that question,so you can quantitatively say what's easy or what's harder. there's all sorts of waysyou can consume the question. you could imagine a teacher, becauseyou're getting all of this data. you know exactly how long it takesfor a certain type of student to complete a question, whetherthey get it right or wrong.
you can imagine a history teacher,once you get this huge data, this huge library questions,like you know what, i want to administer an exam tomy history students tomorrow, i'm going to bring themto the computer lab. i want 50 questions and i don'twant more than a 95% chance that more than two peoplearen't going to finish that exam and we'll have all of thedata, because it'll know for that level of studenthow long it'll take them, it'll do the statistical modeland then, bam, (unintelligible).
that's for the core and youcan do that for anything. (unintelligible) the more nuanced stuff,that's an open question. i'm open to ideas. i think there's things,a peer review of essays, or you could argue throughessays and have people vote on who's argument is better. there's all sorts of things you can do. that hasn't been -
foreign language, i wasthinking, i have a 15 month old. skype exists, the best foreign language, i'm going to have him chatfor a half an hour a week with somebody in mexicocity, dubai, hong kong. half an hour a week and he'll just chat. that's foreign language. i think there's some very simple things with found technology right now thatthat could solve some of these problems. the immersion, as opposedto (unintelligible).
oh, yeah? audience member: verycurious to hear your thoughts about the texas board of education and - sal: what about education? audience member: have you heardabout the texas board of education? sal: oh, the texas boardof education (crosstalk) audience members: they'retrying to rewrite the textbooks, especially around american history. i'm wondering if that madeyou rethink about your mission
with khan academy or - sal: oh yeah, that's interesting. he's asking about thetexas board of education, the rewriting of americanhistory and all that. from a personal level, that'sthe power of this thing. i could all of a sudden - i do every now and thengo on google blog search and i see who's usingthese khan academy videos. obviously, it's a lot ofhomeschoolers and all that,
but actually, a white supremacy grouploves the khan academy. (laughter) because they're like, "thisis quality instruction, "our kids aren't gettingit, this is the only way "that they're going to be ableto compete with the asians "and the jews and the - (laughter) they could've gotten me to teach them, or they would've let me get intotheir house and teach their kid, but because they recognizethat this is better than what they're getting in the school,
i'm able to get into thosekids' heads, (laughter) which would never(unintelligible) (laughter) i think that whole texas thing,i think it's a non-issue. i think with the internetand all that, they can debate night and day over whatshould be in the textbook, no one's going to careat the end of the day, because they're going to getinformation from other sources. the reality is most ofthe kids aren't learning what they're trying to put in thetextbooks, so who cares? (laughter)
audience member: are you able to - different people learn in different ways. for example, some peoplemight learn with examples and other people want tolearn more deductively. can you identify theways that people learn and then adjust the instructionto match the special need? sal: yeah, the question was canyou adjust to deductive learners versus people who need examples? we could.
i don't do it yet, i don't claimthat it's already done and all that, but it definitely could be doneand that's another value of this. some people say, "oh, are youteaching it this new way or regrouping "or are you teaching theold way with borrowing "or are you teaching theabstract manipulative way "or are you teaching ... let's do both, let's do both,and people can pick and choose what they want or do both. in a traditional classroom, youhave to make these decisions
because you have a scarce amount of time, scarce amount of resources, you haveto teach it this way or that way, but here you can teach them both ways. teach it the deductive wayand then do a lot of examples, because that's the waymost people enjoy it. there's nothing to stop you from doing it. audience member: i saw the graph earlier, where you show how youget from (unintelligible) all the way through and that'show the (unintelligible) works
to build a foundation and youcan extend that even further by anything that's factual,history or geography, by taking the graph (unintelligible). you can imagine, after you justlearned about the french revolution, and a lesson about french geography, you could then beoffered to take a look at (unintelligible) for exampleand since you've learned about (unintelligible) you canlearn about (unintelligible) can get you to (unintelligible)you can always relate it
to something you already know. sal: especially once youhave a question library and you think about thebest way to navigate it. right now it's just a curriculum. you can say, "i want to learnabout the french revolution," but (unintelligible) organically,what you're suggesting. i haven't built it, butit can be built today, and it's kind of sillythat it's not being built, especially considering howleverage-able something like that is.
audience member: have youthought about teaching parents, educators, teachers on how to teach, how to communicateeffectively to your audience, how to troubleshoot wherethe hangups, the gaps are. sal: the questions is have iever thought about teaching to educators or parents how to - there's an interesting story there, too. this was when khan academy was like, i was happy if anyonecalled me (unintelligible)
three years ago, miamidade contacted me and said, "can we use your videosfor teacher training?" i was like, "yeah, sure, that's awesome." you all are real educators andyou all want to use my videos. i was like, "but why don't you let thestudents use them, too." (laughter) there is a reality now where ... i think there would bevalue in telling teachers, "look, you don't have tobe perfect, just be human, "and just be conversational,and don't talk down,
"and don't try to use thisas a power exertion type "of classroom managementtype of thing, use it - i don't blame teachers,because the system forces them to manage a classroom, demand respect, if they have a gap in theirknowledge, you can't be honest. if you do that in a classroom thenthey're going to undermine you. there's all these otherdynamics that occur why a teacher can't be human. we're in a world where we'llprobably get a subset of teachers
who are scaling up and then therest of the teachers leverage this to become the human one-on-one mentor and i think that'll be the value. it'll just be kind of example if they say, "hey, let's watch this together,"and then, "did you get it? "if you don't then i'll be yourmentor to walk you through it." i'd be happy to if anyonewas interested in me doing (unintelligible) anythinglike that, but yeah. audience member: i'm curious aboutthe peer-to-peer interactions
and community and how you've gotand where you could be going. you've touched on a couple of thingsi think are just world-changing. intergenerational,knowledge transfer, wisdom, you talked about one room schoolrooms, but what about justthe usual parent-child. there's so much that couldhappen using this model. where are you? sal: yeah, so the peer-to-peer, that's at the most on thedrawing board right now.
i haven't ... i've tried it outwith my cousins as a one off basis. i feel like it'll definitely workand the data is being set off, but knock on whatever thisis made out of. (laughter) hopefully in the next few weeks we'll get some resources to start doing - that's the most technically intensiveand we want to get to the point where not only are we doing peer-to-peer and it's seamless and it's capturing data, but it's also recordingthose interactions.
you need some - once you record, thenit's (unintelligible). i agree with the whole teacher, with the - i want a world where you can - if your parents wantto take you to europe, it's not going tocompete with your school. you go on that trip to europe and maybe there's anotherone room schoolhouse part of the same networkthat you can go to
and meet the kids inparis while you're there. on the plane you can do - (unintelligible) i agreewith you completely. the peer-to-peer is the hardest to do and it's going to be theone that maybe happens, but i think it'll happenin the next five years. i think the potential is large. audience member: what'sthe optimal length or range for the different modulesand how many times
do you have to retape it to satisfyyourself that it's good enough? sal: the question is,what's the optimal length for i'm assuming the videos especially and how many times do i have to retape it? when i started off, youtubelimits you to 10 minutes. now they let me do longer videos,i'm a partner and all of that, but when you start off it's 10 minutes, so my first video was 11 minutes andthey didn't let me upload it. (laughter) i paid attention to theclock and i made my videos
around nine or ten minutes. that's the singlebiggest point of feedback that that is the ideal amount oftime to get one concept across while being able to pay attention. i've gotten letters from people. i've talked to some peopleat google who have done some user interfacestuff and how long people can pay attention and they said,"yeah, it's about ten minutes." i think that might evenbe what motivated youtube
to figure out the limit, butthat forced me to do that. one, that's a granular chunk thatallows people to pay attention. it turns it into one of thesedavinci code dan brown type novels where each chapter's threepages and you're like, "okay, i'm done," and youget a sense of completion, as opposed to a 90 minute lecture that you can only watch40 minutes and you feel - this also makes it very granular. you can really just jump inand get exactly what you need
as opposed to sittingthrough a 90 minute lecture and trying to find - i've found that ten minutesis now i'm getting to - actually, i've done a few videosnow that are pushing 20 minutes, but that's just my fault. i really think everythingcan be done in ten minutes. your other thing is howmany times do i do it? audience member: do you have totape and retape several times to get it to where you want it (crosstalk)
sal: oh quality? audience member: yeah. sal: how many times do i have to - when i started, sinceit was for my cousins, i literally would recordit and put up. (laughter) there's actually one or two videoswhere i'm answering telemarketer calls. (laughter) i didn't care. why re-record it? it's for nadia.
this is free (unintelligible). that's another takeaway. people like thatinformality, they like that - i make mistakes. i would say one out of every12 videos i make on arithmetic, i forget a negative numberor i forget a negative sign or i say 16-8 is 5. i do something bizarre and peoplereally like that. (laughter) or i start a problem and goaround the wrong path first
and i say, "oh wait, that's not right." i've gotten positive feedback onthose things because you never see it. when you go to a class, theteacher they have the script, they've done that problem 20times and they just do it perfect. you're an idiot if you didn'tsee this, obvious steps, but when you see someone whokind of knows what they're doing, they follow their intuition inone direction, it was wrong, that's okay, that's the art ofmathematics, of solving a problem. then you go down a complete other path.
i try now that it'sgetting more notoriety, i feel like, "oh my god, so manypeople are going to watch this video, "let me re-do it," but i findthe more i think that way it hurts the quality of the video. audience member: does kaplan or any of the other paid for onlineservices, phoenix university, do any of them view you as acompetitive threat, being free? sal: they should. (laughter) i've actually spoke to theresearch group at apollo group,
which is the parent companyfor the university of phoenix. i think they're aware of it now. i've talked to princetonreview about potentially - because i would like to do something with their materials and all of that. in any of these groups,even textbook publishers, i've talked to a few, there's a groupthat's very insecure about everything, but there's another view that'slike, "look, this is out there "and this could evenempower your instruction."
if i do all the satproblems out of your book, hopefully you'll donateto me and i can do them, that's not going toprevent people from taking your princeton review course. it's going to make your timein your princeton review course way more productive, because thekids will have done the problems, get all their questionsanswered by this sal guy and then they can havethis one-on-one interaction with the guy you're paying to teach.
i've talked to princetonreview and the apollo group who definitely believe that, butthen they have to fight the inertia of we have this billion dollarfranchise, we can't mess it up. once they really view me as a competitor, then i'll start gettingsecurity. (laughter) audience member: hi, i'mwondering if you can speak on anything more to how you devisedthe content for your videos. are you looking at textbooksand here's the order of content, because you're not a fan of textbooks.
i'm wondering if there's apotential to reconceptualize the way knowledge isorganized to begin with, because i was lucky, i couldfollow textbooks very well, but our natural curiositiesdon't follow (unintelligible). early childhood educators arevery often going to have that. you go to class(unintelligible) and it's killed many children's natural curiosities. how can you link thebasics that you're teaching to the real world questions that we have?
sal: the question is howdo i build the curriculum? how do i decide what to cover,given that i'm not a fan - i'm not a fan of a textbookas a primary learning source for someone who's - i do look at textbooks. i don't go straight, section 12-1, nextlecture, section 12-2, next lecture. actually, there's a textbook publisher who wanted me to do videos for their book and they said, "i want you to doexactly this concept on this video."
i looked at that concept, i was like,"there's no way i can get myself - it would be so boring. it would be just so mundane itwould just kill all my credibility if i were to do a video on that. my thing is i try to get the big picture (unintelligible) withthe french revolution. spend some time, mull over it, andthen once you have the big picture, then try to take the nuggets thatwill build you the big picture. then maybe use thetextbook or something else
just to make sure you're comprehenseive. the kid's going to seeeverything that they might face when they take an assessment. i use a ton of wikipedia, i use textbooks, i use other opencourseware sometimes, to - a lot of stuff is hard to consume. you really have to be motivated to consume or already even have a foundationto consume a lot of these things. i feel my value-addedis taking all of that
and distilling it downto something useful. audience member: in your model up thereyou have a figure (unintelligible) how have you thought of lookingat environmental (unintelligible) stress can be measured inlearning by somebody blinking or - sal: the question is rightnow i'm just tracking when they're doing the problem,how long it's taking them, what time of day they're doing it,did they get it right or wrong, did they click the hint,all that type of stuff, but there's this whole whatwas their blood pressure
while they were doingthe problem, (laughter) where did their eyes move, what is their cycles intheir brain doing on an eeg? i think that'd be awesome. you could do some biofeedback stuff. that's a whole frontier that i think is - audience member: the onlybad thing is you don't want to hook somebody up toan eeg (unintelligible) sal: (unintelligible) if it can bedone, try it out and see if it -
camerawoman: there's researchalready in effective computing in media lab and rosalind picard. that's what (crosstalk)she was my adviser. then they do sweat measurementand eye dilation (crosstalk) sal: i think the fun thing aboutthis, what i view as the khan academy, right now it's very content focused. everything i talked about was the videos and these modules and all ofthat, but where i see it going is the content is really just seeding.
in order to do all this datastuff, you need the content, but once you have a critical mass ofcontent, then the data takes over. you can start doing somereally fascinating things with experiments that youcouldn't have done before and real-time learning and biofeedbackand all this other (unintelligible) audience member: in thebeginning of your talk you said your visitors to yourwebsite surpass mit (unintelligible) is that true? sal: i'll give you all my data.
sal: how do i know thatit's past (unintelligible) sal: there's a couple things. there's an interesting thing about - just on youtube, which i thinkis the primary consumption area for most open courseware,it has passed up mit. you can actually look at the cumulativeupload views on a daily basis and just measure the delta and you can see that khan academy'sgetting about 60 or 70,000, mit's getting about 30 to 40,000.
i think next up is berkeley's getting15,000 or something like that. that's just on that. i don't have access tomit's actual weblogs. (unintelligible) definitely onthe internet, i'm not 100% sure. audience member: it'sdefinitely a different audience. sal: that's the other interesting thing. there's all this open courseware, people spend millions ofdollars putting cameras in, but khan academy's really onlythe game in town for k through 12,
which is where all the need is. it's not like i'm like, "oh, mit ... they're catering to us. if we took 1806 already and wejust want to review something, we can consume that, butthat's not for a ninth grader who wants to learn how to learn a matrix. for k through 12, ithink that's one reason why khan academy has picked up, because it is directlyaddressing its audience.
i think, on top of that, theopen courseware movement, which i think is awesome,but their first time it let's put videos in classrooms. they didn't rethink the form factor. they didn't say, "well, is thisreally the best way to learn online?" we're just saying, "we'regoing to show you what goes on in an mit classroom,"which i think is awesome. audience member: when it firstcame out, it was revolutionary. sal: no, it was.
that was my proudest day as an alum. audience member: what ifound is that it seemed to be dumping all this information. sal: yeah, there's two thingsthat it's going when you go there. one is it's hard to navigate. you don't know what to watch next, so it's not for someonewho's never seen it before. there's no direct pathand the other thing is even when i was making videos i wasdoing the (unintelligible) video.
this whole time i was like(unintelligible) mit stuff. i went to the thermodynamics lectures and i had to sift throughso much non-entropy talk to finally get to entropyand then finally there was an exactly what i want, soit's very hard to consume. even when you watch the video, i'm not - i think it was an awesome thing, especially because in 2000 or 2001, every other university was doing some type
of for profit, universityof phoenix-ish type of thing and mit, it's going to be free. i think it's awesome, butif you watch the videos, i probably watched, i thinkmore than any of you all, i watched more open courseware videos. every now and then, theprofessor puts up a slide that's like, "this is a copyrightissue, you cannot see what this - i'm like, "oh my god, how am igoing to understand?" (laughter) it happens one out of every five videos.
or the camera is justnot where it should be. you can't see - i think it's awesome forother universities to see what mit is doing and ittakes a lot of security on the part of mit to do that - audience member: it takes someconfidence to release (crosstalk) everything you have, basically. sal: because it says there'sa value of being here and also it's, "look,this leaves us open,"
because before this, everyone would think all these secret things arehappening at mit that are special, but now you know exactly what's happening and it loses the (unintelligible)mystique of (unintelligible) i agree, but i talked to open courseware and i'm open to talking tothem as much as possible. i asked them, "hey, howabout khan academy becomes "your k through 12 offering? "because this will help recruitkids, get them interested,
"i'm an alum and we could get other - they were putting video camerasin the classrooms right now, but hopefully that changes. i think i would love torecruit other professors to do this form factor,get other students, get you all to do whateveryou all want to do. audience member:(unintelligible) (laughter) sal: the questions is how muchof the success of the site related to the incompetency of teachers?
this is a sensitive subject. (laughter) sal: how many teachersare in the - (laughter) the reality is thereare incompetent anybody, there are incompetent x, andunfortunately, some kids, that's all they get and ithink there's a lot of that. just look at the - the only comments i delete from the site are when there's profanelanguage and even those, some i don't get to.
these are uneditedcomments and you'll see, on pretty much every video,at least half the comments are some kid who's angry thatthey're paying x thousand dollars in tuition to whatever andthey spent a whole semester trying to learn it and theythought it was impossible to learn, they spent ten minutes on somevideo and that's all it was and they're angry, they genuinelywant to kill somebody. (laughter) i think that's out there. a professor in montrealwrote a blog post about this.
it's just like even if you saythis above average instruction. we can debate is it better than90%, is it better than 50%, but even if it's aboveaverage instruction, even if this instructionbetter than 50% of instructors, what's the point of havingthose 50% of instructors. it's a thought experimentthat should happen. even if i'm not the 95th% instructor, someone is. then you get that person, or someone's the 99thpercentile instructor
and if you have that person,then why are you having the other 99 out of 100 deliveringlectures, but there is - it goes into a lot of teachersare a product of the same system. they're products of wherethey don't have the intuitions and the connections and becauseof that, they're passing on this rote, superficial understandingof things for generations. sal: the question, i think,is is it just going to be me or are there other potential people? even the top notch thing, it'snot clear that mit instruction
is better than what you'regoing to get at the local high - those guys are going to be kick butt, in terms of the researchthey're doing and all of that, but instruction that mightbe ideal for an mit student, some of the mit lectures,i think they're confident because they say, "look howhard this class is." (laughter) the question about other instructors, when i started this, i assumedthat was going to be the case. i assumed that i had done30 videos and i was like,
"oh, this is fun andpeople are watching it. "let me get my other buddies frommit and they'll make some videos "and i'll see if they're interested." some were interested,but it was harder for me to get people to do videos thanfor me to just do them myself. after some point, i hadmaybe 100 or 200 videos and i said, "this is algebra,it's pretty much done." i said, "i'm just going to keepgoing," but i'm open to it. if someone wants to -
a couple people have volunteered, i said, "make 50 videos. "if i think those 50 videosare of the same stock - even if they don't getassociated with khan academy, they're going to have value by themselves. they're going to be on youtube,the world can access them. if i think they're of the samestyle and they're coherent with this brand, i'llpush traffic that way, especially there's a guy whojust contacted me about music.
he wants to do a music video. i don't know anything aboutmusic (unintelligible) until you teach me about music. i'm open to it, but i don'tthink it's something i could put a job posting for (unintelligible) the other thing is a lotof very good instructors freeze up a little bitin this form factor, because you are puttingyourself out there. if you're a nobel prizephd, it's a lot harder
when someone criticizes you than me. i'm just a guy, but whenyou're the authority, to put yourself out there in that way, sometimes a little daunting to - audience member: i don'tknow if this is for you or your crack programming team, but how far away are you fromhaving forget the videos, at least the drilling modulefor high school algebra one on a ds or an ipod touch?
sal: how far is the module from a - audience member: havinga complete drill set, all the questions, all the metrics, everything being able to load. put junior in the back of theminivan and do two problems every 15 minutes or every five minutes and teach junior algebra onein the back of the minivan, all year, not having to go to class. sal: oh yeah, ipad(unintelligible) ipad expert.
i think it's literally - we don't have but one developerfor six months devoted on that would probably be able to migrate - audience member: wouldthe data fit, though? how much data would you need for that? sal: i think it's just a matter ofsyncing it and that type of thing, it'd just be implementation,but i think it could easily be consumed and done, especiallyon an ipad or something like that. there's no reason whythat shouldn't be done.
audience member: you're non-profit, right? sal: yes. audience member: doyou see yourself moving to a revenue model andhave another approach? sal: yeah, i'm surprised ittook this long (laughter) it's a not for profit, 51c3,and it was interesting, because about a year ago,before i quit my job, i wanted to quit my job. it was a good job, buti wanted to do this,
and some pcs had contacted me saying, "hey, we love what you're doing "and we think you'redoing education right. "how about we do a hybrid model? "you're going to get asalary and you can do this. " we'll have some premiumcontent or this or that," and i was very close to doing that, but then the more andmore i thought about it, especially when i spoke tothem, i knew that at some point
our desires were going toconflict and my worst nightmare is that all these videos idid, all of this gets acquired by mcgraw-hill (laughter)and it gets thrown in some library someplace andjust disappears from the planet. for me, the real joy of this,even if i'm forced to drive a used honda my entire life, isthat my great-great-grandkids might be able to learncalculus from these videos. that, to me, is the mosttremendously exciting powerful thing. it is not for profit, it was being fundedby my bank account until very recently.
actually, last week, ann and john doerr wrote a $100,000 checkto give me a salary, so you don't have to do too much. i'll take some sandwiches, but i'm okay. i'm talking to some fairlylarge corporations now that they're like, "yeah, this - i tell them, "show me a highersocial return on an investment." get half a million dollar budget,we'll have three, four developers, we can educate millionsof kids for all time.
it's not like this is a charter school and as soon as the moneydisappears the school disappears. as long as someone's paying $50 a month for the web hosting (laughter)it's going to exist forever. now it's being replicated on this, it'sgetting on all sorts of hard drives and we're going to put it onbittorrent and all of that, so we don't even have topay for the web hosting. you're building a library that cancontinue to deliver social returns. you do any type of (unintelligible)it's like $500,000 per year investment,
the social return is going to be,especially if you translate it into five languages, it'slike $5 billion a year, if you associate a video view with $1. it's ridiculous and i hopepeople are seeing that. hopefully we'll get some. sal: i am not going tosell out. (laughter) there might be a movie one day. (laughter) hopefully i won't sell out. audience member: to followon that a little bit,
you told us a numberof ways that education is entrenched in productive ways. is this interesting in away wikipedia's interesting or is it interestingin ways that can end up transforming education as it exists today? this would be good supplementalmaterial (unintelligible) (crosstalk) sal: i'll repeat it. the question is do i envisionthis as being what wikipedia is, which is a tremendousimpact, but a supplemental,
nice to have for theworld, or is this something that could transform education? i think in the short term it's the former, but in the long term, and forme long term is ten years, i think it really is - a big part of this, i viewthis as the operating system with the dna for a physical school and i actually do a little summercamp, really for me to experiment with what you do with therest of the school day,
because i can do this fortwo hours and then it frees the rest of the time for creative things,to actually write computer programs, compose music, paint, toactually build things. my goal is that summercamp's going to get longer and longer and longer. (laughter) actually, the san jose tech museumis going to give us some space. we're going to starthaving a presence there. that physical experiment's goingto get more and more substantial and at some point, we're going to be like,
"we're a real school nowand this is the prototype," and i don't want to do it if it'sonly 5% better outcomes than what - i want kids, "hey, i justpainted that picture, "i just built that robot,and by the way i'm 12 "and i'm learning calculus." i want to see that and i thinkonce you have one example of that, but i think (unintelligible)then i think you set the example and i think the rest of theworld's going to be like, "wait, even our $40,000 a year fancyprivate school is not doing that
"and they're charging $5,000a student," as opposed to - audience member: then thenext question that comes, if you do that, this seems toapply to self-selecting students who seek out the materials andeven in the case of the camp, obviously there's some selection there. how applicable are the ideas then ofgoing through school (unintelligible) sal: right, the idea isthis is right now applicable to a self-selecting pool of students and even the summer campi'm doing and that's true
and how do you apply it to a broader - i think there's two - the summer camp that ishowed the data with, the selection criteria wasnot kids with a certain iq or kids who've done - that was just non-problem kids. this was mainly minoritykids from east palo alto, redwood city, menlo park, who theydidn't have discipline problems, but i think that could describe 80%of students, probably 90% of students.
i'd say problem the subsetthat have discipline problems, a lot of it is because they'renot being addressed in this way that they have these gapsand no one can address it and there's all sorts of social stigmas. it's ridiculous that conceptsand age are associated in this tightly linked way thatif you're 13 and you need help with decimals you're stupid andi'm going to make fun of you. i think that leads to a lot ofsocial acting out and all that. actually, i want to get this into prisons,
because they have alot of time. (laughter) i don't know the exact answer,there would be some experimentation, but i suspect that this is going to appeal to a larger swathbecause it's interactive. everything about it, you'renever having to sit passively. you're learning the way peoplenaturally want to learn. i think a lot of thesehyperactive kids or whatever, they're natural learners, theyjust don't like being lectured to. they like to be interactive andtalk to their peers and whatnot.
we'll see where that goes,but it's an interesting seed. it's an interesting question. if there is something that can reallyaccelerate 90% of the students, but it leaves behind 10%, isthat a good or a bad thing and that's a whole socialdebate that one could have. audience member: in case thereis a movie made about you, can the other salman khanplay this salman khan. sal: oh yeah, he says,"can the other salman - for those who've never donea web search for salman khan,
i encourage you to do so when you go home, because there's an indianactor with my name. i think that's helped me beat mitopen courseware, as well. (laughter) audience member: one issueyou might have thought about, i've watched a secondgrader in the neighborhood who is homeschooled and i'mafraid he's really missing out on the socialization factor. if we transition more tothis method of teaching, how would we not lose that?
sal: right, so the question'son socialization factor. i want to be very clear,this is on the internet, essentially and someone couldbe sitting in an isolated room and doing this and nothaving any socialization, but the goal when i describedthe one room schoolhouse is that it actually is social. for the first time, you actuallydo get socialization in a school. most of us, there's a thousandkids in the schoolhouse, but you don't get any interaction,especially in a meaty way,
except in recess and lunch, butnow this is actually encouraging socialization in the classroom,where you're tutoring each other. that's actually very good socialization. if you think about when do you wantto give someone a big thumbs up? right now, the systemsays, "oh, you're a genius, "you got an 800 on your sat." that's where you get the thumbs up, but the real thumbs upis you're a good teacher, you have empathy, you have maturity.
when other kids respect youfor your ability to communicate and not talk down to them. if you think about whatpeople will get points for, i want it to be that, thatthe data and assessments show that you're a great teacher to your peers. if you're an employer or auniversity, those are the kids you want to admit, not thekid who's just racing ahead and just arrogant and all of that. i think it will actually helpsocialization, especiallythe one room model.
oh, yeah. audience member: whenare you speaking at ted? sal: oh, the question is whenam i speaking at ted? (laughter) i don't know. actually, i just had a - i don't know. if they call me i'll go. (laughter) i'll be speaking at pop - have you all heard of poptech. it's this thing in new hampshire.
it's like a mini-ted. (laughter) audience member: would youspeak at tedx, if they call you? sal: i'll be good anywhere. i speak at the old(unintelligible) (laughter) excellent, thank you. (applause)