JTBD Radio

Moderator: Welcome to the latest edition of jobs to be Done Radio, where we discuss how to apply the jobs to be done framework to understand why consumer switch from one product to another and ultimately how to get more customers to switch to your product and here are your hosts.

Chris Spiek: Welcome to the latest edition of Jobs to be Done Radio, I am Chris Spiek, as always I am here with Bob Muster and Arwen Folkes. Hey guys.

Male speaker: Hey Chris.

Male speaker 2: Hey Chris.

Chris Spiek: It has been a while since we have done one, it’s good to be back. Today, we have Chris Lee on as a guest from Tealeaf Academy. How are you doing Chris?

Chris Lee: Hey Chris, how are you guys doing?

Chris Spiek: Good, good. So, a very little announcement admin content today, I do want to mention, I did a little guest appearance on a very cool podcast called Ecommerce Influence. Couple of weeks ago, it just aired if you are in the web, web app, web space, online sale space, definitely check that out. Chen and Austin are the guys over there. They are definitely – they are quick on the take. So, I feel like they learned a lot very quickly over like a 20-minute conversation and they are already applying the stuff and it’s good to see, so if you are an e-commerce, check out Ecommerce Influence another great podcast. So, Chris give us a little background Tealeaf Academy is a place to learn programming, but expand, tell us what it’s all about?

Chris Lee: Sure, absolutely. So, Kevin and I started Tealeaf academy about two years ago, and it started pretty innocuously in that we didn’t really set out to necessarily build this program. We initially started choosing people, just people we know and then we expanded into just a very small contained pay program and it just went from there in terms of people asking for more on both in terms of what we are getting up from the material as well as more advanced material. So we started just teaching and then it really expanded from there. So, we have been doing it for about two years and in those two years, we tweaked, modified and extended the program a lot to the three course program that we have today. So, starting out from a very friendly introduction to programming to Ruby, which is what we teach for the most part and then introduction to what they have all been that was in our first course and then our second course we have introduction to being railed and the first two courses what we say is always had about 40% of our entire curriculum. So the bulk of it is in the third course, our last course which is production, quality, application to how to build production quality, application and all the concerns that you face when you are trying to build something about production quality I suppose, just more in the prototype and level thing that will simple [inaudible 00:02:55] out.

Chris Spiek: Got it and that’s what makes you – that’s the focus, right. So, I guess in the back of my mind, I am thinking there are a lot of places to learn the program online, but you guys have, I don’t know specialized or really focused on that, don’t just learn to be a script kiddie, like learn to build something really robust and something that is going to function.

Chris Lee: Absolutely and this is I think why we are drawn to jobs to be done framework because there is in this space and this online programming education space. There is such a wide range of options, you have three options court academy or the nine or ten dollars a month program out there, like tree house and court school and obviously you have books as well [inaudible 00:03:44] as well and then on the other hand you have some programs that are charging 15,000 dollars or so and obviously, they are promising a little different and they are not in the offline, but they are in the programming education space and if you get a job out of that program, then maybe that’s an okay price to pay and then even extreme, more extreme than that is quarter million dollar you go to college for four years.

Chris Spiek: Sure.

Chris Lee: Little different, but sort of in our same spectrum, where do you want right and this space that ranges from three or nine dollars a month to such a high number, five digit number like 15,000 dollar. How do you want to position yourself as a company. We started out as a very low end sort of 100 box tissue rail type of program. There was actually another program in this space that is doing that today and they call themselves a one month. It’s a one month program, one month rail, one month finance etc. They charge about 100 bucks or so, for roughly a month worth of content and it’s a very different business model than what we do. So, we definitely could have done that as well, but we decided to not do that and to do what we do today. There are a lot of different ways that we can go forward and I think it’s about canonical example that everyone gives with regards to jobs to be done which is a milk shake example and so people are buying that a lot, we need to go and figure out why. So people come to us a lot, we need to figure that out and I think that was the process that we went through. So, this space has a lot of different options and that we are definitely not the only option in town. I wouldn’t say what is the best option for you. I think we are [inaudible 00:05:57] for everybody. I think we are the best options for a certain profile and I think this is where the jobs were done [inaudible 00:06:03] what profile is that and it is not just talking to our users and I am sure there is some what we found out later on.

Chris Spiek: Yeah, so it feels like filling the gap, so that’s like we have low end, 100 dollar entry level and we have universities in 15,000 dollar programs. When you guys shaped this thing, it feels like, “We are going to fill this gap as we think that the market wants something in the middle.” Is that correct and then at one point you say, “We need to learn more about it, we can’t just rely on that promise”.

Chris Lee: Yeah, sort of it’s in a middle and also I think [inaudible 00:06:49] to what we value the programs that cost 5,000 to 15,000 dollars. They are all [inaudible 00:06:58] program. So it’s a very different execution model, you go into San Francisco, there and that’s the [inaudible 00:07:06] and people have to be there 24/7, you have instructors there. The environment is very different, so that’s really, really great because it’s much easier to focus when you are physically there and you have to charge a lot because you have physical building and physical instructors and everyone’s physically in the place. And also if you think about it, that model fits very nicely with getting a job which is their promise or their job to be done because they have [inaudible 00:07:44] employers there with students, all in one at physical location. For us from the outset, we didn’t want to do that. We wanted to be 100% remote, 100% online, that is just something that from a personal perspective we wanted to [inaudible 00:08:02] just for personal reasons, both do not live in a same city.

So, we are a remote team and we like being flexible in that regard. We also thought that online education was a viable path. Now, the question is how deep do you want to go? If we say if we want to be online, then what is the execution? There are still a lot of different ways to execute this. On one end you have nine dollars a month, producing [inaudible 00:08:29] model and higher sales team and do you [inaudible 00:08:32] a company. So, that was definitely an option that they think about that, but the core value there is going to be a little bit different than what we did today which is a higher service and less in number, less about analysing turn and all those metrics that you value when you have a product and it’s more about catering to each individual student that comes through and try and make sure that they have a good experience and so in this it’s not just a matter of – okay there is subscription model on one end and physical classroom at other end. So, I would just jump in the middle here. Even in the middle, there is a lot of different layer and execution models.

Chris Spiek: Got it, got it, yeah – thank you, I think that clarifies a lot of it.

Male speaker: So, I have a question at what point or have you figured out this point is maybe the job [inaudible 00:09:40]. It says we know exactly what the person is looking for once they have invested the 15,000, the high end course because they want to get directly to a job. When did you have the thought that you needed to find when Tealeaf hit in? While when you started with your first model and was there a day when you were like, “Wait, this is not feeling right”.

Chris: So, you know it’s a funny thing that we didn’t really – when you want to do this – I don’t think we had a market niche that we think about it from that, it was very much people who are coming to us and then finishing our program and just having a great experience or if someone who didn’t finish, we would give him a [inaudible 00:10:24] but the people that did finish, they would give this feedback [inaudible 00:10:20]. For example, when you first start the course, I would say almost every single person who finished said they wanted more. It’s like we just we want. We know how to use rails now but we now want to build a real app and we don’t know how. So, this is how our third course came about, was from that demand, we heard that. I think that was a good first step but in the back of our minds, we never really took time to analyse the job to be done and I think this is why when I heard about it, it really triggers something in my mind of – it would be nice to really understand what our users are really [inaudible 00:11:09], It’s nice that they are repaying us from a product and that’s great, but it would be even better if we understood what exactly they are looking for.

Chris Spiek: So, you are in that position of when they find us, they love us and we can sell them more stuff because they are asking for it right, but how do I get back to the beginning and what were they initially shopping for, what brought them to us because you can only sell things to the existing client base for so long.

Chris: Right, so it’s like we have this product that people buy and it definitely validated a lot of our initial [inaudible 00:11:55], so it’s not just a random stuff we are going to talk. We definitely have some ideas and those ideas were indeed validated. Well, I think what surprised me was as we do this more and more, there is a depth to what people want. It’s not just I want to learn rails, which is what our initial assumption was. There is a depth to how people come to us, why people purchase and I think that’s what jobs to be done, allow us to understand more which is, there is a process and it’s not even to just get a job [inaudible 00:12:32] in Boston earlier, you had mentioned that you know what do they really want when they want a job. There is another layer of depth. So, it’s not just a job, it’s a good job, it’s a job that is stable, it’s a job that pays better than their current job. There is a sense of working towards the future as opposed to being in the industry that is perhaps declining, so you want to switch to industry that is growing and you can invest in the future as [inaudible 00:13:09] that comes with that. So all these things come into play, and I think the longer we do this, the more I realize the depth of what people watch, and we are solving the first order problem which is they want to learn from technology, Ruby, rails [inaudible 00:13:31], but there is actually a lot of depth here.

Chris Spiek: That’s right so to me the thing is that you end up realizing that there is a first order job or there is that almost obvious type of job, but then when you start to almost filling that, it’s like seeing the world in black and white and then all of a sudden, it’s like you see color and you see depth and you see… The thing is that most of those things never relate to the underlying initial job. It’s like icing on the cake that make it special and add a lot of value without a lot of effort and so to me it’s that once you crack through the first level or the functional type jobs or the explicit type jobs and get to the implicit type things of the emotional and social peace, it’s just so cool.

Chris Lee: Absolutely, yeah, and again [inaudible 00:14:23] learning just the technology, there are lot of different ways to do it and we use the think – we viewed everybody as competitors, so there are so many competitors in this space and we really have to crack down on each and what I found out is when we [inaudible 00:14:41] interviews is people who are using all the resources [inaudible 00:14:50] and that for us was another insight into how our customers come to us. So instead of using these [inaudible 00:15:07] tenders, all of them, they could be perhaps channels and that was very, very different, so our mindset changes.

Chris Spiek: So, tell us about like that, the first set of interviews that you did and when you did them what was like – you came to Atlanta and you learned it [inaudible 00:15:26] and you walked away, so, tell us what it was like at that point where it was okay, I learned it now and what do I do I about applying it and what those first couple interviews felt like and then how it got different over time?

Chris Lee: Sure, sure I’ll also say that I actually heard about those jobs to be done even before the business offer conference that you guys presented at which was last year and it was I think [inaudible 00:15:56] and it was pdf to [inaudible 00:16:05) websites and I read it and I thought it was really great. I actually bought the PDF for 20 bucks or something from the Harvard Business School website and I thought that was really great. I searched around for how to proceed. I [inaudible 00:16:21]. This was couple of years ago and I really could not find any testimonials or how to [inaudible 00:16:36] like that. So, it’s always there back in my mind, but I didn’t really know how to proceed forward and then last year in 2013, you guys did I think a workshop. You guys did a presentation in the workshop. The presentation was great, I loved the presentation and I remember kind of point that [inaudible 00:16:59] this is the PDF I have showed you lot back and [inaudible 00:17:10] that’s funny and then Kevin actually attended your workshop I think [inaudible 00:17:15]. So, you guys happened to do a workshop and we thought was perfect. Right, it’s all coming together. I wanted to do it anyway and [inaudible 00:17:30] year workshop in Atlanta, so we had changed that and it was pretty amazing and we just thought – it’s pretty funny because that process, that historical process is what jobs to be done helps you revile. So that was like the third touch point of jobs to be done in my life, third or fourth touch point and so that’s the funny part and I converted, so when we came back, we knew we were going to do this. I [inaudible 00:18:06] couple of mistakes and I will talk about that, first of all, very, very successful. So we did two batches and we did dozen interviews each in two batches, about I will say a week and half apart. Yeah, it could’ve been way too much.

Chris Spiek: By the way, that’s hard for people to understand like okay, I did 24 interviews and I didn’t need that many and most people will be like, “I need 100” and like no you don’t.

Chris Lee: So I will just say when I finished this workshop, I took lot of interviews and some of them were very sceptical of what – we got to be sure here because I don’t see how you can just be without and about part of the interview. So, we scheduled almost 2000 interviews and first batch was great and had couple of mistakes in it, one is some of our interviews that people who had not yet committed, so they converted but our program is so long, it is not as simple as purchasing a book or a coffee maker and that’s your commitment to the coffee maker to use it for a couple of weeks and that’s your commitment. For us, it’s a very long program, four plus months and so you can actually convert or purchase register. I didn’t do anything for couple of weeks, just because you know it’s hard to sit there and watch video encode and it’s hard to get started and so that was a mistake, is to interview who haven’t commit and some of those interviews ,I think were not as [inaudible 00:19:43] we interviewed later on into our program.

Chris Spiek: Why do you think that is? Why do you think that they couldn’t –why weren’t they as impactful?

Chris Lee: Just because we have a pretty level [inaudible 00:20:04] because this whole business of educating, alternative education instead of going to college or after you go to college as well it’s very new thing so people always ask us if our program is right for them and then they will write like an essay and I can give indications, but I will not know for sure because this is, I would say, our program and I would say wearing a code in general, not just our program but learning any skills to a certain level proficiency is hard, not like particle physics, but it’s hard like losing weight. It takes a lot of habit changes, it takes a lifestyle changes, and it takes consistent coding every single day two, three hours every single day. I don’t know that I could do anything for two, three hours everyday for several months.

Chris Spiek: So, it’s a discipline versus like it’s just your brain power. It’s like a discipline that is what makes sense.

Chris Lee: Exactly, it’s not. Absolutely, what we do which is application development it’s not about mental course, it’s more about discipline, it’s more about exactly – and it’s about having a temperament is what we say right, it’s temperament should be above that’s because it’s getting frustrated at [inaudible 00:21:27]. So it’s developing that temperament, developing the habit of tinkering, and people are interested in way how the program because of the sexy marketing but we get into it, it’s called like being a car mechanic or something, you are some mentally moving things or allocating the bits so they doesn’t say you have try another piece.

Chris Spiek: I just have to say how far we have come that programming is sexy. I don’t mean to interrupt. I just that hit me, keep going.

Chris Lee: There is a leak and you got to figure out which two it is coming from. You certainly don’t know good way to do it is that okay check this one, no check this one, no leaking. It’s lot of that and so people don’t like it, I didn’t just type so and boom and I think that’s the part what I mean is only interviewed people for having it committed. It’s more of what they didn’t realize that what it took and so we hear pretty times of that. We don’t want [inaudible 00:22:34], program so interviewing those people is really helpless. It didn’t help us make things better. We were able to convert them, but that’s actually not a good thing. We don’t want to trick.

Chris Spiek: That’s the whole thing to me is, that if you interviewed people too early again we think the commitment is the moment when they purchase and the real commitment is the moment in which they change behavior, and so just because they give us the money it doesn’t mean that they have changed behavior and so if you do it too early they are going to give you the rationale or they are going to backward justify what they did and it’s that conversion of how it helped and do what they wanted to do and so at some point if you are trying to sell them and convince them, you are actually not helping them. They need to convince themselves.

Chris Lee: Absolutely, absolutely so we tried to do that on our – to really talk about how we work, what we do and the fact that we helped a lot of people, a lot of people have gone through this and changed our lives. They are going to be right for you. I am not a 100% sure, but given this a shot I won’t give you – it’s not. So, interviewing the people who have committed and you are absolutely right in that. You have to look at behavior and for us at least it’s not just where infrastructure happens, it’s when the behavior change happens and we can see that through the code of conduct. If previously you haven’t really committed that much load on a daily basis and then after you start our program at some point you started committing very frequently right between four, five times a week then that’s a behavior change and then we interview that so I think that was from camera one – was not figuring just put the improved too liberal art who to interview…

Chris Spiek: And so let me just say, I just want to fix the language. I don’t think it’s a mistake. I think at some point if you are to hire an expert and have bring us in, I think we would have stumbled into the same thing because until you are grounded in the purchase process that people really go through you don’t know – it’s you go to the natural purchase process. They signed up, they gave us their money they bought. That’s the buy right and then well you do five interviews they are like that’s not commitment. They gave us their money, but they are not in yet, and we need to move the time right around, so I think I know you are calling on a mistake. I just want to call it out mistake and say like if other people that are listening to this find themselves in that situation, I think it’s just part of like alright, we are going to shift things around, we are going to keep going.

Chris Lee: Sure yeah.

Chris Spiek: But the two things is, is that, that’s the importance of the timeline is that the timeline based on commitment, not based on the moment that – we usually say the moment they buy but it’s always, it’s got to be back to the moment where they are actually committed and so we always talk about the example of – we did some work in fashion and you found that women would buy dresses, but they weren’t committed to the dress till they cut the tag off, and so you have to shift the whole timeline to the moment that they cut the tag not when they bought it.

Chris Lee: Yeah that’s the point and maybe I shouldn’t call on it mistake, but a hindsight we could have done that part at earlier.

Chris Spiek: I wondered though you went in this though because I have a question, so I go to Tealeaf, I say I want to learn how to code. The guys give me a call and say, “Have you dug in yet”, like no, it’s been five weeks, it’s on my list of things to do. Who’s job is it? That’s poor choice of words, but is it Tealeaf’s responsibility? Is it the consumer’s responsibility to dictate when it’s commitment because the reason that I am asking the bigger question is massive online courses now have the same issue of people who sign up in rows, but they won’t finish the course. So, exactly what point and how do you figure out when they are committed?

Chris Lee: Right. That’s a really good question, and I don’t think we have a definite answer to that but what we do is, we try to be as transparent as possible. For example, we have a book that we have written and it’s completely free and we want people to do that book prior to registry, and that is a self-task. If you can get through that – and just the way we offer our business, which is again in pretty liberal reforms we have a very, very high incentive to be very transparent in other words we are not trying to convert people or inaugurate, so it’s not like a gym model where we sell as many memberships as possible and we hope we don’t come for the [inaudible 00:27:07]. That’s not like it at all, so it’s the idea of trying to sign and identify the right books, trying to give you information before you pay us, before you register. So we have some books that we have written completely free and the first thing we say is do those books, do that book and go through all, it’s pretty wrong, I mean it takes some time to get there, and if you can’t, and then that’s hard, so don’t come to us yet at least and if you can that’s a really good indication. This is not whole truth, but it’s a good indication that you have committed some degree. This book was actually – we just lost it early this year and we didn’t really think about it in terms of as a sculpture or like that but now I think it is a pretty good indicator, so this has the first barrier that you should do before registry.

Chris Spiek: Yeah. That’s awesome, but I want to go around the tape and have a question. Now you said, this is not the gym model, but if I go talk to my wife and it’s like January second she is extremely happy with her gym purchase. It’s new year, she is going to lose weight, she is going to spend 60 bucks a month to join a gym, she is happy having paid that money so tell me why not, that’s a very profitable model people using all around the world? Why not do that?

Chris Lee: Yeah, I just mean I guess from the healthy nature of gym membership, so like they are always selling membership and not every gym but typically gym sales people are very pushy and they are trying that you sign out for a year, two years or some plus years, I am hoping coming to that and I think – just have to work a little bit but that’s not what we are trying to do. We are not trying to convert people for – converting and locking and if you want to leave it’s nearly impossible. It’s like if you demonstrate that you moved by the littering. I have moved around a lot so I have been in this topic a lot of time. You can’t ever go through a gym, it’s only for a month. Like that’s not possible.

Chris Spiek: Right, right, right.

Chris Lee: Or per use, so sometimes you will go some place and say yeah you can come for free [inaudible 00:29:44] like I am not from here. I am just visiting. I mean a place because I have to [inaudible 00:29:44] that I am willing to pay money for it but you won’t let me pay money because you are trying to convert me. It’s always like that. That’s what I mean and sort of trying to trick people into paying for perhaps more than what they want to use without regard to whether or not they actually pop.

Chris Spiek: So I want to jump in here. I have like ten things on the board so the first thing is your wife being happy is not progress. Whatever her goal is, the reason why she signed up is progress and so she could be happy, like I got off my list but she really never intended to make progress within the first place or if she did the fact is, is that she should do it but any static point of happiness is not necessarily measured as any degree of progress for me. So, just because she is happy doesn’t like okay like I might lose weight okay and I know I got to lose more like okay I am happy I lost few weight but I not done and so got more to go and so the whole thing is that it’s this happy and liking metric is not the right metric to be able to measure people on.

Male speaker: Yeah so a part of me wants to disagree. I don’t know if this is the right time to debate this. I think short term happiness can’t be progress. I think she could do every year and still she is happier the day after she signed up for the gym membership than she is the day before. It might not be sustained so here is what I think. I don’t think it’s sustained the progress. I don’t think the gym industry is necessarily doing people a service by doing it but I think at the end of the day they realize that this is how consumers behave and roll to maximize profitability. They sacrifice true progress at the expense of the consumer, right.

Chris Spiek: But Chris is much better off getting people to earn the right to join the class and having because you know they are committed because now when they spend a couple of thousand dollars for the class you know they are ready to do it and they will get the value out of it, but if I charge people and they, “Yeah it wasn’t for me.” First they blame themselves, they blame themselves about the gym and it’s like, “I didn’t have time…” and then they then at some point turns like, “Yeah it wasn’t the gym for me and it’s really crowd”. By having people earn the right to take the class that to me is the thing of knowing that people are going to be committed. Your conversion rate must be like the number of people who finish must be much higher now that you made the book the hurdle than before.

Chris Lee: Right, right and I think gym membership, basically any class model because that’s what class model are, it’s analysing conversion and truly not about all a lot it’s – I heard the intercom interview on your podcast that was really good because intercom I think tried to solve this problem which is helping vendors analyse their customer usage so that they can make sure that customers are maximizing their products and there is very few products that do this. Most of them don’t use anything like Intercom. They are just happy to take all the revenue, which is what gym membership is they are very, very happy to take down the revenue which is that’s hard, that’s coupled, that’s now how we wanted to do.

Chris Spiek: Some people can want that business model but at the end of the day you need to add value and the moment which they pay is not when the value is created. The moment that is created is when they make the progress they were intended to do with the money they paid. It’s not an instantaneous, “I paid $5 and four months later I now can program”. There is no instantaneous, “This is such a great thing”. So, to me it’s that.

Chris Lee: Right. I think that the beauty of [inaudible 00:33:40] and it’s really about that which is it’s not about price, it’s about now it’s understanding what the value is of your product and then you can decide how you want to modify that back. You can say I want people to pay when they receive value or you want to people say I want people to pay no matter what. I am happy to see that. It’s sort of [inaudible 00:34:04] decision. Once you have the information you as a business can make your pricing decision.

Chris Spiek: So can you give us an example of something you learned that from the interviews that 24 that you did what you think away?

Male speaker: So, I want you to answer that question but I just want to point out how I believe this is profound then I think what you described over kind of last five or ten minutes is, you have ultimately made your products more difficult to purchase and I don’t think there is any other way of saying it because you could grease the skids, you could do 30 days free, take out the less, you could go in that direction and I could rattle a 15 ways for you to do it but you have intentionally gone in and said, “I can’t take your money until you read this book,” and you do all these other things and you increase the value that you deliver to the people that sign up and what will bobble through is the progress that they make by doing that and I just think it’s one of the things where you do your interviews and you get the insight, like “Yeah that was a great decision”, but there are people out there that I think need to realize there are times one have the right to do that and you gain that knowledge by doing the interviews, doing the evaluation and saying yeah this is of sound reason. We need to tackle this.

Chris Spiek: All right, so my question to you then is, was the trigger that this was the right thing to do? The fact that coding is so hard to do is it the idea of once you get into the muck and mire of learning how to code you are going to come up with guess lot of obstacles and barriers you got to push through. If you don’t have this strength, the persistence to push through those you are not going to be happy with our course. I am not even won’t let you spend your money versus if you are selling books, the book to read it you just need to buy thee book now and be on your way. I don’t need to make this process harder for you like what’s the distinguish, what’s the trigger to tell you which way you are supposed to go?

Chris Lee: Right, right. How many books do you have that you have never read? I have a tone and I hate them and I didn’t really derive too much out of it so yeah I think again I think the job [inaudible 00:36:07] doesn’t inform you what you should do in terms of how to monetize that knowledge but it gives you the knowledge of where the value is. So for us wear blue straps. It’s of Kevin and I and what we get out of this is incredible enjoyment when we see people have transfer making their lives. We went to a [inaudible 00:36:31] conference recently and we had dinner with a couple of our previous students and it was amazing. We met there and a few others and it was just a great amazing experience having those connections and it’s not – we never go did that we have a [inaudible 00:36:44] so it’s a different – so you can derive different things that we are not venture not, we are not trying to optimize on revenue. We are definitely trying to optimize on bringing value to people as best as we can right and do things like jobs and go and help so illusive or illuminate and to be what the value is.

Male speaker: You started with a philosophy I think that’s the most important part. If your business philosophy or your approach to business isn’t what it is, we want to ultimately provide a value to the consumer and help them make progress and this isn’t going to make a lot of sense I mean the gym membership and I am not judging sales models or gym memberships, but you are not going to find a lot of value in jobs we have done. It’s my belief if it’s just let’s sign them up, lock them in and…

Chris Spiek: So to me it’s really – all we are really doing is unpacking that word value and unpacking it over a time and unpacking over when does it actually occur and how does it occur and what’s the cause of things that happened with it. To me it is ultimately just understanding not how do we deliver value but actually how do your customers realize value and when do they realize the value and how do they realize value, it’s those combinations of things to me that are like – and that’s the amazing part is that you see patterns that some people have values because they buy it first day. Another people have value three years later it’s like, “My god this is awesome”. That’s where you have to understand how to trigger the value so that they understand it but also that they can see it going in because anybody who can’t see the value going in won’t buy so you got to be able to have value going in and on reflection.

Male speaker 2: So Bob like 15 minutes I think you have asked a really good question. Do you want to repeat it? Because I…

Male speaker: I would like to know like one big like takeaway like where you guys kind of like had the interview, you did a couple of like – you put connected some dots and what wholly crap do you have one of those?

Chris Lee: Sure. Yeah we have got several and couple I will share, first of all, I remember when we started doing this Kim and I attended in the conference last [inaudible 00:39:07] they stop doing it some of the less of county guys, just amazing conference. I just had mind blowing experience and it was we have a [inaudible 00:39:15] moment where we were okay we are going to drive the company this way because that was just inspirational. This one speaker talked about him living in Wallstreet and pursuing his own business and we were just really pumped up about it because here where we are struggling with. We are struggling with the fact that we can’t really become a job [inaudible 00:39:41] company because we are more comprehensive and international. There is no way we can play somebody in Taiwan because we don’t know a lot of companies in Taiwan or that’s right so – we ask you is all over the world they are just not on our core and in person who are amongst physical place. They may offer have really, really tight connections with recruiters which then can refer their student to the global companies. That’s just not what we do and also we don’t have very strong [inaudible 00:40:13] this is what all employers are looking for. Give me your smarter student’s, but we teach people skills.

So there is a lot of hire ability factors that we just don’t consider when we let people take our course, what school you graduated from, what previous experience you have, things like that, that normally you can filter out and say hey we have a 100% placement. It won’t be very easy to do that actually. We can say we have a 100% placement rate. I ever sell [inaudible 00:40:44] we let like few people in. We teach everybody and so we thought after attending our conference we were really pumped up when we go you will not teach how to bill applications and be very entrepreneurial, so that’s the task we are going to pursue I propose to the job task in a job task. And then during these interviews we would ask that which one do you prefer, that was one of our questions and everybody wants both. It’s not one or the other. There is no job task and then the option [inaudible 00:41:16] it’s pretty much – yeah I [inaudible 00:41:20] yeah we have been working on my project. So that was a complete bust over guess in terms of what we want to pursue, otherwise we would have fit the company in a very unnatural way I think to be entrepreneurial or intern.

Chris Spiek: So I just want to make sure I got this right so what happened is you built a theory at the market in how it works and it was either people who want to get a better job or people who want to build their own products and you were going to go after those people who want to build their own products and when you did the interview it was like, “Yeah whether I work for somebody I do it up myself I just want to know how to program”.

Chris Lee: Right, it’s and what you said Bob about why did you want to a new job is because I do want a fair life and it’s building your own company will build their own product and I guess that’s fine too. Like I am not going to do that right it’s or if my product doesn’t work out and yeah I will go get a job. So it’s not either or usually.

Male speaker: And they don’t have to work that out before learning the program, I think that’s the other thing it’s like you are asking me this question but I have got some to figure that one out. And I can just imagine like the position like do you want to create your own product? Yeah I don’t know I want to keep my job you know like if I have to you know to me it’s like all this anxiety you are creating because of that position very, very interesting.

Chris Lee: Right so that was a big – okay we definitely don’t want to because if we would have done that we would have added a lot of how do you still do analytic and very weird topics about [inaudible 00:43:06] into our technical group topic and I will say the other sort of the opening was that you got a connection to this, I was looking to provide notes from this workshop and I had to written in I wished I don’t remember but let’s unless there is this four person right and unless the anxiety is actually a really, really powerful force that people can try to address enough.

Male speaker: What do you mean?

Chris Lee: So, for example switching to our program it’s very much about not only the magnitude of getting a job or building an idea but also letting the anxiety of taking a course like ours because when you learn on our own right they are really switching from self-guided learning or learning from low cost resources. There is a lot of anxiety when you are using that and so our curricular helps snuffing that anxiety and I think that’s a big selling point. It’s not just about you used to get a job with our curriculum but also there is a structure and there is going to be people around to help you and you can be sure you are learning the right things right instead of [inaudible 00:44:22] group and reading a book on medical ravings and two months later you still can’t relate the very simple allocation doesn’t feel good. The anxiety of making sure that you are moving forward is actually pretty, it’s actually a pretty large anxiety and we try to address that in our curriculum and it’s not just promising the world afterwards. Sometimes it’s just about listening back there.

Male speaker: Well, yeah and the other thing is over promising actually created anxiety and that’s what a lot of people don’t understand, “We can do this and this and this and we will teach her to slice spread as well” and it’s like “aha”. I think the other thing what you talked about discipline and commitment I think on the anxiety side we always talk about trusting me and trusting the product so it’s like – even if you prove to me that you guys are the best possible teachers of Ruby and robust application development I still don’t know if I trust myself that I can get through it. I think that’s the other thing you have probably showed up by creating the book is like go through the book and get a taste as to what this is going to be like and build the trust in yourself or disqualify yourself and say maybe later or maybe a different time, but I don’t trust myself enough to get through it.

Male speaker 2: So, here is the beauty of this right now. You have people read a book which is more work and it’s literally outlining everything but what you are actually doing when they read the book is they are getting more confidence in what they actually know and we do see the anxiety and so the notion is, is that as much as you are – the people at the first level look at it and go, “My god they are making it so hard”. The reality is that the second level you are actually giving them more confidence to know that they can do it and so they value it even more and it’s the decadency of the people looking at your system and going, “My god they are going to die, they have to read a book”. They are having to do and the person reading the book, “I know that and I know that and oh yeah I know that. I must understand this stuff. I can do this and this.” Is like wholly crap the whole thing of you creating that barrier is actually giving them more confidence in themselves not necessarily more confidence in the product and bam, again it can only see that in Hindsight. You know we can see that in the interview.

Male speaker: So we have got about two minutes here. We are going to lot of wrap up. What do you want to leave us with? Hey Chris what’s kind of the last parting thought?

Chris Lee: I would say what I love about the job to be done framework here, it’s really easy to learn. My first job was in a large, large company I was working and consulting for a while and I totally understand the desire for those large [inaudible 00:47:21] to make simple prophecy difficult because have to fill consultancy hours and why I love that job to be done is because you can attend these workshop and then you can go and do it you can do it quite quickly and quite cheaply and you don’t have to hire army and consultants to implement that practices and come for five month in [inaudible 00:47:42] like that. That’s what I really loved about it and I was really sceptical about what we think to right out of the different focus groups, so actually we derived a lot of value out of it and out of the two sets of dozen interviews – I would say that the second set for the most part was getting repetitive like we fairly how we understand the story, we have seen this profile, the story five, six times already and they make sure enough tell the same story about how they got through us and what resources they used. It doesn’t take a ton of time. We set aside I think but we can have to do this and basically after the first set of eight interviews we had a pattern and I actually called Bob about this and I said are we meeting within here because we are getting the same story and it doesn’t seem right like something smells wrong here.

Male speaker: I remember that call by the way it was like, “Okay I don’t think we are doing this right,” and you recorded and you send it to me like those were awesome.

Chris Lee: Right and so when you said it was as long as you are treating the historical process it’s virtually impossible to lead the witness and when we are very aware so we try not to, but it’s just all the stories dubbed tale into a couple of main narrative and so it was very insightful, we thought we are dealing with just a worldwide audience of different background. We are but at the same time they have a similar journey.

Male speaker: All right so thank you so much for opening the door for me because I have been trying to find where this squeezes in for a while. All right so Bob last week before you and I were sitting in a meeting with a client and they were as usual we always get this question around a ten interviews can’t be enough and you said something around like statistical significance and general population can you… My whole thing is that first of all Chris is not worried about statistical significance right it’s like at what point do I enough confidence that we are headed in the right direction and so to me the first this is that it’s as a developer I am looking forward statistics is not the method in which you built confidence so, some people do and especially in a larger corporations but the reality is that I feel like with ten interviews I get a thousand real data points as opposed with a thousand surveys I get ten real data points and so my thing is that with ten interviews I can actually learn more than I could with a thousand surveys and so that the notion is that these interviews are so rich and they are so – when you start to see the patterns it’s like now you can get and as a developer – develop more not from a marketing perspective but from a development perspective to know how to help people make the trade offs and so to me once you see the patterns you know the trade offs they are willing to make. You know how they think. You know what they did, you know what they had to tell themselves in order to say everything prepared me to get here now I can spend the two grants and take this class. It’s like okay if I get five people to tell me that it’s like, “Have you worked hard enough now you are ready to really do this come see us” and it’s like, “Yes that’s me”. I don’t need such a significant threat.

Chris Spiek: Awesome. Well, Chris thank you so much for the time. I know we tend to run into you guys for at least once a twice a year around the country, I hope that continues because it’s always great catching up with you and so glad you are doing well but thanks. I know you are busy and we have to wrestle schedules to get you on here but I am glad we did because I think the story is really, really valuable to the audience.

Male speaker: Thank you so much.

Chris Lee: Yeah, no problem and look forward to see you in teacher conference somewhere.

Male speaker: How can people follow you, connect with you? What’s the place to go?

Chris Lee: We are on the normal channels, Twitter gotoleat.com, you can find more about us. [inaudible 00:51:24]

Male speaker: Gotoleaf.com is the place to go. Thanks man. Talk to you soon.

Chris Lee: Bye.