Super excited to publish this interview! I recently spoke to Saul Costa, co-founder and CEO of Codevolve, an exciting new company which makes an amazing product for interactively learning computer programming. Go check it out.
HM: Yeah, I’m doing good thanks. It’s nice to have you, thanks for coming on the call.
SC: Absolutely, thanks for having me.
HM: So, tell me about yourself; what’s your background?
SC: I’m a 23-year-old programmer. I’ve been programming since I was about 8 or 9 years old. I started mostly because I really wanted to create video games and my parents said well if you want to play video games, you can start writing them in BASIC. They gave me this little floppy disk of Basic video games.
From there, I went on to study computer science at Norwich University, which is a private military school over in Vermont. I then came out to Silicon Valley and worked for a Y Combinator company called MixRank, doing a lot of data engineering for about a year.
While I was working there, I was in communication with some of the faculty at Norwich and they were mentioning a lot of the problems that they had in the area of teaching computer science.
I really wanted to create video games and my parents said well if you want to play video games, you can start writing them in BASIC.
Whilst I had been at Norwich I had tutored a number of students and found that when they came to me and they would be stuck, it typically wasn’t because of the learning materials or the course they were in. It was just because their tools weren’t that great and the bug messages or error messages they were getting from the compiler were very cryptic and they couldn’t understand them.
So, I was kind of remembering those things whilst I was in communication with the faculty at Norwich.
Eventually, we started a company that hopefully can address those problems and what that’s grown into is this really simple and easy to use online coding environment that also offers bug hints and bridges the gap from that first line of code, all the way through to when you start using a professional development tool.
HM: Right! And how long have you been working on this particular project?
SC: So we started Codevolve just under 2 years ago, so pretty soon after I moved out to Silicon Valley. We went full time on it – when I say we I mean myself and my co-founder Lauren Wyatt - back in March 2015, so full time on it for about a year.
In the past year, we’ve gone through 2 accelerator programs here in Silicon Valley. One was the Plug and Play accelerator programme in Sunny Vale and the other was the world-renowned Imagine K-12 Edtech accelerator, which recently got acquired by Y Combinator.
HM: Oh wow, that’s cool! So, your initial users came from there. Who’s using the product now?
SC: So the way we’ve transitioned the product is rather than partnering with students or teachers directly, we actually partner with content companies that have different types of content for learning to code. So, textbooks, videos, online courses – things like that.
That’s proved really well for both parties, because for us we are able to build a really solid business model and that’s the thing that can be tricky to do in the consumer market sometimes. For them, they instantly have this really competitive online coding environment which they can configure in any way they want and embed into their website or learning platform.
We’d like to see this type of tool become the standard for when you learn to code.
HM: Yeah, I thought your product was a bit unusual in this space. Because there are people either building their own stuff to differentiate or people selling content.
SC: Yeah, what we’d like to do is rather than this as a key point of differentiation, we’d like to see this type of tool become the standard for when you learn to code. So when you sit down and learn to code, it’s one part content, maybe three parts actually sitting down and doing.
Unfortunately, students today trying to write those first few lines of code are using a professional development tool.
They just get very bogged down in it and get stuck in the details; there are lots of unnecessary details for someone just starting out.
So, hopefully we can get really widespread adoption for this tool or tools similar to it, and make the standards and ensure that the quality of tools that students use is really high.
Unfortunately, students today trying to write those first few lines of code are using a professional developer’s tool.
They get very bogged down in it and get stuck in the details; there are lots of unnecessary details for someone just starting out.
HM: So, actually just to go back a little bit, could you describe the product? Tell us in simple words, what it does and how it works with an example or two?
SC: It’s basically a platform for getting hands-on experience with tech skills. Right now, that’s focused on computer programming.
So, there are two sides to the platform; one is this in-browser, coding environment that works with a multitude of languages - web-based languages, scripting languages, things like that. It allows the student to come into the project, write a few lines of code, hit the run button and instantly get an interactive console or website to be viewed which shows up and is their actual program being run in a self-contained coding sandbox in the cloud.
This is really good for students since they don’t need to set anything up locally. They can just come in and they have everything they need right there. As they are going through their code, we are giving them a lot of feedback on it.
When they get stuck, we extract the error messages out of their compiler, and translate it into something that is easier to understand. Really just trying to get more students into programming and lower that barrier to learning.
This is really good for students since they don’t need to set anything up locally. They can just come in and they have everything they need right there.
The flipside to it – and really what makes it a platform – is that we have a content offering environment that allows different content companies to come in and create programming labs, and configure the in-browser code editor in every way possible.
They can load in example code ahead of time, they can customise automated grading, change the theme and the look and when they want to publish it, it’s as simple as embedding a Youtube video into their website or it integrates directly into their existing platform. This makes it more of a platform rather than just a tool.
HM: Ok, so I’ve played a little bit with your product. I guess for most people, the analogy would be it’s like Codecademy.
SC: Yeah, it's pretty similar. Codecademy has a really great approach in the way they’ve done the step-by-step walkthroughs. We have the ability to create a very similar type of project where you can create a step-by-step walkthrough.
But unlike Codecademy, you actually don’t need to write any code to build something like that. So you can do that from the authoring tools and it’s completely web-based.
HM: Oh I see. I suppose if you’re going to create this kind of content you need to know the subject matter, so you probably need to be a programmer, although you don’t actually program.
SC: Lots of the people we see understand coding, but they are not developers. So, they can write a few different programs in C++ for example, but they might not be able to build an entire application for the internet.
That’s typically the makeup of these content teams; they have that subject area knowledge, but they don’t have development experience. So for content companies where the majority of their team is in that position, this is really a game-changer because all of the people that were previously going to need to rely on developers for an environment like this, can now just come in here and create that content without having to rely on developers.
HM: One feature you mentioned is that it provides very helpful error or debugging information. Can you give me any specific examples? Because when I’ve used Codecademy that’s been one of my frustrations some of the time, and I guess there is a limit to how intelligent it can be.
For instance, when you say print this sentence and even if you change the string a little bit, instead of saying print "Hello World", you say "Hello World!" it says “Oh no! That’s wrong, you’ve made a mistake.”
I’m an experienced programmer so I was just testing it out, but I can imagine for someone new that’s very discouraging.
SC: Yeah, definitely. There are two sides to this really. So, the one you’re describing is really when you’re building a tutorial or something on the Codevolve platform and you can configure that to be pretty flexible.
It's based upon the content creator and how specific they want to be. We have some companies where they want a really specific set of instructions, or specific set of grading requirements.
Then they want other ones where they just want to make sure that they’ve used different programming concepts. So, the way that we’ve structured the content tools, based on how you can look at the program's output and also the code itself, adds that layer of flexibility.
Then for the error messages that come from the compiler or runtime, what we are doing there is applying some really basic artificial intelligence that we’re constantly working on improving, to be able to actually look at the error messages that we get out of the programming project, then figure out what the best error message is that we can give to the student at this point.
Not to give them a solution but rather to give them a very general hint essentially, to point them in the right direction.
HM: Ok, so just wondering if you could tell us about the software. How does it actually work? How does the code evaluation work?
SC: Absolutely. So, we really have three main parts to the stack.
On the front-end, it’s about 40,000 lines of React JS, so fully JavaScript based.
We made that switch about 8 or 9 months ago to using React and that’s really when the product started to take off. We found React was this tool where, approaching it, our team didn’t have a lot of experience with JavaScript type frameworks, especially not data driven ones like React.
But once we mastered that and started to incorporate it, it’s just completely changed the way we that we write front-end code and definitely helped us do it a lot faster. So, we definitely recommend that as an alternative to Bootstrap and Angular for front-end frameworks.
Then for the server side, we have a Ruby on Rails app. That serves two purposes: one is to deliver the assets and web pages. Then we also built what we call a code engine.
We made that switch about 8 or 9 months ago to using React and that’s really when the product started to take off.
The code engine is really what ties the front-end and the user interactions back to the third part of the stack, which are these coding sandboxes.
It’s like a translation, or intermediary where the student gets some code input and then we run that in a very specific way through those coding sandboxes and then extract a lot of pieces of data and useful bits that we feed back to the user that’s actually on the webpage.
So that’s a pretty important piece of the overall architecture - being able to do that in a way that’s safe and secure; when you're executing code you have to make sure that you do it in a contained environment.
Also, something that’s very fast and will scale very well. When we work with these companies they start with 20,000 new users, it’s not like us going out and gradually adding new users.
We had to make sure we designed a system that can scale very well; not just in terms of usage, but also adding languages and adding functionality to the code engine itself. So we really built a dynamic assembly line in a sense, that takes the code from the user, runs it, and then passes back the relevant data.
And then what we do at the very last moment is break away in these language-specific classes that control various things that are different between programming languages.
So, looking at what the types of outputs are and specific runtime commands and things like that. That allows us to very quickly add new languages. This has been great because if we are working with a new customer, and they have a specific, new language they need, we’ll go and just add that for them because it only takes 24-48 hours.
If we are working with a new customer, and they have a specific, new language they need, we’ll go and just add that for them because it only takes 24-48 hours.
HM: To add a completely new language?
SC: Yeah, it’s very straightforward because the overall code engine makes it very dynamic and there are only about 100-200 lines of code for adding a new language.
HM: That’s really impressive! It sounds like a pretty complex set-up; what sort of problems or challenges have you faced while building this?
SC: The biggest thing is definitely the code execution. When we originally did this our first pass, about a year and a half ago, was a really simple BASH script that would use Docker containers to execute the code.
Since then, it’s evolved quite a bit. We have a very streamlined process and we're able to interact with our coding sandboxes in a fast and interactive way; so the bash script was kind of static and the new way is very interactive. I think the biggest thing we faced taking it to market – as is the case with lots of companies – is that you never really realise how many different ways there are to break something until users start using it!
I think we are especially subject to this because there are so many different ways to write code and users will find every single possible way to write a piece of software. So, for example, for loops that are stuck, that was something that system didn’t initially handle.
So, as user would go to write a for loop and one user could completely crash the entire system. So, now of course, we’ve addressed that and it handles it very well but it’s a learning process for us; going through and finding all these ways people can break their code.
I think the biggest thing we faced taking it to market is that you never really realise how many different ways there are to break something until users start using it!
HM: Yeah, I mean allowing people to type in any piece of code into your system and executing it is pretty scary. I’ve worked a lot with data analysis systems and even allowing people to upload their own data files creates crazy problems. People upload all sorts of weird stuff and there are so many edge cases.
SC: There are so many things that don’t make sense until you see it. Especially when you’re working with people who are just starting with programming. We’ve been going through and working on improving the error messages and as part of that we’ve been sampling code that causes different ones.
We’ll pull out a syntax error that has 50-100 different combinations and a couple hundred pieces of code that are all broken in a new and unique way. So, it’s pretty interesting and an insight into what a new programmer is going through when they first start out.
It’s pretty interesting and an insight into what a new programmer is going through when they first start out.
HM: Yeah absolutely. I’m also interested to find out a bit more about actual users’ stories. You mentioned you have a certain class of customers who are content providers and don’t necessarily have the tech skills in-house for building this kind of thing.
So, give me some examples of what kind of things people have built with Codevolve and what experiences have their students had?
SC: We have partnered with a variety of companies, all the way up to really large textbook publishers, down to pretty small coding bootcamps and then everything in between.
The feedback so far, when our users have come online (mostly this semester, six weeks in), students love it.
I just received an email actually a couple of days ago from one of our customers who is doing a session with students. They said that they have never seen the students this excited about a course.
Their focus has been mainly on languages like Java and Python and C++ and then some other partners have really started to push the web development languages such as HTML, CSS and JavaScript.
But what we’re really hoping to achieve in the next 6-9 months is to push this out to a few other areas of tech as well. So rather than just coding, I think there are interesting areas around databases and things like that where you can pick the same approach of making a very simple and configurable UI and then open it up to the content creators as well.
They said that they have never seen the students this excited about a course.
The students so far, what they’ve said is that this is solving the problems that we set out to solve. It’s something they understand and is simple to use.
Even from when we initially launched this two years ago, they have absolutely loved the bug hints that we give them. Professors and students have told me that it’s solving the problems we want to solve and that’s really good to hear I think.
HM: That’s the best feedback you can expect, right? One question I still have; you know when you mentioned that a complete beginner starts to get bogged down in setting up the environment and code editor and all that. In that way, a live environment like this, when they can just start typing in the browser really helps.
But having worked as a professional for a while, how do you think about the transition to using a code editor?
SC: Yeah absolutely and this is something we’ve been pushing out more and more is the point at which using a tool like ours starts to have diminishing returns and it would be better to use a professional development tool.
When we initially started, it could handle your first couple of CS courses in college maybe, but now it’s a competitive product through to professional development because we’ve integrated things like graphs and you can interact with the program like you’re running an actual terminal.
I think that timeline has gotten extended quite a bit. But what we are trying to do with all of our products is look at: what’s working in the development community and what are the concepts that people are learning and using.
Then we take those concepts so, for example, Sublime. Our code editor is very similar to Sublime, if Sublime had a Run button. Or a terminal which is what you actually end up using when you’re writing code as a professional developer.
So, we take those concepts and we just try to massage them a bit and make them easier for a beginner to understand and put an emphasis on learning rather than being a developer.
One thing we’ve worked very hard to do is – we’ve gotten a lot of feedback for features that would push us into being more of a development tool and we’ve always pushed back on that because our goal is to stay very focused on being a learning tool rather than a professional development tool.
But hopefully even when students do transition off of that, which we’ll start seeing over the next couple of years as students graduate, they’ll be able to recognise the key concepts that they've learned on Codevolve in the professional development tools they're using.
So, we take those concepts and we just try to massage them a bit and make them easier for a beginner to understand and put an emphasis on learning rather than being a developer.
HM: It’s funny you mention Sublime because I was just thinking would you ever think about making a plug-in or integrating with text editors?
SC: Yes, that’s one of those things that makes us more of a tool designed for developers that we’ve tried to stay away from. I think there are lots of interesting things that could be done in that space to improve it, but right now the main focus is on helping students get started with coding, helping them when they're actually coding and eventually other tech skills as well.
HM: Right, because one other case apart from becoming a development tool is offline these days.
SC: Yes, and I think at some point we are going to have some sort of offline editor and it will be very simple to download it and continue using it as well. But that’s just to kind of support a lot of edge cases or things that aren’t possible to do online. That’s a bit down the road and it’d still be cloud-powered so all the code will be run via our systems so there wouldn’t be any set up required or anything like that.
HM: Right, so, one of my final questions for is what is your vision for tech education, especially online? What do you see happening and how is it going to change?
SC: I think it’s definitely getting very big. If you look at the job growth for any area within tech, it’s well above average.
Especially in the US, there are certain initiatives such as Computer Science For All; it’s a $4bn funding plan that Obama has presented to get Computer Science education into the K-12 space. There are kind of a lot of shifts that are happening right now that are altogether going to form a massive wave in terms of technical education, especially online.
What we are trying to make sure is that all these students that become interested in computer programming in K-12 or when they get to college is that they have a really great tool for learning coding, for learning database, for learning any type of tech skill that they want.
That tool's not a professional developer tool, it’s designed for learning and lowers those barriers so that when they when they start that first CS course they don’t get stuck with something we can help them overcome with technology and that they actually go on to succeed as a programmer or someone else in the tech field.
If you look at the job growth for any area within tech, it’s well above average.
HM: Yeah, I’m really excited with this space. We are quite early in our journey at Learnetto and I’m actually quite curious to try out Code Evolve for one of our courses.
SC: Awesome – would love to have you on the platform.
HM: Great, so that’s all the questions I have for you today. Where can people find you online?
