Made by
Made by
Joe Nash
Training programmes take a lot of commitment but their returns are high. At DevRelCon London 2018, GitHub’s Joe Nash spoke about the benefits of devising tailored programmes, the four learning principles and the achievements of the award-winning GitHub Campus Experts.
Hello, everyone. Thanks very much BB for the lovely intro. As she said, my name’s Joe Nash. I’m the Student Program Manager at GitHub. I help our education team rollout programs to 1.1 million students.
And I’m here today to talk to you about the topic of developer education. Because as developer relations professionals, we work in an industry that is very education reliant. Think about it for a minute, can you think of any other product that is as reliant on educating consumers at the point-of-sale as developer tools and development products? I can think of maybe one, well, maybe two, IKEA Furniture perhaps and power tools. But I think most chainsaws are easier to pick up than our APIs.
So, I want to talk to you today about how we can properly educate our developers and use some established learning principles and the learning sciences to improve access to our products. So to get into that, we have to start with a really fundamental question, which is, how do developers learn?
To answer this question, I’m going to look at some popular developer surveys. The infamous Stack Overflow survey, as well as surveys from two companies that are closer to education, HackerRank, a program for competitive coding and learning through challenges, and freeCodeCamp, an online community of people learning to code. And what we see from this data backs up this intuition that we’re a developer or we’re an education-heavy industry.
What we see is that developers are always learning. For example, in the HackerRank survey we can see that nearly 70% of developers have a formal degree from a school or university. But despite that, 74% of those developers still consider themselves self-taught to some extent. And that shows us that they’re learning throughout their entire careers, that they’re continuing to teach themselves new skills, which again, we see in the Stack Overflow survey, where we see that 87% of developers have taught themselves a language framework or tool, probably one of your frameworks or tools.
So we know that they’re learning, we know that they continue to teach themselves throughout their professional career. How are they doing that? What are the methods they’re using? We’re in freeCodeCamp, in that survey we see that three of four developers are now turning to YouTube for their coding education. This is a growing trend, moving away from our static, online, textual tutorials and references through to more lively media.
And there’s some companies that react to this in really fascinating ways. Microsoft for example, this is Jeff Fritz from the Microsoft Developer Advocacy team, had this series of tweets about the Twitch content that Microsoft is starting to roll out, where they’re look at more than 50 hours of live programming on Twitch a week currently. And they’ve also recently moved Channel Nine over to YouTube. So, some really big developer names, who are building a lot of developer tooling are starting to react to these changes.
And when you look deeper into the surveys, we see other parts of this trend. For example, if we look at the different places, the different forms of education that developers are using to teach themselves, they’re self-teaching. How are they doing that? We see one thing come up time and again, we see references to MOOCs, Massively Open Online Courses, we see references to Codecademy, to Coursera, to edX, to Pluralsight.
Basically, a huge number of developers are using…well over 50% of developers now, are using MOOCs or some kind of complete course, i.e. a front to back learning experience to teach themselves over disembodied tutorials or single-use case tutorials. So what we’re seeing with these trends is that increasingly developers are turning to complete learning experiences. They’re looking for experiences that provide them not only technical information, but context, a method of practice, and a way to get feedback and evaluation on what they’ve learnt.
So where do we fit in with this? How do we as developer relations professionals contribute to this new educational trend, and how do we utilize it? Well, I think it’s fairly obvious that if developers want to learn this way, we should start to equip our developer portals with these experiences so we can get on this train and start to benefit from that.
And some famous courses are already starting to do that for us, for example, Wes Bos who you may be familiar with, he’s an independent content creator who creates a series of really great tutorials for the JavaScript ecosystem. One of his tutorials, the Node.js one really heavily features the mLab API. Well obviously, we can’t rely on his platforms to do our work for us, we need to get on that ourselves. But before we do that, before we go jumping into this trend, I want us to think a little bit about how effective these courses are actually being in their educating of developers.
And to do that I want to turn to a paper by Ada Kim and Andrew Ko at the University of Washington. Their paper, “A Pedagogical Analysis of Online Coding Tutorials,” looked at a series of learning science dimensions and applied those to online, a lot of popular tutorials. They looked at sources of developer community learning such as Stack Overflow, then they looked at things like Codecademy, they looked at Regex Golf, they looked at a whole bunch of things.
And what they found, is that most online coding tutorials are still immature and do not yet achieve many key principles in learning sciences. To put that simply, they’re not following best practices of teaching people. Teaching people is one of the oldest things that humans know how to do. And there’s a lot of established knowledge out there on how to do that. And I think we as an industry could benefit from applying some of that learning science to our work.
So, what are those best practices for teaching? What did Ada and Andrew look at? They looked at four dimensions which I’ve summarized as connect, organize, practice, and reflect. When we talk about connect, we’re talking about building new knowledge on top of prior knowledge. We’re looking at how we set a new knowledge that we’re going to teach people in context of what they already know and build upon that.
Organization is about how we break down and structure concepts so that they’re able to be efficiently and effectively learnt. Practice and feedback; it’s useless to teach someone a new skill if they never get a chance to practice and reinforce that learning, and giving people feedback is fundamental to their continued growth. And reflection is the final step and a very important one, once you’ve learnt something new, you need to think about how to apply that in the real world, transfer that knowledge to where you’re going to use it, and they looked at that as well.
So, let’s start with connect. We’re talking here about prior knowledge, building new knowledge on top of what a learner already knows. This is about setting things in context. If we just start teaching people stuff without taking into account their existing experience level, we’re going to create pain and frustration. So there are two parts to successfully connecting a learner’s prior knowledge to new knowledge.
The first of those is personalization, where we customize our learning experiences based on what users already know. A really obvious example of this is when your courses are broken down into beginner, intermediate, advanced, that allows people to choose a learning path that’s the best fit for them. Another example of this is, well another factor in connection is utilization. Once we’ve taught someone something, does the later part of the tutorial use that knowledge?
Does it allow them to flex those muscles? If you know that a learner has come from a specific background, are you using their existing experiences in the accumulation of new knowledge?
A great example of a platform that does both of these things really well is Microsoft Learn. Microsoft Learn is a new learning experience. I’m letting the Microsoft folks take a picture before I move on. Microsoft Learn is a new learning experience that’s built into the Azure documentation, and it does a lot of things really well in all of the principles we’re going to cover, but particularly in this connection.
One of the great ways to enable personalization of courses is to provide learning paths. Learning paths that a learner picks up…well, they give a learner a paved road through the content that aligns with their experiences and what they want to gather. If we look at the Microsoft course, when you first land on Microsoft Learn, you see that there’s a drop down that selects your background and how that might relate to Azure. Are you a developer, an administrator, a business analyst?
When you select one of those, the page presents you with pathways that will guide you through Azure and teach you about Azure from your experience, and how you might want to know. So for example, I selected the developer one. And one of the courses that caught my eye was communicating between applications with Azure Queue technology… Queue Storage rather. Looking at this page, there’s a couple of other places where we can see personalization come into effect.
For example, at the top here, we can see again, this course is labelled for intermediate, for developers. I as a developer landing on this page, I’m able to set this course in the context of my existing experience. And another way that you can provide personalization is through learning objectives. How many of your documentations and tutorials start with something like, “In this tutorial, you will learn…” Setting expectations for learners is super important, and also allows them to seat your content again within their existing knowledge.
If we look at the Azure tutorial we see right here at the top, “In this module you will.” And before they even go in, students know roughly what…they get an idea of what they’re going to learn, they’re able to reflect on their learning experience after because they can look at this and say, “Did we achieve these things?” And most importantly, they can also look at this and say, “Ooh, actually that all looks a bit scary, maybe I should start with a beginner tutorial or something.” Or maybe, “I know how to do these things, I can go for an advanced tutorial.” It enables that personalization that lets the learner learn.
And then when we get into utilization, there are some great techniques that the Microsoft tutorial uses here as well. One of them is prerequisites. In this case, this module has no prerequisites. But by having training courses that have prerequisites you can build up experiences that use a learner’s prior knowledge.
Maybe we start with a beginning introduction to creating Azure cloud storage devices before coming to this module, and we’re building up new knowledge on top of the knowledge we’ve accumulated previously. And you can see this in the table of contents for this module. For example, in the middle here, we have some steps, Identify a Queue, and then we have Programmatically Access a Queue.
The tutorial teaches us a concept and then immediately follows up with the next stage of learning that utilizes that concept. I can also tell you that in the Add a Message, well, Programmatically Access a Queue, we have to create a cloud storage device. We’re taught how to do that in the Create Storage Account phase. So, that’s connecting. And in that last section, we got into a little bit of structuring and organizing our courses. The second learning principle I want to talk about is exactly that.
And this is about how we structure our information for effective and efficient learning. And broadly speaking, there are two tactics for doing this. We’re talking about breaking problems down into smaller, easier to tackle parts or we’re talking about building concepts up into taking simple concepts and layering them to solve more complex problems. Building concepts up is a very familiar and simple one to… Anyone here who’s ever done a coding tutorial, you’ve probably all seen “Hello, World!”, you probably had that first amazing experience, where you’re like, “Cool, I made a computer do a thing.”
And then they’re like, “Cool, just customize that thing.Here’s some variables.” And then you’re like, “Cool, let’s get user input, is that a prompt?” And then suddenly the scenario has changed. But because we’re still using what we’ve learned already we’re able to adapt to this new tutorial, we’ve changed to a guessing game, but now there are conditionals. Again, we’re taking simple building blocks and stacking them on top of each other to provide the learning experience.
When it comes to breaking problems down, I want to turn to another example, which is TwilioQuest. TwilioQuest is a super fun learning experience, which takes the form of a retro RPG. And TwilioQuest is a really great example of decomposing complicated projects for the learner. For example, one of their projects that occurs very early on in the tutorial is a to-do bar.
At this point, the user has literally just learnt how to send SMS via an API. And suddenly they’re told, “Hey, you’re now going to build a to-do application with your new knowledge.” And that can be quite scary. There’s a couple of things here that are really interesting. First, I want to draw your attention to the objectives, going back to the previous topic of connection, this learning experience is using those principles, and they have learning objectives.
But then over here, we have victory conditions. And those victory conditions clearly state the problem. And it’s quite a large problem for developers to solve at this point in their journey. What the page then does is break that problem down into a series of smaller to tackle chunks so that the learner can look at these and with the knowledge that we’ve taught them already, the limited access to programmatic SMS they’ve had, they can look at these and say, “Oh, okay, this problem isn’t insurmountable. I know how to approach this with what I’ve learnt.”
They take a complex problem, and break it down, and a lot of the most successful tutorials we use both breaking problems down and building concepts up and meet somewhere in the middle. TwilioQuest teaches you how to use SMS through a series of small steps like, hey you’re going to get phone number, now you’re going to go use TwiML Bin, now you’re going to send a message. And then it meets that in the middle, we’re breaking a project down into the concepts that we’ve already met. So, part of this project requires users to practice and apply what they’ve learnt.
So now we’re going to move on to the third learning principle, which is practice and feedback. And what we’re talking about here is allowing learners to reinforce their learning for applying it to some problem, to practicing it. And there are three things here that are really important. One of them is what my manager Vanessa, who is an instructional designer, who’s criteria for mastery, it’s impossible to practice and know you’ve achieved something if you don’t know what mastering it means.
So with learners, when you give them something to practice, or when you give them an exercise to perform, you should tell them what success looks like, what is the criteria for mastery? How do I know when I’ve completed this topic, and when I’ve of my goal? So when we give exercises or something to practice, we should make sure we lay out a criteria for mastery.
Next is learn actionability. Are they actually applying the skills they’ve learned in a relevant and realistic concept? For example, if we’re teaching people to use our API, the practice should be to use the API, right? It shouldn’t be something abstract. It should be to actually go and do the thing. And then do we give feedback? And important here is the timeframe, the immediacy of the feedback is really vital for learners to be able to conceptualize and move on from any challenges they faced. For example, if they’ve gone wrong in the tutorial, if we leave it a week before we tell them that, it’s going to be very hard for them to adapt and continue learning.
If we’re able to give them immediate feedback, it’s much easier for them to continue that learning experience. The example I want to give you is the GitHub Learning Lab. This is built by the training team here at GitHub. And this tutorial does a couple of really cool things. Firstly, it’s all within GitHub. You don’t have to leave GitHub to participate in this experience. And the way it does that is a bot that uses the same actions that a GitHub user does.
So when you start a course on GitHub, this is the introduction to GitHub course, the bot will come and open an issue teaching you the concepts, and you read the issues. So, you’re learning about GitHub whilst actually using GitHub. But what it will then do is follow up with further comments on the issue to provide opportunities to practice. So here it’s teaching us about what GitHub is and what issues are.
And then it asked us to assign ourselves to this issue. What’s really important in this activity is it tells us exactly what success looks like, and what we should be expecting if we meet success, what the bot will then do. And having done that, we get immediate feedback. The next step comes on, the bot says congratulations, I as a learner know that I’ve met the criteria for mastery with immediate feedback, and I’m able to move on. And at that point is when we want to encourage learners to transfer and apply what they’ve learnt to a real-world context.
The example I want to give here is award-winning Campus Experts. The Campus Experts Program is a program that I run at GitHub which is about teaching students to do what you all do. It’s about teaching students to build communities on campus. And it’s a training-led program, we train every student who applies for the program before they can become a campus expert. The training is the gateway. And that training goes through six modules, we teach them what community means, we then give them four skills that is useful in communities such as public speaking, workshop development.
And then at the end, we encourage them to reflect. We take them through the community impact module where we say to them, “Cool, you’ve analyzed the community, you’ve learnt four skills, now how are you going to use those skills to improve the community?” We actually have an explicit exercise that tells them to reflect and transfer that knowledge to a real-world scenario. And that results in some wonderful Campus Experts who are in this audience.
So just to summarize, those four learning principles are connecting a learner’s prior knowledge to the experience you’re going to be providing them. What new knowledge are you teaching them, and how does that relate to what they already know? We spoke about how to structure and organize our new knowledge so that learners can easily and effectively absorb it. We spoke about giving them an opportunity to practice and making our criteria for mastery clear and providing immediate feedback. And we spoke about making sure that they have an opportunity to reflect so that they’re able to take those new concepts out into the real world.
So saying that, telling you all to go run a MOOC or a huge training course is obviously not necessarily a practical thing, “Hey, under resource to DevRel teams, go run Coursera.” So I want to give you some practical tips that we applied, Campus Experts is a good example.
Over 3,000 students have gone through Campus Experts, and it’s all hand reviewed, it’s all hand curated. And I’m mostly doing okay. And so I want to give you some tips that I’ve learnt, those are reuse, automate, and incentivize. So in reuse, I want to quote, I guess someone probably said this, at some point, “Good teachers copy great teachers steal.” You all already have the content for your courses, you’ve already got the documentation, you’ve already got the tutorials.
All you need to do is seat that within the learning principles we’ve spoken about. You need to give them learning objectives. You need to add practice and criteria for mastery. And you need to give some feedback mechanism. You don’t have to reinvent the wheel and recreate all this content. Coming to Campus Experts, I mentioned there’s these six modules. With these modules, we brought a bunch of really knowledgeable people from GitHub into a studio and said, “Tell us what you do.”
A great example of this is Module 5, which is the Introduction to Git module. I stole this, so I just went and got one of our training team and said, “Do the thing here for me please,” and they did that. I didn’t have to rewrite that content, why would I? We have a team dedicated to writing that content who have already mastered and written that content for a decade. I can just take that content and apply it to my course. The next is automate. I mentioned that immediate feedback’s really important.
And that’s a really scary thing, right, how do we give immediate feedback? Especially when feedback is very time intensive. Well, I want to encourage you to use real- world tools, because the great thing about building developer tools and developer products is that it’s related to code. They interface off our products through code, and code is testable. An example of this is Omar at San Francisco State University, this is a real university teacher whose job is to teach people folks, we should learn from these people.
And what Omar is doing is teaching hundreds of students a week and giving them coding assignments. That’s a lot of people hours to review all those students’ assignments. Omar uses Travis CI, he’s giving them coding assignments. He uses the mechanisms that we, as software engineers use to test, to provide immediate feedback and follow up on the exercises. And this is something that we can do as well.
Learning Lab does this. The way GitHub Learning Lab works and provides that immediate feedback is there are already webhooks that went into GitHub for all those actions. When you commit… when you do something with an issue or you commit a new file, GitHub fires off a webhook. Learning Lab captures those webhooks and uses them as an opportunity to teach, and uses them to provide immediate feedback. So make use of robots. There’s a great framework called Probot, which is a framework for building GitHub apps.
That’s actually what Learning Lab is built upon, and also Campus Experts, we’re basically running a very human-intensive MOOC, we use a GitHub Repo for the whole thing, because once you work at GitHub for a little bit every problem starts to look like a GitHub Repo. And we’re just using real-world tools, I’m not building anything, I’m not, like, finding education-specific stuff, I’m using the things that engineers use to teach those engineers.
And if you’re teaching a problem that isn’t fixable by bot, get social. A lot of the MOOCs provide forums to the learners so that they can give each other feedback. This is actually a principal in adult learning, andragogy, that adults should be involved with their own evaluation and assessment. So we’re actually providing those social opportunities for your learners to give each other feedback is actually very…a sound educational practice.
And finally, incentives, very brief one. Campus Experts is a training program that acts as the gateway to a larger program, acts as the gateway to more opportunity. Your training doesn’t have to be the end, you can use that to build other programs. TwilioQuest provides credit to people who complete the course. And all of these ones, all these screenshots have had experience or some kind of giveaway. At Campus Experts we give badges.
Definitely build things like experience and badges into your programs to gamify things, incentivize them. A great framework is Mozilla Open Badges. This is a super cool thing where you can create badges and then issue them, and it does some magic where it bakes, like, the credential into the image. So like anyone who looks at image, the metadata for the fact they’ve earned that is like in the image, it’s very cool. So make it a doorway.
And I’m being told time. So to recap, we looked at how developers learn, they’re self-teaching, they’re moving to MOOCs and YouTube and we should do that too. Make sure you apply the four learning principles, connecting the prior knowledge, organizing and structuring our information for efficient learning, giving opportunities to practice and get feedback, and applying that knowledge to the real world. And then steal shamelessly, use robots, and don’t stop there, your training can be the gateway to something much bigger.
Thank you very much, everyone. I’ve been Joe Nash. I’ll be around after for questions.