Hack Reactor's Response to Recent November 2016 Course Report Reviews
Recently, Hack Reactor Remote was the subject of four negative reviews on Course Report. Although the vast majority of students in those recent classes had great experiences (we know because we actively solicit and track student satisfaction ratings and during this time, our NPS scores ranged from 75 to 100), this is nonetheless the worst spate of negative reviews in our four year history. We built a name as the industry's leading bootcamp by getting better every cohort, not by being perfect all the time, and we see this as a perfect opportunity to improve.
In Hack Reactor style, our team sprang into action to investigate and correct the issues surfaced by these reviews. To stand by our commitment to students, we're evaluating and addressing each concern. Below, we’ve laid out six common threads in the reviews. Although some points are completely warranted and we'll lay out what we're doing about them, we would also like to provide additional context and background on others. We should also acknowledge that there’s no need for drastic action. We track student satisfaction using the NPS system, and our NPS score for 2016 to date is 84 (out of 100), meaning that, as a whole, students are thrilled with our course. For comparison, Apple’s NPS hovers around 70 for their products.
Reasoning: One of the primary lessons taught at Hack Reactor is how to solve unfamiliar engineering problems. This is the single most valued skill-set by employers, so we make it a priority to help students achieve autonomy. That means that when a student comes to an HiR or tech mentor with a problem, the focus of that interaction is to get the student to where they can debug and solve the problem themselves. We actively train HiRs and Tech Mentors to fight the urge to give students the answer. We recognize that without proper training and guidance, this interaction can come off to a student negatively.
A separate mechanism -- office hours with technical staff -- is intended to be more oriented towards tutoring students. We ensure that we have enough office hours every week to meet all demand.
Improvements being made:
Set student expectations early so all parties understand that the single most hireable characteristic for a Software Engineer is their ability to be autonomous on a feature, product, or project.
Ensure office hours are being filled as needed.
- Make additional rounds as students are coding throughout the day.
Reasoning: Fact: Some content is 2.5 years old. Our best, most effective videos naturally stay in circulation the longest, unless the content requires an update. With that said, there are ways we can and will improve on the type of recording - classroom recording vs. “studio” recording (a lecturer talking to the camera). Regarding updates to technologies, e.g. Angular 2.1.0 vs. Angular 1 - we make decisions based on what will get students hired. Oftentimes, the technologies employers are using are time-tested ones, not the latest bug-prone versions. As such in many cases, we would do our students a disservice by teaching a new technology before the industry has shown that this new tech has sustaining relevance. Occasionally, it becomes clear very quickly that a technology is very important, and we waste little time adding it to the curriculum. React is a prime example of this. React was added to the curriculum relatively quickly once employers adopted it and valued it as an employable skill.
Improvement being made:
- Dramatically increase our pace of re-recording video lectures.
Reasoning: Hack Reactor has never and will never encourage or produce “fake”, misleading or untrustworthy reviews of our program. At times, we have encouraged students to leave reviews by offering them a free hoodie, and we always explicitly state that these reviews can be positive or negative. The reviews are never checked or audited before the hoodie is given out. We want reviews because we believe in our program, and we believe honest assessments from graduates will reflect well on us. The hoodie program is a simple way to encourage reviews while providing a gift graduates can wear with pride.
Reasoning: It’s true that one can find instruction on the technologies we teach from a variety of sources. The reason people come to Hack Reactor is to accelerate their knowledge and gain an underlying comprehension of the material that will allow them to quickly adopt new technologies and perform autonomously on the job. It typically takes a self-studier six months to a year before they are ready for a junior-level job. Our graduates take mid-level roles after our three month course and have the skills and knowledge they need to be successful in their long careers as software engineers. We teach students to use technology in situations that resemble those encountered on the job, by professional software engineers. In order to have lasting skills, one must understand both the function and fundamentals underlying a given language, framework or library.
An often-overlooked point is that online resources only teach the technical side of being an engineer, but most professionals would say that soft skills, such as communication and teamwork are just as important. We weave these in throughout the course through pair programming, project work and instruction specific to soft skills.
Reasoning: We actually intentionally removed Hiring Day from the program. We noticed that more students were getting jobs faster through personalized introductions rather than just one hiring day. Having Hiring Day took the resources that otherwise would have been on the phone with companies pitching graduates to having to get people to show up to an event. As such, we sat down and re-crafted a more efficient match-making solution. With that said, there are modifications to the typical “Hiring Day” that would be an even greater improvement to the more efficient match-making solution, and we intend on beta testing new versions of a Hiring Day.
Improvement being made:
- Beta-test a new Hiring Day at several campuses.
Reasoning: Hack Reactor’s curriculum and program structure is built by professional engineers who have worked as Software Engineers at Google, Twitter, YouTube, Facebook, Adobe, and the like. Lecturers and Instructors have to be individuals who know the curriculum forwards and backwards, and who excel at teaching and working directly with students. Among the 2,000+ graduates of our program, we have had some truly exceptional teachers, and some have stayed on as instructors (< 0.1% of them). The ones who stay on are the cream of the crop of graduates from the program. In fact, many students would tell you that they are some of the best teachers of software engineering on the planet. The way that Technical Mentors or Instructors who have industry experience are impactful to students is oftentimes outside of the typical direct instruction. They are helping architect various portions of the course, and thus may not be student-facing. This is all to say there are deliberate decisions being made here, not an oversight.
Improvements being made:
We are working to ensure all students have access to industry-experienced people with office hours, helpdesk escalation, or alternative meaningful and productive ways.
We are training technical mentors to periodically check in with students during pair programing sessions.
It is a part of the culture here at Hack Reactor to be transparent and honest and to do right by people. The overwhelming majority of our students have wonderful, often life-changing experiences at our schools. Indeed, the whole point of what we do day in and day out is to change lives and help our students reach their dream of becoming software engineers. It’s not uncommon to hear students say Hack Reactor is the best education they’ve ever had or even that it was the best three months of their life which we see again and again in the many other reviews of our program. We have gotten to this point by listening to our students, taking ownership and driving forward based on results. It would be much easier to dismiss the negative reviews and move on. That’s not who we are. Those reviews contained valid points, which we believe we have addressed here, and will continue to address going forward. Our ethos is one of empathy and constant improvement, which guides us as we strive to create the best possible experience for students.
How do our students do on the job market? See our third-party verified student outcomes to find out. We would also love your thoughts on how we did in addressing this recent feedback. If you have ideas to share, please do so here.