🏫 Eight Years on ClassCharts

If you have been following this blog for a while, you’ll know that most of what I write about here are things I built for myself, like Metis or MoaiTime. What I never really wrote about is the job that took up the biggest part of my career so far. From November 2015 to December 2023 I worked on ClassCharts, and apart from a couple of paragraphs in my very first post (where I didn’t even name it) and the shorter version on the case studies page, there’s nothing about it here.

So in this post I’ll go through a few of the decisions I made along the way. It’s a long one, eight years is a lot to cover!

Where it started

First, a bit of context. ClassCharts does behaviour management, seating charts and school-to-home communication for UK schools. Teachers log behaviour points against pupils during the lesson, and that data then feeds into detentions, attendance, homework and reports, and goes out to parents and pupils through their own apps. So logging a point has to be quick, ideally quicker than the incident itself, otherwise teachers won’t bother.

I joined Edukey Education, the company behind it, as the very first developer (after the CTO). So there were two of us, and the products already had hundreds of thousands of users! By the time I left, ClassCharts was serving more than ten million pupils, parents and teachers across thousands of schools, and the engineering team had grown to more than a dozen people.

I was also one of four code owners on the main repository by then. Somewhere in between, Edukey was acquired by TES Global.

Just for the record, I was a contractor the whole time, working through my own company, but full-time. At first the contract was with Edukey, and from 2021 it was directly with TES Global.

Replacing jQuery without a fork

The main application was a big CakePHP monolith with a jQuery front end. Once it was clear that the front end had to move to React, we basically had two options. We could start a second application next to the old one and move the schools across to it, or we could replace the old one from the inside.

I went with the second one. I carved React micro frontends out of the monolith, which were self-contained React apps mounted inside the existing CakePHP pages, starting with the dashboard and later the teacher and pupil app surfaces. A screen only moved over when somebody was going to work on it anyway, so we did the migration as part of normal feature work and it never needed a budget of its own.

A fork would have been quicker, but we’d have been running two systems and at some point telling every school to switch on a date we picked. So we lived with two frameworks in one repo for years, which wasn’t pretty, but we never had to stop shipping to do it.

Three clouds

We started on Rackspace with three boxes, then moved to Google Cloud and later to AWS, going from around 20 instances in the middle years to roughly 100 by the end.

Before any of the moving started, I brought Docker into the company (this was the mid-2010s) and containerised development and production first, so everything was already portable before the first move. Kubernetes came later, as the platform grew.

The move from Google Cloud to AWS was really two migrations in one. Files went from Google Cloud Storage to S3, and messaging went from Google Pub/Sub to RabbitMQ, so we swapped a managed queue for one we ran ourselves. The files were mostly copying bytes over, while the queue was harder, because every publisher and consumer changes at once and the delivery semantics aren’t quite the same on both sides.

The biggest single piece was moving more than 40 TB of images and assets between cloud providers, with the platform up the whole time. Yes, 40 TB! I wrote the migration tooling for that myself, and it probably deserves a blog post of its own at some point.

One smaller story from the Kubernetes side. In 2023 the file scanner (a separate service that virus-scans uploaded files, which two colleagues originally wrote in 2020) was moving onto Kubernetes, and its RabbitMQ consumer kept dropping. I couldn’t reproduce it locally and every experiment meant a deploy, so it came down to a lot of instrumenting and bisecting. It turned out the connection was being terminated at around the hour-and-a-half mark, so I just had it reconnect every five minutes, which is cheap.

The data grew along with the servers. Behaviour and trigger events went from tens of thousands a year to tens of millions, on a MySQL estate of a couple of terabytes, with tables of billions of rows. Keeping up with that was mostly adding indexes and reading query plans. By the end the platform was serving around 1,000 requests a second, and the load follows the school timetable, since every school in the country starts the first period at roughly the same time.

When the schools closed

ClassCharts was built teacher-first. When UK schools closed in March 2020, parents and pupils were suddenly at home on their phones, and they needed things that had only ever existed on the teacher side. We spun up the parent and mobile equivalents in a matter of days.

At the same time, teachers could still set homework, but they had no way to collect it. We shipped homework file submission, with virus scanning, in under two weeks! Children were uploading files into a school system, so scanning went into the first version.

In April and May 2020 came parent-teacher messaging, because until then announcements only went one way. I built the front end for it across the web app and the parent and teacher apps (threads, history, unread badges and live delivery through Pusher), against a backend a colleague was building at the same time. Those were a busy couple of months!

Two generations of mobile apps

There are three ClassCharts apps, one each for teachers, pupils and parents, all from one React Native codebase for iOS and Android, so six store listings in total. Before I built them, there was only a native app, and only on Android. The first generation I built were basically shells around the web app. A React Native wrapper hosted a web view pointed at our web app, where the native side handled push notifications, storage, permissions and the store lifecycle, and the web view handled the screens. It was cheap to ship, and at the time I think it was the right call.

By 2018 we’d outgrown it though, and the parent and pupil apps were rebuilt from scratch, module by module (homework, attendance, activity, behaviour, detentions and the rest). That was a lot faster than the web view, and we got proper offline handling.

I also owned the release and signing pipeline for all six listings, along with push notifications. Across those six listings the apps passed a couple of million downloads. One thing I took away from it is that a bad mobile build can’t be reverted the way a web deploy can. It sits in a store review queue, and that made me a lot more careful before pressing the button!

The rest of the job

Over the years the job kept changing. I built the majority of the features across the product range, and later brought in Scrum, then Cypress with the workflows that ran it, next to GitFlow and CI/CD pipelines on GitHub Actions and later Jenkins. I also mentored juniors and the QA engineer who grew the Cypress suite, and onboarded the teams that came in with the acquisition.

By the time I left in December 2023, only one other person had contributed as much across the product range as I had.

Wrapping up

ClassCharts is closed source, so unfortunately there’s no repository to link this time. The shorter, more structured version is on the case studies page, the rest of what I’ve built is on the projects page, and my full background is on the career page. Next up is what came after ClassCharts, so stay tuned!

I hope you enjoyed this blog post and I will see you in the next one!