About 18 months into my career I was asked to make some changes to a website for a well known high street retailer. The site had been live for a number of years and (being generous) was not in the best shape. It had also been built using a set of tools that I had never seen before but there was a deadline and I was the only developer available.
To get started I pulled down the Subversion repository but like several of the other older projects I’d seen at the company it had been used as a dumping ground rather than a record of the project’s history. I had heard of developers who had simply committed all of their work without explanation every Friday before going home and this seemed to tell a similar story.
Not that it mattered, I already knew that there was no chance of the repository ever matching the live site anyway.
The method of deployment used by the company was unforgiving; HTML templates had to be copied and pasted into the CMS one-by-one and assets moved to live and staging servers using FTP or even drag and drop over Remote Desktop. Adding an extra dimension to this risky manual procedure was the knowledge that the client may have edited any number of templates and files since I or my colleagues had last deployed something, meaning all new work began with a few hours eyeballing the contents of the live site, staging site, and SVN repository then making a judgement on which version most suited us.
Once I had the staging site (all development work was performed on a staging site) looking like a reasonable impersonation of the live site I got started on the changes but the JavaScript files I needed to update looked almost alien to me. Neither Google nor Stack Overflow could provide answers and it felt like there were only a few people on the planet who could have ever used this framework (true.) After many frustrating hours poking around I completed the changes and informed the project manager of my success despite having little idea how I’d managed to do it.
Deployment was scheduled for a few days later; as it was a big one we’d have to be in office for 7AM, put up a temporary holding page, then deploy (copy and paste) the changes to the live site before handing it over for a round of manual Q&A.
On the big day I copied and pasted as precisely as I could and the holding page was taken down just a little behind schedule (I am never on time.)
Before lunch the phone started to ring. The client had not processed a single transaction in the hours that had passed since my code went live. My colleagues and I frantically rolled back (copy and pasted) the changes and once order seemed to be restored I was wheeled over to the angry tech director for a brief retrospective:
“You fucked up”
I’m more than a decade into my career now and I hadn’t thought much about this incident until I was recently asked about previous jobs I’d had. In hindsight it seems so ridiculous but at the time I suppose I didn’t think all that much of it (I have worked at more than one company with similar practices.) I was able to fix the problem and we moved on.
If I did learn anything from this experience it’s that I know now to make sure my teams and colleagues feel supported, ensure the tools and processes we use are suitable, and that we’re able to be alerted quickly and respond blamelessly when things inevitably go wrong.