While this year has felt endless, there are projects which will feel like they take forever. As we wrap up our tour of the best of 2020, let's visit an endless project. Original -- Remy
Once upon a time, there lived a man named Eric. Eric was a programmer working for the online development team of a company called The Company. The Company produced Media; their headquarters were located on The Continent where Eric happily resided. Life was simple. Straightforward. Uncomplicated. Until one fateful day, The Company decided to outsource their infrastructure to The Service Provider on Another Continent for a series of complicated reasons that ultimately benefited The Budget.
Part of Eric's job was to set up web servers for clients so that they could migrate their websites to The Platform. Previously, Eric would have provisioned the hardware himself. Under the new rules, however, he had to request that The Service Provider do the heavy lifting instead.
On Day 0 of our story, Eric received a server request from Isaac, a representative of The Client. On Day 1, Eric asked for the specifications for said server, which were delivered on Day 2. Day 2 being just before a long weekend, it was Day 6 before the specs were delivered to The Service Provider. The contact at The Service Provider, Thomas, asked if there was a deadline for this migration. Eric replied with the hard cutover date almost two months hence.
This, of course, would prove to be a fatal mistake. The following story is true; only the names have been changed to protect the guilty. (You might want some required listening for this ... )
Day 6
- Thomas delivers the specifications to a coworker, Ayush, without requesting a GUI.
- Ayush declares that the servers will be ready in a week.
Day 7
- Eric informs The Client that the servers will be delivered by Day 16, so installations could get started by Day 21 at the latest.
- Ayush asks if The Company wants a GUI.
Day 8
- Eric replies no.
Day 9
- Another representative of The Service Provider, Vijay, informs Eric that the file systems were not configured according to Eric's request.
- Eric replies with a request to configure the file systems according to the specification.
- Vijay replies with a request for a virtual meeting.
- Ayush tells Vijay to configure the system according to the specification.
Day 16
- The initial delivery date comes and goes without further word. Eric's emails are met with tumbleweeds. He informs The Client that they should be ready to install by Day 26.
Day 19
- Ayush asks if any ports other than 22 are needed.
- Eric asks if the servers are ready to be delivered.
- Ayush replies that if port 22 needs to be opened, that will require approval from Eric's boss, Jack.
Day 20
- Ayush delivers the server names to Eric as an FYI.
Day 22
- Thomas asks Eric if there's been any progress, then asks Ayush to schedule a meeting to discuss between the three of them.
Day 23
- Eric asks for the login credentials to the aforementioned server, as they were never provided.
- Vijay replies with the root credentials in a plaintext email.
- Eric logs in and asks for some network configuration changes to allow admin access from The Client's network.
- Mehul, yet another person at The Service Provider, asks for the configuration change request to be delivered via Excel spreadsheet.
- Eric tells The Client that Day 26 is unlikely, but they should probably be ready by end of Day 28, still well before the hard deadline of Day 60.
Day 28
- The Client reminds Eric that they're decommissioning the old datacenter on Day 60 and would very much like to have their website moved by then.
- Eric tells Mehul that the Excel spreadsheet requires information he doesn't have. Could he make the changes?
- Thomas asks Mehul and Ayush if things are progressing. Mehul replies that he doesn't have the source IP (which was already sent). Thomas asks whom they're waiting for. Mehul replies and claims that Eric requested access from the public Internet.
- Mehul escalates to Jack.
- Thomas reminds Ayush and Mehul that if their work is pending some data, they should work toward getting that obstacle solved.
Day 29
- Eric, reading the exchange from the evening before, begins to question his sanity as he forwards the original email back over, along with all the data they requested.
Day 30
- Mehul replies that access has been granted.
Day 33
- Eric discovers he can't access the machine from inside The Client's network, and requests opening access again.
- Mehul suggests trying from the Internet, claiming that the connection is blocked by The Client's firewall.
- Eric replies that The Client's datacenter cannot access the Internet, and that the firewall is configured properly.
- Jack adds more explicit instructions for Mehul as to exactly how to investigate the network problem.
Day 35
- Mehul asks Eric to try again.
Day 36
- It still doesn't work.
- Mehul replies with instructions to use specific private IPs. Eric responds that he is doing just that.
- Ayush asks if the problem is fixed.
- Eric reminds Thomas that time is running out.
- Thomas replies that the firewall setting changes must have been stepped on by changes on The Service Provider's side, and he is escalating the issue.
Day 37
- Mehul instructs Eric to try again.
Day 40
- It still doesn't work.
Day 41
- Mehul asks Eric to try again, as he has personally verified that it works from the Internet.
- Eric reminds Mehul that it needs to work from The Client's datacenter—specifically, for the guy doing the migration at The Client.
Day 42
- Eric confirms that the connection does indeed work from Internet, and that The Client can now proceed with their work.
- Mehul asks if Eric needs access through The Company network.
- Eric replies that the connection from The Company network works fine now.
Day 47
- Ayush requests a meeting with Eric about support handover to operations.
Day 48
- Eric asks what support is this referring to.
- James (The Company, person #3) replies that it's about general infrastructure support.
Day 51
- Eric notifies Ayush and Mehul that server network configurations were incorrect, and that after fixing the configuration and rebooting the server, The Client can no longer log in to the server because the password no longer works.
- Ayush instructs Vijay to "setup the repository ASAP." Nobody knows what repository he's talking about.
- Vijay responds that "licenses are not updated for The Company servers." Nobody knows what licenses he is talking about.
- Vijay sends original root credentials in a plaintext email again.
Day 54
- Thomas reminds Ayush and Mehul that the servers need to be moved by day 60.
- Eric reminds Thomas that the deadline was extended to the end of the month (day 75) the previous week.
- Eric replies to Vijay that the original credentials sent no longer work.
- Vijay asks Eric to try again.
- Mehul asks for the details of the unreachable servers, which were mentioned in the previous email.
- Eric sends a summary of current status (can't access from The Company's network again, server passwords not working) to Thomas, Ayush, Mehul and others.
- Vijay replies, "Can we discuss on this."
- Eric replies that he's always reachable by Skype or email.
- Mehul says that access to private IPs is not under his control. "Looping John and Jared," but no such people were added to the recipient list. Mehul repeats that from The Company's network, private IPs should be used.
- Thomas tells Eric that the issue has been escalated again on The Service Provider's side.
- Thomas complains to Roger (The Service Provider, person #5), Theodore (The Service Provider, person #6) and Matthew (The Service Provider, person #7) that the process isn't working.
Day 55
- Theodore asks Peter (The Service Provider, person #8), Mehul, and Vinod (The Service Provider, person #9) what is going on.
- Peter responds that websites should be implemented using Netscaler, and asks no one in particular if they could fill an Excel template.
- Theodore asks who should be filling out the template.
- Eric asks Thomas if he still thinks the sites can be in production by the latest deadline, Day 75, and if he should install the server on AWS instead.
- Thomas asks Theodore if configuring the network really takes two weeks, and tells the team to try harder.
Day 56
- Theodore replies that configuring the network doesn't take two weeks, but getting the required information for that often does. Also that there are resourcing issues related to such configurations.
- Thomas suggests a meeting to fill the template.
- Thomas asks if there's any progress.
Day 57
- Ayush replies that if The Company provides the web service name, The Service Provider can fill out the rest.
- Eric delivers a list of site domains and required ports.
- Thomas forwards the list to Peter.
- Tyler (The Company, person #4) informs Eric that any AWS servers should be installed by Another Service Provider.
- Eric explains that the idea was that he would install the server on The Company's own AWS account.
- Paul (The Company, person #5) informs Eric that all AWS server installations are to be done by Another Service Provider, and that they'll have time to do it ... two months down the road.
- Kane (The Company, person #6) asks for a faster solution, as they've been waiting for nearly two months already.
- Eric sets up the server on The Company's AWS account before lunch and delivers it to The Client.
Day 58
- Peter replies that he needs a list of fully qualified domain names instead of just the site names.
- Eric delivers a list of current blockers to Thomas, Theodore, Ayush and Jagan (The Service Provider, person #10).
- Ayush instructs Vijay and the security team to check network configuration.
- Thomas reminds Theodore, Ayush and Jagan to solve the issues, and reminds them that the original deadline for this was a month ago.
- Theodore informs everyone that the servers' network configuration wasn't compatible with the firewall's network configuration, and that Vijay and Ayush are working on it.
Day 61
- Peter asks Thomas and Ayush if they can get the configuration completed tomorrow.
- Thomas asks Theodore, Ayush, and Jagan if the issues are solved.
Day 62
- Ayush tells Eric that they've made configuration changes, and asks if he can now connect.
Day 63
- Eric replies to Ayush that he still has trouble connecting to some of the servers from The Company's network.
- Eric delivers network configuration details to Peter.
- Ayush tells Vijay and Jai (The Service Provider, person #11) to reset passwords on servers so Eric can log in, and asks for support from Theodore with network configurations.
- Matthew replies that Theodore is on his way to The Company.
- Vijay resets the password and sends it to Ayush and Jai.
- Ayush sends the password to Eric via plaintext email.
- Theodore asks Eric and Ayush if the problems are resolved.
- Ayush replies that connection from The Company's network does not work, but that the root password was emailed.
Day 64
- Tyler sends an email to everyone and cancels the migration.