Like many dev teams, Rubi's team relies heavily on continuous integration. Their setup, like many others, relies on git hooks, and whenever someone pushes a commit to any branch, it automatically runs all the associated unit tests. Good code stays green, and any bugs are immediately revealed. Branches with failing tests cannot be merged into the main branch, which is all pretty reasonable.

Recently, Ruby pushed a commit on a branch up, and pretty much immediately realized that the tests were going to fail because she forgot to update a related code file. Even as she started to amend the commit, she waited for the CI server to cough up an error. And waited. And waited. And waited.

Now, for this particular repository, Rubi wasn't usually doing much development in it. She was helping with a big system upgrade, and so her first thought was that she must have made some other mistake. After all, it's not like automated CI would just get turned off, right?

Well, when she glanced at the YAML file which controlled their test runner:

test-service-integration:
  stage: test
  image: $RUNNER_BASE_IMAGE
  needs:
    - build-prod
 before_script:
 {.... other stuff removed} 
 allow_failure: true 
  tags:
    - run_in_docker
  only:
    - main

allow_failure: true meant that the CI server would let failing code pass, which generally wasn't a good idea. More than that, though, that rule applied to the main branch. In this configuration, tests were never run on other branches, and when they ran on the main branch, failures were allowed.

When she flipped the flag back to allow_failure: false, the unit tests in the main branch failed catastrophically.

Fortunately, their configurations were also source controlled, so it wasn't hard to find out who was responsible. Roger, one of their junior developers. He had made that change ten months ago, and no one else had touched the file since. Rubi pinged him on Slack and started a video call.

"Roger, what is going on with this?"

"Oh, that," Roger said. "Well, my tests were failing, well, not my tests, but one of the tests that somebody else wrote."

"So… you just turned the tests off?" Rubi said.

Roger shrugged. "Well, I had a feature to deliver for that sprint."

"I can't believe Felicia was okay with this," Rubi said. Felicia was the tech-lead on Roger's team. "She had to see this when you submitted your pull request."

"My what?" Roger asked.

"Your pull request? Y'know, for Felicia to review your code before it gets merged into main?"

"Oh, I just do all my work right on the main branch. Is that not okay? Felicia said it was fine, and that she didn't have time to review every change."

Rubi rubbed her temples and sighed. "No, that is not okay. Just… don't push anything else until I've talked to Felicia."

Rubi ended that call, and prepped for another video call, where she and Felicia could have a long discussion about how team leads can help their junior developer actually be successful, how code reviews contribute to that, and why leaving the automated unit tests off for ten months was a terrible idea.

[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.