How I Accidentally Spent a stupid amount on a Test Firewall (And Built a Solution So It Never Happens Again)

Picture this: You’re sitting on a beach in Thailand, coconut in hand, completely disconnected from work. Life is good. Then you make the mistake of checking your Azure bill.

$650

For one month. For a test subscription that should cost maybe $50.

The culprit? One Azure Firewall. Premium tier. Running 24/7 for two weeks while I was building sandcastles instead of tearing down test infrastructure.

I’d spun it up for a «quick network test» before vacation. You know how it goes – just need to validate some routing rules, shouldn’t take more than an hour. Fast forward 14 days, and Azure had been dutifully charged me for each hour for my forgetfulness – great…

Sound familiar? Welcome to the club. Population: every cloud engineer who’s ever said «I’ll just spin this up quickly.»

The Problem: Why Test Environments Drain Your Wallet

Let’s be honest about how testing actually works:

  1. Need to test something quickly (just for 5 minutes, I promise)
  2. Spin up resources in Azure (Premium tier? Sure, it’s just a test)
  3. Get distracted by other priorities 
  4. Forget to clean up (what test resources?)
  5. Get surprised by the bill (surprised Pikachu face)

Why Manual Cleanup Doesn’t Work

We’ve all tried the manual approach:

  • ✅ Create cleanup checklists (ignored them)
  • ✅ Set calendar reminders (snoozed indefinitely)
  • ✅ Write it on sticky notes (lost under pile of other sticky notes)

The problem isn’t discipline – it’s that test environments feel temporary so we don’t apply production-level rigor. Plus, cleanup always happens during busy periods when you’re already overwhelmed.

Reality check: If you can’t remember what you had for lunch yesterday, you’re not going to remember that test VM you spun up last Friday.

The Solution: terraform-azure-money-saver

Module can be found here.

What It Actually Does

Every day at your specified time:

  1. VMs without «keep» tag → Stopped
  2. VM Scale Sets without «keep» tag → Stopped
  3. Azure Firewalls without «keep» tag → Deallocated
  4. Application Gateways without «keep» tag → Stopped
  5. Resources in sandbox subscriptions → Cleaned up (byweekly)

The «Keep» Tag Strategy

Simple rule: Tag anything permanent with keep key, anything else gets nuked.

# This VM survives the purge
tag keep==true
# This VM does not
tags environment ==true

Forces you to make intentional decisions: «Do I really need this to persist overnight?»

Behavioral Changes

  • Conscious tagging: Now I think about resource lifecycle upfront
  • Fearless testing: Spin up whatever you need, cleanup is automatic
  • Better sleep: No more 3 AM «did I shut down that expensive thing?» anxiety

The Real Win

Peace of mind. I can go on vacation without checking Azure costs hourly.

Pros and Cons:

✅ What Works Great

  • Set and forget: Deploy once, works forever
  • Immediate ROI: Saves money from day one
  • Multi-subscription: Works across your entire tenant
  • Safe: Respects tagged resources

❌ What Doesn’t

  • Not for production: Would be a resume-generating event
  • Learning curve: Need to establish tagging discipline
  • Permissions required: Need permissions on target subscriptions

Lessons Learned

  1. Automation beats good intentions 100% of the time
  2. Infrastructure as Code makes solutions shareable
  3. Tag early, tag often – make it muscle memory

Legg igjen en kommentar