Thursday, June 8, 2017

I woke up in Software Hell

This morning, I woke up in software hell.  Condemned to burn for eternity with no way out!

Or was I?  I'm a Christian ... Lutheran to be specific.  So I believe that as long as we have faith there is always a way to heaven.  (Dear God, please stop me from writing blog posts when I am inebriated!  The failure to register my new domain was probably heaven sent!  Now, no one will ever see this post!)

I lead a Business Intelligence team of 10 - 7 full time, and 3 contractors. Well, OK, 1 of the 7 doesn't actually report to me but most of the time the architect listens to me respectfully.

I am putting myself out on a limb here sharing all this information.  I am being very vulnerable here, so please don't laugh!

Up until a year ago, I was a Business Intelligence Developer.  But my boss was overwhelmed by the number of direct reports she had.  And I was keen to try to make a difference for the team that I helped build back up from a count of 2 (post death march), so I made the leap into management.  Of course, I refused the title of "Manager" because who wants that on their title when it makes you seem non-technical, right?

What I quickly discovered is that for every problem that I tried to resolve, there were 10 more underlying problems.  And I love to dig into things, so of course pretty soon the total number of problems became "billions" (Austin Powers reference folks ... milllions just seemed like too dated .. I can still see Scotty's disapproving face).  I used to think my former bosses at this company were ineffectual.  But of course, I now label myself as that too ... OMGoodness the stuff runs deep here.

So, do I throw in the towel?  Heck no!  I am a glutton for punishment.  I've been at this company for nearly FIVE years.  That is a total record folks.  My norm is 3 years before the itch hits ... or layoffs happen.  Shocked my grandparents so much they all are dead now (natural causes, but they were flabbergasted).

I can tell you that there are widespread problems in the way we do software development at my company.  We've got some awesome people.  Really I love them all.  But I am starting to believe that the reason so many Business Intelligence and Data Warehouse projects fail is because no one has figured out how to manage the projects.  I can tell you that the technologists on the team are so overwhelmed with keeping up on all the technological advancements that software process is the last on their list.  Process sucks, right?  And I can tell you that the pool of candidates for what I do is so small that you aren't that picky about the software dev processes they've used because you could never hire anyone and the work is unending!

So tonight, after stewing for a few days on the latest call to the carpet my boss gave me, I stopped thinking about the technologies and I started researching the methodologies.

Some background: I came to my company with a litany of varying SDLC methods.

  • 1st company - 1999 B2B dotcom doing VB6/Classic ASP with 20 yo boss who lived 800 miles away.  Methodology?  We didn't have anyone but developers on our team.  Write code, test it yourself, see if boss likes it.
  • 2nd company - 2001 VB6/COM Watherfall RUP.  Company learning RUP as we went trying to get 6 Sigma certified.  Turns out that was just to make us look good to sell.  Wasted a lot of time building interaction diagrams, but I retained ERDs and I still find Use Case diagram principles helpful.
  • 3rd company - 2002 VB6/Crystal Reports/C# One report/app at a time, custom dev for customers.  Just do it.  Honestly, I think I was best at this.
  • 4th company - 2004 ... well, I went back to training, then tech support where we started building our own ticketing system and I was doing XP on my own, and just as I went back to dev who were doing Scrum ans where I learned TDD.  Bam sold.  Fast forward...
  • 5th company - 2008. tsql / SSIS / Ant / Cold Fusion.  We met every two weeks to share.  Architect was also business stakeholder and called the shots.  People just got stuff done. If one of us saw something, we had the freedom to call it out.  No walls.  I automated everything I could get my hands on!
  • 6th company - 2012. Sql BI stack. "Agile" paceline for 2 years.  Dramatic shift to Kanban/Lean where BI team was accidentally excluded from training.  Welcome to software hell.
So, yeah.  I had no idea what was correct.  I just cared about getting stuff done.  But as I look back, I realize that the less process we had, the better off we were. The less rules we had, the more empowered I was.  The smaller the teams and the less ego on the team, the better we worked together!

I was feeling frustrated tonight, so I broke down and googled "Agile" and went to good old Wikipedia.  Guess what I learned?  While EIS at my company professes to follow Agile ... OMGoodness we are falling sooooo short.  I scored us at 5 of 42 on this test: How Agile Are You? (42 point test).  Some folks are going to argue we are Kanban not Scrum.  But I will argue that we should be.  But find me a Kanban test and I'll take it! Maybe this one How Agile Are You: Let's Measure It.  59 of 300 .. slightly better .. from 12% to 20%

The good news is that taking these assessments is valuable.  It was obvious to me what we are doing wrong.  Next step - let the team figure it out by giving them these assessment tools.

Maybe, just maybe I'll wake up tomorrow and the level of hell will be less deep!