Who's schedule is it anyway?

Delivering new products on schedule is always difficult. At a recent Agile Software seminar, someone asked the crowd if they had ever had any home remodeling done. A lot of people raised their hands. Then she asked if it went according to plan; very few people raised their hands. The point she was making was: if something as well understood as remodeling can't go according to plan, what makes us think that we can do something that has never been done before, and predict the future outcome?

What can you do if your schedule is dictated? This happens in many or perhaps most organizations, and it's not going to change anytime soon. But you can manage it.

I remember an executive who once told me: once your organization gets to a certain size, you are expected to just figure out how to make the dictated schedules work. Whether this is successful or not depends on what percentage of your responsibilities has this attribute. If schedule is the number one requirement for 10% of your work, indeed, you should be able to manage it. Somewhere between that ideal and 100%, you will hit a wall unless you have a method to deal with the unsolvable problem.

I advocate for a highly transparent system to communicate the decisions you are going to have to make. Specifically, you really only have a couple of variables to play with to meet the schedule: resources and function (some people will say quality too is a variable -- depends on your environment). You may need to decrease function or adjust schedule in another area to free up resources to meet the schedule on a higher priority project. The transparency depends on your ability to rank the projects in a way that meets the corporate strategy. You will need to articulate and defend the priority of each; depending on the number of items, you may need a spreadsheet and a numerical method for ranking that can easily be explained. Once you have the ranking, you can justify hurting the lowest priority projects in function or time in order to fund the highest priority projects.

This seems self evident as I write this, but in practice it can be really hard to do when the project you are hurting is going to affect someone else's bonus. Once ranked, socialize the ranking before adjustments need to be made, so that your decision criteria has been vetted before you need to hurt someone's favorite project. To be really transparent, make sure the stakeholders of those lowest priority projects understand the risk involved.

Software Product Lines

I decided to cover this subject since the December issue of Communications of the ACM is dedicated to the subject matter. I first got introduced to the terminology last year when I was talking to a San Diego employer, who was looking for management talent with this skill set. I didn't find much information on it then, but bought a (largely boring) book on it.

If you haven't heard of it, it's kind of the holy grail of software development. What if we could stop re-implementing stuff, and use the stuff we have already invested huge money and effort into making correct? People talk about re-use all the time, but it really doesn't happen significantly in real life. So we pay our employees to solve similar problems over and over again. Extremely inefficient. I've long been a proponent of solving this problem... imagine what it could do for your quality alone. Can SPL really solve this problem?

I don't yet know the answer to that question. A significant amount of information comes from the SEI at CMU. In particular, check this out. Google of "software product line methodology" turns up a bunch of references. Maybe I didn't do it right last year, but it appears that the methods have a lot more traction now. CMU is even offering a certificate program in it.

The articles in the December Communications of the ACM were mostly written in academia. One was written by a leader at Nokia. I mentioned that a San Diego employer is changing their organization and development practices to embrace SPL. It must have enough evidence of success to be worth it, or maybe it's just another management fad. The challenge is that it takes additional work to break your products into product lines; different practices, organizations, processes, roles, etc. The initial change will be hard, and even after that there is an investment hurdle to get over when determining new product lines.

What the methods still seem to need in my mind, is a good grasp on the future. I am dubious about the abilities of the average organization to do a good job at that. I suppose applying it to the cash cows makes sense, except that by definition they already exist, are stable, and making money. So to change it you have to believe you need to make significant additional change, or maybe your current software inventory is too hard to change (either in development effort or product stability).

I believe this method is applicable to larger organizations with good Voice-of-Customer (VOC) data. If you really know where you have to go, you may be able to make the right investment decisions to create software product lines.

Sustainable Advantage

Because I am such a fan of innovation, one of my former bosses once chided me, saying: "The only sustainable competitive advantage is execution". The context of the discussion was about Apple Computer's innovations, and whether they were innovating or simply producing ... well .. art. This quote from fastcompany.com inspired the conversation:

If your cool new thing doesn't generate enough money to cover costs and make a profit, it isn't innovation. It's art.

That article, written in January 2004, may have to eat its words because Apple has proven to be an impressive execution engine. Two incredible examples in my opinion are completely switching over to Intel ahead of schedule, with great quality, and changing the way music is distributed and sold. These are really hard things to do, and they executed amazingly well. It didn't hurt that the products are world-class cool, but what gets lost is how those great ideas could have been simply ideas. The difference is they set audacious goals and then they executed.

That's a long introduction to what I want to talk about, which is taking a fresh look at your business from the standpoint of execution. We complain about how offshoring is making things so difficult, how technology and open source and infrastructure has enabled a lot of competition. Yet we rarely take a disciplined, open, introspective look at our value chain and ask "why" at each step. If we did so regularly, as part of strategic reviews, we could deliver better products faster to customers, increasing customer satisfaction and our brand's reputation. In the process we can literally crush the competition, and then we get to call the shots instead of constantly reacting.

You can use tools from six sigma and lean, to evaluate what you are doing today, where your value is created, where effort is expended for little or no value, and where inventory queues (not necessarily physical inventory) prevent faster flow of value creation. The objective is to get objects through the value creating engine faster, spending enough administrative oversight to ensure the process is working in terms of quality. To do this, you have to start measuring, and that might be difficult if your organization doesn't want to do that extra work. I advocate you start by creating a value chain map so that at least you know what you want to measure and have enough information to make a compelling case for why the organization should do the work to collect the data.

I want to mention, you don't have to fear six sigma. You can use the tools without making the whole organization embrace it as a normal part of their operations.

If you can get that far, and can ensure the integrity of the data, the rest should go a lot easier. You'll still need to involve stakeholders every step of the way, who may feel threatened if something they currently do is no longer necessary. In the end though, you should be able to bring everyone together if the improvements fit into your overall strategy. Using the data, the tools, strategy, communication, and change management, your company too can be a world class value producer.

Innovation and middle management power

Rosabeth Moss Kanter wrote an article for Harvard Business Review in 1982 that was reprinted in 2004, entitled "The Middle Manager as Innovator". She points out that middle management drives changes that increase organization capacity.

Among the many things she says that I agree with, she talks about how lack of power in these ranks can interfere with the innovative process, because it inhibits collaboration. When power is unclear, unfocused, or unevenly distributed, managers are more concerned about protecting their turf than collaborating for the greater good. The successful middle manager in these environments uses their power to persuade their peers to support their cause. This is the good side of politics, influencing others to accomplish the greater good. In order to be successful at doing things out of band, almost surely these managers will need extra resources, and that will likely come from the manager's peers, and he or she must have a little support from higher up to prevent a veto from stopping the project.

One of my coaches pointed out that in order to make things happen, you have to have the right combination of Power, Influence and Authority. The required mix depends on the particular objective. I think in Ms Kanter's article, some times she is talking about influence and authority rather than power. Influence is the ability to change someone else's direction based on a number of skills including communication, persuasion, trust, vision, etc. Authority is that property which is granted to you by someone who has higher authority - it is what the manager is officially allowed or expected to do. Power is the ability to inflict consequences, good or bad, on others.

I have worked for companies where collaboration was really difficult, and I think that it was the lack of power anywhere in the organization that was a key cause (when this is the case, authority and influence must play a larger role, leaving things unbalanced). Some of my peers an I analyzed what it was we were actually empowered to decide. It was almost nothing, limited to who gets promoted and how much to compensate them. This was an intentional part of the culture, Darwinism. One influential leader told a group of new managers, "If you want to get something done, don't ask for approval, just do it". This was felt to foster an innovative environment, and in older times it worked. But when power is fleeting and temporary, and everyone is trying to make their own project successful, you will see very little collaboration.

Collaboration is key, in my opinion, to long term success and market leadership with innovation that improves not only what is delivered, but how. To accomplish that, make sure your middle management is empowered.

Entrepreneurship and Insubordination

Robert Nardelli is quoted as saying "there's only a fine line between entrepreneurship and insubordination". I found this gem in the Oct 2006 issue of Harvard Business Review, in an article by David Garvin and Lynne Levesque. I'll leave the details to your reading pleasure, but there were a few key points that resonate with me, regarding how to innovate within a corporation.

First, quite a bit of innovation is going to lack data, and what you do have is likely to be ambiguous. Another quote: "it's hard to find marketplace insights for markets that don't exist"; this is the innovator's dilemma. You need "Mavericks" that can cope with that ambiguity and drive the organization forward anyway. You'll need to attach them to seasoned operational experts to help them design experiments that validate or negate hypotheses, and change course accordingly. This totally makes sense and seems to bypass a lot of risk and leverages the strengths of the umbrella corporation.

A perhaps counter-intuitive recommendation in the article is to "learn from small samples" rather than statistically significant large focus group learning. Highly successful examples include Intuit's "Follow Me Home" program, where employees actually watch customers use Intuit's products in their natural environment. Nokia, P&G, and Starbucks have similar programs which are leading to new sights.

The article also talks about skills acquisition and building. This is a variant of the build or buy decision, in this case whether you build the team internally or borrow or acquire a team with the right skills already in place. This is a critical decision but generally can be made by evaluating whether you have the competence to grow the needed skills, how critical it is to have the skills on board, and how long it will take to develop those skills.

Sirens' call: Agile

I consider myself an early adopter; I am usually the first to try new things. When it comes to Agile software development, however, I have pretty much waited until it entered the mainstream. We are there now. Large, respectable companies are moving to some form of Agile. They had to. It is clear that what we have been doing hasn't led, in general, to higher quality, predictability, efficiency and effectiveness. The systems are too complex and the world is changing too quickly.

One of the reasons I didn't adopt right away, is that Agile seemed like a step in the wrong direction, for the wrong reasons. If you haven't seen the Agile Manifesto, check it out at http://www.agilemanifesto.org/principles.html.

The first principle I struggle with is "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation." The statement is true, and frankly, is the reason developers love Agile. Many developers I have worked with view documentation as a non-value-add activity. My aversion to this principle is that it ignores a huge problem in software development, which is keeping maintenance costs and quality under control. One of my clients had over 5 million lines of code and no idea how it worked. The people who created it had left the organization. The reasons why things had been done certain ways were lost. Adding anything which required change to the base code was a recipe for unquantifiable risk to the entire release, and many hours debugging under stressful conditions. I'm sure you developers out there will say that documentation is best buried in the code. The problem with that argument is that only developers will read it; and if I may say so, despite their technical prowess, I have seen some incredible oversights committed by developers (including myself).

The second principle I struggle with is that we should "welcome changing requirements, even late in development". It is clear that requirements do change and will change and we must adapt. But I fear that without attention to detail in requirements, people will be lazy about thinking things through, and any organization which behaves that way will waste inordinate amounts of energy refactoring.

Now that I've shown my negativity towards the principles, let me state that I think Agile brings a lot to the table. The number one thing it brings in my opinion has nothing to do with efficiency; it is risk management. In an out-of-control Agile environment, you may have the artist's problem - "it will be done when it's done, and I can't tell you when that is". Assuming you don't have that environment, then you have planned what you need and when you need it to meet your customer's requirements. Using Agile, you define mini-releases along the way to get there. Each mini release could actually solve the customer's challenge in some useful way. Now, if something ends up harder than you planned, and it will, you still have something to ship. You still have goals, you still have a sense of urgency.

My point is this: if you approach this as a potential efficiency drain to create the extra releases, but know that in return you have a ship date that you have confidence in, as well as feedback along the way about how "done" you really are, then delivering "working software frequently" is totally worth it.

Efficiency Metric

I'm inspired by a very simple metric I've seen in a presentation from the lean software institute, which they refer to as "Staff Productivity". It is a deceptively simple equation:

Value Added Effort
----------------
Total Hours

This number cannot exceed 1.0. Value Added is defined in the sense found in "Re-Engineering the Corporation"; it is that value that is tangible to the end customer. I don't believe anyone can build an argument that says that maximizing this value is a bad thing. And therein lies the beauty of the metric.

If you can get your organization to agree that maximizing the metric is a good thing, then you can motivate people to actually measure it. It should appeal to engineers because they value creation, and it should appeal to management because it is a measure of output relative to expense.

Of course, here comes the tricky part: defining what is and is not value add. For example, if your development process includes "refactoring", then you either can include the original development time, or the refactoring time into Value Added Effort, but not both. The customer incurs no tangible benefit by your team re-doing something that is already done. If it needs to be re-done, then you cannot claim the customer got benefit from the original implementation. This is clearly true if the product has yet to go out the door, and perhaps arguable to some percentage if it has.

My point is this: your team undoubtedly has some bureaucracy that can be streamlined or removed. It isn't reasonable to assume you can get this number to 1.0, because you need to invest in the future effectiveness and efficiency of your organization as well as your short term efficiency. But if you define value add correctly, you can both remove unnecessary process steps and identify careless work habits and poor management practices. With appropriate care, you'll be able to justify that new development practice.

Metrics and Radiators

We all know that what gets measured gets done. So why is it that so many software organizations don't have data on their performance? My theory is that people are afraid to be measured. I have personally seen aversion to ascribing defect rates to individuals. The argument against it was that managers are not capable of interpreting the subtlety in the data, afraid that if one takes on harder work and incurs inherently higher defects rates, that managers won't know how to parse that out.

One of the better components of Agile Software Development is the information radiator. This is the prominent public display of data relevant to the project. I argue that a combined team and individual defect rate radiator will inherently drive the right behaviors in an organization. Management need take no additional action other than to keep the radiator current. Capable, normal Individuals and teams want to do better. They can rationalize why they might have worse metrics than other individuals or teams, but they will work to make it better if it's visible.

Other metrics I have worked on include those for efficiency and effectiveness of an organization. The first time you sit down with your team to define these, they probably won't get it. Mine came back and wanted to measure the number of meetings with agendas and number of meetings which finished on time. They chose those because meeting time seemed very wasteful to people. While interesting, this wasn't what I had in mind for measuring how well an organization is doing. So I went after another aspect of quality: delivering products on time.

One regular challenge to delivering products on time was dealing with change. Schedule changes from dependent components, requirements changes from customers or the market, design changes due to oversight,etc. What we agreed to measure was the number of contingency plans we had relative to the number we needed, and whether those contingency plans actually worked when needed. The other part related to that was how well we executed relative to plan. In order to do this you need a fairly granular plan. And a radiator. Also not a perfect measurement system, but one which should get the proper results.

Strategy and Mission

Much has been written on strategy. It can be a fuzzy and cosmic subject. I advocate a very simple approach to setting and communicating a strategy, and it can be done at all levels of the organization where choices are made.

A simple strategy is one that is written specifically for the purpose of review by other parts of the organization. It should cover what could be done, what will and won't be done, in what order, and why. Writing this down invites discussion of the rationale for investment. It can make the case for dial-up or dial-down investment in an area. In a healthy business there is more that could be done than you can afford to do. Without a written strategy, your department may subject itself to direction changes from other departments, customers and others.

That doesn't mean you won't change your mind. When you do, the simple strategy provides a level of rigor and review to make sure you haven't compromised something else by doing so.

A written strategy is a mission statement on steroids.

Corporate Life Cycle

I am a fan of Adizes' model for the corporate life cycle. In his book Corporate Lifecycles, he describes four categories of roles that each executive should have reporting to him or her, in order to bring good conflict to the table that can be used for good decision making. The four roles are PAEI:


Performing. In software product development, this is most of what we think about the business; viz creating code which provides a value which translates to money.

Administering. These roles provide predictability, repeatability, quality in what you work on today.

Entrepreneuring. These roles ensure the long term success of your business, looking ahead to what you need to do in the future.

Integrating. These roles are also long term, making sure that you can afford to work on what you need to in the future.

The essence of the model is that you can predict how an organization will perform based on how well balanced these roles are. It is a fascinating read.

Approachability

One of the best ways you can get input on all aspects of your organization is to make sure people want to talk to you. This seems self evident, yet many leaders don't invite input except from superiors. You can't just say "give me your input" and expect it to happen, you have to demonstrate that you care.

What I do is make others feel respected and important. Everyone. The janitor. The receptionist. My developers. My peers. My entire network. I get to know their name and commit it to memory. I banter with them when I first see them during the day. I enjoy them as a person for a couple of minutes on every interaction.

I remember a former manager and mentor who told me that he respects anyone who does their job well, irrespective of the status of their position. If an individual takes to his or her job with pride, if s/he is the best XYZ that s/he can be, that deserves respect and admiration. That feedback stuck with me, and as I became more people oriented as my career went along, I put it into practice.

I'm sure some executives will say this is poor time management. I disagree. As an executive you will get more back from your organization by doing this. Two benefits resulting from this attitude and behavior are trust and loyalty, and it is both exponential and infectious.

The one true way

You've all seen it: religious wars over technical solutions. emacs versus vi. C++ versus Java. VxWorks versus Embedded Linux. These are decisions you often can't make right because there is no right solution. And whichever you choose, you will alienate a percentage of your technical people. I've tried various ways to solve this dilemma with disappointing results.

What should you do when faced with these?

You will have to do due diligence to make sure that there isn't a clear winner because of some property you haven't yet thought of. Pick a senior person or a small team as technical advisors, and have them research the options, but give them a due date and set the expectation for how much resources they should spend on it. You want to understand whether you can get mindshare or not. If you can't you want to discover that quickly, recognize there probably isn't a "right answer", then simply decide or appoint someone to decide.

Here's the critical part though: make sure the team knows that the decision will be final, who will make the decision, when, and what any known decision criteria is. Let them know that you will support the decision and that you expect the entire team to support it as well. Moreover, that there will be negative consequences for anyone who doesn't go along with the decision. Make sure you are captain of your ship and that the crew isn't going to sabotage the decision. They are either on your ship or not. Their choice. All for one and one for all.

Then you can go on to working on things that add value.

The most important thing about leadership

What I have noticed when technical people complain about bad leadership (as opposed to lack of leadership), is that there is really a lack of trust. This typically occurs because the leader has either exhibited repeated selfish behavior or is failing to communicate openly and honestly. I have seen some leaders holding back information in the interest of appearing dignified or more intelligent than others. I have also observed leaders spewing the party line to cheer up the audience, when the party line is not credible. Neither of these behaviors will lead to long term success.

In my experience, people will respect you and follow you when they understand that you have their well being in mind at all times. The only way I know of to establish that understanding is to drop your guard and let others know where you stand. I'm not advocating that you don't speak the party line, but when you do, make sure you add what you really think (perhaps softened a little if it is a sensitive issue). That way people know you are real, and when you say things are really good, or that they could be really good if only the company or individual did something, they will believe.

You need to go out of your way to help others, in and out of your team, when it is the right thing to do for the group. If this makes the team stronger or the product better, everyone wins. You'll get credit for doing the right thing even though it hurt you, and when you need their help, they'll be there to help you. You want to be the leader that will be there for them when they need you, and you need to demonstrate that every chance you get.

The consequences of losing trust in a high tech firm are dire. Once you lose trust, you lose a lot of the communication that needs to happen. Your team needs to be able to tell you when you are heading for a mine field. They will do that when you do the same for them. As a leader in high tech, you have to accept the fact that you don't have all the answers, and you will need effective communication to help your group get to the right destination.

If your company is healthy, individual well being is typically best served by the company's well being. So if you focus on what is right for the company, while respecting the individual, people will notice. If your company isn't healthy, that is a totally different situation. You need to decide what is best for each individual and let them know why you believe the things you do. If you are a leader in an unhealthy company, you have to think of the individuals first. Anything less than that isn't leading, its pandering.

In the end, it's all about trust.