The BudgetView experiment: Introducing Problem Driven Development

I have been working for quite a while now on a personal project - building an application called BudgetView, the product of a five years experiment in which I have been exploring two topics:

• how to make personal budgeting, which is far from being the most attractive and engaging activity, as painless and straightforward as possible.
• how to use agile and lean principles to solve this problem and build an application that fully satisfies its users.

This personal project has brought me a few very important insights. The main one is that I have come to understand how to apply Lean principles to Product Management - the kaizen of software, in a sense.

This has led me to propose an approach dubbed “Problem Driven Development”, which is about using a structured problem-solving activity to improve the software in such a way that it completely solves the problems faced by its users, without adding any waste in their use of the tool.

The genesis

In may 2006, I was looking for a way to get a clear vision of my budget - clearer than the lengthy listings of operations made available on my bank’s website. I had a look at all the major players in the rather crowded “personal accounting software” arena, but I couldn’t even start imagining actually using one of those.

I asked a few people around me how they were managing their own budget, and it turned out that only a handful were using Money or any other tool, a few others had a custom-made Excel spreadsheet, but most of them were just not managing their money at all. Most of the people I talked with said they would be interested in an easy to use tool for managing their finances.

The underlying message? The problem of “personal money management” had not yet been cracked by the software industry!

It was not long before I started working on this with my partner, Marc Guiot. We had had a few great successes on our agile software development projects, and we were convinced that using an iterative approach we would find a much better way to handle this “personal budgeting” problem.

We thus happily started hacking our way into this new field, mostly as a side project since we were both maintaining our day to day coaching and project activities.

One “idea” to rule them all

In the first months, we were just looking for a minimum version of a tool that we could use ourselves. The problem we were addressing at the time was: how could we provide a clear vision of one’s budget?

The general idea was to give a one-page overview of one’s income and expenses, with a month-by-month view of the operations. However, we thought at the time that it was not enough to differentiate our product. We then came out with “The Idea”: there would be a shared server which would learn about all the categorizations made by the whole community of users, up to a point where the system could be able to automagically categorize operations. This was very “web 2.0”, despite the fact that we were building a desktop application.

We did a few paper prototypes and started working on this first version of the tool:

We kept on iterating and improving the software, based on our own ideas and the feedback of a handful of people. About a year and a half later, it was looking like this:

A few people were actually using it, but without any real passion. The feedback we were getting was, let’s say... polite. We were improving it step by step, but we didn’t realize that we were stuck in a local optimum.

The first clear signal, however, came from my wife. She had been using the tool for a few months when she once asked me: “That’s all very nice, but... what is the purpose of using it? What do I do once I see all these operations and all these categories?”.

Oooops! Our product was useless. On top of that, it turned out that the initial auto-categorization of operations was not such a great idea, since the main problem users were facing was how to manage their budget.

We were not clear enough on what constituted value in the eyes of our users. This had led us on the one hand to deliver something that had not enough value for them, and on the other hand to produce useless features (an example of overproduction, the worst kind of waste).

At this stage it was no longer a matter of “kaizening” our application anymore. A “kaikaku” (or “radical overhaul”) was in order.

First pivot

We had been thinking for two years about what it meant to manage a family budget - trying out ideas, looking at competitors, reading a few books on the topic, etc. However, our ideas were still confused, and we were both surprised by the complexity of the topic.

So we went back to square one and spent a whole week brainstorming on our initial question: “what does it mean to manage a family budget?”. We came up with the three concepts that still are the pillars of BudgetView today:

• center the tool around the high-level “envelopes” that constitute a budget, instead of the atomic operations downloaded from the bank.
• help users create a budget at the beginning of the month, and then monitor their progress against this plan during the month (a budget being what I would now call a “target condition”, in lean terms)
• provide a forecast of the budget in the next few months, to help users work on mid-term plans to realize their personal projects.

We created paper prototypes for all the screens, iterated on them, and came up with a new vision that looked like this:

We shared our new paper prototypes with a few users to get their input, but it was difficult for them to really get a feeling of how it would work in practice. We thus decided to start implementing the first elements of this new design to get more feedback.

At that time Marc and I decided to put our freelance activities on hold for a few months, to work full time on our project. We were coding six or seven days a week, rewriting large parts of the application, looking for a minimum set of features that would allow users to import their operations, categorize them in various envelopes, and visualize all the envelopes in a consolidated budget view. We also decided to remove all server-related features, mainly because our users had been telling us that they preferred their financial data to stay on their own computers instead of a remote website.

After three months, we had what we thought was a first usable version of the new tool. Its main screens were looking like this:

When the main usage scenarios were implemented, we sent the application to a dozen new people. The overall feedback was terrible. They found the concept “interesting”, but:

• without any external guidance, it was really difficult for them to understand what the application was really about, what they were supposed to do, and how they were supposed to manage their budget with it - the interface didn't guide them enough, and the overall logic was too different from that of other tools.
• they had many complaints about missing features that we had deliberately postponed in order to build a “minimum viable product”. We were trying to have a minimum feature set, but the minimum acceptable level was much higher than expected.

The reactions were too negative to launch anything. We were far from ready, and our four-months sprint was over. This was a tough period for us, but we were committed to continue getting feedback from users and fixing things one bit at a time.

First glimmer of hope

We thus resumed our work, mostly on evenings and week-ends, relentlessly iterating and improving the application based on user feedback - which mostly resulted in implementing everything we had deliberately postponed.

At that time we had a hard time getting lots of feedback for the tool, since most new users would quickly tune out, but we managed to get enough to keep going. We kept on fixing things, considering everything which caused a problem or even a question for the user as an “error” in the design, and following Christopher Alexander’s insight in Notes on the Synthesis of Form:

We would hone the product until it was dead simple and got the job done without any problem.

The first clear sign of love for our product came in mid-2009, about three years after we started. One of our main users, who had had the patience to struggle with the first versions of our software, once told us he was now able to manage his budget in twenty minutes a month, down from two hours with his previous tool. He even said it was a pleasure to use, and started telling his friends about it. Woohoo!

Users don’t want to spend time with our software, and that’s great!

On top of much needed congratulations, our friend had just given us a precious hint on what was the main benefit of our tool in his eyes: saving time in managing a budget.

In lean terms, our goal was to reduce the lead time of a “personal budget management session”, by removing any unnecessary steps. This might look quite a trivial statement, but it radically changed our approach to product management:

This is a powerful statement, which is not at all limited to our application.

Just think about it one moment: who really wants to stay in Google, for instance? When you use Google, your main goal is to go as fast as possible to the target site, right? In other words, you use Google not to enjoy its numerous features, but because it reduces the time it takes to complete a search session.

If you follow that line of thought, you can consider that:

• The Google home page, with its zen-like simplicity, is a lead time reduction strategy - it just helps you avoid scanning useless messages or controls and go straight to your search.
• The Google infrastructure, consisting of hundreds of thousands of computers, is a lead time reduction strategy - bringing you a response in about one-tenth of a second.
• The Google indexing & search algorithm is a lead time reduction strategy - if the best results didn't come first, you'll have to waste time in back-and-forth navigation
• The Google "auto-correction" feature, which for instance gives you answers for "lead time" even when you mistakenly type "lead tme", is a lead time reduction strategy - saving the time you would spend fixing your tipo.

In our case, we could thus reframe our users expectations like this: give us value (the ability to better manage our money) and remove any waste in the process (entering operations manually, learning how to use the tool, performing useless operations, making mistakes, etc.)

Our next challenge was then to better understand the obstacles faced by users, so that we could reduce both the lead time of their discovery process, and that of a typical usage session.

We kept on iterating and improving the software, but it was not working. Despite our efforts, people still had a hard time learning how to use it, and we had very few users adopting the application. We were stuck again.

Lean remedy 1 : Going to the Gemba

At that time, the way I was seeing this project was changing deeply as a result of my lean training and my activity as a lean coach at Operae Partners. A first benefit of lean thinking was that I gradually learned to better focus on what we had to do rather than what we wanted to do. Our discussions with Marc were more and more focused on a few important questions. What was our goal? What was the next level we needed to reach? What was the main thing preventing us from reaching it? What was the simplest thing we could do to solve that?

An important thing we realized was that the way we were getting feedback from our users was not working. When they described their problems, it was very difficult for us to know what steps they had gone through to get into the situation they were describing. And once we had implemented a solution, we had no clear way to know if it was working for other people.

Since it was a money management application, we were a little embarrassed at asking users to let us see how they were using the software, since this would have meant looking at their personal accounts, income levels, etc. As a result, we had learned to rely on emails and phone calls as the main vehicle for user feedback.

What was happening in reality was that despite our supposed focus on user feedback, we were reacting to what we thought their problems were, based on what they thought their problems were. We didn’t see ourselves what was really happening. We had failed to go to the “gemba”, the location where the action takes place - in this case next to the people using our software for the first time.

Once it became clear enough for us that we needed to see users in action, we went back to “usability 101”, and set up what we called “observations”. We would organize an 1-hour meeting with a potential user, and ask her to discover the tool with a demo account, in front of us. We would ask a set of predefined questions at key steps of the process, and spend most of the time silently watching and taking notes.

The first observation was an eye opener, or rather a cold shower. Despite our efforts to make the application “intuitive” and “user friendly”, the new user experience was disastrous, to say the least. I went out of this first meeting in a rather depressed mood, but with a long list of problems that the user had faced - most of them caused by what we could have considered trivial details.

The double curse

The main thing we learned from these observations was that it was not possible for us to know in advance how people would react to a design. I already had some experience in user interface design, which had always been an important center of interest for me. However, until now I had only been writing enterprise software, where users were comfortable with using computers and ready to invest at least some effort in understanding how things worked. In this case, with consumer software, the bar was much higher. People would not read more than two or three consecutive words on the screen, and they would deliberately ignore anything remotely looking like a help message, discovering the application in seemingly random patterns.

As designers of the application, we were thus the subjects of a double curse:

• The Curse of Knowledge: we knew too much about our application, and were thus unable to even imagine what it was like to discover it (for a thorough description of this phenomenon, I encourage you to have a look at the Heath brothers' excellent book Made to Stick).
• The Curse of Ignorance: we knew very little about the user expectations, context and experience. We could not even know what we had to fix in the tool, or whether our fixes were relevant or not.

Lean remedy 2: the PDCA wheel

What this all meant is that even the “solutions” we would implement as a result of our observations could be flawed, so we needed to test everything we did with new observations. In other words, we had to treat every single evolution as a PDCA:

• Plan: What problem are we trying to solve? Did we really go and see its causes? What is the simplest thing we could do to check if this is the right cause, and maybe fix the problem? How will we know if it’s really fixed?
• Do: Implement the solution.
• Check: Test the solution through a new observation.
• Adjust: Draw the right conclusions from the experiment. What do we learn about users’ reactions? What do we learn about user interface design, or personal budgeting? What do we learn about our product development process?

Introducing Problem Driven Development (PDD)

All these insights led to what I have dubbed “Problem Driven Development”, which is a rigorous product management approach consisting in using "PDCA + Gemba" to completely solve the problem of the user.

This is a radical shift in how we view software development, in very much the same way that we have stopped thinking that dumping kilo-lines of code as fast as possible was a good thing. The goal of a software development project is not to pump out features as fast as possible, it is to completely solve the problem faced by users without introducing any new problem with the system.

The main sequence of PDD is the following:

• Understand value from the point of view of users. This means sitting next to them, understanding how they work with and without the system, and defining where the application brings a real benefit.
• Measure performance. You cannot improve what you don’t measure. We measure the number of users and things like conversion rates, for sure, but we are also interested in the time it takes for users to get up to speed with the software during our observations, for instance.
• Visualize problems. In our case this mostly means sitting next to them to see what they are going through.
• Solve problems one by one, managing every evolution like an experiment built as a PDCA cycle.
• Learn the right lessons. Every PDCA cycle is an opportunity to learn more about the domain, about how users discover and interact with an application, and about the development process itself.

On of the consequences of all this is that Marc and I are not guided by a scrum-like backlog of features anymore. We track the list of problems faced by our users instead, along with counter-measures we imagine based on the causes we identify for these problems.

Lessons for enterprise software

The lessons from PDD are not just valid for consumer products and startups - they also make perfect sense for enterprise software projects.

Many projects I have come across are driven by laundry lists of features imagined by various stakeholders, without any real connection with the reality faced by users every day. Features are dumped over features, without anyone from IT or business teams ever checking that all these features bring real benefits to the end users once put in production.

At a time when many IT departments are considering Lean or Lean Six Sigma initiatives to improve their project management processes, why not start from the beginning and clarify the goals of the projects themselves?

Some questions will help start a project on the right foot:

• Who are the users of our systems, and why do we think an IT solution is absolutely needed? Is there a cheaper solution users could put in place before rolling out any new piece of software? (hint: what about considering a lean approach for the users activity in the first place?)
• What measurable benefits is the IT solution supposed to bring? How many hours of work will be saved every month, for instance? How will be able to measure these savings once the solution is deployed?
• What wil be our process for checking after every release that the new features bring measurable improvements in the users activities?
• When and where will take place the next observation?

We’ll have made some progress in this industry when IT projects will be started with problem solving A3s!

Next steps

This PDD approach has done wonders for us. We have removed lots of clutter, clarified the concepts, streamlined the discovery process for new users. In our last observations, people were able to discover the software much faster, and they found it fairly easy to use.

However, we are not there yet - even if it’s getting better, we still have users who have a difficult time getting to grips with the application in some places. Our plan is to keep on polishing the whole user experience, from the time someone discovers our website to the regular usage and recommendation phases, with always the same approach: engage users, understand what obstacles they are facing, and remove these obstacles one by one.

By the way, if you can read French, why don’t you give BudgetView a try and keep in touch with us? Or better yet, if you live near Paris, what about contacting me to participate to an "observation"? We'll talk a bit about PDD, and you'll have a firsthand experience in feeding a few PDCA cycles!

One last thing: when do you plan to conduct the next user observation for your own project?