The need for data-driven product development
Product development has many nuances and dependencies. Every product development cycle or SDLC imagined or put forth is found to be inadequate. Every proposed solution falls into one of the many traps present and has proved to be ineffective in one or the other phase of the product’s life.
SDLC started with the waterfall method which expected all requirements to be listed before design started, design to be completed before any coding started, coding to be completed before the testing started and so on. This obviously was inadequate because nothing can be 100% done when fuzziness is in play and product development is as fuzzy as it gets. The waterfall model was modified to the iterative model to account for the deficiencies. It tried to correct the problem by allowing to step back to the previous steps in case of problems. But, the obvious problem that this bought out was that iteration could happen in any phase of the development. So, you put down the requirements, you design, you start coding. During coding someone comes up with a very vague question which becomes the tip of the iceberg and you have to go back to the drawing board and start from requirements all over again. A whole lot of time is wasted. The SDLC corrected itself to the spiral method, the V model from them and then the agile method, then the scrum method of agile, each with their own deficiencies.
The current standing methodology is the scrum or agile method, but this has its own traps. When you create a product for the first time or add an enhancement for the first time, you need to have substantial functionality thought out and designed. You cannot do a piecemeal from the beginning of the product development. You need to devote the appropriate time thinking through. If you do not devote the time to think through, you have a patch work of a whole lot of functionality, stitched together to seem like a whole. A product built in such a manner stands on very fragile foundation and is bound to fall apart the minute the foundation is challenged. Where no foundation was thought through and laid correctly, the chances of the foundations being challenged is very high and the chance of the foundation collapsing is even higher. While the scrum methodology with its epics and stories are good for later phases of the life of the product, it does not augur well with the first development or major enhancement phases. It should be recognized that if most of the time in a product development cycle is devoted to bug fixing and maintenance instead of major new enhancements, then the product has failed, because the returns on such products are largely spent on maintenance and not real R&D.
Sure, we can keep patching the SDLC and keep fixing the problems as they occur. But, I think, what we need to ask ourselves a very simple question, “Why has it taken so many years and iterations of SDLCs and yet we are not able to come up with a comprehensive way of developing products?” I think it is time to ask the question “Are we ignoring some phase of the product development which needs to be a part and parcel of the product development life cycle?” The current SDLCs always only includes a play with these 5 phases: requirements analysis, architecture/design, coding, testing and deployment, however they may be arranged, waterfall, spiral, V or short cycles.
But, it should be recognized that a software product’s lifecycle is not purely product development. It is a conglomeration of various disconnected cycles: product marketing cycle, product development cycle, customer engagement, support cycle and such, strung together by the abstract fragile thread called “the product idea”. In the current way of working, each of these cycles are considered to be separate and follow their own processes. It should be appreciated that while some of the activities in these cycles can be run simultaneously, all cycles are tightly tied to each other and more specifically, tightly tied to the product development cycle. I find that, it is because we do not recognize and integrate these into the product development cycle, we keep running into the problems of changing or unknown requirements, changing UX designs or changing functionality designs. The change in any or all of these needs to be factored into the product development cycle.
For e.g., the product development has to easily react to the changes in product marketing. It is possible that a certain market segment was identified and targeted, during the market study of the product or feature. But as the marketing cycle continues it could have emerged that the product or feature is easily accepted by a second market segment rather than the identified market segment. If, this change is not fed back into the product development cycle and changes incorporated immediately, we are looking at a whole lot of wastage. The product development has to also easily react to responses from the customer engagement cycles. Customer engagement cannot be an activity that happens only at the end of every product release. It has to be an on-going effort where the customer can be engaged at every step of the product development to reduce wastage. If wastage needs to be reduced, a product development cycle has to be an integrated cycle with the product life cycle and not disparate cycle. This is exactly a data-driven product development.
Data-driven product development
How does a data-driven product development work? We start with the simplest product cycle:
In the current landscape, the creation of the MVP, adaptation of the MVP or modifying the features of the product are considered to be product development and the SDLC is kicked off once the ideation is completed or the analysis is complete. But, as I have said before, the phases of product development are as fuzzy as they can get. There are no clear-cut lines between ideation and product development or no clear-cut lines between studying the statistics of the product and modifying the product based on the results. You can think you have got all the major features identified, you can think you have explored all possibilities and ironed out most of the problems, but that is just in thought and documentation. Most times, it just needs the questioning of a single assumption or bringing to light a single preconceived notion before a whole new dimension to the idea is exposed allowing you to explore new possibilities. When what will break down the barriers that prevented you from seeing new dimensions or break down the whole feature as unnecessary, cannot be predicted in any manner, it is as unknown as the future.
In such an environment, having product development go off and develop something based on its own rigid cycle does not work well for anyone involved in the product life cycle, neither the business nor the technology. What is required is for the product development cycle to be integrated with the product life cycle, not as a single “Develop product box” that can be spawned off separately and forgotten about until the development is completed. Product development needs to be broken down into its constituents phases and the phases that get affected by variations in business decisions due to market response or customer interactions be designed to absorb the variations as and when they occur. While scrum attempts to do this using short stories, it should be recognized that just splitting the scope of work into smaller pieces while the work remains the same as before, is not the solution, it just introduces other problems.
It should be recognized that integration of product cycles CANNOT be just a pure process change. It should be accepted that when we talk about change in target market segment or changes due to altered customer perceptions, we are talking about an impact in the product development all the way from the definition stage to coding. While the precepts of the product may remain the same, such changes usually mean a change in focus of functionality, a change in the prioritization of the functionalities or such, that impact inherent product development focus. This implies that the product design and development needs to be highly reactive to marketing cycles and customer feedback, which is not process related. It is thorough rethinking of the process, architecture and design of the product.
The first and foremost is that, the product needs to be inherently incorporated with data collection ability to be able to react effectively. This means that product analytics cannot be considered as an afterthought added using third party analytics tools to monitor the usage. It needs to be designed for, from the beginning of requirements definition. For e.g., if a feature being defined is considered to be a very basic feature of the product, then it needs to be thought at the point of definition of the requirement as to “what will indicate the feature is not as basic and the appropriate monitoring capabilities added to track the assumption”. Let’s say for e.g., in a multi-channel web retailer’s shipping product, we identified the basic feature as “combining shipments across channels”. What is important to study as a data point to validate this requirement is “how many shipments are created with orders from multiple channels”. Just adding analytics to study how many shipments are created using URL based monitoring does not validate this requirement. Throughout the product development, such data monitoring needs has to be taken into consideration and related functional monitoring capabilities built in to validate the thoughts and assumptions.
This helps in multiple ways. Product analysis after the fact using the incorporated analytics help rethink requirements and redefine product features. Another important use of such a monitoring identification is that it will help identify the features that need to change when input comes from marketing research or customers.
In the above e.g., of multi-channel web retailer’s shipping product, we could have assumed that a “common inventory” is used across all channels. But a survey or talking to potential customers can reveal that most of them just have multiple inventory sets for each channel. In this case adding a feature to combine orders from multiple channels does not work well for the customer. In such a case we can very easily look up all those features for which we added the appropriate monitoring capabilities and with remove them or modify them appropriately.
Thus what is needed, is a highly reactive product development cycle. A highly reactive product development cycle is one where it interacts with the other phases of product lifecycle and product development morphs with the input from them. Below:
It should be noted that marketing changes and customer reactions almost always impact the product requirements and design and very rarely impacts just the code. Yet, in all our product development cycles we spend most of time in coding and testing as opposed to product design.
Hence, to be able to react fast, the time devoted for requirements analysis, architecting and designing must be maximized so that it is only the drawing board designs that are changing and once we know that design is more or less done, the time to code, test and take the product to market should be minimized to avoid interim changes.
Thus the two most important changes in product development to achieve data-driven product development is: One, incorporate data analytics in all phases of the product development life cycle starting from the requirements phase and two minimize coding and testing to the most minimal possible and increase the product design phase to the maximum and go to market fast.
Published on Java Code Geeks with permission by Raji Sankar, partner at our JCG program. See the original article here: Data-driven product development Opinions expressed by Java Code Geeks contributors are their own. |
