- Setting up Auditing & Logging of Files/Objects Using Native Windows File Server Tools - 16th October 2020
- Designing Key Performance Indicators (KPI) - 15th July 2020
- DDOS Attacks and Website Hacking - 6th July 2020
Programming Fundamentals: Functional Decomposition
Improving software engineering by breaking a problem down into more easily solvable parts
Functional decomposition is a rather grandiose and pretentious term for breaking the problem down into smaller more easily solved problems. In this article I’d like to cover how spending time doing this, if needed twice (once after the initial requirements acceptance, and then again at the time of feature design) has benefits. Sometimes, in order to speed development up, you first need to slow it down – assess the situation, the design, and the options available. I’ll also cover how slowing down to speed up improves the development processes that managers are responsible for managing and how this also benefits organisations in general. Finally, I’ll cover techniques of decomposition, how engineering team managers and Agile coaches can assist in functional decompositions, and briefly touch upon why a timely delivery with a smaller feature set is preferred to a delayed one with a complete one.
Although I’m no longer a software engineer myself, often times some of the software engineers or development team leads in the programmes I’ve managed have come up to me saying they have a problem that requires more time than they previously allocated in the sprint or that a feature would have to be delayed to the next release. This is usually because they need more time to solve the problem (not to implement the feature per se, but to solve a technical issue presented by implementing the requirement).
Other than the obvious reason of coming to me because they’re escalating as I’m either in charge of the release train or programme, or I'm their team/department lead (and a problem shared is a monkey off their back), I think the reason the engineers come to me is that I’m a former dev my self and it’s sometimes good to bounce ideas of someone who understands what they’re doing but is abstracted away enough to look at things from a fresh perspective.
Timeliness vs Functionality
I’ve also always encouraged the team to escalate any deadline/release date impacting issues immediately because sometimes it’s better to be timely, but with some features absent, than be late but with all features implemented. This is because sometimes the business need to be first to market is more important than having every single feature. This is something not always called out in non-functional requirements so the programme lead or departmental heads must determine this through their relationships with the business sponsors and wider organisation.
This need to be first to market is sometimes very acute because, even though at the time of product design the market researchers may have thought certain features are critical, it may turn out those features aren’t as popular with the customers as first thought. But beating a competitor to market share is.
Besides, it’s not always possible for the business, sponsors, or sales teams, to judge the quality of a product immediately but they sure can, and often do, judge and complain if something is late!
In these situations, I of course, work to understand what, technically, the problem is, before deciding on the next option. And one of the things I encourage the escalators to do at this point is to use this as an opportunity to further decompose the problem, with me as a fresh perspective.
I’ve found that sometimes doing this breaks the problem down enough to present a solution when previously there was none and pulling a feature or delaying a release was where everyone was heading to (this may also be because it’s possibly the first time, after my cajoling, the engineers attempted functional decomposition after the initial higher level requirements analysis sessions in a planning sprint).
Real World Example
A somewhat distraught engineer once came to me saying that he’s looked more closely at a requirement he’s just begun working on and it will take longer than expected to implement. Which means the other team of mine that in turn is dependent on this function he’s creating will have to delay one of their features into another release.
After assuring him that the world was not ending and that programmes all over the world accommodate such issues daily and so too would this one, I asked him to spend some time with me analysing what the actual requirement was that had to be implemented and help me understand, technologically, what was attempting to be done.
Now, I’m no longer as technical as I used to be, which is possibly why this helped, because I asked him to explain to me the issue as he would to a “small child, or large dog” (© Jeremy Irons in Margin Call). And through that process we asked enough questions about the overall problem at hand to see that the a certain form of the solution would be implementable in time.
How Managers Can Help: Help the Engineer Better Understand the Requirement
“I need to delete something from a group of things. The deletion has to be quick, otherwise the app looks like it’s hanging and the user experience degrades enough where users might move over to rival apps. But the processing on the phone, I’m finding out now, is too slow, and I need more time to figure out a solution.”
This is the point where we started to discuss questions, approaches and ideas around what is the actual requirement and what can be done to meet it. Because the underlying question to me is not that this will be delayed so what do we do next, but rather, why do you think it will take longer to build, how long will it take you to build it and how accurate do you think that estimate is?
Those are the critical pieces of information we would need if we were to plan a rescheduling of activities to accommodate this potential setback. And just by looking at the original requirement – create a function that deletes something from a group of things quickly – in my opinion, it was impossible to make any accurate estimate as to how long that will take.
My approach here was to spend some time on functional decomposition to arrive to a better estimate and make a decision on options based on that. These were the kind of questions we thought through:
- What type of group is it: “A bunch of transactions”
- How are they being received: “Passed through a function call”
- So what’s actually being passed to you – i.e. what data is it that you’re being given by the other team: “A linked list of transaction IDs and some numerical data”
- Is that sensitive user data: “No”
- Singly or doubly linked: “That doesn’t matter”
- So what does matter? “Time. How long it takes to find the right transaction. That’s crucial.”
- Ok, so does resource usage, particularly memory, i.e. space complexity, matter: “No, the data itself isn’t that large, but the app’s on the phone and so is the data and we can’t use up a lot of memory on the phone. But we could always do the processing on our servers if we want”
We now have a different problem to when we started (“delete a transaction from a group of transactions”). We now know that the transactions are in a list, the list is made up of transaction IDs and some supporting numerical data, time complexity is critical but space complexity is not (i.e. speed is critical but memory usage isn’t), and we could, crucially, do the processing on our own servers where we have more memory if needed by sending data from the phone.
With this newly thought through information to hand, the approach for both the engineer and my self was simple. The engineer would programme the function to send data to our servers, and programme our server side software to process the deletion and send the data back. I would coordinate with the other team to ensure that the deletion feature would first check to see if an appropriate data connection to our servers existed before calling the function the engineer was programming to process the deletion.
Paper Based Algorithmic Problem Solving
This leads to a technique I’ve guided engineers towards, which I think sometime get’s forgotten with the progress made in git versioning helping to quickly go back to previous commits and modern day IDEs. Always solve the problem algorithmically first on paper.
Back in the old days of quants being quants and developers being developers, an investment bank I worked in specifically broke down the roles of the person creating the mathematical algorithm for trading (the quant), and the person implementing that algorithm (the developer), into two distinct roles.
Now, that doesn’t mean every organisation must have two distinct roles, one for designing the function and another for implementing it (although some often do). But it does mean that a lesson we can learn is that, to ensure all possible approaches have been looked at, developers should:
- Always spend time, talking through an approach to the problem (perhaps with another team member, requirements manager or agile coach)
- Use a small sample set of data to serve as a guide
- Step by step (no double conditionals!) and explaining each step, write a bit of pseudocode on paper (or whiteboard).