Arguably the Automated Builds and Integration Builds (2 of the 12 Basic Software Development Practices) form the hinge upon which software development succeeds or fails.
By having an integration build which is automated you are in the position to close the door on some Level 2 Ignorance. To recap Level 2 Ignorance is when you do not know something and at the same time you are unaware of it. (For more see Five Levels Of Ignorance).
The automated integrated build informs you immediately when certain bugs have been introduced. As such you are no longer Level 2 Ignorant of those bugs because you know of them. Granted you may not be able to fix them and so you are at Level 1 Ignorance, but this is an improvement.
Either you caused them, in which case you will learn something you are doing wrong, which remedies something that would otherwise be Level 3 Ignorance. The same is true if someone else caused the bug, they are not aware of the bug and they are not aware they are doing something wrong which is causing bugs.
Absent this build process, your team would be at Level 3 Ignorance of things that cause bugs. A very dangerous place to be. If you remain there your project will fail.
Showing posts with label Software Development Practices. Show all posts
Showing posts with label Software Development Practices. Show all posts
Monday, August 18, 2008
Thursday, July 17, 2008
Backing Down A Bug
So a bug is found in production.
In spite of your following all the best practices.
You had separate environments for Development, Integration, QA (testing), UAT and Production.
You have been doing versioned releases and following your documented change control procedures.
You had a QA staff that did it's best to test for and find all bugs.
You have three levels of help desk or technical support.
And in spite of this a bug made it all the way out to production and it must be fixed. And the business side of the house is flipping out and the pressure is on. Fix the bug now!!
How you handle this will determine if your company grows and gets a tiny bit better. Or should you fail, you the company will plant the seeds of disruptive behavior that will haunt you for the rest of your tenure.
You must avoid the temptation to storm in and take things over. You need to Back This Bug Down From Production with the care you would back a jumper from a ledge.
First make sure that the help desk or technical support was engaged. If not, engage them. Use this as learning opportunity for the support staff to learn how to mitigate or work around bugs in production. Should you find a work around, you must insist to the suits that it is over. A work around has been found. The work around is in place. Both the bug and issue (the issue is how the bug made it to production) are being researched and will fixed in later release. End of Story. Do not be stampeded on this.
Now chances are good that a work around is not immediately found. In this case, the backing down starts. QA must take a role here. At this point QA has tested the application and their job is to get the bug reproduced in the QA environment. Again, you must resist the desire to jump in and take over. QA has an important role in defining the bug. If QA can reproduce the bug in the QA environment, you should then be able to use this information to reproduce the bug in the Development environment. When you have fixed bug, QA will now be in a position to test the fix. This would not be possible if you had ripped control away from Tech Support or QA.
But this is not the end of the story. There is still the open issue of how the bug survived your processes. Inevitably something is missing in terms of testing or the processes were not followed. Fixing this issue will prevent future bugs.
In spite of your following all the best practices.
You had separate environments for Development, Integration, QA (testing), UAT and Production.
You have been doing versioned releases and following your documented change control procedures.
You had a QA staff that did it's best to test for and find all bugs.
You have three levels of help desk or technical support.
And in spite of this a bug made it all the way out to production and it must be fixed. And the business side of the house is flipping out and the pressure is on. Fix the bug now!!
How you handle this will determine if your company grows and gets a tiny bit better. Or should you fail, you the company will plant the seeds of disruptive behavior that will haunt you for the rest of your tenure.
You must avoid the temptation to storm in and take things over. You need to Back This Bug Down From Production with the care you would back a jumper from a ledge.
First make sure that the help desk or technical support was engaged. If not, engage them. Use this as learning opportunity for the support staff to learn how to mitigate or work around bugs in production. Should you find a work around, you must insist to the suits that it is over. A work around has been found. The work around is in place. Both the bug and issue (the issue is how the bug made it to production) are being researched and will fixed in later release. End of Story. Do not be stampeded on this.
Now chances are good that a work around is not immediately found. In this case, the backing down starts. QA must take a role here. At this point QA has tested the application and their job is to get the bug reproduced in the QA environment. Again, you must resist the desire to jump in and take over. QA has an important role in defining the bug. If QA can reproduce the bug in the QA environment, you should then be able to use this information to reproduce the bug in the Development environment. When you have fixed bug, QA will now be in a position to test the fix. This would not be possible if you had ripped control away from Tech Support or QA.
But this is not the end of the story. There is still the open issue of how the bug survived your processes. Inevitably something is missing in terms of testing or the processes were not followed. Fixing this issue will prevent future bugs.
Thursday, May 22, 2008
How to think of QA
Supppose you had a Friend.
You know, pretend you are back in school taking some really hard courses with difficult assessments. Now suppose all your friend did was check your homework before you turned it in.
How would you feel about that? How would it change things?
It is odd that all Software Developers do not feel this way about QA. But you can tell the Software Developers who have reached maturity because they understand QA is their best friend. And over time they have figured out the homework check dynamic and it changes the way the operate.
Clearly it is next to worthless getting your homework checked right before the bell rings. If your friend finds anything you will only have time to change the most simple stuff, spelling, plus or minus sign, grammer, elimination of extraneous solutions. If you have something fundamental wrong you may need to change everything.
The first step towards QA acceptance is when Software Developers start proactively getting their code to QA early. Some even start to pester QA to test their stuff first. QA loves this. Just like your friend who checked you Algebra homework in highschool, everyone likes to be needed.
The next phase follows naturally, Software Developers start getting their work to QA often. As Software Developers mature from a Feature focus to a Bug focus on their work they realize, it is easier to get Feature implemented Bug free in isolation. As such they want each individual Feature tested as soon as it is available. They also soon encounter Daisy Chain Bugs and Regression Bugs.
Daisy Chain Bugs, Regression Bugs, Requirement Bugs and more exotic Bugs takes us into the realm of Software Development Management. Software Development Managers that have reached maturity understand how critical good QA is to sustained success in software development. The key their is the sustained. It is possible to get lucky and implement a bugless feature. The difficulty arises when you want to add another and add another. The final maturity revelation that is reached is that:
Good QA can save you from bad programming, but no amount of good programming will save you without QA.
You know, pretend you are back in school taking some really hard courses with difficult assessments. Now suppose all your friend did was check your homework before you turned it in.
How would you feel about that? How would it change things?
It is odd that all Software Developers do not feel this way about QA. But you can tell the Software Developers who have reached maturity because they understand QA is their best friend. And over time they have figured out the homework check dynamic and it changes the way the operate.
Clearly it is next to worthless getting your homework checked right before the bell rings. If your friend finds anything you will only have time to change the most simple stuff, spelling, plus or minus sign, grammer, elimination of extraneous solutions. If you have something fundamental wrong you may need to change everything.
The first step towards QA acceptance is when Software Developers start proactively getting their code to QA early. Some even start to pester QA to test their stuff first. QA loves this. Just like your friend who checked you Algebra homework in highschool, everyone likes to be needed.
The next phase follows naturally, Software Developers start getting their work to QA often. As Software Developers mature from a Feature focus to a Bug focus on their work they realize, it is easier to get Feature implemented Bug free in isolation. As such they want each individual Feature tested as soon as it is available. They also soon encounter Daisy Chain Bugs and Regression Bugs.
Daisy Chain Bugs, Regression Bugs, Requirement Bugs and more exotic Bugs takes us into the realm of Software Development Management. Software Development Managers that have reached maturity understand how critical good QA is to sustained success in software development. The key their is the sustained. It is possible to get lucky and implement a bugless feature. The difficulty arises when you want to add another and add another. The final maturity revelation that is reached is that:
Good QA can save you from bad programming, but no amount of good programming will save you without QA.
Monday, May 12, 2008
Separation of Concerns
Software developers can test do all their own testing.
But it will not work out.
Software developers can also plan and envision the product.
But this overtime will go astray.
What drives sustained success in software development is Separation of Concerns.
Which sounds like Specialization but is not.
The best example is Accountants and Auditors. An accountant can by changing the rules, change the profitability of the company (think depreciation). So over time accountants have showing a profit, and ever growing one at that, as their main concern. Auditors on the other hand are only concerned with how the accounting processes and practices are done. Auditors have no concern for the whether a profit is shown but rather that the profit and loss number was arrived at in an acceptable and consistent way. When your accountant becomes your auditor as well, or when you auditor accepts at face value all that your accountant does you can expect an Enron type outcome.
Software Development is different because it has three concerns that need to be separated; Specification, Implementation and Validation. It is better to think in these terms rather than the roles that they are usually mapped onto; Business Analyst, Software Developer and Tester/QA. Business Analyst implies some concern for the business as whole. Test is close although it emphasis is on bugs and Quality Assurance takes the focus on Verification.
Specifying what is to be created must be completely separate from Implementation, which is how it will be created. Time and time again the pattern repeats of software developers pushing back and not wanting to implement something because it is hard. Or it does not lead to using the technologies or practices that are currently fashionable. And by fashionable, I mean well rewarded by the market place. If you hold both concerns of Specification and Implementation at the same time the temptation to make a comprise on Specification to benefit Implementation is immense.
When it comes to holding Implementation and Validation concerns at the same time the problem is even worse. Testing may reveal a problem. Knowledge of what it takes to fix it hinders the desire to immediately log it and raise it as a concern. Here again compromising Validation to benefit Implementation is the easiest route.
To keep this in the proper length I will state the next four and return to them later with separate articles:
Specification is about serving the customer.
Testing is about isolating the problem.
Development is about solving the problem.
The process provides the framework. The process is the automated assembly line of software development.
permalink
But it will not work out.
Software developers can also plan and envision the product.
But this overtime will go astray.
What drives sustained success in software development is Separation of Concerns.
Which sounds like Specialization but is not.
The best example is Accountants and Auditors. An accountant can by changing the rules, change the profitability of the company (think depreciation). So over time accountants have showing a profit, and ever growing one at that, as their main concern. Auditors on the other hand are only concerned with how the accounting processes and practices are done. Auditors have no concern for the whether a profit is shown but rather that the profit and loss number was arrived at in an acceptable and consistent way. When your accountant becomes your auditor as well, or when you auditor accepts at face value all that your accountant does you can expect an Enron type outcome.
Software Development is different because it has three concerns that need to be separated; Specification, Implementation and Validation. It is better to think in these terms rather than the roles that they are usually mapped onto; Business Analyst, Software Developer and Tester/QA. Business Analyst implies some concern for the business as whole. Test is close although it emphasis is on bugs and Quality Assurance takes the focus on Verification.
Specifying what is to be created must be completely separate from Implementation, which is how it will be created. Time and time again the pattern repeats of software developers pushing back and not wanting to implement something because it is hard. Or it does not lead to using the technologies or practices that are currently fashionable. And by fashionable, I mean well rewarded by the market place. If you hold both concerns of Specification and Implementation at the same time the temptation to make a comprise on Specification to benefit Implementation is immense.
When it comes to holding Implementation and Validation concerns at the same time the problem is even worse. Testing may reveal a problem. Knowledge of what it takes to fix it hinders the desire to immediately log it and raise it as a concern. Here again compromising Validation to benefit Implementation is the easiest route.
To keep this in the proper length I will state the next four and return to them later with separate articles:
Specification is about serving the customer.
Testing is about isolating the problem.
Development is about solving the problem.
The process provides the framework. The process is the automated assembly line of software development.
permalink
Saturday, May 3, 2008
12 Basic Software Development Practices
The gap between programming or writing code and developing software is growing.
This paradox is driven by the proliferation of easy to use tools, libraries, frameworks and OSes designed to ease the burden of software development.
Regardless of why, my experience has been that the typical software development shop is not executing on basic fundamental software development practices. A trend of late that I find even more alarming is quite often many of the programmers and managers are not even aware of or do not insist upon implementing them. It reached the point that I have enumerated 12:
12 Basic Software Development Practices
1. Source Control
2. Issue Tracking
3. Bug Tracking
4. Work Item Tracking
5. Environments
6. Integration Builds
7. Automated Builds
8. Unit Testing
9. Regression Testing
10. QA
11. Versioned Releases
12. Setups
So for a while my thinking was I can just enquire about these 12 before taking a new contract or engagement and all will be well. It is not surprising that I was always told "oh yeah we have all of that.." And it is also not surprising that expectations did not match reality. So I formulated follow on questions to determine the depth of practice.
Questions to determine depth of practice
1. Source Control
Where is the code kept?
Central location?
Individual machines?
Which version is authoritative?
2. Issue Tracking(Issues are a catch all term for everything that does not fall into the bug or work item buckets. They include but are not limited to 3rd party software issues, network issues, development machine issues.)
Is there a Central reporting tool / repository for issues?
Is there an established work flow?
3. Bug Tracking
Is there a Central reporting tool / repository for bugs?
Is there a bug workflow which includes states for:
Initial
Approved
Denied – Feature, works as designed
Denied – Not reproducible
Fixed
Verified
Reopened
Closed
4. Work Item Tracking
Is there a Central reporting tool / repository for new feature and enhancements?
Is there a work flow which includes states for:
Initial
Coded
Test
Failed
Accepted
5. Environments
Do separate environments exist for each of?
Development
Integration
Quality Assurance
User Acceptance Testing
Production
Is developer access to environments higher that Development limited?
Is developer access to environments higher than Integration denied?
Does QA or Change Management own the process of moving releases from one environment to the next?
6. Integration Builds
Does the code build from source control?
Does it build on a machine that is not a development machine?
7. Automated Builds (hourly/daily)
Is the code being built a regular basis?
Is this build done by an automated process?
8. Unit Testing
Do developers check in unit tests with their code?
9. Regression Testing
When code is built for release are all unit tests run?
Bug, Feature and QA tests?
10. QA
Is QA done by someone other than developers?
Does QA write automated or unit tests to demonstrate bugs?
Does QA write automated or unit tests to test common test cases for a feature?
Does QA write automated or unit tests for security and other required practices?
11. Versioned Releases
Is there a version number definition?
Are the products tied to the version number with each release?
Is the version number incremented with each release, both internal and external?
Does QA report bugs and test features according to version number?
12. Setups
Does each application have an install program?
If it does not have an install program are there instructions documented for manually installing the application?
After the install does the program run or is more intervention required?
Footnote. Microsoft Team System can provide them all.
permalink
This paradox is driven by the proliferation of easy to use tools, libraries, frameworks and OSes designed to ease the burden of software development.
Regardless of why, my experience has been that the typical software development shop is not executing on basic fundamental software development practices. A trend of late that I find even more alarming is quite often many of the programmers and managers are not even aware of or do not insist upon implementing them. It reached the point that I have enumerated 12:
12 Basic Software Development Practices
1. Source Control
2. Issue Tracking
3. Bug Tracking
4. Work Item Tracking
5. Environments
6. Integration Builds
7. Automated Builds
8. Unit Testing
9. Regression Testing
10. QA
11. Versioned Releases
12. Setups
So for a while my thinking was I can just enquire about these 12 before taking a new contract or engagement and all will be well. It is not surprising that I was always told "oh yeah we have all of that.." And it is also not surprising that expectations did not match reality. So I formulated follow on questions to determine the depth of practice.
Questions to determine depth of practice
1. Source Control
Where is the code kept?
Central location?
Individual machines?
Which version is authoritative?
2. Issue Tracking(Issues are a catch all term for everything that does not fall into the bug or work item buckets. They include but are not limited to 3rd party software issues, network issues, development machine issues.)
Is there a Central reporting tool / repository for issues?
Is there an established work flow?
3. Bug Tracking
Is there a Central reporting tool / repository for bugs?
Is there a bug workflow which includes states for:
Initial
Approved
Denied – Feature, works as designed
Denied – Not reproducible
Fixed
Verified
Reopened
Closed
4. Work Item Tracking
Is there a Central reporting tool / repository for new feature and enhancements?
Is there a work flow which includes states for:
Initial
Coded
Test
Failed
Accepted
5. Environments
Do separate environments exist for each of?
Development
Integration
Quality Assurance
User Acceptance Testing
Production
Is developer access to environments higher that Development limited?
Is developer access to environments higher than Integration denied?
Does QA or Change Management own the process of moving releases from one environment to the next?
6. Integration Builds
Does the code build from source control?
Does it build on a machine that is not a development machine?
7. Automated Builds (hourly/daily)
Is the code being built a regular basis?
Is this build done by an automated process?
8. Unit Testing
Do developers check in unit tests with their code?
9. Regression Testing
When code is built for release are all unit tests run?
Bug, Feature and QA tests?
10. QA
Is QA done by someone other than developers?
Does QA write automated or unit tests to demonstrate bugs?
Does QA write automated or unit tests to test common test cases for a feature?
Does QA write automated or unit tests for security and other required practices?
11. Versioned Releases
Is there a version number definition?
Are the products tied to the version number with each release?
Is the version number incremented with each release, both internal and external?
Does QA report bugs and test features according to version number?
12. Setups
Does each application have an install program?
If it does not have an install program are there instructions documented for manually installing the application?
After the install does the program run or is more intervention required?
Footnote. Microsoft Team System can provide them all.
permalink
Subscribe to:
Posts (Atom)