The Four Faces of a Software Application

Ever feel that no two people see your Software Application the same way?


Unless it is notepad or you are not going to share your software with anyone you will notice four distinct collection of scenarios for your application; Installation, Configuration, Administration and User.

Installation is the easiest to overlook because every well maintained program makes installation a breeze. An ease which masks the difficulty of the problem. Today's applications are rapidly developed in days, sometimes just hours and for the talented perhaps in just minutes. The rich feature set that is immediately available comes from leveraging powerful and extensive frameworks. Getting those frameworks on the target machine amounts to an install within an install. Which brings up a hard unalterable rule of Software Development. "Create the Install program before you have anything to install." Less mature developers and nontechnical managers will not understand this focus on the install first. If you have good QA they will save a company from itself by insisting on an installers.

After an application is installed it will be configured to its environment. Initially an ever growing amount of configuration will live in the installer. Time will reveal that this bloats the installer and turns testing the installer into a nightmarish complex exercise. Eventually, separating the installation from configuration is required. Where those configuration components live then becomes a delicate trade off. If you can rely upon your users, you may be able to leverage a large amount of the System OS configuration components. If not, you will need to wrap them and expose them in your administrative interfaces.

Skipping over Administrative for a moment, once the application is installed and configured it should run without any care and feeding. Thus all of the things related to the Install, Configuration and Administration are merely noise to the end user. The user interface should focuses only on how to get work done on a day to day basis.

There should be a separate interface for Administration which is now defined as those exception or unusual cases of interacting with the program. Troubleshooting or diagnosing failures is also included here. The Administrative interfaces should anticipate that there will be failures and provide some means of working within the app to report them at the very least.

How to think of QA

Supppose you had a Friend.

You know, pretend you are back in school taking some really hard courses with difficult assessments. Now suppose all your friend did was check your homework before you turned it in.

How would you feel about that? How would it change things?

It is odd that all Software Developers do not feel this way about QA. But you can tell the Software Developers who have reached maturity because they understand QA is their best friend. And over time they have figured out the homework check dynamic and it changes the way the operate.

Clearly it is next to worthless getting your homework checked right before the bell rings. If your friend finds anything you will only have time to change the most simple stuff, spelling, plus or minus sign, grammer, elimination of extraneous solutions. If you have something fundamental wrong you may need to change everything.

The first step towards QA acceptance is when Software Developers start proactively getting their code to QA early. Some even start to pester QA to test their stuff first. QA loves this. Just like your friend who checked you Algebra homework in highschool, everyone likes to be needed.

The next phase follows naturally, Software Developers start getting their work to QA often. As Software Developers mature from a Feature focus to a Bug focus on their work they realize, it is easier to get Feature implemented Bug free in isolation. As such they want each individual Feature tested as soon as it is available. They also soon encounter Daisy Chain Bugs and Regression Bugs.

Daisy Chain Bugs, Regression Bugs, Requirement Bugs and more exotic Bugs takes us into the realm of Software Development Management. Software Development Managers that have reached maturity understand how critical good QA is to sustained success in software development. The key their is the sustained. It is possible to get lucky and implement a bugless feature. The difficulty arises when you want to add another and add another. The final maturity revelation that is reached is that:

Good QA can save you from bad programming, but no amount of good programming will save you without QA.

UAT Environment

User Acceptance Testing, UAT for short, also goes by the name Demo or Demonstration.

If you have a somewhat typical set of environments setup to support software development, they will correspond roughly to the following:

Development (DEV)
Build
Integration (INT)
QA/Test
UAT
PreProduction
Production (PROD)

How these are used also varies slightly from one company to the next but some key concerns are addresses at each level.

Build ensures the code is in Version Control and always builds.

Integration ensures the code under development and third party code, software and legacy systems all still work together.

QA or Test validates the ever increasing feature development and bug fixes.

So it may seem like UAT is overkill. But it is precisely because the previous (lower) environments have checked out and passed on all these details that allows UAT to provide the first chance for the Customer to provide their feedback or approval. The system is in a state where it could be deployed , all of the bugs and issues have been resolved.

But it is precisely because the Customer is providing feedback here that Software Developers will be most insistent on gaining access to the system.

One reason is familiar, they need to fix something. The answer to this is the same as it was on QA, fixes should flow throught the process to keep the version integrity. (Although QA should have already found the bug so if this is a new bug, some investigation should be done to determine why QA missed it.)

Another reason is the Customer is changing their mind. Here Acceptance is called for, not only will the Customer change their wishes, they may never stop. Sometimes they flip flop between several implementations. Sometimes there is more than one customer and they cannot agree. In this case you will need to find a way to make the different options configurable. This will need to be documented in a Use Case and follow the process of every other enhancement.

But the best reason for keeping Developers off of UAT is to see "Can this solution live without the constant care and feeding by Developers?" Even better "Can mere mortal users actually use this system?"

permalink

The QA Enviroment

The QA Enviroment

In Seven Development Environments:
http://andrewboland.blogspot.com/2008/05/seven-software-development-environments.html

I asserted that developers access to higher environments should be restricted. And above the QA, developers should have no access.Developers do not understand this and object. Since the objections are plentiful and compelling, a reasoned response is warranted. Both the objections and the responses vary by the environment. QA, UAT and Prod have similar but differing reasons given for developer access.Thus I will group these by environment and take on the QA ones first.

So the top three compelling reasons for access QA (Quality Assurance, also known as Test) fall into one of:

1. I need access to QA so I can install the program.
2. I need access to QA so I can fix the bugs they find.
3. I need access to QA so I can understand or diagnose the problems QA finds.

What I like about these is they are versions of "I need it to get the job done."

1. I need access to QA so I can install the program.

This is common and reveals a deeper problem or problems with the software development process.Either there is no installer or setup process for the software. Or the installer/setup is broken. Or the software requires prerequisites or extensive configuration.
Taking the last of these first, the need for prerequisites or configuration post install.The reply here is that this means the installer or the setup process is incomplete. It is fairly easy to have the installer program check for prerequisites. These days is also very easy to have the installer program include the prerequisites and install them if they are missing.Likewise, configuration post install means that there is some documentation missing. The QA team should be provided with setup instructions. If it is too complex for QA then it will be too complex for your customers.

2. I need access to QA so I can fix the bugs QA finds.

This sounds noble. And usually there is a follow on objection, "I need to fix this bug so QA can keep on testing."
But this starts a really bad habit. If the developer can change the release being tested by QA, what is it that QA is testing?
Say the version of the release that QA is testing is "1.0.23".
While that version is being tested the developers should be working on "1.0.24".
So what do we call this modified version?And then what is the QA supposed to do? Both during the time the developer is messing around in the QA environment and after?Is QA really testing that "1.0.23" release or is it now something else?
The best outcome is for the developer to spend a brief amount of time to understand the problem and then go work up a solution in the development environment.

3. I need access to QA so I can understand or diagnose the problems QA finds.

Here is where the developer needs to hold back and let QA do it's job. When QA finds a bug it should isolate the bug and provide reproduction steps.If the bug cannot be reproduced it is not a bug. Insisting on reproducible steps will help in diagnosing the problem.

permalink

Seven Software Development Environments

How many environments are enough?

Well one is definitely not enough. It is usually a sign of deeper problems that developers do not insist on separate environments for development and testing. One problem could be simple ignorance. In which case a few definitions are in order. So here are a typical seven.

1. Development
2. Build
3. Integration
4. QA
5. UAT
6. Staging
7. Production

They go by slightly different names from company to company but you should be able to recognize them by their function. It is also possible that several have been collapsed into one.

Development: This is the wild west. Developers get to do anything here and they will. As such anything proven here is suspect. Typically the name is shortened to "Dev".

Build: Hopefully source control is in use, if it is there is a chance automated builds or integration builds will occur on a machine that is not a development machine.

Integration : As the name suggests this is were everything comes together. A typical software solution now routinely involves multiple applications under development, a few legacy applications or libraries, some third party tools and servers. And to set it all up usually requires some configuration. This is the first enviroment were application installed. Team leads worth their salt will restrict access to this environment. Perhaps read only access will be allowed to team members but the lead should insist the problems be fixed in dev. Sometimes goes by the name "Int".

QA: Another name is "Test". Developers should have no roll in this environment. If the QA team is good they will not even let development leads have access. QA testors should be able to handle the deployment to QA without the developers. If they can't then you have bugs or lack of documentation.

UAT: The longer name is "User Acceptance Testing". Here is were the client gets a chance to provide feedback. Since QA was able to install on QA, either QA or if you are lucky enough to have a Configuration Management Team will handle this transfer. Note at this point developers are no longer needed to keep the application working.

Staging: Sometimes this go by the name "Preproduction" or PreProd. This environment is handy for reducing down time and handling security issues. It is also common for the owners of the site to refuse the developers of the site any access to this environment.

Production: Shortened to "Prod" and this is the final enviroment, the real environment.

permalink

12 Basic Software Development Practices

The gap between programming or writing code and developing software is growing.
This paradox is driven by the proliferation of easy to use tools, libraries, frameworks and OSes designed to ease the burden of software development.

Regardless of why, my experience has been that the typical software development shop is not executing on basic fundamental software development practices. A trend of late that I find even more alarming is quite often many of the programmers and managers are not even aware of or do not insist upon implementing them. It reached the point that I have enumerated 12:

12 Basic Software Development Practices

1. Source Control
2. Issue Tracking
3. Bug Tracking
4. Work Item Tracking
5. Environments
6. Integration Builds
7. Automated Builds
8. Unit Testing
9. Regression Testing
10. QA
11. Versioned Releases
12. Setups

So for a while my thinking was I can just enquire about these 12 before taking a new contract or engagement and all will be well. It is not surprising that I was always told "oh yeah we have all of that.." And it is also not surprising that expectations did not match reality. So I formulated follow on questions to determine the depth of practice.

Questions to determine depth of practice

1. Source Control

Where is the code kept?
Central location?
Individual machines?
Which version is authoritative?

2. Issue Tracking(Issues are a catch all term for everything that does not fall into the bug or work item buckets. They include but are not limited to 3rd party software issues, network issues, development machine issues.)

Is there a Central reporting tool / repository for issues?
Is there an established work flow?

3. Bug Tracking

Is there a Central reporting tool / repository for bugs?
Is there a bug workflow which includes states for:
Initial
Approved
Denied – Feature, works as designed
Denied – Not reproducible
Fixed
Verified
Reopened
Closed

4. Work Item Tracking

Is there a Central reporting tool / repository for new feature and enhancements?
Is there a work flow which includes states for:
Initial
Coded
Test
Failed
Accepted

5. Environments

Do separate environments exist for each of?
Development
Integration
Quality Assurance
User Acceptance Testing
Production

Is developer access to environments higher that Development limited?
Is developer access to environments higher than Integration denied?
Does QA or Change Management own the process of moving releases from one environment to the next?

6. Integration Builds

Does the code build from source control?
Does it build on a machine that is not a development machine?

7. Automated Builds (hourly/daily)

Is the code being built a regular basis?
Is this build done by an automated process?

8. Unit Testing

Do developers check in unit tests with their code?

9. Regression Testing

When code is built for release are all unit tests run?
Bug, Feature and QA tests?

10. QA

Is QA done by someone other than developers?
Does QA write automated or unit tests to demonstrate bugs?
Does QA write automated or unit tests to test common test cases for a feature?
Does QA write automated or unit tests for security and other required practices?

11. Versioned Releases

Is there a version number definition?
Are the products tied to the version number with each release?
Is the version number incremented with each release, both internal and external?
Does QA report bugs and test features according to version number?

12. Setups

Does each application have an install program?
If it does not have an install program are there instructions documented for manually installing the application?
After the install does the program run or is more intervention required?

Footnote. Microsoft Team System can provide them all.

permalink