The QA Enviroment

The QA Enviroment

In Seven Development Environments:
http://andrewboland.blogspot.com/2008/05/seven-software-development-environments.html

I asserted that developers access to higher environments should be restricted. And above the QA, developers should have no access.Developers do not understand this and object. Since the objections are plentiful and compelling, a reasoned response is warranted. Both the objections and the responses vary by the environment. QA, UAT and Prod have similar but differing reasons given for developer access.Thus I will group these by environment and take on the QA ones first.

So the top three compelling reasons for access QA (Quality Assurance, also known as Test) fall into one of:

1. I need access to QA so I can install the program.
2. I need access to QA so I can fix the bugs they find.
3. I need access to QA so I can understand or diagnose the problems QA finds.

What I like about these is they are versions of "I need it to get the job done."

1. I need access to QA so I can install the program.

This is common and reveals a deeper problem or problems with the software development process.Either there is no installer or setup process for the software. Or the installer/setup is broken. Or the software requires prerequisites or extensive configuration.
Taking the last of these first, the need for prerequisites or configuration post install.The reply here is that this means the installer or the setup process is incomplete. It is fairly easy to have the installer program check for prerequisites. These days is also very easy to have the installer program include the prerequisites and install them if they are missing.Likewise, configuration post install means that there is some documentation missing. The QA team should be provided with setup instructions. If it is too complex for QA then it will be too complex for your customers.

2. I need access to QA so I can fix the bugs QA finds.

This sounds noble. And usually there is a follow on objection, "I need to fix this bug so QA can keep on testing."
But this starts a really bad habit. If the developer can change the release being tested by QA, what is it that QA is testing?
Say the version of the release that QA is testing is "1.0.23".
While that version is being tested the developers should be working on "1.0.24".
So what do we call this modified version?And then what is the QA supposed to do? Both during the time the developer is messing around in the QA environment and after?Is QA really testing that "1.0.23" release or is it now something else?
The best outcome is for the developer to spend a brief amount of time to understand the problem and then go work up a solution in the development environment.

3. I need access to QA so I can understand or diagnose the problems QA finds.

Here is where the developer needs to hold back and let QA do it's job. When QA finds a bug it should isolate the bug and provide reproduction steps.If the bug cannot be reproduced it is not a bug. Insisting on reproducible steps will help in diagnosing the problem.

permalink

Seven Software Development Environments

How many environments are enough?

Well one is definitely not enough. It is usually a sign of deeper problems that developers do not insist on separate environments for development and testing. One problem could be simple ignorance. In which case a few definitions are in order. So here are a typical seven.

1. Development
2. Build
3. Integration
4. QA
5. UAT
6. Staging
7. Production

They go by slightly different names from company to company but you should be able to recognize them by their function. It is also possible that several have been collapsed into one.

Development: This is the wild west. Developers get to do anything here and they will. As such anything proven here is suspect. Typically the name is shortened to "Dev".

Build: Hopefully source control is in use, if it is there is a chance automated builds or integration builds will occur on a machine that is not a development machine.

Integration : As the name suggests this is were everything comes together. A typical software solution now routinely involves multiple applications under development, a few legacy applications or libraries, some third party tools and servers. And to set it all up usually requires some configuration. This is the first enviroment were application installed. Team leads worth their salt will restrict access to this environment. Perhaps read only access will be allowed to team members but the lead should insist the problems be fixed in dev. Sometimes goes by the name "Int".

QA: Another name is "Test". Developers should have no roll in this environment. If the QA team is good they will not even let development leads have access. QA testors should be able to handle the deployment to QA without the developers. If they can't then you have bugs or lack of documentation.

UAT: The longer name is "User Acceptance Testing". Here is were the client gets a chance to provide feedback. Since QA was able to install on QA, either QA or if you are lucky enough to have a Configuration Management Team will handle this transfer. Note at this point developers are no longer needed to keep the application working.

Staging: Sometimes this go by the name "Preproduction" or PreProd. This environment is handy for reducing down time and handling security issues. It is also common for the owners of the site to refuse the developers of the site any access to this environment.

Production: Shortened to "Prod" and this is the final enviroment, the real environment.

permalink

12 Basic Software Development Practices

The gap between programming or writing code and developing software is growing.
This paradox is driven by the proliferation of easy to use tools, libraries, frameworks and OSes designed to ease the burden of software development.

Regardless of why, my experience has been that the typical software development shop is not executing on basic fundamental software development practices. A trend of late that I find even more alarming is quite often many of the programmers and managers are not even aware of or do not insist upon implementing them. It reached the point that I have enumerated 12:

12 Basic Software Development Practices

1. Source Control
2. Issue Tracking
3. Bug Tracking
4. Work Item Tracking
5. Environments
6. Integration Builds
7. Automated Builds
8. Unit Testing
9. Regression Testing
10. QA
11. Versioned Releases
12. Setups

So for a while my thinking was I can just enquire about these 12 before taking a new contract or engagement and all will be well. It is not surprising that I was always told "oh yeah we have all of that.." And it is also not surprising that expectations did not match reality. So I formulated follow on questions to determine the depth of practice.

Questions to determine depth of practice

1. Source Control

Where is the code kept?
Central location?
Individual machines?
Which version is authoritative?

2. Issue Tracking(Issues are a catch all term for everything that does not fall into the bug or work item buckets. They include but are not limited to 3rd party software issues, network issues, development machine issues.)

Is there a Central reporting tool / repository for issues?
Is there an established work flow?

3. Bug Tracking

Is there a Central reporting tool / repository for bugs?
Is there a bug workflow which includes states for:
Initial
Approved
Denied – Feature, works as designed
Denied – Not reproducible
Fixed
Verified
Reopened
Closed

4. Work Item Tracking

Is there a Central reporting tool / repository for new feature and enhancements?
Is there a work flow which includes states for:
Initial
Coded
Test
Failed
Accepted

5. Environments

Do separate environments exist for each of?
Development
Integration
Quality Assurance
User Acceptance Testing
Production

Is developer access to environments higher that Development limited?
Is developer access to environments higher than Integration denied?
Does QA or Change Management own the process of moving releases from one environment to the next?

6. Integration Builds

Does the code build from source control?
Does it build on a machine that is not a development machine?

7. Automated Builds (hourly/daily)

Is the code being built a regular basis?
Is this build done by an automated process?

8. Unit Testing

Do developers check in unit tests with their code?

9. Regression Testing

When code is built for release are all unit tests run?
Bug, Feature and QA tests?

10. QA

Is QA done by someone other than developers?
Does QA write automated or unit tests to demonstrate bugs?
Does QA write automated or unit tests to test common test cases for a feature?
Does QA write automated or unit tests for security and other required practices?

11. Versioned Releases

Is there a version number definition?
Are the products tied to the version number with each release?
Is the version number incremented with each release, both internal and external?
Does QA report bugs and test features according to version number?

12. Setups

Does each application have an install program?
If it does not have an install program are there instructions documented for manually installing the application?
After the install does the program run or is more intervention required?

Footnote. Microsoft Team System can provide them all.

permalink