UAT Environment

User Acceptance Testing, UAT for short, also goes by the name Demo or Demonstration.

If you have a somewhat typical set of environments setup to support software development, they will correspond roughly to the following:

Development (DEV)
Build
Integration (INT)
QA/Test
UAT
PreProduction
Production (PROD)

How these are used also varies slightly from one company to the next but some key concerns are addresses at each level.

Build ensures the code is in Version Control and always builds.

Integration ensures the code under development and third party code, software and legacy systems all still work together.

QA or Test validates the ever increasing feature development and bug fixes.

So it may seem like UAT is overkill. But it is precisely because the previous (lower) environments have checked out and passed on all these details that allows UAT to provide the first chance for the Customer to provide their feedback or approval. The system is in a state where it could be deployed , all of the bugs and issues have been resolved.

But it is precisely because the Customer is providing feedback here that Software Developers will be most insistent on gaining access to the system.

One reason is familiar, they need to fix something. The answer to this is the same as it was on QA, fixes should flow throught the process to keep the version integrity. (Although QA should have already found the bug so if this is a new bug, some investigation should be done to determine why QA missed it.)

Another reason is the Customer is changing their mind. Here Acceptance is called for, not only will the Customer change their wishes, they may never stop. Sometimes they flip flop between several implementations. Sometimes there is more than one customer and they cannot agree. In this case you will need to find a way to make the different options configurable. This will need to be documented in a Use Case and follow the process of every other enhancement.

But the best reason for keeping Developers off of UAT is to see "Can this solution live without the constant care and feeding by Developers?" Even better "Can mere mortal users actually use this system?"

permalink

Seven Software Development Environments

How many environments are enough?

Well one is definitely not enough. It is usually a sign of deeper problems that developers do not insist on separate environments for development and testing. One problem could be simple ignorance. In which case a few definitions are in order. So here are a typical seven.

1. Development
2. Build
3. Integration
4. QA
5. UAT
6. Staging
7. Production

They go by slightly different names from company to company but you should be able to recognize them by their function. It is also possible that several have been collapsed into one.

Development: This is the wild west. Developers get to do anything here and they will. As such anything proven here is suspect. Typically the name is shortened to "Dev".

Build: Hopefully source control is in use, if it is there is a chance automated builds or integration builds will occur on a machine that is not a development machine.

Integration : As the name suggests this is were everything comes together. A typical software solution now routinely involves multiple applications under development, a few legacy applications or libraries, some third party tools and servers. And to set it all up usually requires some configuration. This is the first enviroment were application installed. Team leads worth their salt will restrict access to this environment. Perhaps read only access will be allowed to team members but the lead should insist the problems be fixed in dev. Sometimes goes by the name "Int".

QA: Another name is "Test". Developers should have no roll in this environment. If the QA team is good they will not even let development leads have access. QA testors should be able to handle the deployment to QA without the developers. If they can't then you have bugs or lack of documentation.

UAT: The longer name is "User Acceptance Testing". Here is were the client gets a chance to provide feedback. Since QA was able to install on QA, either QA or if you are lucky enough to have a Configuration Management Team will handle this transfer. Note at this point developers are no longer needed to keep the application working.

Staging: Sometimes this go by the name "Preproduction" or PreProd. This environment is handy for reducing down time and handling security issues. It is also common for the owners of the site to refuse the developers of the site any access to this environment.

Production: Shortened to "Prod" and this is the final enviroment, the real environment.

permalink

12 Basic Software Development Practices

The gap between programming or writing code and developing software is growing.
This paradox is driven by the proliferation of easy to use tools, libraries, frameworks and OSes designed to ease the burden of software development.

Regardless of why, my experience has been that the typical software development shop is not executing on basic fundamental software development practices. A trend of late that I find even more alarming is quite often many of the programmers and managers are not even aware of or do not insist upon implementing them. It reached the point that I have enumerated 12:

12 Basic Software Development Practices

1. Source Control
2. Issue Tracking
3. Bug Tracking
4. Work Item Tracking
5. Environments
6. Integration Builds
7. Automated Builds
8. Unit Testing
9. Regression Testing
10. QA
11. Versioned Releases
12. Setups

So for a while my thinking was I can just enquire about these 12 before taking a new contract or engagement and all will be well. It is not surprising that I was always told "oh yeah we have all of that.." And it is also not surprising that expectations did not match reality. So I formulated follow on questions to determine the depth of practice.

Questions to determine depth of practice

1. Source Control

Where is the code kept?
Central location?
Individual machines?
Which version is authoritative?

2. Issue Tracking(Issues are a catch all term for everything that does not fall into the bug or work item buckets. They include but are not limited to 3rd party software issues, network issues, development machine issues.)

Is there a Central reporting tool / repository for issues?
Is there an established work flow?

3. Bug Tracking

Is there a Central reporting tool / repository for bugs?
Is there a bug workflow which includes states for:
Initial
Approved
Denied – Feature, works as designed
Denied – Not reproducible
Fixed
Verified
Reopened
Closed

4. Work Item Tracking

Is there a Central reporting tool / repository for new feature and enhancements?
Is there a work flow which includes states for:
Initial
Coded
Test
Failed
Accepted

5. Environments

Do separate environments exist for each of?
Development
Integration
Quality Assurance
User Acceptance Testing
Production

Is developer access to environments higher that Development limited?
Is developer access to environments higher than Integration denied?
Does QA or Change Management own the process of moving releases from one environment to the next?

6. Integration Builds

Does the code build from source control?
Does it build on a machine that is not a development machine?

7. Automated Builds (hourly/daily)

Is the code being built a regular basis?
Is this build done by an automated process?

8. Unit Testing

Do developers check in unit tests with their code?

9. Regression Testing

When code is built for release are all unit tests run?
Bug, Feature and QA tests?

10. QA

Is QA done by someone other than developers?
Does QA write automated or unit tests to demonstrate bugs?
Does QA write automated or unit tests to test common test cases for a feature?
Does QA write automated or unit tests for security and other required practices?

11. Versioned Releases

Is there a version number definition?
Are the products tied to the version number with each release?
Is the version number incremented with each release, both internal and external?
Does QA report bugs and test features according to version number?

12. Setups

Does each application have an install program?
If it does not have an install program are there instructions documented for manually installing the application?
After the install does the program run or is more intervention required?

Footnote. Microsoft Team System can provide them all.

permalink