During testing we use requirements to guide our activities. We
check to see that the program is doing what it is supposed to be doing. But how
often do we stop and wonder how much more it is doing? Why does the install
process take longer and longer every week? Why is build 210 three times larger
than build 209? Why does issue 3197 reference a component of the product I've
never even heard of??BR>
Checking requirements doesn't answer these
questions. Instead we need a way to work backwards from what we have, to what we
thought we should have. This is the realm of asset management, and our part of
this is asset reconciliation. Here, I provide a simple version that helps remove
some of the unknown variables, or the wildcards of product quality, and also
prevents unnecessary product bloat.?BR>?BR>There is no magic in this
technique--it is a simple four-step process: 1) locate all the assets, or parts,
of the application package; 2) identify what each part could be used for, or
what its immediate purpose is; 3) determine if this part is actually needed by
the application; and finally 4) ensure these parts are being
tested.?BR>?BR>Remember, the goal is not merely to catalog assets. The goal is
to track down and eliminate unneeded bloat and make sure all remaining assets
have been properly tested. The accounting need only be as rigorous as your
project necessitates.?BR>
Step 1: Locate the assets
Your first
step is to locate all of the parts of the application. Note that you are
interested primarily in the final version--what comes out of a ZIP file or an
installation package. You want to identify initially all of the directories and
files that the product comprises. This is a simple starting point that consists
of a few dir commands at a shell prompt.
But don't stop there. Often,
additional assets are hiding inside compressed, or archive files (some of the
usual suspects end with ZIP, JAR, TAR, and CAB). You should also understand the
manner in which UI resources are stored. Something as simple as a single DLL
file can contain numerous dialogs, hundreds of images, strings, and other
nondescript data items. Be sure to track down all the shared libraries the
install program put in or the directories. And never
forget about the final tidbits stored in the registry or placed in a file in the
directory.
Now that you have a big list of assets you are ready
to move on, but don't be afraid to come back to this list and add more items or
further detail the contents of a file you later learn is an archive, such as
ZIP. Also don't be afraid to move on before exploring every compressed file or
configuration item. This can be treated as an iterative process: with each pass,
you can further explore one area of the assets, or go one detail level deeper in
the analysis.
Step 2: Identify their purpose
For each item
you have in your list you need to determine what it is. Is this an image file?
Is this a library component? Is this a help reference?
Focus on
identifying the general purpose of the item, rather than obtaining a detailed
description of its use. You are interested, at this stage, in understanding what
each of the items could be used for, where it could be used, or how it could be
referenced. Don't yet concern yourself with whether the item is actually
necessary for the program to operate--that is the next step.
If there
are some items on your list that you really can't identify, be sure to ask other
members of the development team. Unidentified items will prove troublesome to
further analyze. If nobody is able to identify what the item is, then you know
you are dealing with a suspect asset.
Step 3: Determine if they are
needed
Once you've identified each of the items, you then need to
determine if they are actually needed by the product. A large part of this step
is already completed simply by knowing the identified purpose of the items. It
is nonetheless an important and longer step, with a distinct purpose from the
previous one. This is when you determine the concrete use for each item on your
list, determining when, where, and how each is used.
You want to know
where each dialog appears in the program. Images may be used in the default
interface, or perhaps they are selectable by means of a user-selectable skin.
Check each shared library to see whether it is needed, or whether it is a
remnant from a previous version. Review each configuration setting in the
registry, or configuration files, to ensure that it is applicable to the product
(configuration bloat can be just as much of a problem as file bloat). Be
watchful of items that look like they might be used but simply aren't (for
example an extra JPG in the directory which is actually never
referenced).
The information you collect needs to be enough to
determine when, where, and how the item is used. When ensures that you've
considered all the variables that may impact the use of the item. It is quite
possible that certain items, such as language resources or device drivers, are
used only in certain configurations. Where and how will give you the insight
you need to test the item. They can also further identify those cases where an
item is used, but only in another component that is not used (perhaps an icon
that is displayed only in a dialog that is no longer
referenced).
Beyond merely determining that the item is used by the
program, you need to determine if it is supposed to be used by the program. You
should be able to state which requirement describes, or requires, each of the
items on your list. Often, requirements are quite strict in what they state,
other times they are open. Sometimes a directory filled with selectable images
will have no specifics, but other times the requirement may specifically state
only three skins that can be chosen. Your work here may be of interest to the
requirements management team as they strive to improve their system: don't leave
them out of the loop!
At this point, there are three situations that
will result in change requests for the product. First, if an item is both not
used by the product and not described by the requirements, you will likely want
to create a deletion request for this item. Second, if an item is used by the
program but not included in the requirements, you will need to speak with the
team to determine whether the item should be added to the requirements, or
removed from the program. Third, if an item is described by the requirements but
not used by the program, in this case you need to determine whether the program
is to be adjusted to use the item, or whether the item and the requirement for
the item both need to be dropped.
Step 4: Ensure they are
tested
This last step seems obvious at first, but it plays an important
role when you have fairly open requirements. When a requirement allows many
items to be added to a product, normal testing of the requirement may only
result in one of these items being tested. This step is where you formalize the
inclusion of all of these items into the testing process.
A common area
in this regard is that of examples, or samples. Typically, the requirements
don't specify every example that will be shipped with the program, and the
examples end up being a collection of those created for testing requirements,
debugging the application, or showing off the GUI. Here it is important that you
really want to make sure all the examples demonstrate correctly what they are
supposed to, and that each example meets the quality standards of your
product.
Conclusion
It is clear that the first few
iterations of this process will likely be the most difficult; it is a daunting
task to go from tracking zero assets to the thousands, or even tens of thousands
of assets in a reasonably sized project. Don't be aggressive in your first
attempts; instead, concentrate on tailoring the technique to your needs and
creating the tools you will need to track this work. With each iteration, you
can improve your tools and the level of detail you track. It should be clear
that a simple editor with one EXE, a few DLLs, and some examples is not going to
require the same effort or tools as a modern real-time strategy game with some
40,000 image files. In the latter case, you are advised to try to integrate this
technique into your standard asset management process (late reconciliation may
be too costly for your project).
The simplest form of this process does
not require any other changes in development or testing. You can implement this
over several weeks, or months, and not worry that it might interfere with your
existing process. Once you've started, the confidence you have in your product
should improve. You can feel more secure that there aren't any hidden aspects of
the program left untested, or that some spurious library isn't going to cause
the program to crash. Ultimately, this gives one more little boost to product
quality.
(Note: This process does not replace asset management, rather
it is best seen as the quality control process for asset management. Use as much
information as possible from your asset management system--it may reduce the
workload of this activity.)
About the Author
The author has
been involved with software development for more than ten years, ranging from
software design, to implementation, to management. Currently mort-ora-y is
trying to improve the quality of a product that transforms some 36,000
development files into a deployed package of around 16,000 files. This process
is a formalized accumulation of the techniques mort-ora-y has discovered so far.
|
|
