January 2002 • Table
of Contents
Emerging Sectors
Measuring Quality In the Department of Defense
Events of Sept. 11th reinforce the finance and accounting
division's transition to a customer focused, strategy based organization
by Michael W. Dugan
A quote from Joseph M. Juran in the June 2001 issue of
Quality Progress suggests the next steps in the evolution of quality will
be into the "enormous service industries" that were previously "largely
immune" to the quality movement: education, healthcare and government.
In reality, there has been a quiet transformation in the delivery of government
services since the fall of the Soviet Union.
The collapse of government sponsored economies gave theoretical
impetus to the idea that government services are best delivered by private
industry. For the Department of Defense (DoD), that has translated into
a call for revitalizing and transforming its establishment and infrastructure.
The thought is that functions not directly linked to the
war fighting mission of the DoD should be outsourced or done in partnership
with private firms. The other change is to the paperless environment of
electronic commerce. For instance, payments to citizens and businesses
are faster and cheaper when electronic funds transfers are the norm.
Nonetheless, criticism of financial management in the Pentagon
remains. In July 2000, Tom Bloom, our director, saw the need for the Defense
Finance and Accounting Service (DFAS) to move to a customer focused, strategy
based organization, structuring our efforts along business and product
lines. At that time, I was a senior line manager with a staff of more
than 6,000 engaged in supporting Army financial management. One aspect
of the DFAS Business Evolution, as we have named it, was the establishment
of a quality and performance assessment group, which I volunteered to
lead.
Organization background
DFAS is the accounting arm of the DoD. It was established
in 1991 to improve financial management and reduce cost for the department.
The deputy secretary of defense directed the consolidation
of the finance and accounting operations of the various military departments
and defense agencies into a single business entity that charges fees for
the services provided. This helps the department identify the full cost
of finance and accounting activities and focuses our efforts to improve
the service.
As our men and women in uniform and our civilians face
the challenges of the war on terror, it is our responsibility and duty
to make sure we support them in every way we can. DFAS pays all military
and civilian employees with few exceptions. We pay all contractors and
vendors providing supplies, equipment and services to DoD.
The volume of payments and the associated accounting requirements
are staggering. Every workday DFAS disburses an average of $1 billion.
DFAS pays over 1 million invoices each month to thousands of companies
and firms, not to mention disbursing 2 million paychecks. Those statistics
do not include the several hundred thousand monthly intergovernmental
payments.
DFAS must keep track of each payment and record it against
any one of more than 230 appropriations. Approximately 19,000 military,
civilian and contractor personnel make DFAS one of the largest accounting
organizations in the world.
During World War II, a relatively unknown senator, Harry
Truman, conducted hearings on the quality of supplies and equipment reaching
our Armed Forces. These hearings proved instrumental in the war effort.
Quality in all support functions will be just as important in the new
war on terror. But how do you measure quality in an accounting organization?
Forming the team
Quality improvement specialists Tim Clark and Mike Serowik
and I were given the task of establishing the quality organization and
program. Both Clark, author of Success Through Quality, and Serowik,
a certified Baldrige examiner, were longtime quality professionals. I,
on the other hand, had no formal training in quality theory. In previous
positions, I "allowed" my staff to form quality circles and go to total
quality management training, but I was only cursorily familiar with the
precepts.
We faced a daunting challenge to develop a quality program
that would support the vision and help achieve the goals of the agency.
Accusations of erroneous payments and inaccurate bookkeeping spoke of
a difficult and complex financial management challenge. We needed to gauge
the level of quality in our various processes and implement a program
to improve the accounting and payment processes.
The concept that captured my imagination was the cost of
poor quality. This was the key to translating quality theory into concrete
business practices. We pointed out to line managers that if DFAS disburses
$1 billion a day and has an accuracy rate of 98%, then $20 million is
disbursed each day inaccurately. Put in those terms, prior performance
standards were no longer acceptable.
By all estimates, rework was a very high cost in our accounting
business. Thousands of individual transactions may process through several
systems daily. If a transaction is correct, it flows through for less
than a dollar in total cost. Backlogs result when financial transactions
are inaccurate or do not process automatically. If a transaction rejects,
goes into suspense or recycles, the cost grows exponentially because human
intervention is required to correct it. *
Collecting and comparing data
In 1997 DFAS was placed under a performance contract with
the Defense Management Council--a kind of board of directors for the DoD
headed by the deputy secretary of defense. This contract met the goals
of the Government Performance and Results Act in establishing a measurement
of DFAS based on results. Each one of our outputs is defined in the time
required, accuracy demanded and unit cost target.
Our
performance management information system (PMIS) was the basis for Serowik's
work in pulling data for us to review. Table
1 (p. 91) is a monthly PMIS report. Notice we have one month of data
by site for our major sites (networks composed of many smaller sites)
in which each metric is shown in green, yellow or red depending on whether
we met the goal, just missed it or failed to meet the goal that month.
As we reviewed the data, the question of comparability
arose. For instance, if one network had eight greens, two yellows, and
three reds, was it better or worse than a sister network with seven greens,
five yellows and one red? How should we compare quality between business
lines, for example, accounting vs. bill paying? What about the trends:
Was DFAS getting better?
Looking at the data over a two-year period, we noticed
things got better at the end of each fiscal year. But at the beginning
of the next fiscal year there was an immediate and dramatic worsening
of the metrics.
We discussed the reasons behind this, namely the invariable
delay in establishing new appropriations and implementing new legal and
regulatory requirements. That indicated a special cause situation that
camouflaged the common cause variations. Still, I was unable to discern
whether DFAS was improving its service.
Are we getting better?
Clark and Serowik began discussing the potential of using
a capability process model (Cpk) to form the basis for an index
that might give us a picture of what was happening over time. We wanted
a dashboard to answer: Are we getting better?
The Cpk model had been used by a variety of
companies to test the quality of their production processes but to our
knowledge had never been used in the manner Clark and Serowik were proposing.
We were a bit nervous about it--not only about whether it would work but
also whether it was intellectually sound.
Serowik
devised a method to insert the data into the Cpk model. Figure
1 provides the Quality Report over an 18-month period. This index is
the combination of four indexes that we developed: accounting, personnel
pay, commercial pay and travel pay.
Each index has five to six metrics that have been factored
into a Cpk formula to achieve individual scores, which are
averaged to give us the composite index and associated indexes. The advantages
of this Quality Report are substantial:
1. It provides trend data for both managers and customers.
We can see and judge the effect of changes in processes or environment.
2. It can be linked to Six Sigma. The metrics are
linked to defects, and defects are linked to the cost of poor quality.
The good news is we approximate a 99.6% average on our index.
3. It allows us to compare our different business
lines. We can compare the quality of a payment made with the quality of
the accounting for that payment.
Looking ahead
The Quality Report is just the beginning. We are working
to instill a passion for quality in everything we do. We have a Six Sigma
scholarship for some of our topflight personnel; we are prototyping the
application of the Baldrige criteria for organizational performance at
one of our sites; we facilitated individual improvement projects focusing
on the return on investment; and we established a DFAS quality Web site
with standard improvement models and resource materials.
The slogan "good enough for government work" originated
when the government had strict standards for surveying. It meant a product
was precise and of high quality.
We hope to bring that same meaning back to the quality
of DoD accounting. Our soldiers, sailors, airmen and marines, our civilians,
our nation and our taxpayers deserve nothing less.
MICHAEL W. DUGAN is a director of quality and
performance assessment at the defense finance and accounting service of
the Department of Defense in Indianapolis, IN. He earned a master's degree
in business from Syracuse University. Dugan is an ASQ member.
If you would like to comment on this article, please post
your remarks on the Quality
Progress Discussion Board, or e-mail them to editor@asq.org.
