This article describes a technique called the "Technology Artifact Review" which can be used to determine whether a Year 2000 compliance project or set of projects will complete as required.
Introduction
Professional liability, officers liability, and business interruption
insurance underwriting requires careful underwriting practices,
if satisfactory returns are to be obtained by the insurer. In
ever increasing numbers, policy holders provide or rely on information
technology in the routine conduct of their business. In two earlier
articles published National Underwriter, Michael Gantt
(November 11, 1996) and Vito Peraino (February 17, 1997) did an
excellent job of defining the problem. To summarize the effect
of the "Millennium Bug" in terms of risk: The litigation
fallout is estimated to be as high as $1 trillion in the United
States alone. This risk is shared by technology intensive companies
conducting their own business (such as insurance companies and
service providers), software and consulting organizations who
fail to properly protect their clients, and hardware vendors who
already have sold products that will fail. I might add that the
U.S. is probably the best positioned, so the litigation risk is
even higher for Western Europe, the Far East or Australia. The
underwriting assessment of this risk is at best haphazard and
superficial for most insurers. At worst it is absent.
This paper presents a cost effective method of assessing information
technology (IT) risk, based on evaluation techniques that have
been used in the field since 1987. These were originally developed
to provide a benchmark for the improvement of internal IT organizations,
many of them from within the Property and Casualty insurance industry.
It assesses technology risk from all sources, including the Millennium
Bug. While no assurance of absolute reliability is possible, these
methods have proven to be flexible and effective tools in a wide
range of industries and types of organizations. Together they
are called the Technology Artifact Review (TAR).
The Technology Artifact Review- Defined
The TAR is a highly structured, time compressed, assessment of
the strengths and weaknesses of an IT intensive organization as
an indication of the future performance of the organization in
support of defined objectives. Employing both off-site elements
and on-site inspections, the product of such a review is an objective
assessment of the future risk of IT related failure of Key Business
Functions (KBFs). The KBFs of the organization may be either the
provision of IT related services or IT intensive business products
(such as claims administration, medical care, or financial services).
Since severe failure of an IT project, such as Year 2000, always
prevents accomplishment of the business functions, professional
liability exposures and business interruption risks are high.
To the extent that Directors and Officers of the affected organization
are held to be liable for such failures, they are likewise exposed.
Methodology
Since the purpose of a TAR is to quickly and economically determine
the technological risk environment of the client, without unduly
burdening them, the pre assessment activity is emphasized. Although
the Pre-Assessment Package may appear to be large, it is
important to note that the documents requested are usually already
available in well run IT environments. If the prospective client
must spend an undue amount of time to assemble them (more than
3 days), it is legitimate to question whether this is an assumable
risk.
With the information from the pre-assessment activity in-hand,
an experienced IT professional should be able to make a preliminary
assessment of the IT environment that likely exists, and identify
areas for on-site investigation. At that time the targeted interviews
are requested and scheduled. It is very important that the reviewer
establish control over the client with respect to what he/she
will have access to when on-site.
The on-site portion of the assessment is essential because IT
organizations are aware of the importance and appearance of key
documents. At worst, these can be falsified. The norm is that
sound methodologies exist on paper, but not in practice. The IT
organization that actually follows the guidelines that it publishesis
the exception to the rule. Therefore on-site inspection and interviews,
under the control of the reviewer represent the only viable safeguard
against intentional or unintentional misrepresentation by the
prospective client. Depending on the size of the organization
and the number of verification points identified by the reviewer,
the on site phase can be accomplished in 1 to 2 business days.
The final phase is the analysis of the information obtained and
preparation of the briefing documents for the underwriter. While
the primary purpose of these analyses is to enable the underwriter
to make an informed decision about the case, there is an important
secondary purpose for those cases actually written. The secondary
purpose is to inform the new insured as to steps that need to
be taken to reduce the risk of IT failures in mission critical
applications.
Components of the IT Risk Assessment
Built on the time honored principle of trust with verification,
the TAR has three components:
The pre-assessment package- forms and requests for information
to be provided to the Chief Operating Officer of an insured or
prospective insured.
The on-site component- interviews and inspections of documents
that were produced as a consequence of past information technology
activity.
The underwriter's briefing package- an objective assessment
of the projected risk of future IT activity of the insured or
prospective insured.
Each component of the assessment can be configured to the business
of the prospective insured and to your own company's needs. They
must contain certain elements. For instance, the pre-assessment
package must elicit from the client enough details to enable the
reviewer to identify potential trouble spots, and to identify
the most likely places to look to disprove or verify the problematic
area. The on-site component must be comprehensive enough to enable
verification of IT capabilities based on historical performance
and structured interviews of key people in the organization, some
of them outside the IT department. The underwriter's briefing
package must concisely, in non-technical language, describe not
only the strengths and weaknesses found, but also differentiate
between the important and the cosmetic.
The Pre-Assessment Package as completed by the client contains
ten information items. A Client Organization Chart, down
to the level of department heads and in the case of the IT organization,
down to the team leader level, enables the reviewer to identify
potential interviewees and correlate the other documents in the
package. The Hardware/Systems Software and the Applications
SoftwareSummaries must include each major component,
the computer language used, how supported, number of lines of
computer instructions it contains and when each component was
placed into service. The IT Strategic Plan and Year
2000 Compliance Summaries are important planning documents
that enable an initial assessment of the relative sophistication
of the client with respect to technology management and give early
indications of areas requiring closer attention. The IT Systems
Development Methodology and IT Quality MethodologySummaries are paper indicators of the reliability of the
development processes, but the reviewer should note that the majority
of paper methodologies are not followed in actual practice and
that relatively new ones (in use less than 2 years) have not yet
had the chance to stabilize. The IT Project History, IT
Maintenance History, and IT Voluntary Staff Turnover History
should contain enough quantitative and descriptive information
to enable the reviewer to get a "feel" for the work
load and staff issues within the IT organization. Any areas found
to be factually wanting or absent by the reviewer are follow-up
items in the on-site stage of the review.
The On-Site Component is absolutely essential to the accuracy
of a TAR. The basic principle, which in 25 years of commercial
data processing I have yet to see fail, is that all well run (meaning
low risk) IT environments spin off objectively verifiable artifacts
as a byproduct of their development and support activities. These
artifacts are reliable because they are, with the exception of
some government contracts, not the end product and therefore tend
not to be "doctored". Even status reports, which are
often exercises in creative writing, eventually become reliable
as deadlines near. Interviews, direct observations, and document
reviews are the main tools of this component. Primary Document
Reviews are the inspections of the technology plans, methods
documents and historical data which were provided by the client,
in summary form, during the previous step. It is unreasonable
to request the detail documents to be provided in the pre- assessment
stage, due to their size and complexity. Therefore an on-site
inspection is required. Artifact Inspections generally
include selected project planning, reporting and tracking documents,
status reports, specifications, test results, and systems support
request paper files. Targeted Interviews, both within and
outside the IT organization, are used to verify historical performance
and determine if systems users are able to effectively work with
systems development personnel. It is reasonable to assume that
interviewees may be coached prior to the interview, but an experienced
reviewer can generally detect when this has occurred and determine
the facts.
The Underwriter's Briefing should contain at least 4 sections
and, for accepted risks, be amended to include a 5th section.
The Organizational Assessment is a narrative of those characteristics
that pertain to the reliable use of technology. Typically these
characteristics are evenly distributed between executive management,
departmental management, and the IT department itself. The Methodology
Assessment is an analysis of the completeness and maturity
of the methods used to develop and support technology. Mature
methods, consistently used, mean low risk of a Year 2000 project
failure. The Performance Assessment is predicated on the
premise that the past is a good predictor of the future. This
component of the briefing package summarizes historical facts
from previous IT projects that may be predictive of future performance.
The Risk Analysis is a synthesis of the previous three sections
into a strengths/weaknesses snapshot of the client. It translates
the entire review into quantitative and qualitative measures of
risk. The 5th component should be added only after the risk is
accepted. The Remediation Recommendations are specific
changes to the IT project environment that if made will significantly
reduce Year 2000 and other IT related risk.
Due to space limitations, details on the contents of the three
components have been omitted. As a service to National Underwriter
readers, more extensive information on the components are available
from Systems Managment Resources, Inc.. Send E-Mail to SMR, Inc.
Staff Requirements
The manpower requirements to support a review are small, in that
a single reviewer can and should handle the entire case. A prospective
client's time investment is from 2 to 5 man-days to assemble documents
and participate in interviews. The reviewer will require 3.5 to
5.0 man days to take a case through the entire process, assuming
that the framework for such reviews is already developed.
However, only a experienced manager with a broad IT background
is capable of performing such a review. A combination of people
skills and technical skills is essential to organize and execute
these assessments in a way that is non-threatening to the prospective
client.
Fortunately, many insurance companies have such people already
in their employ. Our recommendation is that mid to upper level
IT managers be utilized to perform these reviews, on a part time
basis. Many IT managers possess these skills and also desire the
opportunity to contribute more directly to the profitability of
their company. It is also very career broadening to perform such
reviews; and because of this, such assignments may prove to be
highly sought after within the IT management staff. It is important
that their other responsibilities be adjusted to allow their participation.
Should IT management personnel not be available to perform these
studies, it is possible to "farm out" the studies to
a third party. Should this route be taken, it is important to
verify the credentials of the specific reviewers assigned. This
is not a job for junior level people, and not many firms have
a track record in these assessments. The point is that, if the
3rd party route is taken, expect to spend some time identifying
the right resource to use.
Pitfalls of the Information Technology Risk Assessment
First you will insure fewer risks. Prospective insureds are often
reluctant to expose their internal operations to outside scrutiny,
particularly those with problematic situations. Expect to lose
some applicants.
Second you will be using a scarce resource within your own company,
e.g. the experienced business person with an IT background and
good client skills. Expect some adjustment discomfort as this
resource is made available.
Third these reviews are at their core, subjective in nature. While
very grounded in verifiable data, it is often a "judgment
call" as to the prognosis of a given IT situation. No doubt
some bad risks will be accepted, if enough cases are evaluated.
There is also a possibility of multiple losses due to the use
of a common IT services provider for a solution to a common problem,
such as year 2000 compliance.
Conclusion
This paper has presented a structured method for conducting a
Technology Artifact Review, which is both effective and economical.
While it is important to follow a pre-defined process and use
appropriately experienced reviewers, the demands placed on an
insurer to implement a TAR program are manageable.
Though certain drawbacks exist with TAR's, the end result should
be that the book of business written by the insurer will be profitable,
and the risk of multiple catastrophic loss greatly mitigated.
Reflecting the shift to information based products and services,
these types of reviews will become as essential and commonplace
as casualty risk assessments and risk management programs are
today.
About the Author
Ed Praytor, CDP CCP began his insurance information technology
career in 1973, as a systems engineer for Electronic Data Systems
Corporation. Leaving EDS in 1979, he subsequently led the software
support and professional services organizations for Insurance
Systems of America (ISA) and Advanced Technology Systems (ADTEC).
In 1985 he started Systems Management Resources, Inc.(SMR), a
highly specialized management consulting group based in Atlanta.
Since 1985, SMR has concentrated on the diagnosis and correction
of the organizational and methodology related causes of IT quality
and performance problems. In 1987, SMR introduced the information
technology SnapshotTM, the organizational assessment and improvement
methodology on which the Technology Artifact Review described
in this paper is based. Since 1987 he and his associates have
performed numerous SnapshotTM reviews, both inside and outside
the insurance industry.
Certified by the Institute for Certification of Computer Professionals
and the American Arbitration Association, he also arbitrates IT
disputes. He has published several feature articles, authored
a monthly column on IT performance issues and is currently finishing
a book on this topic which will be published in 1998.
