Subset Overview / Details
______________________________________________________________
Requirement Set: CMMI
Subset: Validation (VAL)
______________________________________________________________
Notes:
·
The contents of this web page were extracted from
the following document: Capability Maturity Model® Integration
(CMMISM), Version 1.1, Continuous Representation,
CMU/SEI-2002-TR-011, March 2002 (CMMI-SE/SW/IPPD/SS). Copyright 2002 by
Carnegie Mellon University. NO WARRANTY.
·
Ignore the identifiers
in square brackets that appear at the end of paragraphs.
·
The formatting may not
be the same as in the printed CMMI document. The web page is best viewed in
Internet Explorer.
·
In the CMMI, a subset is known as a "Process
Area (PA)" and a requirement is known as a "Practice". The
specific practices are referred to as SPs and the generic practices are
referred to as GPs.
·
This web page contains the text for SPs and GPs as
it appears in Chapter 7 of the CMMI document, in the section corresponding to
the process area named in the heading of this page. This web page does not
include the detailed description of the GPs that appears in a separate chapter
of the CMMI document; the detailed
description of the GPs is available in a separate web
page. (Note: Using the hyperlink provided here will open that web page in a
separate window.)
______________________________________________________________
Category: Engineering
Purpose
The purpose of Validation is to demonstrate that a
product or product component fulfills its intended use when placed in its
intended environment. [PA149]
Introductory Notes
Validation activities can be applied to all aspects of
the product in any of its intended environments, such as operation, training,
manufacturing, maintenance, and support services. The methods employed to
accomplish validation can be applied to work products as well as to the product
and product components. The work products (e.g., requirements, designs,
prototypes) should be selected on the basis of which are the best predictors of
how well the product and product component will satisfy user needs. [PA149.N105]
The validation environment should represent the intended
environment for the product and product components as well as represent the
intended environment suitable for validation activities with work products. [PA149.N106]
Validation demonstrates that the product, as provided,
will fulfill its intended use; whereas, verification addresses whether the work
product properly reflects the specified requirements. In other words,
verification ensures that “you built it right;” whereas, validation ensures
that “you built the right thing.” Validation activities use approaches similar
to verification (e.g., test, analysis, inspection, demonstration, or
simulation). Often, the end users are involved in the validation activities.
Both validation and verification activities often run concurrently and may use
portions of the same environment.
[PA149.N102]
Refer to the Verification process area for more information about
verification activities. [PA149.N102.R101]
Where possible, validation should be accomplished using
the product or product component operating in its intended environment. The
entire environment may be used or only part of it. However, validation issues
can be discovered early in the life of the project using work products. [PA149.N103]
When validation issues are identified, they are referred
to the processes associated with the Requirements Development, Technical
Solution, or Project Monitoring and Control process areas for resolution. [PA149.N104]
The specific practices of this process area build on each
other in the following way. The Select Products for Validation specific
practice enables the identification of the product or product component to be
validated and the methods to be used to perform the validation. The Establish
the Validation Environment specific practice enables the determination of the
environment that will be used to carry out the validation. The Establish
Validation Procedures and Criteria specific practice enables the development of
validation procedures and criteria that are aligned with the characteristics of
selected products, customer constraints on validation, methods, and the
validation environment. The Perform Validation specific practice enables the
performance of validation according to the methods, procedures, and criteria. [PA149.N107]
Refer to the Requirements Development process area for more information
about requirements validation. [PA149.R101]
Refer to the Technical Solution process area for more information about
transforming requirements into product specifications and for corrective action
when validation issues are identified that affect the product or
product-component design. [PA149.R102]
Refer to the Verification process area for more information about
verifying that the product or product component meets its requirements.
[PA149.R103]
Specific Goals
SG 1 Prepare
for Validation [PA149.IG101]
Preparation for validation is conducted.
SG 2 Validate
Product or Product Components
[PA149.IG102]
The product or product components are validated to ensure that they are suitable for use in their intended operating environment.
Generic Goals
GG 1 Achieve
Specific Goals [CL102.GL101]
The process supports and enables achievement of the specific goals of the process area by transforming identifiable input work products to produce identifiable output work products.
GG 2 Institutionalize
a Managed Process [CL103.GL101]
The process is institutionalized as a managed process.
GG 3 Institutionalize
a Defined Process [CL104.GL101]
The process is institutionalized as a defined process.
GG 4 Institutionalize
a Quantitatively Managed Process
[CL105.GL101]
The process is institutionalized as a quantitatively managed process.
GG 5 Institutionalize
an Optimizing Process [CL106.GL101]
The process is institutionalized as an optimizing process.
Practice-to-Goal Relationship Table
SG 1 Prepare for Validation
[PA149.IG101]
SP 1.1-1 Select Products for Validation
SP 1.2-2 Establish the Validation Environment
SP 1.3-3 Establish Validation Procedures and Criteria
SG 2 Validate Product or Product Components [PA149.IG102]
SP 2.1-1 Perform Validation
SP 2.2-1 Analyze Validation Results
GG 1 Achieve Specific Goals [CL102.GL101]
GP 1.1 Perform Base Practices
GG 2 Institutionalize a Managed Process [CL103.GL101]
GP 2.1 Establish an Organizational Policy
GP 2.2 Plan the Process
GP 2.3 Provide Resources
GP 2.4 Assign Responsibility
GP 2.5 Train People
GP 2.6 Manage Configurations
GP 2.7 Identify and Involve Relevant Stakeholders
GP 2.8 Monitor and Control the Process
GP 2.9 Objectively Evaluate Adherence
GP 2.10 Review Status with Higher Level Management
GG 3 Institutionalize a Defined Process [CL104.GL101]
GP 3.1 Establish a Defined Process
GP 3.2 Collect Improvement Information
GG 4 Institutionalize a Quantitatively Managed Process [CL105.GL101]
GP 4.1 Establish Quantitative Objectives for the Process
GP 4.2 Stabilize Subprocess Performance
GG 5 Institutionalize an Optimizing Process [CL106.GL101]
GP 5.1 Ensure Continuous Process Improvement
GP 5.2 Correct Root Causes of Problems
Specific Practices by Goal
SG 1 Prepare for Validation
Preparation for validation is conducted. [PA149.IG101]
Preparation activities include selecting products and
product components for validation and establishing and maintaining the
validation environment, procedures, and criteria. The items selected for
validation may include only the product or it may include appropriate levels of
the product components that are used to build the product. Any product or
product component may be subject to validation, including replacement,
maintenance, and training products, to name a few. [PA149.IG101.N101]
The environment required to validate the product or
product component is prepared. The environment may be purchased or may be
specified, designed, and built. The environments used for product integration
and verification may be considered in collaboration with the validation
environment to reduce cost and improve efficiency or productivity. [PA149.IG101.N102]
SP 1.1-1 Select Products for Validation
Select
products and product components to be validated and the validation methods that
will be used for each.
[PA149.IG101.SP101]
Products and product components are selected for
validation on the basis of their relationship to user needs. For each product
component, the scope of the validation (e.g., operational behavior,
maintenance, training, and user interface) should be determined. [PA149.IG101.SP101.N104]
The requirements and constraints for performing
validation are collected. Then, validation methods are selected based on their
ability to demonstrate that user needs are satisfied. The validation methods not
only define the technical approach to product validation, but also drive the
needs for the facilities, equipment, and environments. This may result in the
generation of lower level product-component requirements that are handled by
the requirements development processes. Derived requirements, such as interface
requirements to test sets and test equipment, may be generated. These
requirements are also passed to the requirements development processes to
ensure that the product or product components can be validated in an
environment that supports the methods.
[PA149.IG101.SP101.N101]
Validation methods should be selected early in the life
of the project so they are clearly understood and agreed to by the relevant
stakeholders.
[PA149.IG101.SP101.N102]
The validation methods address the development,
maintenance, support, and training for the product or product component as
appropriate.
[PA149.IG101.SP101.N103]
Typical Work Products
1. Lists of products and
product components selected for validation [PA149.IG101.SP101.W101]
2. Validation methods for each
product or product component [PA149.IG101.SP101.W102]
3. Requirements for performing
validation for each product or product component
[PA149.IG101.SP101.W103]
4. Validation constraints for each
product or product component [PA149.IG101.SP101.W104]
Subpractices
1. Identify the key principles, features, and
phases for product or product-component validation throughout the life of the
project. [PA149.IG101.SP101.SubP101]
2. Determine which categories of user needs
(operational, maintenance, training, or support) are to be validated. [PA149.IG101.SP101.SubP102]
The product or product component
must be maintainable and supportable in its intended operational environment.
This specific practice also addresses the actual maintenance, training, and
support services that may be delivered along with the product. [PA149.IG101.SP101.SubP102.N101]
An
example of evaluation of maintenance concepts in the operational environment is
a demonstration that maintenance tools are operating with the actual product. [PA149.IG101.SP101.SubP102.N102]
3. Select the product and product components to
be validated. [PA149.IG101.SP101.SubP105]
4. Select the evaluation methods for product or
product-component validation. [PA149.IG101.SP101.SubP103]
5. Review the validation selection,
constraints, and methods with relevant stakeholders.
[PA149.IG101.SP101.SubP104]
SP 1.2-2 Establish the Validation Environment
Establish
and maintain the environment needed to support validation. [PA149.IG101.SP102]
The requirements for the validation environment are
driven by the product or product components selected, by the type of the work
products (e.g., design, prototype, final version), and by the methods of
validation. These may yield requirements for the purchase or development of
equipment, software, or other resources. These requirements are provided to the
requirements development processes for development. The validation environment
may include the reuse of existing resources. In this case, arrangements for the
use of these resources must be made. Examples of the type of elements in a
validation environment include the following:
[PA149.IG101.SP102.N101]
· Test tools interfaced with the product being validated (e.g., scope, electronic devices, probes)
· Temporary embedded test software
· Recording tools for dump or further analysis and replay
· Simulated subsystems or components (by software, electronics, or mechanics)
· Simulated interfaced systems (e.g., a dummy warship for testing a naval radar)
· Real interfaced systems (e.g., aircraft for testing a radar with trajectory tracking facilities)
· Facilities and customer-supplied products
· The skilled people to operate or use all the above elements
· Dedicated computing or network test environment (e.g., pseudo-operational telecommunications-network testbed or facility with actual trunks, switches, and systems established for realistic integration and validation trials)
Early selection of the products or product components to
be validated, the work products to be used in the validation, and the
validation methods is needed to ensure that the validation environment will be
available when necessary.
[PA149.IG101.SP102.N102]
The validation environment should be carefully controlled
to provide for replication, analysis of results, and re-validation of problem
areas. [PA149.IG101.SP102.N103]
Typical Work Products
1. Validation environment [PA149.IG101.SP102.W101]
Subpractices
1. Identify validation environment
requirements. [PA149.IG101.SP102.SubP101]
2. Identify customer-supplied products. [PA149.IG101.SP102.SubP102]
3. Identify reuse items.
[PA149.IG101.SP102.SubP103]
4. Identify test equipment and tools. [PA149.IG101.SP102.SubP104]
5. Identify validation resources that are
available for reuse and modification. [PA149.IG101.SP102.SubP105]
6. Plan the availability of resources in
detail. [PA149.IG101.SP102.SubP106]
SP 1.3-3 Establish Validation Procedures and Criteria
Establish
and maintain procedures and criteria for validation. [PA149.IG101.SP103]
Validation procedures and criteria are defined to ensure
that the product or product component will fulfill its intended use when placed
in its intended environment. Acceptance test cases and procedures may meet the
need for validation procedures.
[PA149.IG101.SP103.N101]
The validation procedures and criteria include test and
evaluation of maintenance, training, and support services. [PA149.IG101.SP103.N102]
Examples of sources
for validation criteria include the following: [PA149.IG101.SP103.N103]
· Product and product-component requirements
· Standards
· Customer acceptance criteria
· Environmental performance
· Thresholds of performance deviation
Typical Work Products
1. Validation procedures [PA149.IG101.SP103.W101]
2. Validation criteria [PA149.IG101.SP103.W102]
3. Test and evaluation
procedures for maintenance, training, and support
[PA149.IG101.SP103.W103]
Subpractices
1. Review the product requirements to ensure that
issues affecting validation of the product or product component are identified
and resolved. [PA149.IG101.SP103.SubP101]
2. Document the environment, operational
scenario, procedures, inputs, outputs, and criteria for the validation of the
selected product or product component. [PA149.IG101.SP103.SubP102]
3. Assess the design as it matures in the
context of the validation environment to identify validation issues. [PA149.IG101.SP103.SubP103]
SG 2 Validate Product or Product Components
The product or product components are validated to
ensure that they are suitable for use in their intended operating environment. [PA149.IG102]
The validation methods, procedures, and criteria are used
to validate the selected products and product components and any associated
maintenance, training, and support services using the appropriate validation
environment.
[PA149.IG102.N102]
Perform
validation on the selected products and product components. [PA149.IG102.SP101]
To be acceptable to users, a product or product component
must perform as expected in its intended operational environment. [PA149.IG102.SP101.N101]
Validation activities are performed and the resulting
data are collected according to the established methods, procedures, and criteria. [PA149.IG102.SP101.N102]
The as-run validation procedures should be documented and
the deviations occurring during the execution should be noted, as appropriate. [PA149.IG102.SP101.N103]
(For users of the continuous representation, this is a capability
level 1 specific practice. Validation processes at capability level 1 or 2 may
not include procedures and criteria, which are created in the Establish
Validation Procedures and Criteria specific practice at capability level 3.
When there are no procedures or criteria established, use the methods
established by the Select Products for Validation specific practice to
accomplish capability level 1 performance.)
[PA149.IG102.SP101.N104]
Typical Work Products
1. Validation reports [PA149.IG102.SP101.W101]
2. Validation results [PA149.IG102.SP101.W102]
3. Validation cross-reference
matrix [PA149.IG102.SP101.W103]
4. As-run procedures log [PA149.IG102.SP101.W104]
5. Operational demonstrations [PA149.IG102.SP101.W105]
SP 2.2-1 Analyze Validation Results
Analyze
the results of the validation activities and identify issues. [PA149.IG102.SP102]
The data resulting from validation tests, inspections,
demonstrations, or evaluations are analyzed against the defined validation
criteria. Analysis reports indicate whether the needs were met; in the case of
deficiencies, these reports document the degree of success or failure and
categorize probable cause of failure. The collected test, inspection, or review
results are compared with established evaluation criteria to determine whether
to proceed or to address requirements or design issues in the requirements
development or technical solution processes.
[PA149.IG102.SP102.N101]
Analysis reports or as-run validation documentation may
also indicate that bad test results are due to a validation procedure problem
or a validation environment problem.
[PA149.IG102.SP102.N102]
Typical Work Products
1. Validation deficiency
reports [PA149.IG102.SP102.W101]
2. Validation issues [PA149.IG102.SP102.W102]
3. Procedure change request [PA149.IG102.SP102.W103]
Subpractices
1. Compare actual results to expected results. [PA149.IG102.SP102.SubP101]
2. Based on the established validation
criteria, identify products and product components that do not perform suitably
in their intended operating environments, or identify problems with the
methods, criteria, and/or environment. [PA149.IG102.SP102.SubP102]
3. Analyze the validation data for defects. [PA149.IG102.SP102.SubP103]
4. Record the results of the analysis and
identify issues. [PA149.IG102.SP102.SubP104]
5. Use validation results to compare actual
measurements and performance to intended use or operational need. [PA149.IG102.SP102.SubP105]
Generic Practices by Goal
(Note: The detailed description of the GPs is available in a separate web page. Using the hyperlink provided here will open that web page in a separate window. However, the GP elaborations pertinent to the process area of this web page are available below.)
GG 1 Achieve Specific Goals
The process supports and enables achievement of the specific goals of the process area by transforming identifiable input work products to produce identifiable output work products.
Perform
the base practices of the validation process to develop work products and
provide services to achieve the specific goals of the process area. [GP102]
GG 2 Institutionalize a Managed Process
The process is institutionalized as a managed process.
GP 2.1 Establish an Organizational Policy
Establish
and maintain an organizational policy for planning and performing the
validation process. [GP103]
Elaboration:
This policy establishes organizational expectations for
selecting products and product components for validation; for selecting
validation methods; and for establishing and maintaining validation procedures,
criteria, and environments that ensure the products and product components
satisfy user needs in their intended operating environment. [PA149.EL101]
Establish
and maintain the plan for performing the validation process. [GP104]
Elaboration:
Typically, this plan for performing the validation process
is included in (or referenced by) the project plan, which is described in the
Project Planning process area. [PA149.EL116]
Provide
adequate resources for performing the validation process, developing the work
products, and providing the services of the process. [GP105]
Elaboration:
Special facilities may be required for validating the
product or product components. When necessary, the facilities required for
validation are developed or purchased.
[PA149.EL111]
Examples of
other resources provided include the following tools: [PA149.EL103]
· Test management tools
· Test-case generators
· Test-coverage analyzers
· Simulators
· Load, stress, and performance tools
Assign
responsibility and authority for performing the process, developing the work
products, and providing the services of the validation process. [GP106]
Train
the people performing or supporting the validation process as needed. [GP107]
Elaboration:
Examples of
training topics include the following: [PA149.EL104]
· Application domain
· Validation principles, standards, and methods
· Intended-use environment
Place
designated work products of the validation process under appropriate levels of
configuration management. [GP109]
Elaboration:
Examples of
work products placed under configuration management include the following: [PA149.EL105]
· Lists of products and product components selected for validation
· Validation methods, procedures, and criteria
· Validation reports
GP 2.7 Identify and Involve Relevant Stakeholders
Identify
and involve the relevant stakeholders of the validation process as planned. [GP124]
Elaboration:
Select relevant stakeholders from customers, end users,
developers, producers, testers, suppliers, marketers, maintainers, disposal
personnel, and others who may be affected by, or may affect, the product as
well as the process. [PA149.EL113]
Examples of
activities for stakeholder involvement include the following: [PA149.EL114]
· Selecting the products and product components to be validated
· Establishing the validation methods, procedures, and criteria
· Reviewing results of product and product-component validation and resolving issues
· Resolving issues with the customers or end users
Issues with the customers or end users
are resolved particularly when there are significant deviations from their
baseline needs for the following:
[PA149.EL115]
· Waivers on the contract or agreement (what, when, and for which products, services, or manufactured products)
· Additional in-depth studies, trials, tests, or evaluations
· Possible changes in the contracts or agreements
GP 2.8 Monitor and Control the Process
Monitor
and control the validation process against the plan for performing the process
and take appropriate corrective action. [GP110]
Elaboration:
Examples of
measures used in monitoring and controlling include the following: [PA149.EL109]
· Number of validation activities completed (planned versus actual)
· Validation problem report trends (e.g., number written and number closed)
· Validation problem report aging (i.e., how long each problem report has been open)
GP 2.9 Objectively Evaluate Adherence
Objectively
evaluate adherence of the validation process against its process description,
standards, and procedures, and address noncompliance. [GP113]
Elaboration:
Examples of
activities reviewed include the following: [PA149.EL110]
· Selecting the products and product components to be validated
· Establishing and maintaining validation methods, procedures, and criteria
· Validating products or product components
Examples of
work products reviewed include the following: [PA149.EL112]
· Validation methods, procedures, and criteria
GP 2.10 Review Status with Higher Level Management
Review
the activities, status, and results of the validation process with higher level
management and resolve issues. [GP112]
GG 3 Institutionalize a Defined Process
The process is institutionalized as a defined process.
GP 3.1 Establish a Defined Process
Establish
and maintain the description of a defined validation process. [GP114]
GP 3.2 Collect Improvement Information
Collect
work products, measures, measurement results, and improvement information
derived from planning and performing the validation process to support the
future use and improvement of the organization’s processes and process assets. [GP117]
GG 4 Institutionalize a Quantitatively Managed Process
The process is institutionalized as a quantitatively managed process.
GP 4.1 Establish Quantitative Objectives for the Process
Establish
and maintain quantitative objectives for the validation process that address
quality and process performance based on customer needs and business
objectives. [GP118]
GP 4.2 Stabilize Subprocess Performance
Stabilize
the performance of one or more subprocesses to determine the ability of the
validation process to achieve the established quantitative quality and
process-performance objectives. [GP119]
GG 5 Institutionalize an Optimizing Process
The process is institutionalized as an optimizing process.
GP 5.1 Ensure Continuous Process Improvement
Ensure
continuous improvement of the validation process in fulfilling the relevant
business objectives of the organization.
[GP125]
GP 5.2 Correct Root Causes of Problems
Identify
and correct the root causes of defects and other problems in the validation
process. [GP121]