Back

Perform Quality Management on all newly developed or modified systems.


CONTROL ID
01100
CONTROL TYPE
Testing
CLASSIFICATION
Detective

SUPPORTING AND SUPPORTED CONTROLS



This Control directly supports the implied Control(s):
  • Initiate the System Development Life Cycle development phase or System Development Life Cycle build phase., CC ID: 06267

This Control has the following implementation support Control(s):
  • Evaluate system development projects for compliance with the system requirements specifications., CC ID: 06903
  • Establish, implement, and maintain a system testing policy., CC ID: 01102
  • Establish, implement, and maintain system testing procedures., CC ID: 11744
  • Test quality control procedures for proper implementation., CC ID: 06610


SELECTED AUTHORITY DOCUMENTS COMPLIED WITH



  • An independent party (e.g. the quality assurance function, the TRM function or the technology audit team), which is not involved in the project development, should conduct a quality assurance review of major technology-related functions if necessary. This review is to ensure compliance with the proj… (4.2.3, Hong Kong Monetary Authority: TM-G-1: General Principles for Technology Risk Management, V.1 – 24.06.03)
  • O67: The organization shall establish development and modification procedures for assuring validity of the implementation. O90.2(3): The organization should conduct verification tests on programs, system design specifications, and other contractor-created information to ensure functional requirement… (O67, O90.2(3), FISC Security Guidelines on Computer Systems for Banking and Related Financial Institutions, 7th Edition)
  • Financial institutions should have a methodology in place for testing and approval of ICT systems prior to their first use. This methodology should consider the criticality of business processes and assets. The testing should ensure that new ICT systems perform as intended. They should also use test… (3.6.2 70, Final Report EBA Guidelines on ICT and security risk management)
  • The measures thus derived are reviewed regularly during the project and reassessed in case of changes to the assessment criteria. (C, I, A) (1.2.3 Additional requirements for high protection needs Bullet 1, Information Security Assessment, Version 5.1)
  • Testing should involve not only the end users, but the workflows, connectivity to other systems, and security. When testing is complete, checkpoints should be used to decide on the readiness to go-live or launch in production. (§ 3.3 (Testing and Go-live), IIA Global Technology Audit Guide (GTAG) 12: Auditing IT Projects)
  • Quality Assurance of the system development methodology should include confirming that security controls (e.g., policies, methods, procedures, devices or programmed mechanisms intended to protect the confidentiality, integrity or availability of information) agreed during the information risk assess… (CF.17.03.02c, The Standard of Good Practice for Information Security)
  • The systems development lifecycle should cover testing the desktop application. (CF.17.01.02-4, The Standard of Good Practice for Information Security)
  • Tests should involve testing the complete system environment (e.g., end-to-end testing or compatibility testing) to identify any conflicts or dependencies with other systems. (CF.18.04.06, The Standard of Good Practice for Information Security)
  • Quality Assurance of the system development methodology should include confirming that security controls (e.g., policies, methods, procedures, devices or programmed mechanisms intended to protect the confidentiality, integrity or availability of information) agreed during the information risk assess… (CF.17.03.02c, The Standard of Good Practice for Information Security, 2013)
  • Tests should involve testing the complete system environment (e.g., end-to-end testing or compatibility testing) to identify any conflicts or dependencies with other systems. (CF.18.04.06, The Standard of Good Practice for Information Security, 2013)
  • Tools that are used to verify different material goods at the same time shall not interfere with the verification of any of the products. (§ 4.5.4.3.5, ISO 12931:2012, Performance Criteria for Authentication Solutions Used to Combat Counterfeiting of Material Goods, First Edition)
  • § 7.3.5: The organization shall verify that the design and development outputs have met the design and development input requirements. The organization shall keep records of the verification results and any actions that are needed. § 7.3.6: Before delivery or implementation of the product, the org… (§ 7.3.5, § 7.3.6, ISO 13485:2003 Medical devices -- Quality management systems -- Requirements for regulatory purposes, 2003)
  • The organization shall isolate the part of the system that is causing non-compliance to perform validation, as appropriate to the organizational objectives or agreement terms. (§ 6.4.8.3(b)(4), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • An independent evaluator should test the security functions of the system to ensure they operate as specified. The evaluator should either retest a subset of the tests in the test documentation or rerun every test for verification. The evaluator should also come up with other tests to run. (§ 18.4, ISO 15408-3 Common Criteria for Information Technology Security Evaluation Part 3, 2008)
  • Testing should be developed to ensure all security functions are covered by a test and the procedures include any prerequisites, the steps to conduct the tests, and the expected results. Each test should be linked to at least one security function and all tests combined should cover all the security… (§ 11.8.2, § 12.9.2, § 13.9.2, ISO 18045 Common Methodology for Information Technology Security Evaluation Part 3, 2005)
  • verification activities are conducted to ensure that the design and development outputs meet the input requirements; (8.3.4 ¶ 1(c), ISO 9001 Quality Management systems - Requirements, Fifth edition 2015-09-15)
  • The organization should test changes to system components in order to minimize the risk of adverse effects on protecting personal information. (Table Ref 1.2.6, Generally Accepted Privacy Principles (GAPP), CPA and CA Practitioner Version, August 2009)
  • The organization must perform and approve unit, integration, and system testing according to the test plan and include a sufficient range of valid and invalid conditions in the testing. (CSR 6.3.9, Pub 100-17 Medicare Business Partners Systems Security, Transmittal 7, Appendix A: CMS Core Security Requirements CSR, March 17, 2006)
  • A medical device manufacturer shall establish and maintain procedures to validate the device design. Validation shall be conducted under defined operating conditions on the initial production lots, batches, or units and shall ensure the devices conform to user needs and intended uses. It shall inclu… (§ 820.30(g), 21 CFR Part 820, Subchapter H - Medical Devices, Part 820 Quality System Regulation)
  • Evaluate the adequacy of development activities by assessing: ▪ The adequacy of, and adherence to, development standards and controls; ▪ The applicability and effectiveness of project management methodologies; ▪ The experience of project managers; ▪ The adequacy of project plans, particularl… (Exam Obj 5.1, FFIEC IT Examination Handbook - Development and Acquisition)
  • System auditing utilities should be incorporated into new and existing ICS projects. These auditing utilities should be tested (e.g., off-line on a comparable ICS) before being deployed on an operational ICS. These tools can provide tangible records of evidence and system integrity. Additionally, ac… (§ 6.2.3 ICS-specific Recommendations and Guidance ¶ 6, Guide to Industrial Control Systems (ICS) Security, Revision 2)
  • Determine level of assurance of developed capabilities based on test results. (T0058, National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, NIST Special Publication 800-181)
  • Perform validation steps, comparing actual results with expected results and analyze the differences to identify impact and risks. (T0183, National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, NIST Special Publication 800-181)
  • Determine level of assurance of developed capabilities based on test results. (T0058, Reference Spreadsheet for the Workforce Framework for Cybersecurity (NICE Framework)”, July 7, 2020)
  • The organization requires the developer of the information system, system component, or information system service to verify that the scope of security testing/evaluation provides complete coverage of required security controls at {organizationally documented depth of testing/evaluation}. (SA-11(7), Security and Privacy Controls for Federal Information Systems and Organizations, NIST SP 800-53, Deprecated, Revision 4, Deprecated)
  • Provide evidence of meeting the quality metrics [Selection (one or more): [Assignment: organization-defined frequency]; [Assignment: organization-defined program review milestones]; upon delivery]. (SA-15(1)(b), Security and Privacy Controls for Federal Information Systems and Organizations, NIST SP 800-53, Revision 4)
  • The organization requires the developer of the information system, system component, or information system service to demonstrate the use of a system development life cycle that includes [Assignment: organization-defined state-of-the-practice system/security engineering methods, software development… (SA-4(3) ¶ 1, Security and Privacy Controls for Federal Information Systems and Organizations, NIST SP 800-53, Revision 4)
  • Provide evidence of meeting the quality metrics [Selection (one or more): [Assignment: organization-defined frequency]; [Assignment: organization-defined program review milestones]; upon delivery]. (SA-15(1) ¶ 1(b), Security and Privacy Controls for Information Systems and Organizations, NIST SP 800-53, Revision 5)
  • [Assignment: organization-defined software development methods; testing, evaluation, assessment, verification, and validation methods; and quality control processes]. (SA-4(3) ¶ 1(c), Security and Privacy Controls for Information Systems and Organizations, NIST SP 800-53, Revision 5)
  • Provide evidence of meeting the quality metrics [Selection (one or more): [Assignment: organization-defined frequency]; [Assignment: organization-defined program review milestones]; upon delivery]. (SA-15(1) ¶ 1(b), Security and Privacy Controls for Information Systems and Organizations, NIST SP 800-53, Revision 5.1.1)
  • [Assignment: organization-defined software development methods; testing, evaluation, assessment, verification, and validation methods; and quality control processes]. (SA-4(3) ¶ 1(c), Security and Privacy Controls for Information Systems and Organizations, NIST SP 800-53, Revision 5.1.1)
  • Conduct Testing (Developmental, Functional and Security): Systems being developed or undergoing software, hardware, and/or communication modification(s) must be tested and evaluated prior to being implemented. The objective of the test and evaluation process is to validate that the developed system … (§ 3.2.3.6, Security Considerations in the Information System Development Life Cycle, NIST SP 800-64, Revision 2)
  • Bank management should thoroughly test new technology systems and products. Testing validates that equipment and systems function properly and produce the desired results. As part of the testing process, management should verify whether new technology systems operate effectively with the bank's olde… (¶ 37, Technology Risk Management Guide for Bank Examiners - OCC Bulletin 98-3)