Back

Perform a final system test prior to implementing a new system.


CONTROL ID
01108
CONTROL TYPE
Testing
CLASSIFICATION
Detective

SUPPORTING AND SUPPORTED CONTROLS



This Control directly supports the implied Control(s):
  • Initiate the System Development Life Cycle implementation phase., CC ID: 06268

This Control has the following implementation support Control(s):
  • Involve all stakeholders in the final acceptance test., CC ID: 13168
  • Conduct a final security audit prior to implementing a new system., CC ID: 06833
  • Establish, implement, and maintain system acceptance criteria., CC ID: 06210
  • Document the acceptance status for all products passing the System Development Life Cycle implementation phase., CC ID: 06211
  • Control products that do not conform to the system acceptance criteria., CC ID: 06212


SELECTED AUTHORITY DOCUMENTS COMPLIED WITH



  • AIs should adopt and implement a full project life cycle methodology governing the process of developing, implementing and maintaining major computer systems. In general, this should involve phases of project initiation, feasibility study, requirement definition, system design, program development, … (4.2.1, Hong Kong Monetary Authority: TM-G-1: General Principles for Technology Risk Management, V.1 – 24.06.03)
  • App 2-1 Item Number III.5(8): User-acceptance testing must be conducted in an environment that is similar to the production environment. This is a control item that constitutes a greater risk to financial information. This is an IT general control. App 2-1 Item Number III.5(9): Test cases must be pr… (App 2-1 Item Number III.5(8) thru App 2-1 Item Number III.5(12), Appendix 1 Correspondence of the System Management Standards - Supplementary Edition to other standards)
  • All application systems need to be tested before implementation in a robust manner regarding controls to ensure that they satisfy business policies/rules of the bank and regulatory and legal prescriptions/requirements. Robust controls need to be built into the system and reliance on any manual contr… (Critical components of information security 11) c.3., Guidelines on Information Security, Electronic Banking, Technology Risk Management and Cyber Frauds)
  • A robust security screening and testing of the API should be performed between the FI and its third parties before it is deployed into production. The FI should log the access sessions by third parties, such as the identity of the party making the API connections, date and time, as well as the data … (§ 6.4.6, Technology Risk Management Guidelines, January 2021)
  • Financial institutions should have a methodology in place for testing and approval of ICT systems prior to their first use. This methodology should consider the criticality of business processes and assets. The testing should ensure that new ICT systems perform as intended. They should also use test… (3.6.2 70, Final Report EBA Guidelines on ICT and security risk management)
  • The testing of the high-risk AI systems shall be performed, as appropriate, at any point in time throughout the development process, and, in any event, prior to the placing on the market or the putting into service. Testing shall be made against preliminarily defined metrics and probabilistic thresh… (Article 9 7., Proposal for a Regulation of The European Parliament and of The Council Laying Down Harmonized Rules On Artificial Intelligence (Artificial Intelligence Act) and Ameding Certain Union Legislative Acts)
  • Acceptance test of the quality of the services rendered according to the functional and non-functional requirements agreed upon (Section 5.11 BEI-02 Basic requirement ¶ 1 Bullet 3, Cloud Computing Compliance Controls Catalogue (C5))
  • A methodology for testing applications prior to their first use and after material modifications shall be defined and introduced. The scope of the tests shall include the functionality of the application, the security controls and system performance under various stress scenarios. The organisational… (II.6.41, Circular 10/2017 (BA): Supervisory Requirements for IT in Financial Institutions, 14.09.2018)
  • The organization should develop, formally document, and use test scripts, which are related to the user requirements specifications and the functional specifications, to show that the system has been installed and is operating and performing satisfactorily. (¶ 13.4, Good Practices For Computerized systems In Regulated GXP Environments)
  • The system should be thoroughly tested and verified that it is capable of producing the desired results before it is put into use. (¶ 7, PE 009-8, Guide to Good Manufacturing Practice for Medicinal Products, Annex 11, 15 January 2009)
  • During acceptance testing for the new system, all security features should be fully tested. (Pg 12-IV-18, Protection of Assets Manual, ASIS International)
  • Rigorous and documented acceptance criteria should be met before new systems are promoted into the live environment. (CF.18.06.01, The Standard of Good Practice for Information Security)
  • Before new systems are promoted into the live environment, reviews should be performed by implementation staff and business owners. (CF.18.06.02, The Standard of Good Practice for Information Security)
  • Before new systems are promoted into the live environment, checks should be carried out to ensure that performance and capacity requirements can be met. (CF.18.06.03b, The Standard of Good Practice for Information Security)
  • Rigorous and documented acceptance criteria should be met before new systems are promoted into the live environment. (CF.18.06.01, The Standard of Good Practice for Information Security, 2013)
  • Before new systems are promoted into the live environment, reviews should be performed by implementation staff and business owners. (CF.18.06.02, The Standard of Good Practice for Information Security, 2013)
  • Before new systems are promoted into the live environment, checks should be carried out to ensure that performance and capacity requirements can be met. (CF.18.06.03b, The Standard of Good Practice for Information Security, 2013)
  • A program for the systematic monitoring and evaluation to ensure that standards of quality and security baselines are being met shall be established for all software developed by the organization. Quality evaluation and acceptance criteria for information systems, upgrades, and new versions shall be… (CCC-03, Cloud Controls Matrix, v3.0)
  • The organization shall ensure that there is an enabling system available for verification purposes. (§ 6.4.6.3(b)(1), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • The organization shall demonstrate that the system is installed properly. (§ 6.4.7.3(b)(3), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • The organization shall demonstrate that the new system can deliver the required services. (§ 6.4.7.3(b)(5), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • The organization shall demonstrate the system services are sustainable by the enabling systems. (§ 6.4.7.3(b)(6), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • The actual test results should be compared with the expected test results. Any differences should be analyzed to determine if the product needs to be fixed or the tests need to be rerun with a larger sample size or a change in the test. The final report should contain the test configurations, the nu… (§ 10.8.2.4.5, § 10.8.2.4.6, § 11.8.3.4.12, § 11.8.4.4.5, § 11.8.4.5.3, § 12.9.4.4.12, § 12.9.5.4.5, § 12.9.5.5.3, § 13.9.4.4.12, § 13.9.5.4.5, § 13.9.5.5.3, ISO 18045 Common Methodology for Information Technology Security Evaluation Part 3, 2005)
  • § 5.5.3: For software systems assigned to Class B and Class C software safety classes, the medical device manufacturer shall develop software unit acceptance criteria to ensure the software units meet the acceptance criteria before being integrated into larger software items. § 5.5.4: For software… (§ 5.5.3, § 5.5.4, § 5.5.5, ISO 62304 - 2006 Medical device software - Software life cycle processes, 2006)
  • Does the formal software development life cycle process include acceptance testing? (§ I.2.7.1, Shared Assessments Standardized Information Gathering Questionnaire - I. Information Systems Acquisition Development & Maintenance, 7.0)
  • Each system should be accredited before becoming operational. (§ 2-3.a(10), Army Regulation 380-19: Information Systems Security, February 27, 1998)
  • § 820.80(c): A medical device manufacturer shall establish and maintain acceptance procedures for ensuring the in-process products meet the specified requirements. These procedures shall ensure the in-process product is controlled until it has been inspected and tested or other verification process… (§ 820.80(c), § 820.80(d), 21 CFR Part 820, Subchapter H - Medical Devices, Part 820 Quality System Regulation)
  • Tests of new technology, systems, and products before deployment to validate functionality, controls, and interoperability. (App A Objective 12:10 c., FFIEC Information Technology Examination Handbook - Management, November 2015)
  • Performs tests associated with QA and QC independent of the programming function, and whether the QA and QC procedures incorporate user acceptance testing programs. (App A Objective 13:6 c., FFIEC Information Technology Examination Handbook - Management, November 2015)
  • The organization should test all configuration changes prior to moving the system and/or software to the production environment. (§ 4.2.5, Guidelines on Cell Phone and PDA Security, NIST SP 800-124, October 2008)
  • Assist in assessing the impact of implementing and sustaining a dedicated cyber defense infrastructure. (T0348, National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, NIST Special Publication 800-181)
  • After the container technology has been designed, the next step is to implement and test a prototype of the design before putting the solution into production. Be aware that container technologies do not offer the types of introspection capabilities that VM technologies do. (6.3 ¶ 1, NIST SP 800-190, Application Container Security Guide)
  • Registries also provide an opportunity to apply context-aware authorization controls to actions. For example, organizations can configure their continuous integration processes to allow images to be signed by the authorized personnel and pushed to a registry only after they have passed a vulnerabili… (4.2.3 ¶ 3, NIST SP 800-190, Application Container Security Guide)
  • The security engineering principles must include conducting a final security audit before authorizing the system to operate in order to verify the system adheres to the security requirements. (SG.SA-8 Requirement 8, NISTIR 7628 Guidelines for Smart Grid Cyber Security: Vol. 1, Smart Grid Cyber Security Strategy, Architecture, and High-Level Requirements, August 2010)
  • Assist in assessing the impact of implementing and sustaining a dedicated cyber defense infrastructure. (T0348, Reference Spreadsheet for the Workforce Framework for Cybersecurity (NICE Framework)”, July 7, 2020)