Back

Monitor and evaluate system performance.


CONTROL ID
00651
CONTROL TYPE
Monitor and Evaluate Occurrences
CLASSIFICATION
Detective

SUPPORTING AND SUPPORTED CONTROLS




This Control directly supports the implied Control(s):
  • Establish, implement, and maintain logging and monitoring operations., CC ID: 00637

This Control has the following implementation support Control(s):
  • Disseminate and communicate monitoring capabilities with interested personnel and affected parties., CC ID: 13156
  • Disseminate and communicate statistics on resource usage with interested personnel and affected parties., CC ID: 13155


SELECTED AUTHORITY DOCUMENTS COMPLIED WITH




  • Management should ensure that the necessary improvements to control activities, based on their actual performance, have been appropriately designed and are operating correctly. (Practice Standard § I.5(1), On the Setting of the Standards and Practice Standards for Management Assessment and Audit concerning Internal Control Over Financial Reporting, Provisional Translation)
  • In addition, it is necessary to continually analyze confirmed results of usage situations and then to enhance the performance and functionalities of systems and review the system combinations. (P47.1. ¶ 2, FISC Security Guidelines on Computer Systems for Financial Institutions, Ninth Edition, Revised March 2020)
  • In a system where its reliability is enhanced by redundant configuration, it is necessary to confirm regularly that the redundantly configured devices are working normally. (P71.3. ¶ 2, FISC Security Guidelines on Computer Systems for Financial Institutions, Ninth Edition, Revised March 2020)
  • It is necessary to monitor the operational conditions of computer systems according to the importance of individual systems. This measure especially is indispensable for systems that deal with personal data. (P102.1. ¶ 1, FISC Security Guidelines on Computer Systems for Financial Institutions, Ninth Edition, Revised March 2020)
  • Banks providing internet banking should be responsive to unusual network traffic conditions/system performance and sudden surge in system resource utilization which could be an indication of a DDoS attack. Consequently, the success of any pre-emptive and reactive actions depends on the deployment of… (Critical components of information security 26) a., Guidelines on Information Security, Electronic Banking, Technology Risk Management and Cyber Frauds)
  • To ensure that IT systems and infrastructure are able to support business functions, the FI should ensure that indicators such as performance, capacity and utilisation are monitored and reviewed. (§ 7.5.1, Monetary Authority of Singapore: Technology Risk Management Guidelines)
  • System owners monitor each system, and associated cyber threats, security risks and security controls, on an ongoing basis. (Security Control: 1526; Revision: 1, Australian Government Information Security Manual, March 2021)
  • The use of benchmarks to assess performance as well as any monitoring reports to establish performance indicators is called for. (§ H.4.2, The DIRKS Manual: A Strategic Approach to Managing Business Information, rev. July 2003)
  • continuously monitor the security and functioning of all ICT systems; (Art. 16.1. ¶ 2(b), Regulation (EU) 2022/2554 of the European Parliament and of the Council of 14 December 2022 on digital operational resilience for the financial sector and amending Regulations (EC) No 1060/2009, (EU) No 648/2012, (EU) No 600/2014, (EU) No 909/2014 and (EU) 2016/1011 (Text with EEA relevance))
  • The Management Board must report the reasons for any deviation between the actual results and the planned results. (¶ 3.4, German Corporate Governance Code ("The Code"), June 6, 2008)
  • The entity has established policies and procedures that prevent, detect and react to system outages, incidents and events that disrupt system processing, or results in the loss, accidental disclosure or unauthorized modification of the entity's PI. (S7.4 Continuity of physical and environmental protections, Privacy Management Framework, Updated March 1, 2020)
  • The internal assessments should be evaluated over time for their performance, and adjustments should be made, as necessary. (¶ 620(g), Basel II: International Convergence of Capital Measurement and Capital Standards - A Revised Framework)
  • Establish an IT steering committee (or equivalent) composed of executive, business and IT management to: - Determine prioritisation of IT-enabled investment programmes in line with the enterprise's business strategy and priorities - Track status of projects and resolve resource conflict - Monitor se… (PO4.3 IT Steering Committee, CobiT, Version 4.1)
  • Periodically review performance against targets, analyse the cause of any deviations, and initiate remedial action to address the underlying causes. At appropriate times, perform root cause analysis across deviations. (ME1.4 Performance Assessment, CobiT, Version 4.1)
  • Develop senior management reports on IT's contribution to the business, specifically in terms of the performance of the enterprise's portfolio, IT-enabled investment programmes, and the solution and service deliverable performance of individual programmes. Include in status reports the extent to whi… (ME1.5 Board and Executive Reporting, CobiT, Version 4.1)
  • Produce reports of service desk activity to enable management to measure service performance and service response times and to identify trends or recurring problems, so service can be continually improved. (DS8.5 Reporting and Trend Analysis, CobiT, Version 4.1)
  • A key factor for ensuring the Information Technology Service Continuity (ITSC) strategy and plans are appropriate as the organization and its environment changes is to ensure that service levels are reviewed on a monthly basis at Board meetings. (§ 5.6 ¶ 2(e), PAS 77 IT Service Continuity Management. Code of Practice, 2006)
  • The performance of business applications, Information Systems, and networks should be monitored by reviewing current utilisation of systems at normal and peak periods. (CF.10.05.01b, The Standard of Good Practice for Information Security)
  • The performance of business applications, Information Systems, and networks should be monitored by investigating bottlenecks / overloads. (CF.10.05.01e, The Standard of Good Practice for Information Security)
  • System / network availability (i.e., response and up-time) should be measured from the perspective of business users (e.g., by monitoring Information System and network performance). (CF.10.05.04, The Standard of Good Practice for Information Security)
  • The performance of business applications, Information Systems, and networks should be monitored by reviewing current utilisation of systems at normal and peak periods. (CF.10.05.01b, The Standard of Good Practice for Information Security, 2013)
  • The performance of business applications, Information Systems, and networks should be monitored by investigating bottlenecks / overloads. (CF.10.05.01e, The Standard of Good Practice for Information Security, 2013)
  • System / network availability (i.e., response and up-time) should be measured from the perspective of business users (e.g., by monitoring Information System and network performance). (CF.10.05.04, The Standard of Good Practice for Information Security, 2013)
  • The organization should monitor the medical network for performance feedback and operational feedback, such as high error rates, user feedback, speed problems, and malicious software attacks. (§ 4.6.1 ¶ 2(b), Application of risk management for IT-networks incorporating medical devices Part 1: Roles, responsibilities and activities, Edition 1.0 2010-10)
  • Monitoring of Security Awareness Programs. An organization should monitor security awareness programs by: • periodic performance evaluations - to determine the effectiveness of an awareness program by monitoring security related behavior and identify where changes affecting the program delivery mi… (¶ 10.2.3, ISO 13335-3 Information technology - Guidelines for the management of IT Security - Part 3: Techniques for the management of IT Security, 1998)
  • The organization shall implement and monitor methods for evaluating the effectiveness of the risk treatment. (§ 6.3.4.3(e)(2), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • The organization shall monitor the operation of the system to verify it is operating in accordance with the operations plan. (§ 6.4.9.3(c)(2), ISO 15288-2008 Systems and software engineering - System life cycle processes, R 2008)
  • It is a best practice for ICT disaster recovery service providers to perform performance measurements to quantify how effective the provided ICT disaster recovery services are. These measurements should be supplied to the organization as agreed upon in the relevant SLA or other arrangement. The perf… (§ 9.3, ISO 24762 Information technology - Security techniques - Guidelines for information and communications technology disaster recovery services, 2008)
  • feedback on the information security performance, including trends in: (§ 9.3 ¶ 2 c), ISO 27001:2013, Information Technology - Security Techniques - Information Security Management Systems - Requirements, 2013)
  • the IT asset performance; (Section 9.1 ¶ 3 bullet 1, ISO/IEC 19770-1, Information technology — IT asset management — Part 1: IT asset management systems — Requirements, Third Edition, 2017-12)
  • feedback on the information security performance, including trends in: (§ 9.3.2 ¶ 1 d), ISO/IEC 27001:2022, Information security, cybersecurity and privacy protection — Information security management systems — Requirements)
  • Revise target performance: An organization may choose to revise the target performance level to reflect a better understanding of the reasonableness of potential performance outcomes and the corresponding severity of risks to the business objective. (Integrating Reviews into Business Practices ¶ 2 Bullet 4, Enterprise Risk Management - Integrating with Strategy and Performance, June 2017)
  • SL 1 – Monitor the operation of the components of the IACS, and respond to incidents when discovered, by collecting and providing the forensic evidence when queried. (10.1 ¶ 1 Bullet 1, Security for Industrial Automation and Control Systems, Part 4-2: Technical Security Requirements for IACS components)
  • SL 1 – Ensure that the component operates reliably under normal production conditions and prevents denial-of-service situations caused by the casual or coincidental actions of an entity. (11.1 ¶ 1 Bullet 1, Security for Industrial Automation and Control Systems, Part 4-2: Technical Security Requirements for IACS components)
  • SL 2 – Ensure that the component operates reliably under normal and abnormal production conditions and prevents denial-of-service situations by entities using simple means with low resources, generic skills and low motivation. (11.1 ¶ 1 Bullet 2, Security for Industrial Automation and Control Systems, Part 4-2: Technical Security Requirements for IACS components)
  • The organization includes as part of security control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [A… (CA-2(2) ¶ 1, StateRAMP Security Controls Baseline Summary Category 3, Version 1.1)
  • The organization includes as part of security control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [A… (CA-2(2) ¶ 1, StateRAMP Security Controls Baseline Summary High Sensitivity Level, Version 1.1)
  • The entity's system availability and security performance is periodically reviewed and compared with the system availability and related security policies. (Availability Prin. and Criteria Table § 4.1, Appendix B: Trust Services Principles and Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy, TSP Section 100 Principles and Criteria)
  • The entity's system processing integrity and security performance is periodically reviewed and compared with the system processing integrity and related security policies. (Processing Integrity Prin. and Criteria Table § 4.1, Appendix B: Trust Services Principles and Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy, TSP Section 100 Principles and Criteria)
  • The entity's system confidentiality and security performance is periodically reviewed and compared with the system confidentiality and related security policies. (Confidentiality Prin. and Criteria Table § 4.1, Appendix B: Trust Services Principles and Criteria for Security, Availability, Processing Integrity, Confidentiality, and Privacy, TSP Section 100 Principles and Criteria)
  • The organization should compare expected performance to actual performance to ensure it is within the acceptable risk tolerance parameters. (Pg 36, COSO Enterprise Risk Management (ERM) Integrated Framework (2004))
  • CSR 2.1.10: The audit trail must log all hardware fault control routines to determine if recovery is possible. CSR 10.2.3: The organization must review bandwidth utilization rates, network traffic, border defense devices, and alert notifications at least daily and on demand to identify anomalies. Th… (CSR 2.1.10, CSR 10.2.3, Pub 100-17 Medicare Business Partners Systems Security, Transmittal 7, Appendix A: CMS Core Security Requirements CSR, March 17, 2006)
  • Performance measures must be used for all information technology assets and measure how well the technology supports the agency's programs. (§ 5123(3), Clinger-Cohen Act (Information Technology Management Reform Act))
  • § 4.5.2 (MED0290: CAT II) The Information Assurance Officer, for Access Control Lists, for all networked medical devices, will ensure performance is not affected. § 5.2 (MED0500: CAT II) The Information Assurance Officer, for hardware and/or software that validates the proper operation of networke… (§ 4.5.2 (MED0290: CAT II), § 5.2 (MED0500: CAT II), Medical Devices Security Technical Implementation Guide, Version 1, Release 1)
  • Determine whether the financial institution and service provider use a layered anti-malware strategy, including integrity checks, anomaly detection, system behavior monitoring and employee security awareness training, in addition to traditional signature-based anti-malware systems. (TIER I OBJECTIVES AND PROCEDURES Cyber Resilience Objective 10:5, FFIEC Business Continuity Planning (BCP) IT Examination Handbook, February 2015)
  • Determine whether IT management develops satisfactory measures for defining and monitoring metrics, performance benchmarks, service level agreements, compliance with policies, effectiveness of controls, and quality assurance and control. Determine whether management developed satisfactory reporting … (App A Objective 13, FFIEC Information Technology Examination Handbook - Management, November 2015)
  • The Board of Directors and management should periodically evaluate the effectiveness of the e-banking strategy by comparing the actual performance with the organization's goals and expectations. The Board should also determine if the proper procedures and policies are in effect and if the risks are … (Pg 21, FFIEC IT Examination Handbook - E-Banking, August 2003)
  • The organization should establish performance benchmarks and monitor them on a regular basis. (Pg 33, Pg 34, Exam Obj 5.1, FFIEC IT Examination Handbook - Management)
  • Assess management's effectiveness in using help desk information to improve overall operations performance. ▪ Identify whether management has effective tools and processes in place to effectively identify systemic or high-risk issues. ▪ Determine whether management identifies systemic or high-ri… (Exam Tier II Obj G.6, FFIEC IT Examination Handbook - Operations, July 2004)
  • The organization includes as part of security control assessments, [FedRAMP Assignment: at least annually], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [Assign… (CA-2(2) High Baseline Controls, FedRAMP Baseline Security Controls, 8/28/2018)
  • The organization includes as part of security control assessments, [FedRAMP Assignment: at least annually], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [Assign… (CA-2(2) Moderate Baseline Controls, FedRAMP Baseline Security Controls, 8/28/2018)
  • Include as part of control assessments, [FedRAMP Assignment: at least annually], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; security instrumentation; automated security test cases; vulnerability scanning; malicious user testing; insider threat assessment; per… (CA-2(2) ¶ 1, FedRAMP Security Controls High Baseline, Version 5)
  • Do the digital signature procedures include following up on unusual session activity or errors? (IT - Authentication Q 21c, Automated Integrated Regulatory Examination System (AIRES) IT Exam Questionnaires, version 073106A)
  • Pre-trained models which are used for development are monitored as part of AI system regular monitoring and maintenance. (MANAGE 3.2, Artificial Intelligence Risk Management Framework, NIST AI 100-1)
  • Include as part of control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; security instrumentation; automated security test cases; vulnerability scanning; malicious user testing; insider threat assessment… (CA-2(2) ¶ 1, Control Baselines for Information Systems and Organizations, NIST SP 800-53B, High Impact Baseline, October 2020)
  • Measuring the performance of their C-SCRM initiatives toward desired outcomes (3.5. ¶ 1 Bullet 3, Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations, NIST Special Publication 800-161, Revision 1)
  • Include as part of control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; security instrumentation; automated security test cases; vulnerability scanning; malicious user testing; insider threat assessment… (CA-2(2) ¶ 1, Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations, NIST Special Publication 800-161, Revision 1, Appendix A, C-SCRM Level 3 Controls)
  • Analyze Log Data. (§ 5.2, Guide to Computer Security Log Management, NIST SP 800-92)
  • The organization includes as part of security control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [A… (CA-2(2) ¶ 1 High Baseline Controls, Guide to Industrial Control Systems (ICS) Security, Revision 2)
  • Monitor changes to a system and its environment of operation. (T0960, National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, NIST Special Publication 800-181)
  • Provide analyses and support for effectiveness assessment. (T0782, National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, NIST Special Publication 800-181)
  • Diagnose network connectivity problem. (T0081, National Initiative for Cybersecurity Education (NICE) Cybersecurity Workforce Framework, NIST Special Publication 800-181)
  • Diagnose network connectivity problem. (T0081, Reference Spreadsheet for the Workforce Framework for Cybersecurity (NICE Framework)”, July 7, 2020)
  • Monitor and report client-level computer system performance. (T0502, Reference Spreadsheet for the Workforce Framework for Cybersecurity (NICE Framework)”, July 7, 2020)
  • Monitor network capacity and performance. (T0153, Reference Spreadsheet for the Workforce Framework for Cybersecurity (NICE Framework)”, July 7, 2020)
  • The organization includes as part of security control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [A… (CA-2(2) ¶ 1, Security and Privacy Controls for Federal Information Systems and Organizations, NIST SP 800-53, High Impact Baseline, Revision 4)
  • The organization includes as part of security control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [A… (CA-2(2) ¶ 1, Security and Privacy Controls for Federal Information Systems and Organizations, NIST SP 800-53, Revision 4)
  • Include as part of control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; security instrumentation; automated security test cases; vulnerability scanning; malicious user testing; insider threat assessment… (CA-2(2) ¶ 1, Security and Privacy Controls for Information Systems and Organizations, NIST SP 800-53, Revision 5)
  • Include as part of control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; security instrumentation; automated security test cases; vulnerability scanning; malicious user testing; insider threat assessment… (CA-2(2) ¶ 1, Security and Privacy Controls for Information Systems and Organizations, NIST SP 800-53, Revision 5.1.1)
  • The organization includes as part of security control assessments, [Assignment: organization-defined frequency], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [A… (CA-2(2) ¶ 1, Supply Chain Risk Management Practices for Federal Information Systems and Organizations, NIST Special Publication 800-161, April 2015)
  • The organization monitors the status of systems and components, and communicates out-of-bounds and out-of-spec performance to [Assignment: organization-defined system integrators, suppliers, or external service providers]. (MA-7 Control:, Supply Chain Risk Management Practices for Federal Information Systems and Organizations, NIST Special Publication 800-161, April 2015)
  • Management should monitor and measure the performance of technology-related products, services, delivery channels, and processes in order to avoid potential operational failures and to mitigate the damage that may arise if such failures occur. Established controls should identify and manage risks so… (¶ 42, Technology Risk Management Guide for Bank Examiners - OCC Bulletin 98-3)
  • Testing and monitoring regularly the effectiveness of key controls, systems and procedures; and (§ 646A.622(2)(d)(B)(iv), Oregon Revised Statutes, Chapter 646a, Sections 646A.600 thru 646A.624, Identity Theft Protection Act, 2007 Statutes)
  • The organization includes as part of security control assessments, [TX-RAMP Assignment: at least annually], [Selection: announced; unannounced], [Selection (one or more): in-depth monitoring; vulnerability scanning; malicious user testing; insider threat assessment; performance/load testing; [Assign… (CA-2(2) ¶ 1, TX-RAMP Security Controls Baseline Level 2)