menu

Education

What is a compliance framework?

And more importantly, what isn’t a compliance framework?

(Just because the authors call something a framework, doesn’t mean it is actually a framework.)

Contents

What is a Compliance Framework?

Why do Compliance Frameworks Exist?

The Definition of a Compliance Framework

Identifying Mandates

Tracking Authority Documents

Extracting Citations

Identifying the Mandates

Identifying Auditable Mandates

Mapping Mandates to Like Mandates

Crosswalking

Transformation and Harmonization

Standardizing Audit Questions

Which of these are frameworks (and which are not)?

Closed Versus Open Frameworks

Structure

Why do compliance frameworks exist?

Compliance simply means following the rules that are set by people other than ourselves. In more specific terms, compliance is ensuring that the requirements of laws, regulations, industry codes, and organizational doctrines are met. This also applies to contractual arrangements to which the business process is subject.

Most organizations fall under multiple Authority Documents (laws, regulations, standards, audit guides, etc.). Someone in your organization is bound to say “hey, figure out what we need to do to comply with these various documents”.

Multiple Authority Documents

Introducing multiple Authority Documents

Once those multiple Authority Documents are examined, of course they are going to have many mandates in them, some of those Mandates seemingly (but not word-for-word) the same.

Examples of multiple mandates

Oh boy, multiple (and maybe overlapping) mandates

Here’s what we know so far:

    • you’ll have multiple documents you need to be in compliance with,
    • that have many mandates per document that you’ll need to interpret and apply, and
    • somebody is going to want you to self-assert or audit you to provide you’ve implemented methods and practices to comply with those mandates.

That means that you are going to want to achieve three things:

    • Identify the Mandates within the Citations of each Authority document.
    • Map Mandates to like Mandates (to eliminate duplication).
  • Create an audit methodology to prove you’ve implemented the mandates.

How do you do that? There are three ways of ways of doing it.

The first is to just wing it and hope nobody notices. Seriously. We see this done all the time. Not a great idea.

The second is to pick a known standard, such as an international standard like one of the ISO standards, an audit standard like CobiT, or a national standard like those the NIST 800-53 standard as the cornerstone, and interpret every mandate according to that cornerstone.

The third is to follow a compliance framework designed specifically to integrate multiple documents into a cohesive whole.

The first two methods don’t work.

The definition of a compliance framework

Let’s start with the definition of framework (that applies here, as opposed to creating an actual building); a basic structure underlying a system, concept, or text. So, a framework is a structure, a schema, method of organization and configuration to accomplish something.

When dealing with compliance frameworks, that structure and schema focuses on aggregation of compliance rules, first and foremost. Once aggregation and harmonization are complete, those requirements need a structure for integration in to the organization’s processes as well. Therefore, we can define compliance frameworks as such:

A compliance framework is a structured set of guidelines to aggregate and harmonize, then integrate, all compliance requirements applicable to an organization.

In other words, a complianceframework is a methodology for compiling multiple authority documents into a cohesive whole.

It provides a structure for harmonization.

It provides a structure for integration.

The goal of any compliance framework is to reduce the burden of following multiple guidelines by finding commonality between mandates and then maintaining accordance with those common mandates using the organization’s compliance processes and tools.

We now have the defining requirements for compliance frameworks:

identify MandatesDefine rules to extract Mandates from Citations within Authority Documents
map Mandates to like MandatesMap Mandates from all Citations to other Mandates like them
standardize auditsLeverage a standardized structure for auditing the implementation of the selected set of mapped Mandates

Let’s look at each of the defining requirements of a compliance framework, in more depth.

Identifying Mandates

Authority Documents are comprised of Citations, some of which have Mandates, with some of those Mandates being auditable. Let’s break this sentence down into its constituent parts and examine it.

Authority DocumentsAnything from laws through regulations, safe harbors, standards, self-regulatory body guidelines, etc.
have CitationsThe basic unit of content, such as numbered sections or paragraphs of documents.
that have MandatesThe actual “go and do this” parts of each Citation.
and some of those Mandates are auditableSome mandates are informational, others are auditable.

The first identifying characteristic, therefore, has a structure and schema for tracking Authority Documents, extracting their Citations, deriving the Mandates from those Citations, and providing audit questions that match those mandates.

Tracking Authority Documents

First and foremost, there needs to be some structure and rules for tracking the sources of the Citations and Mandates.

Tracking authority documents

Tracking Authority Documents

This can be as simple as the framework providing a register of all of the Authority Document sources, to a full-fledged data structure for Authority Document library management.

Extracting Citations

Citations live inside of each of the Authority Documents. There compliance framework needs to provide a methodology for Citation selection, i.e., which Citations in an Authority Document can be ignored, and which must be included into the organization’s list of Citations to work with.

Citations from Authority Documents

Citations from an Authority Document

Not only does the compliance framework need to provide a structure by which to extract and include Citations, it should also provide a structure to identify and track each Citation. Only some documents will have well-structured formats that number their Citations. Others merely put them onto a page and require some form of paragraph and page or line identification.

Identifying the Mandates

A Mandate is a declarative independent clause (a predicate and a subject) that says “do this” or “do that” or “test that this was done”. Mandates must be extracted from each Citation (or at least each Citation that forms a complete sentence). Therefore, the compliance framework must have guidelines and methods for Mandate identification, extraction, and tracking.

At bare minimum it must provide guidance and methodologies for how each Citation’s primary verbs are extracted from the Citation;

Analyzing verbs

Analyzing verbs

as well as guidance and methodologies for how each Citation’s primary nouns are extracted.

Analyzing Nouns

Analyzing nouns

Identifying auditable Mandates

Not all mandates are auditable. Most of them are, but some of them are written so vaguely that they are open to incredible interpretation and therefore auditing them is a waste of time. But for the 95% of Mandates that are auditable, the compliance framework should provide a methodology for identifying which Mandates are auditable.

Mapping Mandates to like Mandates

Remember that the primary goal of any compliance framework is to reduce the burden of following multiple guidelines by finding commonality between Mandates. Once Mandates have been identified and extracted from Citations, the compliance framework must provide a suite of rules and methods for determining their commonality.

This has been called crosswalking, harmonizing, mapping or unifying one regulation or standard to another. For the sake of sanity, we are going to call this process, in general, mapping because crosswalking and harmonizing are both a part of mapping and unifying is too broad. There are four basic methodologies to achieve the goals of defining what data matches and what data doesn’t;

  1. semantic mappings,
  2. crosswalking rules,
  3. transformation, and
  4. harmonization.

Each of these are combined into either crosswalking or harmonization models. Any form of mapping or unifying compliance must first have a set of relationship rules. Even the most basic system must have a set of rules to map single data sets to differing data sets.

Crosswalking

Crosswalking’s goal is to determine if concept 1 matches concept 2. But it doesn’t end there. Crosswalking is a one-to-one task. Given 4 concepts, crosswalking must be performed 6 times to determine which concepts match (or don’t match) each other.

Crosswalking

Crosswalking

Crosswalking is task intensive. The calculation below shows that 50 concepts creates 1,225 crosswalking tasks. 100 concepts crosswalked to each other creates 4,950 tasks.  Too many tasks to be truly leverageable as a standalone methodology.

Crosswalking Tasks = (N*N-1)/2

Transformation and harmonization

Unlike crosswalking where each Mandate is determined to be like any other, transformation strips each Mandate down into its declarative kernel (predicate subject pair) and connects the Mandate to a matching, stripped down version of itself. Transformation is a one-to-one ratio of concepts to concepts being transformed as shown below.

Transformation of a mandate

Transformation of a Mandate

Once a concept has been transformed, the original concept is mapped to it. Harmonization is the process of either crosswalking new concepts to the transformed one or creating additional transformed concepts if no crosswalk exists.

Harmonization

Harmonization

Therefore, each concept is either crosswalked to an existing harmonized man-in-the-middle concept, or a new harmonization takes place. Given 4 concepts that match the same transformed concept, there would only be 5 tasks. The number of additional tasks depends upon how many new transformations have to take place. Whatever the number, it is surely less than (N*N-1)/2.

Standardizing audit questions

Most Authority Documents don’t write their own audit questions. The PCI DSS is one of the very few that does. So did the FTC’s Red Flag Rules guidance. Other than that, almost nothing.

Most audit questions are created by either a working group in a framework committee or (worse) audit management software teams (where did you think they came from, the audit fairy?).

Only one framework to date has a published standard on methodologies and structures for creating audit questions – the Unified Compliance Framework. Audit Questions to meet ABA requirements, must be asked in Yes/No format and involve either examination, interviews, observation, or testing.

There are two types of auditing methodology that a compliance framework might provide – the simple format and the evidential-based format.

Simple audit questions are often stated as yes or no questions.

  • Are vendor-supplied defaults always changed before installing a system on the network?
  • Have configuration standards been developed for all system components?

Evidential-based audit questions provide a methodology to answer the question as well as force the organization being audited to rely on evidence in order to come to the conclusion.

  • Test a sample of All Assets to ensure the configuration item of Default configuration settings is "Account and password settings do not match vendor supplied defaults." Is this configured correctly?
  • Examine the System Security Configuration standard. Does it ensure that it covers all known configuration items and system components?

Evidential-based audit questions format the question method to the subject being audited. How do you employ these additional elements of evidence? By formatting the audit question methods into test, observe, examine, and interview type questions. Evidential items to support the answers can then be linked to each question.

  • Test applies to testing systems, testing computations, etc.
  • Observe applies to watching processes happen.
  • Examine applies to records or assets that can be scrutinized.
  • Interview is reserved for speaking to individuals or groups.

Which of these are frameworks (and which are not)?

We now know that to be considered a framework the document or structure must provide these three things:

Identify MandatesDefine rules to extract Mandates from Citations within Authority Documents
Map Mandates to like MandatesMap Mandates from all Citations to other Mandates like them
Standardize auditsLeverage a standardized structure for auditing the implementation of the selected set of mapped Mandates

Here is a list of documents people frequently think are frameworks, but are not:

  • CobiT
  • NIST Cybersecurity Framework
  • PCI-DSS
  • CIS Controls
  • NIST 800-53
  • ISO 27000 series

These documents, while they might even say framework in the title, are written without regard to Mandate extraction, harmonization, or audit standardization. While PCI-DSS does have its own audit guidelines, that doesn’t make it a framework.

Here is a short list of widely known frameworks:

  • HITRUST
  • Shared Assessments
  • Unified Compliance Framework
  • Proposed NISTIR 8204

Each of these frameworks were built by extracting Mandates from source Authority Documents (and have rules for doing so), harmonizing those Mandates into the framework, and then creating audit guides from the resulting de-duplicated harmonized controls.

Closed versus open frameworks

Closed frameworks, such as HITRUST and Shared Assessments provide content, but they

  • have no published standard on mandate extraction, harmonization, or audit question creation;
  • apply the controls as provided by the framework and wait for the committee to update the framework; and
  • provide no published methodologies for tailoring and “self-reduction” of mandates
  • provide audit questions but no published methodologies for how those questions were created.

Open frameworks, such as the Unified Compliance Framework and the proposed NISTIR 8204 provide either content as well as

  • published standards on mandate extraction, harmonization, audit question creation;
  • the framework itself provides mapping tools to extend the framework by anyone trained to do so; and
  • published methodologies and tools to tailor the mandates to only those necessary for each organization

Structure

The structure of most closed frameworks is simple. They provide tracking for Authority Documents, their Citations, and audit questions.

Closed Framework Structure

Closed framework structure

NIST’s proposed 8204, while an open structure, provides only elements for Issuers (where Authority Documents can be found), the Authority Documents, and their Citations. It doesn’t, at this writing, provide a structure for audit questions.

NISTIR 8204 Structure

NISTIR 8204 structure

The Unified Compliance Framework has a very complex structure that stretches from tracking Issues through Authority Documents, Citations, Common Controls, a Dictionary, and various role, asset, record, activity, events, and audit questions.

Unified Compliance Framework Structure

Unified Compliance Framework structure