Compliance Spreadsheets Suck

Analyzing frameworks-as-spreadsheets

We’ve all seen the framework-as-spreadsheet model, such as the one initially created by us at Unified Compliance back in 2006. They have three main elements:

1. The identify source Authority Documents by name and version (and hopefully a direct link to the document online).

2. They provide an array of Citation References denoting the individual citations in each Authority Document that align with the reference control.

3. (a) They provide a list of reference controls such as the Common Controls from the Unified Compliance Framework, CobiT’s controls, NIST 800–53, IS 2700, or others.

4. (b) They provide a structure for linking a tagged Mandate to a matched reference control.

This is a pretty common format. There are several organizations that provide framework-as-spreadsheet or table output formats (with both NIST OLIR and the UCF providing JSON outputs as well):

• AICPA Trust Services Criteria
• HITRUST
• NIST 8204 & 8278 (OLIR)
• PCI-DSS
• Shared Assessments
• Secure Controls Framework
• Nymity Privacy Framework
• Unified Compliance Framework

All these frameworks are in use, to some extent, within the Governance, Risk, and Compliance community. Let’s go through each one and see how they represent the requirements of a compliance framework with two new additions.

Because we are now talking about output we need to add that the framework should also have a structure for providing the reader with the Citation Reference (e.g., the §1.2 part) and the Citation Guidance (the actual text). Without the Citation reference, you won’t know what Citation is in play. Without the actual text, the reader won’t be able to verify the framework’s assertion of compliance mapping. Here is the updated table:

So now we know the defining requirements of a compliance framework. How well does a framework-as-spreadsheet hold up to these requirements? First, let’s go back to a term we’ve used above, non satis probandi – an ancient Latin term for not enough proof. You may be more familiar with onus probandi – the burden of proof.

Onus probandi – the burden of proof

Major thanks to my co-Founder and brilliant attorney Marcelo Halpern, here’s what he has to say about onus probandi. As he states it, “here’s how Black’s Law Dictionary defines “burden of proof” (as linked from “onus probandi”):

burden of proof (18c) 1. A party’s duty to prove a disputed assertion or charge; a proposition regarding which of two contending litigants loses when there is no evidence on a question or when the answer is simply too difficult to find. • The burden of proof includes both the burden of persuasion and the burden of production. — Also termed evidentiary burden; evidential burden; onus probandi. See SHIFTING THE BURDEN OF PROOF. Cf. STANDARD OF PROOF. 2. Loosely, BURDEN OF PERSUASION. — Abbr. BOP.

In the past the term ‘burden of proof’ has been used in two different senses. (1) The burden of going forward with the evidence. The party having this burden must introduce some evidence if he wishes to get a certain issue into the case. If he introduces enough evidence to require consideration of this issue, this burden has been met. (2) Burden of proof in the sense of carrying the risk of nonpersuasion. The one who has this burden stands to lose if his evidence fails to convince the jury — or the judge in a nonjury trial. The present trend is to use the term ‘burden of proof’ only with this second meaning …” Rollin M. Perkins & Ronald N. Boyce, Criminal Law 78 (3d ed. 1982).

The expression ‘burden of proof’ is tricky because it has been used by courts and writers to mean various things. Strictly speaking, burden of proof denotes the duty of establishing by a fair preponderance of the evidence the truth of the operative facts upon which the issue at hand is made to turn by substantive law. Burden of proof is sometimes used in a secondary sense to mean the burden of going forward with the evidence. In this sense it is sometimes said that a party has the burden of countering with evidence a prima facie case made against that party.” William D. Hawkland, Uniform Commercial Code Series§ 2A-516:08 (1984).

He goes on to point out that there is also the concept of “Standard of Proof” which is defined as:

standard of proof (1857) The degree or level of proof demanded in a specific case, such as “beyond a reasonable doubt” or “by a preponderance of the evidence”; a rule about the quality of the evidence that a party must bring forward to prevail. — Also termed degree of proof. See BURDEN OF PERSUASION. Cf. BURDEN OF PROOF.

And looking at it from the “proof” side, there’s the concept of Due Proof:

due proof (16c) Sufficient and properly submitted evidence to produce a result or support a conclusion, such as an entitlement to benefits supported by an insurance policy. • The evidence need not be the best proof possible. Metropolitan Life Ins. Co. v. Frisch, 65 N.E. 2d 852, 855 (Ind. App. 1946).

Clearly, if any person or organization is going to present a harmonized compliance framework, it is upon them to carry the burden of proof that the Citations do in fact correlate to the reference framework’s controls. So, does a framework-as-spreadsheet hold up to the burden of proof?

A review of the frameworks

Let’s look at each of these frameworks alphabetically for how the requirements are represented.

AICPA Trust Services Criteria

The AICPA Trust Services Criteria is a well-known framework that has mappings to a very small set of Authority Documents¹. Their Trust Services Criteria is downloadable as a spreadsheet which has three data elements to it:

HITRUST

According to HITRUST itself, the foundation of all HITRUST® programs and services is the HITRUST CSF®, a certifiable framework that provides organizations with a comprehensive, flexible, and efficient approach to regulatory compliance and risk management². Their CSF is downloadable (to those they feel like allowing to have it) as a set of spreadsheets. Their mapping spreadsheet contains items 1 through 3a. They do not show the actual Citation references in the spreadsheet.

Notice that HITRUST doesn’t even present the reader with the Citation’s Guidance text. There is no way, on the face of it, to check HITRUST’s work.

NIST’s Derived Relationship Mapping table

The Derived Relationship Mapping (DRMs) Analysis Tool³ provides Users the ability to generate downloadable spreadsheets for up to four Authority Documents at a time with the Cybersecurity Framework as the Focal Document (selection criteria shown below):

NIST’s DRM table exports as both JSON (which we’ll describe later) and as a spreadsheet. What’s great about their export is that they also show how the Citation in question maps to the reference control (3b Provide a structure for measuring correlation between Mandates or a reference control) as shown below:

Nymity Privacy Framework

Nymity’s frameworks aren’t downloadable as spreadsheets, but rather, a PDF showing a simple table format⁴. Each of the Nymity PDFs is mapped to a single Authority Document, so there is no need for individual Authority Document identification within their tables. However, what is wild about their “framework” is that they don’t provide any identifiers for their reference controls!

PCI-DSS

The Payment Card Security Standard Council’s Data Security Standard (PCI-DSS) is a widely adopted control list that is being used more and more as a reverse engineered compliance framework. What we mean by that is that the Payment Council uses that framework’s controls to them show correlation to other standards without regard to changing its structure. And there’s nothing wrong with that. Because the PCI committee releases these documents on a PCI-to-X document basis, there is no need in their tables to document information about the Authority Document per se, as this is a mapping of that Authority Document’s Citations to PCI-DSS.

Notice that PCI doesn’t present the reader with the Citation’s Guidance text either. There is no way, on the face of it, to check PCI’s work.

SANS Critical Security Controls

AuditScripts formed a relationship with SANS Institute and have created and published a framework for mapping SANS’ Critical Security Controls to a host of Authority Documents. Like many of the others, it has no Citation text.

Secure Controls Framework

According to their mission statement, Secure Controls Framework (SCF) is designed to empower organizations to design, implement and manage both cybersecurity and privacy principles to address strategic, operational and tactical guidance⁵. Like many of the others, it has no Citation text.

Shared Assessments

According to their website, using a comprehensive set of questions (content library), the SIG gathers information to determine how security risks are managed across a 18 risk control areas, or “domains”, within a service provider’s environment. The library houses comprehensive risk and cybersecurity frameworks as well as industry-specific controls⁶. Their spreadsheet also contains very little information.

Unified Compliance Framework’s spreadsheet output

It should be noted here that the UCF’s spreadsheet is not indicative of its structure. This is merely a reformatting and simplified output of the UCF into spreadsheet format. The UCF’s structure was never meant for a spreadsheet as you’ll see when we cover its JSON format.

Summary of support

Not one of the frameworks as spreadsheets provide enough support to meet the burden of proof. Not. A. Single. One.