10 Things to Know About Privacy Metrics

Surprisingly, ask people what “privacy metrics” mean to them and you’ll often get very different answers.

1. In my view, in the context of privacy, a “metric” is the measurement of a specific activity that presents qualitative or quantitative characteristics about an organization’s personal information holdings or an information-related process; a “metrics program” is the organized collection of a series of such measurements.

2. Privacy metrics provide evidence of compliance with legislative/regulatory requirements as well as internal privacy policies and procedures.  They are useful to boards to enhance organizational governance and support management in making privacy-related decisions.

3. Within organizations, privacy-related activities consist of governance and operational activities.  “ Governance activities” involve identifying privacy risks and making decisions to mitigate those risks through the design and implementation of privacy programs, policies and procedures. “Operational activities” concern monitoring an organization’s performance to ensure day-to-day adherence to privacy-related policy and procedures. Privacy metrics can be used in both areas.

4. Implementing a privacy metrics program has four phases:

i.     Metrics Development. Defining privacy metrics, including benchmarks.
ii.    Information Collection. Identifying or confirming data sources and ensuring that they are accurate and consistent.
iii.   Report Development. Developing procedures to record and analyze the metric information; providing suitable formats for both detailed and summary reports.
iv.    Implementation. Adopting the procedures and reporting requirements.

5. In designing metrics, consider the organization’s core activities as well as its privacy-related expectations or objectives. What may be important for one organization may not be of sufficient interest to another and, accordingly, privacy metrics may differ between organizations.

6. Determine your “benchmarks”.  Benchmarks provide “comparators” against which performance is measured and can be positive or negative.  Since organizational privacy requirements are often unique, internal benchmarks evolve over time.  If there is a “high” benchmark, a “low” number in the early periods of information collection may not reflect a “bad” metric. This simply indicates the need to re-adjust the benchmark to what, over time, is recognized as the historical norm.  In some instances, there may be no benchmark or no desire to have a benchmark.

7. Carefully consider how you will build a metrics report. Think about the degree of difficulty in sourcing the information to populate your metrics report. Also think about reporting in terms of “frequency”, “collection” and “reporting”. “Frequency” reflects the time period through which the metric is be measured, e.g. monthly, quarterly or annually. “Collection” may vary from “reporting” in that metrics may be collected monthly for privacy management purposes but compiled quarterly into senior management reports.

8. Monitor your privacy metrics program and assess the relative maturity of the metrics. You may find that, over time, one or more metrics may “fall away” in importance because of the “smooth running” of that activity. Revise your privacy metrics program as appropriate.

9. Whatever metrics you choose, don’t advertise them. Why? Think in terms of the observer effect where people change their behavior when aware of being watched. Publishing the metrics makes employees more aware of their actions and, invariably, will begin to influence reporting.

10. A common adage in business is “ you can’t manage what you don’t measure”. Performance measurement is considered an important component of business management ( here is a section-by-section guide on the best business related plans) and, in an age of data breach notification, should cover privacy. Privacy metrics provide the key performance indicators for measurement of your organization’s privacy performance.

Some examples of privacy metrics discovered in the course of my travels are below. Please feel free to supplement the list by leaving a comment.

• Average privacy document “age”.
• Number of days between on-boarding and completion of basic Privacy and Security training.
• Number of privacy risks that are outstanding after allocated mitigation period.
• Number of Privacy Assessments Completed (preliminary, full, “delta” assessments).
• Number of incidents, by origin, by organizational unit, by project, by severity level.
• Mean time to initiate response to an incident.
• Mean time to complete response to an incident.
• Percentage of organizational budget dedicated to privacy.
• Percentage of privacy personnel with recognized privacy certifications.
• Percentage of staff receiving privacy training.
• Average cost of an incident.
• Percentage of “high-sensitivity” solutions with encryption, anonymization, or pseudonymization capabilities.
• Percentage of “high-sensitivity” solutions with monitored audit trails.