Data Privacy Engineering with Google Tag Manager Server Side and Consent Mode

ePrivacy

The ePrivacy Directive is about accessing or recording data on your computer (or "terminal equipment"). It will eventually be replaced by the ePrivacy Regulation, the current draft of which allows exemptions for statistical uses, but it is not EU law yet. If it does get ratified then a lot of cookie consents may be unnecessary. Until then the text to rely on is Article 5(3).

From feedback I’ve got, ePrivacy compliance is the least clear as it changes from country to country and is currently under active review pending replacement, so interpretations of it are most varied. Cookie consent banners are now well established, but some privacy hawks consider work arounds like using localStorage instead of cookies valid, whilst privacy doves consider any information read or written in a persons browser is in need of a consent, including for example a browser’s user-agent.

ITP / Browser based data protections

ITP is the most rapidly changing in its reach and capabilities, but as its motivated by the richest company in the world’s market position (Apple) and not legislation, privacy hawks are more eager to circumnavigate around it, making it more like an arms race in measures and counter measures. ITP is a leader in the general trend of web browsers to respect privacy, with Firefox, Brave and even Chrome heading down the same path, and are arguably more powerful than governments in enforcing user privacy.

Apple’s moral authority in privacy is not helped whilst they are targets for complaints to do with violating the ePrivacy above, creating an advertisment ID (IDFA) for every iPhone on the planet with user consent.

Each iPhone runs on Apple’s iOS operating system. By default, iOS automatically generates a unique “IDFA” (short for Identifier for Advertisers) for each iPhone. Just like a license plate this unique string of numbers and characters allows Apple and other third parties to identify users across applications and even connect online and mobile behaviour (“cross device tracking”). Apple’s operating system creates the IDFA without user’s knowledge or consent. After its creation, Apple and third parties (e.g. applications providers and advertisers) can access the IDFA to track users’ behaviour, elaborate consumption preferences and provide personalised advertising. Such tracking is strictly regulated by the EU “Cookie Law” (Article 5(3) of the e-Privacy Directive) and requires the users’ informed and unambiguous consent.

GDPR

Feedback to me is that GDPR is usually the easiest to handle internally even if far reaching, since its well defined and fairly static in what it expects - you have to have issues such as being inflexible due to large scale or have IT problems to suffer from its consequences.

As I understand it, GDPR’s main purpose is for legal checks with the big internet companies such as Google, Facebook and Apple to protect EU citizens. The EU has horrors such as the holocaust to remember why protecting personal identity is important, vowing never again that personal identity can be used to discriminate against people. Remembering this is a useful principle when dealing with user privacy: leaking personal data is considered personal harm, and that harm has potential legal and ethical repercussions. It also features into privacy hawks calculus, since what the data is used for is important context: if you are using data to market medicine at children you will carry a much harsher penalty for wrong doing than if you use data to optimise your sales funnel.

GDPR Code examples

For GDPR, your main concern is to make sure that personal data is only collected with consent for it and its purpose, and that personal data is protected. There are different categories of data:

• personal information - also known as PII. Things that can be read to identify one person, such as emails, names, or addresses. It is best not to pass this unless absolutely necessary, as you will have to keep track of where it is stored so you can retrieve or delete it again later.
• pseudoanonymous - Use these instead if you can: its an ID that on its own gives no personal info, but once linked to other data can uniquely ID a user. UserIds, CookieIDs, IP addresses fall under here. GDPR encourages use of pseudoanonyminity as it means you can delete the personal data in one place, then destroy the psuedoanonymous keys to securely protect data, but you do need to protect that pseudoanonymous ID as you would personal data until that link is impossible.
• anonymous - Data that can not be used to identify (or re-identify) a user. This is useful as if you can make your data truly anonymous then GDPR does not apply. There is a great resource on what anonymisation you can do from the UK’s ICO that covers a lot of techniques.

===== GTM CLIENT
const claimRequest = require('claimRequest');
const getRequestPath = require('getRequestPath');
const getRequestQueryParameter = require('getRequestQueryParameter');
const getRequestHeader = require('getRequestHeader');
const returnResponse = require('returnResponse');
const runContainer = require('runContainer');
const getRemoteAddress = require('getRemoteAddress');

// get info from request
const requestPath = getRequestPath();
const queryParameters = getRequestQueryParameters();
const ua = getRequestHeader('user-agent');
const referer = getRequestHeader('referer');
const ip = getRemoteAddress();

// you send requests here instead of to client side endpoints
// e.g. gtm.markedmondson.me/custom-endpoint?custom1=my-special-id
if (requestPath == "/custom-endpoint")  {
  claimRequest();
  
  // construct the event to send to your server side services
  const event = {
    event_name: "CustomIDEvent",
    custom1: queryParameters.custom1,
    custom2: queryParameters.custom2,
    ...etc...
    
    ip_override: ip,
    user_agent: ua,
    referer: referer,
  };
  
  // runs the actual container, with a tag you specify below
  runContainer(event, () => returnResponse());
  
}

========GTM TAG
// a custom tag for a service could then look like:
const getAllEventData = require('getAllEventData');
const sendHttpGet = require('sendHttpGet');
const setResponseBody = require('setResponseBody');
const setResponseHeader = require('setResponseHeader');
const setResponseStatus = require('setResponseStatus');
const encodeUriComponent = require('encodeUriComponent');

const collectDomain = "https://www.api-service.com";
const event = getAllEventData();

var url = collectDomain + '/api-call?' +
    'custom1=' + encodeUriComponent(event.custom1) + '&' +
    'custom2=' + encodeUriComponent(event.custom2) + '&' +
    ...etc...

var headers = {};
headers["referer"] = event.referer;
headers["user-agent"] = event.user_agent;
headers["x-forwarded-for"] = event.ip_override;

sendHttpGet(url, (headers) => {
  setResponseHeader('cache-control', headers['cache-control']);
}, {headers: headers });

data.gtmOnSuccess();

const isRequestMpv1 = require('isRequestMpv1');
const extractEventsFromMpv1 = require('extractEventsFromMpv1');
const claimRequest = require('claimRequest');

// Check if request is Measurement Protocol
if (isRequestMpv1()) {
  claimRequest();
  const events = extractEventsFromMpv1();
  events.forEach((event, i) => {
  
    // if event is a page_view
    if(event.event_name === "page_view"){
      // we have suspected email address
      if(event.page_location.contains("@")){
        event.page_location = "[redacted]"
      }
    }
    
    ...etc...
  });
  
  
}

...etc...
// Get IP from incoming request
const ip = getRemoteAddress();

// anonymise the ip address yourself
ips = ip.split(".")
anon_ip = ips[1]+"."+ips[2]+"."+ips[3]+".000"

// Check if request is Measurement Protocol
if (isRequestMpv1()) {
    // Claim the request
    claimRequest();

    const events = extractEventsFromMpv1();
    const max = events.length - 1;
    events.forEach((event, i) => {

        // Overwrite IP and UA with correct values
        if (!event.ip_override && ip) event.ip_override = anon_ip;
        //.. etc..
    }
});

const claimRequest = require('claimRequest');
const getRemoteAddress = require('getRemoteAddress');
const getRequestHeader = require('getRequestHeader');
const sha256Sync = require('sha256Sync');

// Get User-Agent and IP from incoming request
const ua = getRequestHeader('user-agent');
const ip = getRemoteAddress();

// pick your own salt - can be anything
salt = 'dontknowwhattoputhere'

// Create a hashed ID from the IP and User Agent
const paid = sha256Sync(salt + ip + ua);
...etc...

ePrivacy Code Examples

Google Consent Mode is a new product which integrates with services such as cookie consent banners to turn off cookies for both Google Analytics and Google Ads. Google Consent Mode offers a way to block cookies but preserve at least an anonymous hit for statistical use.

<script>
window.dataLayer = window.dataLayer || [];
function gtag() { dataLayer.push(arguments); }

// these should be changed upon user consent/denied
gtag('consent', 'default', 
    { 'ad_storage': 'denied', 'analytics_storage': 'denied', 'wait_for_update': 500 });
gtag('set', 'url_passthrough', true);
</script>

When Consent Mode is active and set to denied cookies, GA hits still occur and request a cookie but the responses do not include them. This means that every hit will generate a new cookieId (cid). Its just launched so more features are coming, but the hints are that Google will use machine learning to estimate sessions and goals with this data, without having personal data. See this guide from Simo on the nuts and bolts.

The requests also add a gcs=G100 parameter which when set means no data shows up in your Google Analytics account (aside the real-time reports). This may be ok for the privacy doves, at least those who don’t object to data still being seen by Google, but the more privacy hawkish may consider this non-consented, but anonymous data as value to also push to their own Google Analytics.

If you modify the gcs parameter, you can get the data into your GA account, but with the proviso that it will be 1 user, 1 session for each pageview as the cid is changing each time. For this reason you may want to send the data to a different view or property. Since we can change the measurementId in the request and don’t have cookies to worry about, you can modify that directly via the x-ga-measurement_id parameter.

This sets up separate data streams for your opt-in and opt-out users:

Thanks to Gunnar Griese, my colleague at IIH Nordic for help with the below

...etc...
if (isRequestMpv1()) {
    // Claim the request
    claimRequest();

    const events = extractEventsFromMpv1();
    const max = events.length - 1;
    events.forEach((event, i) => {

        // Make unconsented hits appear in GA
        const consentMode = event['x-ga-gcs'];
        if (consentMode == "G100") {
          event['x-ga-gcs'] = "G111";
          // change the measurementId to one for receiving opt-out users 
          event['x-ga-measurement_id'] = "UA-12345-3"
        }
        ...etc...
    })

}

ITP Code Examples

ITP at the moment seems largely aimed at the cross-domain trackers that are building user profiles across many websites, such as display advertising networks. This means that measures are largely aimed at their techniques such as 3rd party cookies and CNAME cloaking, mostly by limiting cookie timeouts to 7 days. Privacy hawks can avoid being affected by those measures by using GTM Server Side to re-write cookies to HTTP (not affected by ITP).

The easiest method is to just activate the FPID in the default Google Analytics client in GTM, which migrates cookies over to HTTP. However, you may want to change the expiry period or have more control over these cookies, in which case see the script below taken from this post by Simo

const claimRequest = require('claimRequest');
const extractEventsFromMpv1 = require('extractEventsFromMpv1');
const getCookie = require('getCookieValues');
const getRemoteAddress = require('getRemoteAddress');
const getRequestHeader = require('getRequestHeader');
const isRequestMpv1 = require('isRequestMpv1');
const returnResponse = require('returnResponse');
const runContainer = require('runContainer');
const setCookie = require('setCookie');
const setPixelResponse = require('setPixelResponse');
const setResponseHeader = require('setResponseHeader');

// Get User-Agent and IP from incoming request
const ua = getRequestHeader('user-agent');
const ip = getRemoteAddress();

// Check if request is Measurement Protocol
if (isRequestMpv1()) {
  // Claim the request
  claimRequest();
  
  const events = extractEventsFromMpv1();
  const max = events.length - 1;
  events.forEach((event, i) => {
    // Unless the event had IP and user-agent overrides, manually
    // override them with the IP and user-agent from the request
    // That way the GA collect call will appear to have originated
    // from the user's browser / device.
    if(!event.ip_override && ip) event.ip_override = ip;
    if(!event.user_agent && ua) event.user_agent = ua;
    
    // Pass the event to a virtual container
    runContainer(event, () => {
      
      if (i === max) {
        
        // Rewrite the _ga cookie to avoid Safari expiration.
        const ga = getCookie('_ga');
        if (ga && ga.length) {
          setCookie('_ga', ga[0], {
            domain: 'auto',
            'max-age': 63072000,
            path: '/',
            secure: true,
            sameSite: 'lax'
          });
        }
        setPixelResponse();
        
        // Make sure no CORS errors pop up with the response
        const origin = getRequestHeader('Origin');
        if (origin) {
          setResponseHeader('Access-Control-Allow-Origin', origin);
          setResponseHeader('Access-Control-Allow-Credentials', 'true');
        }
        returnResponse();
      }
    });
  });
}

As another browser based measure, you can also self-host your analytics script to avoid some Ad Blockers blocking your tracking scripts. Soon you will be able to host the Google Tag Manager script as well.

// analytics.js
if (getRequestPath() === '/analytics.js') {
  claimRequest();
  
  getGoogleScript('ANALYTICS', (script, metadata) => {
  
    for (let header in metadata) {
      setResponseHeader(header, metadata[header]);
    }
    
    const origin = getRequestHeader('Origin');
    if (origin) {
      setResponseHeader('Access-Control-Allow-Origin', origin);
      setResponseHeader('Access-Control-Allow-Credentials', 'true');
    }
    
    setResponseHeader('content-type', 'text/javascript');
    setResponseBody(script);
    returnResponse();
  
  });
  
}