The Safety Dance Risk Assessment for DevOps Practitioners DevOpsDays São Paulo 2020

Your Safety Dancer Daniel Maher Twitter: @phrawzty Email: daniel.maher@datadoghq.com Web: www.dark.ca

Your Safety Dancers Daniel Maher ← that’s me! Andrew Krug ← shout out!

What we’re going to do today

Risk: A Crash Course

What is risk? What is safety science? Kaplan and Garrick (1989) What can go wrong? What is the likelihood? (probability) How bad could it be?

What is risk? What is safety science? Classic calculation R=f(s,p,c) Risk = f (scenario, probability, consequence)

Qualitative vs Quantitative Qualitative reasoning ranks likelihood using a scale score metric. 👍 👎 Speedy Light data gathering? Easy Low precision Light data gathering!

Qualitative vs Quantitative Quantitative reasoning uses data to reason about probabilities and consequences. 👍 👎 Accuracy Time Data Relative risks (adjacent)

Hybrid Models T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832019302649?via=ihub. [Accessed: 15-Oct-2020].

What is risk? What is safety science? Hybrid calculation R=f(s,p,c,k) Risk = f(scenario, probability, consequence, knowledge) knowledge = (mix of both quantitative and qualitative data)

The Knowledge Dimension (This is where you come in!)

Risk in the age of tech Triforce of fear! DDoS Failed Deploy Certificate Issues Load Issues Reputation Information Security Tampering SQL Injection Impersonation Authentication Bypass Productivity Data Exfiltration SQL Injection XSS Financial

Risk in the age of tech Triad of consequences! Headline news? Brand damage Customer trust Reputation Information Security How long to restore? How much forensics? Time to rollback? Employee Safety Doxxing Productivity Fines! Legal damages Identity insurance Financial

Risk in the age of tech What we know vs. what we feel Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely

Risk in the age of tech What we feel High 3 + Maximum 4

7?

Risk in the age of tech What we know 0.5 confidence 0.9 confidence 0.7 confidence High 3 Maximum 4 Hi/Max 3.5 1.5 + 3.6

5.1

Real risks in the real world

Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].

Risk in the age of tech, in all industries G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].

Risk in the age of tech, in tech itself G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].

Risk budgets (Hope for the best; plan for the worst)

You may have heard this song before…

What is your risk budget based on?

Base your budget on your data!

Risk budgets You already have the data… Tools Signal Visibility & Detection Signal Time-based risk

Risk budgets You just need to think about it differently

Qualitative risk = Time-based risk Comprehensive Visibility

Using your data to level up!

Rapid risk assessment event_probability = ( recommendations * max_impact ) a.k.a the more things are wrong the more likely an incident will occur

Use the common language and framework SEV-1 SEV-2 SEV-3 SEV-4 SEV-5 INFO WARN ERROR CRITICAL Low 1 Medium 2 LOW MED HIGH High 3 Maximum 4

Dynamic Risk A method for calculating : Assign a weight to the number of findings Low 1 Medium 2 High 3 WARNING = 1 point CRITICAL = 20 points likelihood = ( points * 0.25 ) Since we’re using a 4 point scale “id”: “W41”, “type”: “WARN”, “message”: “S3 Bucket should have encryption option set”, Maximum 4

Aggregate the data

Use the data

So in summary…

In summary… tl;dr – The best risk assessments balance both speed and accuracy, with a healthy dose of testable fact – Your environment is already giving you risk signals – You are an important piece of the security puzzle! – Risk budgets are a tool that organisations can add to their toolbox in order to help their business go fast and stay safe

In summary… Keep the party going! Mozilla Rapid Risk Assessment: https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html Rapid Risk Assessment Training https://www.youtube.com/watch?v=jxpuafW-H8U Better Reliability Through SLOs (Fique em Casa Conf 2020) https://www.youtube.com/watch?v=JOFYhFbrsK8 Verizon Data Breach Investigations Report 2020 https://enterprise.verizon.com/resources/reports/dbir/ @phrawzty | daniel.maher@datadog.com | www.dark.ca

Daniel Maher // @phrawzty // dark.ca And remember: We can dance—everything’s under control.