Did you accept the risk? Dynamic risk metrics in your environment.

Did you accept the risk? Dynamic risk metrics in your environment. SnykCon 2020

a.k.a. The Safety Dance You can dance if you want to—but you can’t leave risk behind. SnykCon 2020

Your Safety Dancers for today Andrew Krug Daniel Maher

The dance card – Risk management (a classic!) – Real-world risk analysis (explosions!) – Risk in the context information technology (cyber!) – Virtual-world risk analysis (routing tables!) – How to shift-left to avoid specific risks ( !) – Walk through a sample application (dance party!)

Crash Course in Risk Risk 101

What is risk? What is safety science? Classic calculation R=f(s,p,c) Risk = f (scenario, probability, consequence)

What is risk? What is safety science? Kaplan and Garrick (1989) What can go wrong? What is the likelihood? (probability) How bad could it be?

Qualitative vs Quantitative Qualitative reasoning ranks likelihood using a scale score metric. 👍 👎 Speedy Light data gathering? Easy Low precision Light data gathering!

Qualitative vs Quantitative Quantitative reasoning uses data to reason about probabilities and consequences. 👍 👎 Accuracy Time Data Relative risks (adjacent)

Hybrid Models T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832019302649?via=ihub. [Accessed: 15-Oct-2020].

What is risk? What is safety science? Hybrid calculation R=f(s,p,c,k) Risk = f(scenario, probability, consequence, knowledge) knowledge = (mix of both quantitative and qualitative data)

Risk analysis in the real world

Real-world risk analysis Industrial Modeling T. Aven, “Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?,” Reliability Engineering & System Safety, 28-Sep-2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S095183201 9302649?via=ihub. [Accessed: 15-Oct-2020].

Real-world risk analysis Oil Drilling 0. Initiating Event “Drive-off” 5. Outcome 1. Stop drive-off No damage 2. Manual EDS Potential wellhead damage 3. Automatic EDS 4. Drive-off Some wellhead damage Wellhead damage

Real-world risk analysis Oil Drilling CONSEQUENCES 0. Initiating Event Drive-off CONTROLS 5. Outcome

Stop drive-off No damage
Manual EDS Potential wellhead damage
Automatic EDS
Drive-off Some wellhead damage Wellhead damage

Pictures = 1000x words S. E. Services, Inc., “Offshore Drilling Design & Analysis: Stress Engineering,” Offshore Drilling Analysis Testing, 2020. [Online]. Available: https://www.stress.com/capabilities/upstream/offshore-drilling [Accessed: 15-Oct-2020].

Bowtie Model (Hybrid) Purpose Developed to reason about known and unknown risks where the impact is extremely high (ex. loss of life). What’s new? Describes control impacts on risk over time.

Risk in operations, development, and cyber security

Risk as it relates to operations Risk Impact Likelihood The site is down No sales Maybe? Front page of the orange site Strong maybe because metrics? Wake. Up. EVERYONE. 97p based on uptime over the past two years…

Risk as it relates to development Risk Impact Likelihood Failed feature release Time to rollback Reputation Lost revenues How often? Dropped sessions Lost revenue Troubleshooting Bugfix How much test coverage? UI Bugs Reputation Bugfix How much frontend testing?

Risk as it relates to cyber security Risk Impact Likelihood Supply chain vulnerability Time to patch Firefighting Panic ? SQL injection Damage to reputation ?? XSS User data theft Fines Lawsuits ???

Parallels to oil drilling “[A]ccount for issues of human/machine interface, stress, shortage of time, and rare-event experience, which may lead the operator into failure and increase the probability of an accident.” — Nicola Paltrinieri (1996)

Examples of risk in the context of information technology

Risk in the age of tech Triforce of fear! DDoS Failed Deploy Certificate Issues Load Issues Reputation Information Security Tampering SQL Injection Impersonation Authentication Bypass Productivity Data Exfiltration SQL Injection XSS Financial

Risk in the age of tech Triad of consequences! Headline news? Brand damage Customer trust Reputation Information Security How long to restore? How much forensics? Time to rollback? Employee Safety Doxxing Productivity Fines! Legal damages Identity insurance Financial

Risk in the age of tech Less Damage More Damage 1 Low 2 Medium 3 High 4 Maximum Low 1 Medium 2 High 3 Maximum 4 Less Likely More Likely

Risk in the age of tech G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].

Risk in the age of tech, in tech itself G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].

Shifting-left in order to deal with specific risk

Where can we shift left? Or get some tooling maybe? ● Web Applications ○ Mostly vulnerabilities ● Miscellaneous Errors ○ Mostly misconfigs and misfires ● Everything Else ○ Mostly phishing G. Basset, S. Widup, P. Langlois, A. Pinto, and C. D. Hylender, “2020 Data Breach Investigations Report,” Verizon DBIR, 2020. [Online]. Available: https://enterprise.verizon.com/resources/reports/dbir/2020/summary-of-findings/. [Accessed: 15-Oct-2020].

Shifting-left with regards to risk Risk budgets for some? Tools Signal Visibility & Detection Signal Time-based risk

Shifting-left with regards to risk Risk budgets for most?

Qualitative risk = Time-based risk Comprehensive Visibility https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html

Shifting left with regards to risk What is your risk budget based on?

Shifting left with regards to risk Base your budget on your data!

It’s demo time!

Demo time! event_probability = ( recommendations * max_impact ) a.k.a the more things are wrong the more likely an incident will occur

Demo time!

Show some tools that add risk SEV-1 SEV-2 SEV-3 SEV-4 SEV-5 INFO WARN ERROR CRITICAL Low 1 Medium 2 LOW MED HIGH High 3 Maximum 4

Example : CFN_Nag

Dynamic Risk A method for calculating : Assign a weight to the number of findings Low 1 Medium 2 High 3 WARNING = 1 point CRITICAL = 20 points likelihood = ( points * 0.25 ) Since we’re using a 4 point scale “id”: “W41”, “type”: “WARN”, “message”: “S3 Bucket should have encryption option set”, Maximum 4

Ship that to an aggregator

R=f(s,p,c,k)

So in summary…

In summary… a.k.a. the tl;dr – The best risk assessments incorporate qualitative and quantitative metrics – Your environment is already giving you risk signals – Risk is a tool that software development teams can add to their toolbox in order to help the business go fast and stay safe. – It’s not always wrong to accept SOME risk depending on your risk budget.

In summary… Keep the party going! https://infosec.mozilla.org/guidelines/risk/rapid_risk_assessment.html We’re going to have a free risk assessment training with one of the creators (@kangsterizer) of Mozilla RRA. Email or follow us on Twitter for more info! andrew.krug@datadog.com // @andrewkrug daniel.maher@datadog.com // @phrawzty