CONTROLLED CHAOSThe Inevitable Marriage of DevOps & Security

Kelly Shortridge (@swagitda_) S4x20

@swagitda_

Hi, I’m Kelly

2

@swagitda_

“Chaos isn’t a pit. Chaos is a ladder.”

― Petyr Baelish, Game of Thrones

3

@swagitda_

Software is eating the world. It’s on the amuse-bouche course in ICS.

4

@swagitda_

Infosec has a choice: marry DevOps or be rendered impotent & irrelevant

5

@swagitda_

Denying the future & the benefits of modern systems will only hurt ICS

6

@swagitda_

How should infosec control chaos & make a marriage to DevOps last?

7

@swagitda_

1. DevOps Dominion

2. The Metamorphosis

3. Time to D.I.E.

4. A Phoenix Rises

8

DevOps Dominion

@swagitda_

DevOps is not automation or “agile”

10

@swagitda_

DevOps is a mindset that unifies responsibility and accountability.

11

@swagitda_

Infosec can join DevOps or take a back seat to the future of systems

12

@swagitda_

Chaos & resilience is infosec’s future

13

What are DevOps’s priorities?

@swagitda_

Optimization of software delivery performance so tech delivers value

15

@swagitda_

Stability & speed don’t conflict –resilience & innovation are bffs

16

@swagitda_

Security drives stronger DevOps results. Now ICS security must evolve.

17

The Metamorphosis

@swagitda_

Partitioning of responsibility & accountability engenders conflict

19

@swagitda_

After this evolution, DevOps will be held accountable for security fixes

20

@swagitda_

What goals should infosec pursue in this evolution?

21

@swagitda_

And… why should infosec goals diverge from DevOps goals?

22

@swagitda_

Infosec has arguably failed, so “this is how we’ve always done it” is invalid

23

The Security of Chaos

@swagitda_

“Things will fail” naturally extends into “things will be pwned”

25

@swagitda_

Security failure is when security controls don’t operate as intended

26

@swagitda_

What are the principles of chaotic security engineering?

27

@swagitda_

1. Expect that security controls will fail & prepare accordingly

28

@swagitda_

2. Don’t try to avoid incidents – hone your ability to respond to them

29

@swagitda_

What are the benefits of the chaos / resilience approach?

30

@swagitda_

Benefits: lowers remediation costs & stress levels during real incidents

31

@swagitda_

Benefits: minimizes service disruption & improves confidence

32

@swagitda_

Benefits: creates feedback loops to foster understanding of systemic risk

33

@swagitda_

What other ways can infosec become more strategic?

34

Time to D.I.E.

@swagitda_

We need a model promoting qualitiesthat make systems more secure

36

@swagitda_

Enter the D.I.E. model: Distributed, Immutable, Ephemeral

37

@swagitda_

Distributed: multiple systems supporting the same overarching goal

38

@swagitda_

Distributed infrastructure reduces risk of DoS attacks by design

39

@swagitda_

Immutable: infrastructure that doesn’t change after it’s deployed

40

@swagitda_

Servers are now disposable “cattle” rather than cherished “pets”

41

@swagitda_

Immutable infra is more secure by design – ban shell access entirely

42

@swagitda_

Unlimited lives is better for security than game over upon death

43

@swagitda_

Ephemeral: infrastructure with a very short lifespan (dies after a task)

44

@swagitda_

Ephemerality creates uncertainty for attackers (persistence = nightmare)

45

@swagitda_

Installing a rootkit on a resource that dies in minutes is a waste of effort

46

@swagitda_

ICS attacks take months to plan; ephemerality constantly disrupts it

47

@swagitda_

Optimizing for D.I.E. reduces risk by design & supports resilience

48

A Phoenix Rises

@swagitda_

Harness failure as a tool to help you prepare for the inevitable

50

@swagitda_

Game days: practice risky scenarios

51

@swagitda_

Prioritize game days based on potential business impacts

52

@swagitda_

Decision trees: start at target asset, work back to easiest attacker paths

53

@swagitda_

Determine the attacker’s least-cost path (hint: it doesn’t involve 0day)

54

Architecting chaos

@swagitda_

Begin with “dumb” testing before moving to “fancy” testing

56

@swagitda_

Think digital twins, analytics services, or O365… not field-level SCADA

57

@swagitda_

Controlling Chaos: Distributed

58

@swagitda_

Distributed mostly overlaps with availability in modern infra contexts

59

@swagitda_

Chaos Monkey: inject random instances failures to test resilience

60

@swagitda_

Infosec teams can use these tools but make attackers the source of failure

61

@swagitda_

Multi-region services present a fun opportunity to mess with attackers

62

@swagitda_

Shuffle IP blocks regularly to change attackers’ lateral movement game

63

@swagitda_

Controlling Chaos: Immutable

64

@swagitda_

Volatile environments with continually moving parts raise the cost of attack

65

@swagitda_

Create rules like, “If there’s ever a write to disk, crash the node”

66

@swagitda_

Attackers must stay in-memory, which hopefully makes them cry

67

@swagitda_

Metasploit Meterpreter + webshell:Touch passwords.txt & kaboom

68

@swagitda_

Infosec teams can build Docker images with a “bamboozle layer”

69

@swagitda_

Mark garbage files as “unreadable” to craft enticing bait for attackers

70

@swagitda_

Potential goal: self-healing edge devices with immediate reversion

71

@swagitda_

Test: inject attempts at writing to disk to ensure detection & reversion

72

@swagitda_

Controlling Chaos: Ephemeral

73

@swagitda_

Most infosec bugs are stated-related – get rid of state, get rid of bugs

74

@swagitda_

Reverse uptime: longer host uptime adds greater security risk

75

@swagitda_

Test: retrograde libraries, containers, other resources in CI/CD pipelines

76

@swagitda_

Leverage lessons from toll fraud –cloud billing becomes security signal

77

@swagitda_

Test: exfil TBs or run a cryptominerto inform billing spike detection

78

Conclusion

@swagitda_

Security cannot gatekeep DevOps. It must marry it.

80

@swagitda_

Chaos/resilience are natural homes for infosec & represent its future.

81

@swagitda_

Infosec must now evolve to unify responsibility & accountability.

82

@swagitda_

ICS is already cloudy – get ready now before OT migrates as well.

83

@swagitda_

Giving up control isn’t a harbinger of doom. Resilience is a beacon of hope.

84

@swagitda_

“You must have chaos within you to give birth to a dancing star.”

― Friedrich Nietzsche

85

@swagitda_

@swagitda_

/in/kellyshortridge

kelly@greywire.net

86