1

Model Checking One Million Lines of C Code

Hao ChenDrew Dean (SRI International)

David Wagnerwith

David Schultz, Geoff Morrison, Ben SchwarzJacob West, Jeremy Lin

2

Outline

• Overview of MOPS• Engineering efforts

Make MOPS work on lots of real world applications

• ExperienceSecurity properties and findings

3

MOPS (MOdel checking Programs for Security properties)

• A static analysis tool that checks source programs for temporal safety properties

• Main features– Pushdown model checking– Inter-procedural analysis– Control flow centric

4

Algorithm: Pushdown Model Checking

: set of security operations– e.g. = { seteuid(!0), seteuid(0), execl() }

• B*: sequences of security operations that violate the property– B is a regular language– The user describes B by an FSA

• T*: sequences of security operationsfrom feasible traces in the program– T is a context-free language– MOPS automatically builds T from the program

• Question: Is B T = ?

T

B

5

The MOPS Process

Parser ModelChecker

Program

SafetyProperty

CFG

FSA Program OK

Error Traces(they may violate the property)

FSA: Finite State AutomatonCFG: Control Flow Graph

6

Integrate MOPS into the Build Process

• Goal: build a CFG for each target executable in the source package

• Low-level programs– mops_cc1: parse each foo.c into foo.cfg– mops_ld: link all *.cfg together

• How to use these programs to build CFGs for each package conveniently?

7

First Attempt: Edit Makefile

• Feasible for packages with simple Makefiles– OpenSSH

• Problems– Some Makefiles are automatically generated –

they will be overwritten when the package is rebuilt

– Too many Makefiles to edit – some packages generate one Makefile in each sub-directory

– Some Makefiles are very complicated

8

Second Attempt: Interpose on the Build Process to Generate CFGs

• Set the env variable GCC_EXEC_PREFIXto let gcc use– mops_cc1 for cc1– mops_ld for ld

• Therefore– Each source file foo.c is parsed into a CFG file

foo.o– Each executable file a.out contains a linked CFG– Note that machine code is not generated

9

Second Attempt: Advantages and Problems

• Advantage: works even if the build process– Moves files into other directories (mv)– Renames files (mv)– Creates libraries (ar)

• Problem– Sometimes machine code is needed

• By autoconf to test the functionality of the compiler• By the build process to generate C header files

– But we let gcc create CFGs instead of machine code

10

Third AttemptBuild Both Machine Code and CFGs

• Solution– Build both machine code and CFGs. For each

foo.c, build both foo.o and foo.cfg

• Problem– Machine code and its corresponding CFG may be

separated if the build process• Moves and renames files (mv)

• Creates and uses libraries (ar)

– We need to “bind” machine code with its CFG

11

Final Attempt

• Place both machine code and its CFG in the same object file– The ELF object format allows user-defined

sections– For each source file foo.c

• gcc’s cc1 generates machine code foo.o

• mops_cc1 generates CFG foo.cfg

• Run objcopy to place foo.cfg into foo.o

12

Now, as Easy as This

• mops –m property.mfsa -- gcc foo.c bar.c

• mops –b rpm –m property.mfsa -- rh9/*.src.rpm

13

Experiment: Checking Security Properties on Large Packages

• Properties– Drop privilege before making unsafe system calls– Avoid race condition in file system access– Create root-jail safely– Create temporary files safely– Avoid stderr vulnerability

• Programs– 7 programs, total one millions LOC

14

Property: Drop Privilege before Executing Untrusted Programs

• Setuid-root programs need to drop root privilege before executing untrusted programs– exec...(), system(), popen()

• Otherwise, the untrusted program will run with the root privilege

15

Property: Avoid Race Condition in File System Access

• It is suspicious to make two system calls in a program path that use the same filename

• E.g. – access(f) followed by open(f)– stat(f) followed by create(f)

• Vulnerability– Symbolic links

16

Property: Create root jail securely

• Rule: chroot() must be followed by chdir() immediately

• A vulnerable program

• The adversary sends in ../../etc/password from the network

// cwd==/var/ftpchroot(“/var/ftp/pub”);filename = read_from_network();fd = open(filename, O_RDONLY);

17

Property: Avoid Race Condition in Creating Temporary Files

• victim.c:filename = mktemp(template);fd = open(filename, …);

• But an adversary can create a file with the same name between the two statements

• Then, victim.c will– Open the adversary’s file, or– Fail to create the temporary file (with the O_EXCL

flag)

18

Rules for Avoiding Race Condition in Creating Temporary Files

• Never use the unsafe functions:mktemp(), tmpnam(), tempname(), tmpfile()

• Never reuse the parameter f in mkstemp(f) in any other function call, such as stat(), chmod(), etc– Because the same name may refer to a different

file (tmpwatch)

19

Property: Avoid stderr Vulnerability

int main(){ // fd 0 and 1 are open, but fd 2 is closed … f = open(“/etc/passwd”, O_RDWR); // now /etc/passwd is opened to fd 2 fprintf(stderr, “%s”, user_input); // oops, have written to /etc/passwd}

20

Rule: Avoid stderr Vulnerability

• Property– The first three file descriptors should not be

opened by any file except /dev/null and /dev/zero

21

Property: stderr vulnerability

Package LOC Running Time

# Error Traces

Real Bugs Total

Sendmail 221K 13m32s 0 2

Postfix 97K 58.2s 0 2

OpenSSH 58K 3m1s 3 7

Apache 240K 53.0s 2 2

BIND 280K 3m13s 1 1

At 5.1K 7.7s 2 2

Cron 3.7K 5.0s 2 2

22

Property: create temporary files safely

Package LOC Running Time

# Error Traces

Real Bugs Total

Sendmail 221K 43.9s 0 0

Postfix 97K 28.8s 0 0

OpenSSH 58K 49.0s 0 1

Apache 240K 47.4s 1 1

BIND 280K 67.3s 1 1

At 5.1K 5.3s 0 0

Cron 3.7K 4.4s 0 0

23

Experiment: Build CFGs for All RPM Packages in Redhat 9.0 Linux

• Number of packages: 840• Time: 3 days 14 hours on 1.5G Pentium• Preliminary results:

– Successful: 514 packages– Failed: 326 packages

• Reasons for failure– Lack of prerequisite RPM packages

(the major reason)– Bugs/limitations in MOPS’s parser– The package was not written in C

24

Summary

• Engineering efforts in making MOPS work on lots of real world applications

• Experience in checking security properties on large applications