Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions

/ 32Hong,Shin @ PSWLAB

Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions

by D. Engler, B. Chelf, A. Chou, S. Hallem

published in OSDI 2000

Hong,Shin

23年 4月 21日

1Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions


Contents• Introduction• Meta-level Compilation• Checking Assertion Side-effect• Temporal Ordering• Enforcing Rules Globally• Linux Mutual Exclusion• Conclusion

23年 4月 21日


2


Introduction 1/4• System software must obey many rules.– e.g. “check user permission before modifying

kernel data structures”– A code that does not obey these rules may crash

the system.

• There are several methods to find violations of system rules.– Model checking– Testing– Manual inspection

23年 4月 21日


3


Introduction 2/4• Model checking– Pros: rigorous checking– Cons: models are difficult and costly to construct

• Testing– Pros: working with an actual code– Cons: not scalable, finding the cause of a test

failure can be difficult

• Manual inspection– Pros: easy adapt to ad hoc coding conventions and

system rules.– Cons: impossible for complex codes, reliability of

manual inspection is erratic.

23年 4月 21日


4


Introduction 3/4• One possible alternative is to use static

compiler analysis to find rule violations.– Models are not needed.– Static analysis can examine much more paths than

testing.– Reduces the need to construct numerous test

cases

Compilers can be used to enforce system rules because many rules have a straightforward mapping to program source.

23年 4月 21日


5


Introduction 4/4

Meta-level compilation– Extends compilers with lightweight, system-specific

checkers.– System implementers write extensions in a high-

level state-machine language, metal.– Using metal, system implementers can specify

system rules as code patterns.

– The written extensions are dynamically linked into an extensible compiler, xg++.

– The extended compiler detects any rule violation in an input code by matching the code to specified patterns.

23年 4月 21日


6


Meta-level Compilation1/6• Many system rules describe legal orderings of

operations or specific contexts where these operations can or cannot occur.

• A meta-level compiler extension can check these rules by searching for the corresponding operations and checking that codes obey the given ordering or contextual restrictions.

• Metal– Compiler extensions are written in a high-level, state-

machine language, metal.– xg++ compiler translates each input function into its internal

representation, the extensions are applied down every possible execution path in that function.

23年 4月 21日


7


Meta-level Compilation2/6Rule templates• “Never/always do X”

e.g. Never use floating point in kernel.• “Always do X before/after Y”

e.g. Always check that user given pointers are not null before using

in kernel.• “Never do X before/after Y”

e.g. Never acquire a lock twice.• “In situation X, do Y”

e.g. While interrupts are disabled, do not call functions that can sleep.

• “In situation X, do Y rather than Z”e.g. If code does not share data with interrupt handlers, then

use spin locks rather than interrupt disabling.

23年 4月 21日


8


Meta-level Compilation3/6Ex. a metal state-machine to detect (1) when interrupts

disabled using cli() are not re-enabled using either sti() or restore_flags() and (2) duplicate enable/disable calls.

23年 4月 21日


9

{# inlcude “linux-includes.h” }sm check_interrupts { decl {unsigned} flags ; pat enable = { sti() ; } | {restore_flags(flags) ; } ; pat disable = { cli() ; } ;

is_enabled : disable ==> is_disabled | enable ==> {err(“double enable”);}; is_disabled: enable ==> is_enabled | disable ==> {err(“double disable”);}; | $end_of_path$ ==> {err(“exiting w/intr disabled!”);} ;}

Variables used in patterns

Patterns

States


Meta-level Compilation4/6

23年 4月 21日


10

is_enable

d

is_disable

derror

enable

disable

enable/”double enable”

disable/”double disable”

end of path/“invalid exit”


Meta-level Compilation5/6

23年 4月 21日


11


Meta-level Compilation6/6• Metal can specify whether a rule should be applied

either down all paths (flow sensitive) or linearly through the code (flow insensitive).

• Caching is used to prune redundant code paths where state-machine instances follow code paths that reach the same state.– The system represents the state of an state-machine as a

vector holding the value of its variables. – For each node in the input flow-graph, it records the set of

states where it has been visited.– If an state-machine arrives at a node in the same state as a

previous instance, the system prunes it.

23年 4月 21日


12


Checking Assertion Side-effects1/2• We can find incorrect uses of C assert macros using

meta –level compilation.• Assertions should not have non-debugging effects.

If an assert condition has important side-effects, these will disappear when the assertion is removed and the program will behave incorrectly.

• If an assertion expression has any assignment operations or function invocations, the assertion may have side-effects.We can write a metal checker that inspects

assertion expressions for side-effects.In Xok’s ExOS library OS, the extension found 16

violations in 199 assertions.

23年 4月 21日


13


Checking Assertion Side-effects2/2{ #include <assert.h> }

sm Assert flow_insensitive {

decl { any } expr, x, y, z ;

decl { any_call } any_fcall ;

decl { any_args } args ;

start:

{ assert(expr); } ==>

{mgk_expr_recurse(expr, in_assert);} ;

in_assert:

{any_fcall(args)} ==> {err(“func call”);}

| {x = y} ==> {err(“assignment”);}

| {z++} ==> {err(“post-increment”);}

| {z--} ==> {err(“post-decrement”);} ;

}23年 4月 21日


14

apply the extension linearly over input functions

metal provides a set of generic types for matching different classes of types

a metal procedure call to apply the state-machine to the expression in expr in the in_assert state


Temporal Orderings1/8• Many system operations must (or must not)

happen in sequence.• This constraints are well-suited for compiler

checking since sequences of operations are encoded as literal procedure calls in a code.

- Checking copyin/copyout- Checking memory management

23年 4月 21日


15


Temporal Ordering 2/8

Checking copyin/copyout• Most operating system guard against application

corruption of kernel memory by using special routines to check system call input pointers and to move data between user and kernel space.

• A meta-compilation extension can find errors in such code by finding paths where an application pointer is used before passing through the checking routines.

23年 4月 21日


16


Temporal Ordering 3/8• At each system call definition, the extension uses a

special pattern to find every pointer parameter, which it binds to a tainted state.

• The only legal operations on a tainted variable are being (1) killed by an assignment or (2) passed as an argument to functions expecting tainted input (e.g. kprintf).

• A tailored version of this checker for Xok exokernel code found 18 errors from 187 distinct user pointers in the exokernel.

23年 4月 21日


17



Checking memory management• An extension checks four common rules to check

memory management:(1) Since memory allocation can fail, kernel code must check whether the returned pointer is not null before using it.(2) Memory cannot be used after it has been freed.(3) Paths that allocate memory and then abort with an error should typically deallocate this memory before returning.(4) The size of allocated memory cannot be less than the size of the object the assigned pointer holds.

23年 4月 21日


18


Temporal Ordering 5/8• We can write metal state-machine to check the

memory management conditions.– Pointers to allocated storage can be in exactly one of four

states: unknown, null, not_null, freed, or okay.– A variable is bound to the unknown state at every allocation

site. – When an unknown variable is compared to null, the extension

sets the variable’s state on the null path to null and on the non-null path to not_null.

– Pointers passed to free transition to the freed state.– The checker only allows dereferences of pointers in not_null

state.

• This extension found 132 errors in Linux.

23年 4月 21日


19


Temporal Ordering 6/8sm null_checker {

decl {scalar} sz;

decl {const int} retv;

decl {any_ptr} v1;

state decl {any_ptr} v;

start, v.all:

{((v=(any)malloc(sz))==0)} ==>

true=v.null, false=v.not_null

| {((v=(any)malloc(sz))!=0)} ==>

true=v.not_null, false=v.null;

| {v=(any)malloc(sz)} ==> v.unknown ;

23年 4月 21日


20


Temporal Ordering 7/8 v.unknown, v.null, v.not_null:

{(v==0)} ==> true=v.null, false=v.not_null

| {(v!=0)} ==> true=v.not_null, true=v.null ;

v.unknown, v.not_null: {return retv;} ==>

{if (mgk_int_cst(retv) <0) err(“Error path

leak!”);} ;

v.null, v.unknown: {*(any *)v} ==>

{err(“Using ptr illegally!”);};

v.unknown, v.null, v.not_null: {free(v);} ==> v.freed;

v.freed: {free(v)} ==> {err(“Dup free!”);}

| {v} ==> {err(“Use after-free!”);} ;

v.all: {v = v1 } ==> v.ok ;

}23年 4月 21日


21



23年 4月 21日


22


Enforcing Rules Globally1/6• The extensions described thus far have been

implemented as local analyses.

• Many system rules are context dependent and apply globally across functions in a given call chain.

• Following two Linux rules are checked by global analysis: - Kernel code cannot call blocking functions with

interrupts disabled or while holding a spin lock. - A dynamically loaded kernel module cannot call

blocking functions until the module’s reference count has been properly set.

23年 4月 21日


23


Enforcing Rules Globally2/6Computing blocking routines• We build a list of possibly blocking functions in three

passes.(1) Make a list of blocking functions manually.

e.g. kernel memory allocators called without the GFP_ATOMIC flag

, routines to move data to or from user space.(2) The metal extension marks a function as a potentially

blocking function if the function directly calls blocking functions in the list.

(3) Make a global call graph for the entire kernel and perform a depth first traversal over this call graph calculating which routines have any path to a potentially blocking function.

23年 4月 21日


24


Enforcing Rules Globally3/6Checking blocking deadlock• A metal extension can check both rules by assuming

each routine starts in an enabled state with interrupts enabled and no locks held.

• As it traverses each global code path, if it hits a statement that disables interrupt, it goes to a disabled state.

• If it hits a potentially blocking function in a disabled state, it reports the global code path as an error.

• The extension found real 79 deadlock errors in Linux.

23年 4月 21日


25


Enforcing Rules Globally4/6

23年 4月 21日


26

Disable interrupt

Block function


Enforcing Rules Globally5/6Checking module reference counts• Linux allows kernel subsystems to be dynamically

loaded and unloaded.• A module has its reference count tracking the number

of kernel subsystems using the module.– A module increment s its reference count during loading (by

MOD_INC_USE_COUNT) – decrements it during unloading (by MOD_DEC_USE_COUNT).

• A module must protect against being unloaded while sleeping by incrementing its reference count before calling a blocking function.

• An extension can check for load race condition by tracking if a potentially blocking function has been called and flagging subsequent MOD_INC_USE_COUNT.

23年 4月 21日


27


Enforcing Rules Globally6/6

23年 4月 21日


28

Block function

client must be deallocated!


Linux Mutual Exclusion1/2• The checks for Linux locking conventions are

important to avoid deadlock in kernel.• Each kernel function must satisfy following conditions:

(1) All locks acquired within the function body are released before exiting.(2) No execution paths attempt to lock or unlock the same lock twice.(3) Upon exiting, interrupts are either enabled or restored to their initial state.(4) The “bottom halves” of interrupt handlers are not disabled upon exiting.(5) Interrupt flags are saved before they are restored.

• We can write metal state-machine to check these conditions.

23年 4月 21日


29


Linux Mutual Exclusion2/2

23年 4月 21日


30

&s->lock is not released.

hold &s->lock


Conclusion 1/1• Systems are pervaded with restrictions of what

actions programmers must always or never perform, how they must order events, and which actions are legal in a given context.

• Programmers make mistakes, and often have only an approximate understanding of important system restrictions.

• Meta-level compilation makes it easy for implementers to extend compilers with lightweight system-specific checkers.

• The authors demonstrated meta-level compilation’s power by check real, heavily-used, and test systems.

23年 4月 21日


31


References[1] Checking System Rules Using System-

Specific, Programmer-Written Compiler Extensions by D. Engler et al, OSDI 2000

23年 4月 21日


32

Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions

Documents

Transcript of Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions