KCAL: Kernel-support Cost-effective Audit Logging for ... · KCAL: Kernel-support Cost-effective...
Transcript of KCAL: Kernel-support Cost-effective Audit Logging for ... · KCAL: Kernel-support Cost-effective...
KCAL: Kernel-support Cost-effective Audit Logging for Causality Tracking
Shiqing Ma, Juan Zhai, Yonghwi Kwon, Kyu Hyung Lee, Xiangyu Zhang, Gabriela Ciocarlie, Ashish Gehani, Vinod Yegneswaran, Dongyan Xu, Somesh Jha
Background: Forensics Analysis with Audit Log
1. ….......2. PID=1224, Receives from socket03. PID=1224, Writes to File Taskman4. ….......5. PID=4893, Starts from File Taskman6. PID=4893, Reads file FD7. PID=4893, Sends data to socket18. …....... socket1
4893
TaskmanFD
1224socket0
Linux Audit Framework
• Integrated with many other tools, e.g., Orchids, Prelude siem• Kernel component is shipped with mainline kernel
Most Popular Audit Framework on Linux
• Linux kernel component + user-space tools (e.g., Auditd, go-audit)• Logging all system calls, monitoring file accesses etc.
The Linux Audit Framework
Problems: 1) Slow. 2) Large log files.
Space Overhead: 2 to 40 GB/day
0
200
400
600
800
1000
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
LOG
SIZE
(GB)
TIME (DAY)
Max Avg(Server)Avg(Client) Min
auditd
KernelNetlink
Syscall
User
Exit
Task
Exclude
Application
Disk I/O for logging: 50% of the total
overhead due to large #events
Netlinke events transmission: 45% of the total overhead caused by queueing etc.
Kernel event filtering: 5% of the total overhead
Large log file size:2 to 40 GB/day
Linux Audit System Overhead (Comparing with Linux with Audit disabled)
Overhead caused by the Audit system (generating / transmitting /
logging events), measured by application
benchmarks.
Problem: # Logged events is large
# logged events à large log files
# logged events à filter/transfer/writing
Question: is reducing the # log events possible?
Large files are hard to
investigate.
High RT overhead
makes system not useable.
Log Redundancy
• Audit log is redundant for forensics analysis
• Redundant events: event that represent the same dependency relationships (from the forensics point of view)
1. PID=422, Event = Read (FD4)
2. PID=422, Event = Read (FD4)
3. PID=422, Event = Read (FD4)
4. PID=422, Event = Read (FD4)
5. PID=442, Event = Write(FD5)
6. PID=442, Event = Read (FD4)
7. PID=442, Event = Write(FD5)
8. PID=442, Event = Write(FD5)
9. PID=442, Event = Write(FD5)
KCAL: Do not generate redundant events.
KernelShared
Memory Log File
Modified KAudit
auditd
FileEventsCache
ProcessDependence
Cache
Unit Dep Cache
Application
Performs online log reduction
Modified data structure to help log reduction
Replace Netlink with shared memory
Execution-unit applications
Data Channel: Shared Memory
0
1000
2000
3000
4000
5000
6000
512 1024 2048 4096
TIM
E (C
PU C
YCLE
S)
Size per messsage
NetlinkMessage QueueShared Memory
Dependence Analysis: Execution-unit Based
• Dependence analysis is hard in forensics analysis• Dependence explosion
• Execution-unit• Part of a process execution
• More details, see• BEEP (NDSS’13)• MPI (USENIX Security’17, Distinguished
Paper Award)
File
Firefox
File
Log Reduction: In-unit Redundancy
• The same operation(s) on the same object within the same unit
• Case: Vim loading a file
• Application logic or limited buffer size
1. PID=442, Event=UNIT_ENTER
2. PID=422, Event=Read (FD4)
3. PID=422, Event=Read (FD4)
4. PID=422, Event=Read (FD4)
5. PID=422, Event=Read (FD4)
6. PID=422, Event=Read (FD4)
7. PID=422, Event=Read (FD4)
8. PID=422, Event=Read (FD4)
9. PID=422, Event=Read (FD4)
10.PID=442, Event=UNIT_EXIT
while(content=read(fd) > 0) {
insert_content(root, content)
}
Log Reduction: Cross-unit Redundancy
• Different units are doing the same thing
• Case: repeated operations• Vim save file
• Tracking to one of them is sufficient to build the graph
1. PID=442, Event=UNIT_ENTER
2. PID=422, Event=Read (FD4)
3. PID=422, Event=Read (FD4)
4. PID=422, Event=Write(FD5)
5. PID=442, Event=UNIT_EXIT
6. PID=442, Event=UNIT_ENTER
7. PID=422, Event=Read (FD4)
8. PID=422, Event=Read (FD4)
9. PID=422, Event=Write(FD5)
10.PID=442, Event=UNIT_EXIT
Log Reduction: Temporary Files
• Temporary file• Created/operated/deleted by the same
process
• Case: web resources• If not stored explicitly, most will be
deleted later
• Files that are too large for memory
1. PID=442, Event=UNIT_ENTER
2. PID=422, Event=NewFD(FD5)
3. PID=422, Event=Write(FD5)
4. PID=442, Event=UNIT_EXIT
5. PID=442, Event=UNIT_ENTER
6. PID=422, Event=Write(FD5)
7. PID=442, Event=UNIT_EXIT
8. PID=442, Event=UNIT_ENTER
9. PID=422, Event=Delete(FD5)
10.PID=442, Event=UNIT_EXIT
KCAL: Log Redundancy Detection Components
KernelShared
Memory
Modified KAudit
FileEventsCache
ProcessDependence
Cache
Unit Dep Cache
In-unit redundancy
detection
Cross-unit redundancy
detection
Temporary file redundancy detection
Application
Idea: Dependency relationships are cached.
Redundancy detection logic
KCAL-Space Overhead: <4GB/day
0
20
40
60
80
100
120
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
LOG
SIZE
(GB)
TIME (DAY)
Max Avg(Server)Avg(Client) Min
KCAL: Redundancy AnalysisIn-Unit
RedundencyCross-Unit
RedundancyTemporary
FilesKCAL
EventsMachine 1 69% 7% 16% 8%
Machine 2 71% 10% 9% 10%
Machine 3 56% 10% 21% 13%
Machine 4 21% 46% 24% 9%
Machine 5 29% 43% 13% 14%
Avg. 51% 21% 16% 11%
More evaluation results in paper !
Discussion
• KCAL is for forensics analysis only• Linux Audit framework is a general audit framework
• KCAL requires instrumented applications to support execution-unit based log reduction• Leveraging other online log reduction algorithms is also possible
• KCAL modifies the kernel source code• Porting from 3.19 to 3.2 requires an addition 8 line patch
Summary
• KCAL: Kernel-support Cost-effective Audit Logging for Causality Tracking• Based on the Linux Audit Framework• For forensics analysis• Low runtime, space overhead• Reduce redundant events before generating them
• Reduces overhead caused by transferring, writing, storing redundant events