The Difference between Track and Testing Performance
-
Upload
frisksoftware -
Category
Travel
-
view
577 -
download
0
description
Transcript of The Difference between Track and Testing Performance
![Page 1: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/1.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
The difference between track and testing performance
Roel Schouwenberg, Senior Anti-Virus ResearcherKaspersky Lab [email protected]
![Page 2: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/2.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
About:Roel
Malware analysis AV research Incident response
![Page 3: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/3.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Overview
Testing AV engine Testing AVendor’s response time Product technologies Conclusions
![Page 4: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/4.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Current testing
On-demand WildList (won’t go there) Large (zoo) test bed
Retrospective using x month old product
On-access (not so common or detailed)
![Page 5: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/5.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
On-demand: obvious flaws
Trash files Age of samples Lack of transparency Response time is not a factor Lack of resources to perfect testing Etc.
![Page 6: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/6.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Infectors / Trojanizers
Trojanizers (PE, script) Real infectors Check response time for detection and disinfection
Creating trojanizer test bed can take a long time
![Page 7: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/7.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Online scan services
JottiScan, VirusTotal (and others) Much trash and ‘trash’ False positive issues Additional checks needed
SFX archives and so on
![Page 8: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/8.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Testing vs track performance
Detection on/of packer/crypter Compare results with and without packer detection Differentiate between packers
Regular vs custom packer/crypter Generic vs detecting specific family
Age of samples 1/2/3/6/12 months old
![Page 9: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/9.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Differentiate between malware
Regional malware Malware coming from a region Payload (Banker vs GameThief trojan)
Automagically fabricated samples How many Zlobs do you want in the equation?
![Page 10: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/10.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Response time
Global outbreak Localized outbreak Low priority malware Infectors/trojanizers
![Page 11: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/11.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Retrospective testing
1 second is enough Modified ‘droppers’ Type of samples
![Page 12: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/12.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Product technologies
HIPS-like module Components working together – AV vs IS (Memory scanner)
Not so relevant (in this case): Malware removal Registry cleanup Malware detection on infected system
![Page 13: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/13.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
Conclusions
Other/nicer ways to check out the competition Product technologies make testing-life harder Testing will always be flawed
![Page 14: The Difference between Track and Testing Performance](https://reader033.fdocuments.in/reader033/viewer/2022060115/55756dcfd8b42a2e248b4ea4/html5/thumbnails/14.jpg)
AV Testing Workshop, Reykjavik, 16 May 2007
The end
Thank you for your attention!
Questions or comments?