OpenSolaris and the Direction of Future Operating Systems
Transcript of OpenSolaris and the Direction of Future Operating Systems
![Page 1: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/1.jpg)
OpenSolaris and the Direction of Future Operating SystemsJames HughesSun FellowSolaris Chief Technologist
LISA'08
November 2008
San Diego, CA
![Page 2: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/2.jpg)
Agenda
Operating System TrendsComputer / OS architecture trendsWhy developers matterDirection of the OS
Solaris and OpenSolaris Security features
eZFS, XlofiKey Management Containment
![Page 3: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/3.jpg)
What is an Operating System?
![Page 4: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/4.jpg)
The Future of Operating Systems
Hardware TrendsMoore's Law
Operating Systems TrendsLarge ScaleNUMA
Programming TrendsOpenMPFortressMapReduce (Phoenix, Hadoop)
![Page 5: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/5.jpg)
Clock Speeds
http://www.chem.utoronto.ca/~nlipkowi/
![Page 6: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/6.jpg)
Moore's Law
http://www.intel.com/museum/archives/history_docs/mooreslaw.htm
![Page 7: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/7.jpg)
![Page 8: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/8.jpg)
Multicore Opportunity
Transis
tors
Multicore “Opportunity”
![Page 9: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/9.jpg)
Memory Latency
![Page 10: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/10.jpg)
Multi threading
![Page 11: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/11.jpg)
CMT
Multicore OpportunityLeverages Multicore and Multi threading
![Page 12: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/12.jpg)
Batoka (aka T5440)
256 hardware threads0.5 TB of RAM64 integer units32 floating point units32 crypto accelerators
4u
![Page 13: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/13.jpg)
Searching for Goldilocks Applications
Not to largeBlows iCache
Not to smallUninteresting
Just rightPhenomenal Performance
Pheonix Map ReduceFree, SPARC, CMT
![Page 14: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/14.jpg)
High Thread Count
Memory Ops/sec vsThread count
![Page 15: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/15.jpg)
The future is going to be high thread
counts
![Page 16: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/16.jpg)
The winners will solve the problems
using parallel methods
![Page 17: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/17.jpg)
Programs will parallelize at
runtime
![Page 18: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/18.jpg)
Languages will hide parallelism
from the programmer
![Page 19: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/19.jpg)
Parallel apps are becoming real
HotMap Reduce
HadoopPhoenix
Fortress
NotMPI, OpenMP
![Page 20: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/20.jpg)
Operating Systems provides the...
...glue between the application and the hardware
...application tools and libraries needed to get its job done
...programmer's productivity for development and debuggingOperating Systems have to reduce complexity
While enabling efficiency
![Page 21: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/21.jpg)
Managing NUMA
Helping the programmer be efficientTransparent to the programmer; Simple
Managing locality of memoryOpenSolaris “Memory Placement Option”
![Page 22: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/22.jpg)
ZFS
Revolutionary file systemData IntegrityEncryption (soon)SimpleAllows “Mulligans”
![Page 23: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/23.jpg)
Dtrace
Allows applications debugging on production codeAllows logic to be executed in the trapsTakes a complicated debugging problem and makes it simple.
![Page 24: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/24.jpg)
Operating System Future
Efficiency -and- CapabilityEnable applications that require large memory footprint and high thread countsAllowing applications to be bigger, fasterSolve numerically hard problems
SimulationsFinancial modelsPhysics simulations (aka games)
![Page 25: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/25.jpg)
Languages for Parallelism
Explicit Parallelism for ClustersOpenMPIMap Reduce
Hadoop
Explicit Parallelism for SMPOpenMPCilkFortressMap Reduce
Phoenix
![Page 26: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/26.jpg)
Cilk
Cilk is an algorithmic multithreaded language Cilk is algorithmic
guarantees efficient and predictable performance
Runs on OpenSolaris
cilk int fib (int n){ if (n < 2) return n; else { int x, y; x = spawn fib (n-1); y = spawn fib (n-2); sync; return (x+y); }}
![Page 27: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/27.jpg)
Fortress
A new programming language designed for high-performance computing (HPC) with high programmability.
Implicit parallelismTransactionsFlexible, space-aware, mathematical syntaxStatic type-checking (but with type inference)Definition of large parts of the language in its own libraries
![Page 28: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/28.jpg)
Describe algorithms in math terms
![Page 29: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/29.jpg)
Map Reduce
1)Map input to (key, value)2)Sort by key3)Reduce (key, value) to the solution
Steps 1 and 3 describe functions that are independent to scalingParallelism is differed to run time
Possibly without the programmers knowledge
![Page 30: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/30.jpg)
Map reduce Implementations
HadoopJava basedScales wide
Pheonix (Stanford)C basedScales to high threadsLarge memoryBest when problem fits in memory
Both available for OpenSolaris
![Page 31: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/31.jpg)
“...but my application can't scale”
![Page 32: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/32.jpg)
If you don't parallelize your
applications, your competition will
![Page 33: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/33.jpg)
Information Technology can be a competitive
advantage
![Page 34: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/34.jpg)
Solaris 10 and OpenSolaris
Enterprise quality and supportNew HardwareCompatible change
OpenSolarisNew capabilitiesNew management strategiesCommunity drivenDeveloper support
![Page 35: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/35.jpg)
Solaris 10 update 6
ZFS Root / bootNew SPARC boot loader
ZFS Deligated administrationDefault IP route per zoneSHA 256/512256 hardware threads on x86Performance improvementsUpdates of S10 will continue
![Page 36: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/36.jpg)
OpenSolaris 2008.05
Leading indicator of SolarisZFS rootNew packagingNew patchingFamiliar userland
Support for developersUpdates every 2 weeks
“New” every 6 months
![Page 37: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/37.jpg)
OpenSolaris 2008.11
New features TimeSlider
New and refreshed OpenSourcesudo and others
New repositoriesRedistributable repositoryCommunity repositoryClosed repository
![Page 38: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/38.jpg)
Security matters to the developer
Direction for OpenSolarisEncrypted Storage
Tape (now)eZFSxlofi
Key ManagementHigh Assurance containment
Windows in a Solaris TX labeled zone
![Page 39: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/39.jpg)
Storage contains personal information
California law about data breachesMany examples
Laptops being lostThumb drivesCell phonesStorage contains
All communicationsWork in progress
![Page 40: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/40.jpg)
Today
When the user is not logged in, the administrator can see the data
With ZFS and Enterprise RAID, overwriting a file does not erase the data
Data on RAID is clear on single diskIn m of n, 1/nth of the data on each disk
![Page 41: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/41.jpg)
eZFS
When the user is not logged in, the administrator can not see the data
Zeroing the key erase the dataPermanently
Data is protected regardless of strategyRAID, mirror, etc.
Backup should be under separately managed key so that users are not
vulnerable to key loss
![Page 42: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/42.jpg)
Future
eZFSEncrypted Boot
XlofiTurns file (partition, zVol, etc.) into secure disk
Batoka – 12GBytes/s of AES
“All Storage leaves the datacenter one way or
another, sooner or later”
![Page 43: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/43.jpg)
Key Management
Requirements are simple“Don't lose the keys”“Don't give the keys to the wrong people”
OOB key requirementMany organizations working on this
Companies, Standards, etc.
![Page 44: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/44.jpg)
How do you manage your
keys?
![Page 45: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/45.jpg)
![Page 46: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/46.jpg)
Sample Customer
100,000 individual keysToday!
Auditors having a fitUsed to copy information between servers for batch processing
Do you know a customer that has this problem?Solution is not high tech
Capture, categorize, manage, whole lifetime
![Page 47: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/47.jpg)
Encrypted Storage vs HW Trends
Measured AES, 100MB/s, on LaptopAMD, Intel and Sun will have accelerationBatoka, 12GBytes/s
Single disk performance 40MB/s (not Flash)First access has latency
Subsequent access access in RAM buffer
This level of performance is “free”In the OS is “free”
“Security is an expectation, not a market”
![Page 48: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/48.jpg)
Long Term Prediction of Adoption
Computers are fast enoughOS vendors will add for free
Yes, there are country issues
At least password protected There is no reason not to encrypt
In the future, not encrypting your storage will be like using telnet
instead of ssh
![Page 49: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/49.jpg)
Solaris TX
Military grade Sandbox of Applications (aka Zones)Virtual Machines (VB in TX)Extends to throughout the datacenter over the networks
MLS Applicable to more that GovernmentsServers that handle high value transactions
![Page 50: OpenSolaris and the Direction of Future Operating Systems](https://reader031.fdocuments.in/reader031/viewer/2022013015/61d061c2b4561804524bf91e/html5/thumbnails/50.jpg)
Conclusion
Operating SystemsComputer / OS architecture trendsWhy developers matterDirection of the OS
Solaris and OpenSolarisSecurity features
eZFS, XlofiKey ManagementContaining Windows