®
IBM Software Group
© 2005 IBM Corporation
The Open Internet – Open Source, Open Standards and the Effects on Collaborative Software Development
September 26, 2005, High Performance Transaction Systems Workshop (HPTS)
Dr. Danny SabbahGeneral Manager, Rational Software, IBM
IBM Software Group
2
Software Development Needs to radically change
Why??? Increasing abstraction and forced integration = forced complexity
The Ugly reality of developing distributed, internet based solutions
The pluses and minuses of Moore’s Law and Cheap and pervasive interconnection
Mass Adoption (individual computing) meets Mass Interconnection (internetworking)
How??? Rethinking modularity and granularity of software
Increasing focus on ‘decomposition’ for re-composition
Leveraging network effects introduced by Open Source development methodology
Open Source development is not just about ‘Open’ Source
IBM Software Group
3
Destabilizing forces converge to significantly intensify global competition
-5
5
15
25
35
45
55
65
75
1928 '38 '48 '58 '68 '78 '88 '98 '08 '18
Year
Imp
lied
Lif
etim
e in
S&
P 5
00
Yea
rs
Average Lifetime of S&P 500 Companies
Source: Creative Destruction, Richard Foster
Technology innovation and long-term public policy shifts are consistently destabilizing the marketplace (and it’s accelerating)
Technology systematically reduces interaction costs and extends global reach
Constant worldwide public policy shifts alter regulatory climate and promote or discourage competition
In a world where speed through IT automation is now largely assumed
IBM Software Group
4
How Moore’s Law and Unlimited Bandwidth play: As environmental and economic factors (globalization, risk
management, innovation), force CONSTANT, accelerating fundamental business change (processes,deconstruction/refocus, partnerships)
Chaos results from conflicting goals from multiple generations of ‘captured intelligence’ in the form of code/business rules mixed in with new generations of assumptions (Mainframe to C/S to Peer distributed – and variants)
Technology adds to disruptive forces unless properly managed and accounted for. NO ONE starts from SCRATCH. Software EVOLVES.
The unintended side effects of Moore’s law can become a curse to innovation if an organization’s Information technology isn’t allowed to advance at the same rate.
IBM Software Group
5
E01-EDI
Data Warehouse(Interfaces to and from theData Warehouse are not
displayed on this diagram)
G02 - GeneralLedger
A05 - AP
S01 - SalesCorrections
I01 POReceiving
I03 Return toVendor
I06 WarehouseManagement
Mainframe
PC/NT apps Unix apps
3rd Party Interface
S06 - Credit App
P15 EES EmployeeChange Notice
OTHER APPS - PCAP - Collections/Credit
TM - Credit Card DB
ACCTS REC APPS - PC990CORBad Debt
Beneficial FeesBeneficial Reconcile
JEAXFJEBFAJEBKAJEDVAJESOAJEVSAJEVSFNSF
TeleCredit Fees
INVENTORY CONTROL APPS - PCCode Alarm
Debit ReceivingsDevo Sales
Display InventoryIn HomeJunkouts
Merchandise WithdrawalPromo CreditsRTV Accrual
ShrinkAP Research - Inv CntrlAP Research-Addl Rpts
Book to Perpetual InventoryClose Out Reporting
Computer Intelligence DataCount Corrections
Cross Ref for VCB DnldsDamage Write OffDebit Receivings
DFI Vendor DatabaseDisplay Inventory ReconcileDisplay Inventory Reporting
INVENTORY CONTROL APPS - PCDPI/CPI
IC BatchingInventory Adj/Count CorrectInventory Control Reports
Inventory LevelsInventory Roll
Merchandise WithdrawalOpen ReceivingsPI Count Results
PI Time Results from InvPrice Protection
Sales Flash ReportingShrink Reporting
SKU Gross MarginSKU Shrink Level Detail
USMVCB Downloads
Journal Entry Tool Kit
Scorecard - HR
L02-ResourceScheduling(Campbell)
P09 - P17Cyborg
M02 - Millennium
M03 - Millennium 3.0
Banks - ACH and Pos toPay
Cobra
B01 - StockStatus
S03-Polling
P14 On-line NewHire Entry
CTS
Plan Administrators(401K, PCS, Life,
Unicare, SolomonSmith Barney)
D01 Post LoadBilling
I04 HomeDeliveries
I02 -Transfers
Arthur Planning
I07 PurchaseOrder
I12 EntertainmentSoftware
I05Inventory Info
E13E3 Interface
S04 - Sales Posting
V01-Price ManagementSystem
I10 Cycle PhysicalInventory
I55 SKUInformation
K02Customer Repair
TrackingI35 Early Warning
System
B02 MerchandiseAnalysis
I13- AutoReplenishment
U18 - CTO
Intercept
I09 Cycle Counts
E02-EmployeePurchase
Texlon 3.5
ACH
Stock Options
I17 Customer PerceivedIn-Stock
U16-Texlon
SiteSeer
C02 - CapitalProjects
F06 - FixedAssets
US Bank ReconFile
Star Repair
EDICoordinator
Mesa Data
NEW SoundscanNPD Group
AIG Warranty Guard
Resumix
Optika
Store BudgetReporting
P16 - Tally Sheet
Cash Receipts/Credit
S05 - HouseCharges
Ad Expense
L01-PromoAnalysis
V02-PriceMarketingSupport
BMP - Busperformance Mngt
StoreScorecard
I11 PriceTesting
Valley Media
P09Bonus/HR
I15 Hand ScanApps
Roadshow
POS
S08 - VertexSalesTax
A04 - CustRefund Chks
Equifax
ICMS Credit
CellularRollover
S09 - DigitalSatelliteSystem
NPD,SoundScan
Sterling VANMailbox (Value)
I18SKU Rep
X92-X96Host to AS400
Communication
S02 -Layaways
Washington,RGIS,
Ntl Bus Systems
V04-SignSystem
I14 Count CorrectionsNARM
P01-EmployeeMasterfile
I06 - CustomerOrder
FrickCo
UAR - Universal AccountReconciliation
DepositoryBanks
S07 - CellPhones
S11 - ISPTracking
AAS
Fringe PO
Cash Over/Short
L60 MDFCoop
SKU SelectionTool
SKUPerformance
SupplierCompliance
1
I35 - CEI
ASIS
Misc Accounting/Finance Apps - PC/NTCOBA (Corp office Budget Assistant)
PCBS(Profit Center Budget System)Merchandising Budget
AIMSMerch Mngr Approval
Batch ForcastingAd Measurement
AIMS Admin
AIMSReportingAd
Launcher
V03- MktReactions
SpecSource
CTO2.Bestbuy.com
RebateTransfer
SignSystem
CopyWriter'sWorkspace
ELTPowerSuite
StoreMonitor
AIS Calendar
Stores & Mrkts
Due Dates
Smart Plus
InsertionsOrders
BudgetAnalysis Tool
Print CostingInvoice App
AIS Reports
BroadcastFilter
Smart PlusLauncher
GeneralMaintenance
Printer PO
PrinterMaintenance
VendorMaintenance
Vendor Setup
Connect 3
Connect 3Reports
Connect 3PDF Transfe
Spec SourceSKU Tracking
S20-SalesPolling
Prodigy
PSP
In-HomeRepair
WarrantyBillingSystem
Process Servers(Imaging)
Prepared by Michelle Mills
IT Software Engineering Realities
IBM Software Group
6
Unraveling the Chaotic Mess of conflicting abstraction and undifferentiated monoliths
I see 2 major dimensions: Modularity for the Internet age: Very Loosely coupled architectures and
systems focusing on largely ‘declarative’ I/F’s based on OPEN standards (necessarily stemming from the Internet) -- Decomposable Systems
Standards, Standards, Standards…. For interop NOT portability
Commonly termed “Service Oriented Architectures” based on XML and Web Services standards
Only made possible by the ‘positives’ in Moore’s law (i.e. no one in their right mind would have suggested this 10 years ago)
‘The discipline of Programming in the Very Large’ through internet community growth and governance (e.g. Open Source Communities governed by meritocracy)
Meritocracies force minimal design and controlled extensibility.. A form of collaborative innovation
Exposure early and often forces accountability and better quality
Establishing pervasive, default distribution for STANDARDS
IBM Software Group
7
‘Open’ Computing: Staying Open for BusinessGoals of openness:•Flexibility•Resilience•Avoid limiting network effects •Maximize freedom of action and innovation
Open standards:•Promoting interoperability by using openpublished specifications for APIs, protocols and data and file formats
Open architecture:•Building loosely coupled, flexible, reconfigurable solutions
Open source software:•Leverages community development and collaborative innovationcollaborative innovation•Optimizes network effects and ‘minimalism’ in design
IBM Software Group
8
Controlling/minimizing Complexity
Large, monolithic systems and packaged apps are the result of uncontrolled, undisciplined experimentation driven by quickly dissolving software business models
Persistent software systems (OS’s, Middleware, P. Apps) must practice controlled evolution with simpler, decomposable designs.
Software Components must be driven to ‘just enough’
This is Hard! Simple does not mean ‘simplistic’:
“Designs should be as simple as possible, but no simpler”
This applies to all ‘aspects’ of systems designs: internal and external
IBM Software Group
9
The ‘Open’ Proposition: Industry business models must change
Change From “Control”Change From “Control” To “Sustained Value Add”To “Sustained Value Add”
Own standard
Own customer relationship
Control price
Control pace of development
Benefit from network effects
Use economy of scale
Increase availability of skills
Increase Speed of Innovation
Letting Go of Control
IBM Software Group
10
Emergence of Open Source Software communities are being fueled by:
Internet based cost efficiencies and near universal connectivity Standards Efficient collaboration/community tools (email, Wikis, newsgroups,
Collaborative Development platforms) Social aspects Emergence of “good enough” programming models (Occam’s
razor at work) Perception of innovation opportunities Alignment of Interests (commercial and strategic): Development
expense subsidy and incumbency disruption Growing commercial acceptance
IBM Software Group
11
Community TypesThere are two major open source community archetypes, though most projects
have some characteristics of both:
Voluntary model Apache, Linux projects. 46% do not earn any money for their participation, 16% paid to develop; 18% paid
to administer; 12% paid to support it; 26% indirect financial comp Projects such as Apache and Linux that started out as pure voluntary model are
trending towards a greater commercial aligned interests model and now have the majority of their code contributed by professional commercial developers
Vendor-initiated model Eclipse, Open Office, Ingres, for example Often used for establishing or diverting ecosystem vitality Initially dominated by donor company May mature to broad based aligned interest model May have direct or indirect business model plays Vendor initiated projects may take on greater voluntary model aspects as they
mature
IBM Software Group
12
IBM contributes to 150+ OSS projects
More than 1000 developers involved in OSS projects IBM leads 80+ OSS projects
1999 - 2001 IBM forms Linux
Technology Center – contributions to serviceability, performance
Leads Apache XML projects Xalan Xerces, SOAP
Forms Open Source Steering Committee
Creates OSI-approved IBM & Common Public Licenses
Participation in Mozilla
Participated in founding of Eclipse
2002 Linux contributions
to scalability (8-way+), reliability (stress testing, defect mgmt, doc)
Leads Apache Web Services projects WSIF and WSIL
Leads Eclipse projects GEF (editing), EMF (modeling), XSD (XML Schema)
IBM contributes eServer support for Globus Toolkit 2x
2003 IBM and SuSE
achieve EAL2+ Common Criteria security cert
Leads Apache projects Pluto (Portlet API) and WSRP4J (Remote Portal)
Leads Eclipse projects Hyades (testing), AspectJ, Equinox rich client
Globus Toolkit 3 contributions for OGSA, OGSI
2004 IBM and Novell/SuSE achieve
EAL3+ and Common Operating Environment compliance
Linux additional RAS
Incubates Apache project Derby (Cloudscape Java database)
Dialog components to Apache Jakarta taglibs
Eclipse becomes independent org – IBM contributes UML2, Web Tools, Voice Tools and eRCP
Globus Toolkit 4 to be WS-I compliant
2005 Contributions to Xen hypervisor, Linux accessibility
Contributions to Apache WSDL4J 2.0 (Woden), Web Services Security
• Database extensions to PHP
• Redeploy 30+ developerWorks projects on SourceForge.net
• IBM pledges 500 US patents to OSS
• Geronimo commitment; purchase of Gluecode
IBM Open Source History/Involvement
IBM Software Group
13
An Example: EclipseEclipse continues gaining popularity and a broad community
Eclipse formed as an independent organization in February 2004, establishing scalable governance model for Open Source projects and commercial participation
There have been over 18 million download requests since 4Q2001 that we can track (many more that we don’t)
By attracting more subprojects and third party contributors, the Eclipse community is establishing a de facto standard for an Integrated Development Environment
Founding strategic developers and strategic consumers are Ericsson, HP, IBM, Intel, MontaVista Software, QNX, SAP, and Serena Software
Over 30 companies are committed to Eclipse via IBM WebSphere Studio WorkBench
There are over 40 Eclipse add-in providers, including Borland, Novell, Oracle, Red Hat, VA Software, webMethods, and Wind River
More than 175 tools vendors are delivering tools for the Eclipse platform
IBM Software Group
14
Lessons Learned: Not all Open, Collaborative projects Succeed
Critical Factors for a successful Commercial Open Source CommunityBusiness perspective
Must provide “perceived value" to end customersActive sponsorship and marketing by 1 or more significant sponsorsSolid overall vendor support Critical skills available and actively engagedLong-term plans for improvements - plans, design,
development, test, documentation; rigorous IP policies
Developer perspectivePassionate interest in developing and enhancing code
Significant overlap between set of users and developers - common wavelength
Personal rewards - able to build status and recognition in developer community
Diverse, interactive community
Strong overall project/code leadership (e.g. Linus Torvalds -Linux)
IBM Software Group
15
Licensing: Can Open Source Software co-exist with Commercial Software? YES
Most (not all) open source software licenses allow combination and distribution of open source software and Commercial source code under a commercial license.
Some commonly encountered open source software licenses (BSD, MIT, X11, Apache) don't require modifications to original open source software to be published upon redistribution.
GPL allows commercial applications to be built on top of Linux to remain commercial Application can be licensed under commercial license of choice. No need to disclose source code of such applications.
LGPL Libraries can be dynamically linked to arbitrary commercial code No requirement to release commercial code under LGPL.
Decision to use open source software is just another business decision with risk/reward tradeoff
License terms need to be understood before beginning to work with open source software .
IBM Software Group
16
Internal Community Source (Open Source) Model
An open source-style development environment where consumers can contribute code to shared components
Producers get Collaboration
Broader testing and tuning
More folks using their code
Consumers get Fast access to capabilities
Ability to invest to tune to a unique specification
Broader testing and tuning
IBM Software Group
17
• Provides a collaborative environment accessed via a Web-based portal for shared, distributed development and testing; 225+ projects to date engaging 3,000 developers
• Controlled “white box” approach for development of component software• Explore shared development of components intended for reuse• Seeded by respective development organizations
Benefits Encouraging reuse over reinvention Improving information flow between teams
(availability of source materials, decisions and discussions)
Leveraging broader IBM community skills (technical and non-technical communities)
Improving quality through peer reviews and user feedback (defects and forums)
Positively impacting our ability to deliver more function on shorter schedule (collaboration and contribution)
Most valuable assets get the most attention (based on reuse)
Facilitate development
Community Source: Open Source Within IBM
InternalDevelopmentCommunity
Key Features: Access Control Product builds, fixes and
test drivers Discussion Forums Reference information (API
specs, programming documentation, education, demo, etc.)
Defect Reporting Feature Requests Code Storage and Version
Control
IBM Software Group
18
Community Source Infrastructure: How?
Register the project, meeting established criteria
Set up mailing lists and forums
Establish Project Leads & Committers
Publish materials, which may include: 1. Source (logic, test cases, binaries)2. Documentation, specs3. News, bulletins4. Education materials5. Patches, fixes
Build an interested community
Process requirements and contributions
IBM Software Group
19
Community Source Infrastructure: Who?
Contributors: Developers who submit code changes through
committers Provide additional testing Provide additional documentation Submit patches and features Submit requirements Help fulfill consumer requirements
Committers: Developers with write access to the repositories Often the Project Leaders Write/update source materials Control code contributions from contributors
IBM Software Group
20
Community Source Infrastructure: Who else?
Project Administrators: Individuals responsible for enabling and
regulating the smooth operation of the project
Operations and access control
Sets up mailing lists and forums
Publishes news and announcements
Users: Reviewers of source materials and documentation
May submit defects or feature requests
May be non-technical
May participate in discussion forums
IBM Software Group
21
Community Source Background IBM Internal Open Source Bazaar (IIOSB) was established January 2000
Community Source became a subset of the IIOSB around August 2002 Community Source has been referred to as the “IBM Product License zone” in IIOSB
Community Source was moved out of IIOSB infrastructure to its own dedicated server at the end of June 2005 Today based on open source, collaborative development infrastructure
Alphaworks and DeveloperWorks offer technology preview and an IBM open source infrastructure available for outside participation since 1999
IBM Product License zone
Projects – Community
Source
IBM Internal Open Source Bazaar
(IIOSB)
SourceForge Enterprise
GForge
Community SourceIBM Product
Licenseand SWG projects
IIOSBNon-IBM
confidentialprojects
GForge
2005
IBM Software Group
22
Executive Summary: Statistics & Measurements Important Community Source statistics as of August 15, 2005:
Active project count on Community Source: 224
07/15/05-08/15/05: 33 new project registrations approved
Active member count on Community Source: 3,014
Most active projects (> 50% activity level*): 17 projects
*Note: Activity level refers to the site-wide project usage of the key Community Source features (tracker, CVS, patch, file publisher etc.)
0
100
200
300
400
500
600
700
800
Functions
Downloads
Bugs
Support
Patches
Tracker
Tasks
Sit
e-w
ide
Fu
nc
tio
n U
sag
e
Measuring Usage & Leverage
0
50
100
150
200
250
May-05 Jun-05 Jul-05 Aug-05
Months
Measuring Community Growth
Ne
w R
eg
istr
ati
on
IBM Software Group
23
Project Pattern Usage1. Community Growth2. Information Sharing3. Source Code Development4. Production Environment5. Service & Support Mode
0
10
20
30
40
50
60
70
80
90
100
Functions
Mailing ListsDiscussion ForumsFile PublisherCode Library SystemDefect Tracking SystemPatch Tracking SystemFeature Request TrackingTech Support Tracking
% P
roje
cts
Usi
ng F
unct
ion
1 ….. 2 ….. 3 ..… 4 ..… 5
IBM Software Group
24
Community Source Example: Constellation Project Created to address the need to componentize the IBM Software Development Platform
Toolset
Scope For Caspian focus is (Rational Application Developer, WebSphere Integration Developer,
WebSphere Business Integration Modeler, Rational Software Architect) Cross product usage pain point for Atlantic High degree of overlap and lack of reuse
Goals: Create an internal open source environment (modeled after eclipse)
Governance Rules of Engagement
Define a set of common tools and processes
• Status:• Established cross brand Project Management Committee
• Created component inventory working closely with WebSphere Integration Developer• Set of component classifications/layers
• Sizing component recommendations for Caspian plan
IBM Software Group
25
Building the Community… SWG Developers Portal
Find resources
Learn strategy
“How to” integrate crossbrand
Share components/plans
Developers Developers IntranetIntranet Collaborate
Recommend resources
Locate experts
Share content with other portals
Developers Developers PortalPortal
Customized info by role
Developments tools integrated with Web
Integrated with individual teamrooms/intranets
Developers Developers WorkplaceWorkplace
IBM Software Group
26
EclipseClient
ProjectTeamServer
Team Bench awareness, team places UI, collaboration
Wor
kIte
ms
Bui
ld
Req
uire
men
ts
Rep
ortin
g/H
ealth
File
syst
em/V
CM
Tes
t
RA
S
VerticalServerPlug-ins
VerticalClientPlug-ins
Pla
nni
ng
Gui
dan
ce
…
Wor
kIte
ms
Bui
ld
Req
uire
men
ts
Rep
ortin
g/H
ealth
File
syst
em/V
CM
Tes
t
RA
S
Pla
nni
ng
Gui
dan
ce
…
Project Client API/Extensibility
Workflow Search Collaboration
Web/Portal
Relational data
Project home
Proj. Health
Communication: wiki, blog, etc
Web Services
Admin
VerticalWeb UI’s
….
Tomcat/Geronimo->WAS
ND->WAS XD
Cloudscape->DB2/
Oracle/…
Notification Query Persistence
Login/ACL
Platform API/Extensibility
Localdata
IBM Software Group
27
Platform Goals and Services Network effects/pervasive integration – Global development
Simple but scalable deployment and management
Significantly Improved Communication and collaboration (human and agent)
Basic interoperability with selected external solutions to ease evolution and integration
Loosely coupled workflow with fine grained security – peer to peer distribution model
Fine grained, configurable components through plug-in extensibility architecture
Out-of-the-box schema, metadata, metrics, process, views, roles, security model
IBM Software Group
28
Summary
Business models must adapt to survive in today’s business environment. Openness fosters adaptability and flexibility.
Software Development must promote greater ‘experimental’ discipline, incorporate finer modularity and leverages community based feedback/innovation
Letting go of control and loosening the reins on the development environment leads to greater innovation and business value. Programming in the large must become programming in the VERY large over internet protocols and standards
Fostering community source projects can speed development and ultimately, result in greater advances for the organizations. Even for proprietary software.
Commercial, open source and hybrid software models all deliver distinct value to the marketplace. They are leading to the emergence of new software business models
Top Related