1 © 2013 MathWorks, Inc.

MathWorks & PERFORCE!

New York and Boston Roadshow Nov 5th & 7th 2013

Marc Ullman / Senior Systems Architect / MathWorks, Inc.

2

§  MathWorks is a 2000+ person company dedicated to accelerating the pace of engineering and science

§  We have ~90 products based upon our core platforms

–  MATLAB, the language of technical computing –  Simulink, for simulation and model-based design

§  They serve a wide array of markets –  Aero, Auto, Bio, Communications,

Finance, Medical, etc. §  Full product family is released twice a year from a

unified code base §  Products are used to develop safety-critical systems

–  Quality and correctness are paramount

MATHWORKS OVERVIEW

3

TECHNICAL CHALLENGES

§  Managing an almost 1 million file code base

§  Partitioning it into usable subsets §  Integrating changes from ~1000 developers §  Supporting multiple platforms

(Windows, Mac, Linux) §  Hosting development teams on

three continents §  Dealing with bugs and scalability issues

in underlying infrastructure –  Operating systems (e.g., Windows) –  Tools (e.g., Perforce, Visual Studio)

4

MATHWORKS DEVELOPMENT ENVIRONMENT

§  Developed in-house continuous integration system almost 20 years ago

§  Originally based on RCS, then CVS

§  Built significant infrastructure on top of CVS to support

–  Change sets –  Merge history tracking –  Lightweight (sparse) private

developer branches –  Post-commit (a.k.a. pre-flight)

queue-based operation –  Hierarchical (promotion-based) code flow

5

NEED FOR A NEW SCM SYSTEM

§  Poor performance –  Branching full code base took ~24 hours—hence

done rarely –  Repository operations painfully slow for remote developers

§  Poor reliability –  Code to augment feature set was maintenance headache

§  Poor usability –  Lacked GUI and graphical merge tools

§  Poor scalability –  Used overly strict partitioning to keep branches small

CVS-based System Suffered Significant Drawbacks

Net Effect: Lost productivity due to tooling limitations

6

SEARCH FOR A NEW SCM SYSTEM

§  Must-haves –  Atomic change sets –  Merge history tracking –  Flexible partitioning of large code base –  Hierarchical branching –  Rename support –  Private developer branches

§  Most tools are not designed to handle a unified code base the size of ours –  Accurev and Git have very efficient branching but lack good mechanisms for

partitioning a large codebase §  e.g., “Repo” used with Git for Android development

Hard to Find a Solution That Met All Our Requirements

7

PERFORCE ADDRESSED 3 OF OUR “MUST HAVES”

ü  Atomic change sets ü Merge history tracking

ü  Scalability with partitioning –  Via client and branch views

But it lacked other key features like: –  Hierarchical branching –  Rename (with live/dead merging and resolve) –  Lightweight (sparse) private developer branches

8

PERFORCE REDUX—STREAMS

With streams, Perforce became a better fit for our needs §  Streams are true hierarchical branches

–  Stream (branch) hierarchies are clearly defined and captured within Perforce rather than via customer conventions

–  Their evolution is recorded as part of depot history –  With StreamAtChange, stream shape can be referenced

in a time-consistent manner

§  Integration engine was beefed up to handle move/rename

§  Task streams approximate light-weight branches

9

EXCEPT THAT…

The out-of-the box streams inheritance model of did not mesh well with our desired workflows We wanted: §  Wide-open client views §  The ability to widen a branch (stream)

from below §  Stream (shape) changes to be atomic with

content changes

10

COLLABORATION WAS THE KEY TO SUCCESS Perforce helped us come up with an elegant solution §  Virtual streams are used in conjunction with the Perforce

broker to narrow p4 sync and merge / copy / integrate +  Permits users to operate with wide open views +  Transparent to use of CLI, P4V, and plug-ins

§  A change trigger is used to modify behavior of p4 submit +  Limits scope of submits to active components +  Updates the virtual stream paths if pending change includes modified

component definitions +  Results in changes to the stream content and shape being atomic

But Collaboration didn’t stop there—it went both ways §  We helped Perforce refine behavior of StreamAtChange §  We provided use cases that Perforce incorporated into their

regression test suite

11

BENEFITS WE SEE WITH PERFORCE STREAMS

§  Simple user model –  Developers no longer edit client views or branch specs –  Stream specs are updated automatically by submit trigger –  Perforce generates client and branch views as needed

§  Dynamically partitioned codebase –  Developers can define the width (shape) of a branch by

specifying the set of components they need

§  Improved productivity –  Removed major barrier to using narrow branches –  Developers and release engineers no longer struggle to

make componentization changes –  Able to largely automate our hierarchical code motion

(what Perforce likes to call merge down/copy up)

12

LOOKING AHEAD

§  Continuing to work closely with Perforce

§  Considering use of Task Streams

§  Interested in gaining better insight into development projects and processes

–  Excited to try Perforce Insights

13 © 2013 MathWorks, Inc.

Questions?