High Level Perforce Best Practices

7/23/2019 High Level Perforce Best Practices

http://slidepdf.com/reader/full/high-level-perforce-best-practices 1/10

Abstract

When deploying software version management andversion control tools, implementers sometimes focuson perfecting fine-grained activities, while unwittinglycarrying forward poor, large-scale practices from theirprevious jobs or previous tools. Te result is a well-executed blunder. Tis paper promotes some high-levelbest practices that reflect the authors’ experiences withsoftware version management.

High-Level Software VersionManagement Best Practices



High-Level Software Version Management Best Practices

Table of Contents

Introduction ________________________________________________________________1

1. The Workspace ____________________________________________________________2

2. The Codeline ______________________________________________________________2

3. Child codelines ____________________________________________________________4

4. Change Propagation ________________________________________________________5

5. Builds ___________________________________________________________________6

6. Process __________________________________________________________________6

7. Conclusion _______________________________________________________________7

8. References _______________________________________________________________7

More about how Perforce supports these best practices ______________________________ 8



High-Level Software Version Management Best Practices1

Introduction“A tool is only as good as you use it,” the saying goes. As providers of software version management toolsand as consultants to software companies, we are oftenasked for sound advice on software version managementbest practices—that is, how to deploy software versionmanagement software to the maximum advantage. Answering these requests has created a bounty of directand indirect version management experience from which

Workspaces Where developers build, test, and debug.

Don't share workspaces.

Don’t work outside of managed workspaces.Don’t use jello views.

Stay in sync with the codeline.

Check in often.

Use task branching.

Codelines Sets of related source files.

Have a mainline.

Give each codeline a policy.

Give each codeline an owner.

Child codelines Variants of the mainline.

Organize child codelines on the tofu scale.

Don’t copy when you mean to branch.

Branch on incompatible policy.Branch late.

Branch, instead of freeze

Change propagationMoving changes between codelines.

Use the baseline protcol.

Propagate early and often.Merge down, copy up.

Choose the correct codeline for original changes.Get the right person to do the merge.

BuildsTurning source files into products.

Source + tools = product.

Version all original source.

Separate built objects from original source.Use common build environments.

Build often.

Keep build logs and build output.

ProcessThe rules for all of the above.

Track change sets.

Track the flow of change.

Distinguish change requests from change sets.Give everything an owner.

Use living documents.Learn and use integrations to supporting tools.

to draw. Te direct experience comes from having beendevelopers and codeline managers ourselves; the indirectexperience comes from customer reports of successes andfailures with our product (Perforce) and other SCM andversion control tools.

Te table below lists six general topics of software version

management deployment, and some coarse-grainedbest practices within each of those areas. Te followingchapters explain each item.



High-Level Software Version Management Best Practices 2

1. The WorkspaceTe workspace is where developers edit source files, buildthe software components they’re working on, and testand debug what they’ve built. Most version control orsoftware version management systems have some notionof a workspace; sometimes they are called “views”, as inClearCase and Perforce. In Distributed Version Control(DVCS) systems like Git, an entire repository acts as a workspace. Changes to managed repository files begin aschanges to files in a workspace, or when DVCS systemspush, pull, and merge entire repositories.

Te best practices for workspaces include:

• Don’t share workspaces. A workspace should havea single purpose, such as an edit/build/test areafor a single developer, or a build/test/release areafor a product release. Sharing workspaces confusespeople, just as sharing a desk does. Furthermore,

sharing workspaces compromises the versionmanagement systems’ ability to track activityby user or task. Workspaces and the disk spacethey occupy are cheap; don’t waste time trying toconserve them.

• Don’t work outside of managed workspaces. Your version control system can only track workin progress when it takes place within managed workspaces. Users working outside of workspacesare beached; there’s a river of information flowingpast and they’re not part of it. For instance, versionmanagement systems generally use workspaces

to facilitate some of the communication amongdevelopers working on related tasks. You can see what is happening in other’s workspaces, and theycan see what’s going on in yours. If you need totake an emergency vacation, your properly managed workspace may be all you can leave behind. Useproper workspaces.

• Don’t use jello views. A file in your workspaceshould not change unless you explicitly causethe change. A “jello view” is a workspace wherefile changes are caused by external events beyondyour control. A typical example of a jello view isa workspace built upon a tree of symbolic links to

files in another workspace—when the underlyingfiles are updated, your workspace files change. Jello views are a source of chaos in softwaredevelopment. Debug symbols in executables don’tmatch the source files, mysterious recompilationsoccur in supposedly trivial rebuilds, and debuggingcycles never converge—these are just some of theproblems. Keep your workspaces firm and stable

by setting them up so that users have control over when their files change.

• Stay in sync with the codeline. As a developer, thequality of your work depends on how well it meshes with other peoples’ work. As changes are checkedinto the codeline by others, you should update your

workspace and integrate those changes with yourown changes.

As a software version management engineer, it’sgood to make sure this workspace update operationis straightforward and unencumbered with trickyor time-consuming procedures. If developers find itfairly painless to update their workspaces, they’ll doit more frequently and integration problems won’tpile up at project deadlines.

• Check in often. Integrating your development work with other peoples’ work also requires youto check in your changes as soon as they are ready.

Once you’ve finished a development task, check inyour changed files so that your work is available toothers.

Again, as a software version management engineer,you should set up procedures that encouragefrequent check-ins. Don’t implement undulyarduous validation procedures, and don’t freezecodelines (see Child Codelines, below). Shortfreezes are bearable, but long freezes compromiseproductivity. Much productivity can be wasted waiting for the right day (or week, or month) tosubmit changes.

• Use task branching. In contrast to the long, publiclife of codelines, task branches generally supportshort-lived, private tasks, such as hot fixes or localfeature development. Teir primary task is to helpready a set of changes for propagation into themain codebase. Once the contribution has beendelivered, they can be deleted or forgotten. askbranching brings the power of versioning to yourlocal workspace while keeping your main codebaseclear from metadata clutter.

2. The Codeline

In this context, the codeline is the set of source filesrequired to produce your software. Codelines arebranched into child codelines to support specificvariants of the codeline, such as development and releasecodelines.




Te best practices with regard to codelines are:

• Have a mainline. A “mainline,” or “trunk,” isthe branch of a codeline that evolves forever. Amainline provides an ultimate destination foralmost all changes—both maintenance fixes andnew features—and represents the primary, linear

evolution of a software product. Release codelinesand development codelines are branched fromthe mainline, and work that occurs in branches ispropagated back to the mainline.

Figure 1 shows a mainline (called “main”), from whichseveral release lines (“ver1”, “ver2” and “ver3”) andfeature development lines (“projA”, “projB”, and“projC”) have been branched. Developers work in the

main

projA

projB

projC

ver1 ver2 ver3

rel1.0

rel1.1

rel2.0

rel2.1

rel3.0

projA(ver2)

rel2.0

projB

projC(ver3)

rel1.0 rel1.1

rel2.1

rel3.0

ver1

Figure 2: The Promotion Model

mainline or in a feature development line. Te releaselines are reserved for testing and critical fixes, and areinsulated from the hubbub of development. Eventuallyall changes submitted to the release lines and the featuredevelopment lines get merged into the mainline.

An adverse approach is to “promote” codelines; for

example, to promote a development codeline to a releasecodeline, and branch off a new development codeline.For example, Figure 2 shows a development codelinepromoted to a release codeline (“ver1”) and branchedinto another development codeline (“projA”). Eachrelease codeline starts out as a development codeline,and development moves from codeline to codeline.

Figure 1: The Mainline Model




Te promotion scheme suffers from two significantdrawbacks: (1) it requires the policy of a codeline tochange, which is never easy to communicate to everyone;(2) it requires developers to relocate their work inprogress to another codeline, which is error-prone andtime-consuming. 90% of legacy SCM “process” in the

promotion model is enforcing codeline promotion tocompensate for the lack of a mainline.

Process is streamlined and simplified when you usea mainline model. With a mainline, contributors’ workspaces and environments are stable for the durationof their tasks at hand, and no additional administrativeoverhead is incurred as software products move forwardto maturity.

• Give each codeline a policy. A codeline policyspecifies the fair use and permissible check-ins forthe codeline, and is the essential user’s manual forsoftware version management codeline control.

For example, the policy of a development codelineshould state that it isn’t for release; likewise, thepolicy of a release codeline should limit changes toapproved bug fixes1. Te policy can also describehow to document changes being checked in, whatreview is needed, what testing is required, and theexpectations of codeline stability after check-ins. Apolicy is a critical component for a documented,enforceable software development process, and acodeline without a policy, from a software versionmanagement point of view, is out of control.

• Give each codeline an owner. Having defined apolicy for a codeline, you’ll soon encounter special

cases where the policy is inapplicable or ambiguous.Developers facing these ambiguities will turn to theperson in charge of the codeline for workarounds. When no one is in charge, developers tend to enacttheir own workarounds without documenting them.Or they simply procrastinate because they don’t haveenough information about the codeline to come up with a reasonable workaround. You can avoid thistrap by appointing someone to own the codeline,and to shepherd it through its useful life. With thisbroader objective, the codeline owner can smooththe ride over rough spots in software development

by advising developers on policy exceptions anddocumenting them.

1 Some sensible codeline policies: Development codeline: interim codechanges may be checked in; affected components must be buildable.Release codeline: software must build and pass regression testsbefore check-in; check-ins limited to bug fixes; no new features orfunctionality may be checked in; after check-in, branch is frozenuntil entire QA cycle is completed. Mainline: all components mustcompile and link, and pass regression tests; completed, tested newfeatures may be checked in.

3. Child codelinesTe mainline is typically branched into child codelinesspecialized to support a variety of different tasks. Tecollection and relationships of the mainline and its childcodelines can be visualized on a flow graph diagram.

Here are best practices for organizing and creating newchild codelines:

• Organize codelines according to the Tofu Scale. Much like tofu in the supermarket which comesin different measures of firmness, codelines alsofall somewhere on the firmness scale of increasingstability. Release codelines appear above themainline and seek minimal change. Developmentcodelines are arranged below the mainline and seekadditional change.

Te ofu Scale for codeline firmness:

• Don’t copy when you mean to branch. Analternative to using your software version

management tool’s branching mechanism is tocopy a set of source files from one codeline andcheck them in to another as new files. Don’t thinkthat you can avoid the costs of branching bysimply copying. Copying incurs all the headachesof branching—additional entities and increasedcomplexity—but without the benefit of yoursoftware version management system’s codelinesupport. Don’t be fooled: even “read-only” copiesshipped off to another development group “forreference only” often return with changes made. Useyour software version management system to makea new codeline when you spin off parts or all of an

existing codeline.• Branch on incompatible policy. Tere is one

simple rule to determine if a codeline should bebranched: it should be branched when its users needdifferent check-in policies. For example, a productrelease group may need a check-in policy thatenforces rigorous testing, whereas a developmentteam may need a policy that allows frequentcheck-ins of partially tested changes. Tis policy

Release 1 Release 2

Mainline

Develop ment 1 Development 2 Development 3

Personal Dev 1

Firmer

Softer




divergence calls for a codeline branch. When onedevelopment group doesn’t wish to see anotherdevelopment group’s changes, that is also a formof incompatible policy: each group should have itsown codeline.

• Branch late. o minimize the number of changes

that need to be propagated from one branch toanother, put off creating a new codeline as long aspossible. For example, if the mainline contains allthe new features ready for a release, do as muchtesting and bug fixing in it as you can beforecreating a release codeline. Every bug fixed in themainline before the release branch is created is oneless change needing propagation between codelines.

• Branch instead of freeze. On the other hand, iftesting requires freezing a codeline, developers who have pending changes will have to sit on theirchanges until the testing is complete. If this is thecase, branch the codeline early enough so that

developers can check in and get on with their work.

4. Change PropagationBranching codelines creates the task of propagatingfile changes between codelines. Te propagation ofchanges throughout a flow graph is known as the Flowof Change. Here are some things you can do to keep itmanageable.

• Use the baseline protocol. Stabilizing change flowscontinually to softer codelines. Changes from thesofter codelines flow up to the baseline at points

of completion. Changes do not flow up from thebaseline.

• Propagate early and often. When it’s feasibleto propagate a change from one codeline to

Often the mainline will be the baseline.One exception is a collaborative bugfix branch branched as a development(i.e. softer) codeline from a release

codeline. Here, the release branch isthe baseline, and change flows at pointsof completion to the release line. (Temainline will eventually receive thechanges via the flow of change from therelease codeline.)

Change flows:

firmer codelines

baseline

softer codelines

continually

at points of completion

another, do it sooner rather than later. Postponedand batched change propagations can result instunningly complex file merges.

• Merge down, copy up. Create safer merges bymerging changes down to the softer codeline, andcopying up to a firmer codeline. Te softer codeline

can better accommodate the risk of merging because(1) instability is more acceptable there and (2) codein the softer codeline is further from the release date. A typical workflow would be:

1. Merge from the baseline to the softercodeline.

2. est the merged result.

3. Copy from the softer codeline to thebaseline, now that we are assured asuccessful merge.

Merge down, copy up minimizes the risk of introducingdestabilizing changes into the firmer/more stable

baseline.• Choose the correct codeline for original changes.

You may have dozens of codelines in your codelinegraph. How do you decide where to make anoriginal change? Make original changes intended tostabilize a codeline above the mainline, and all otherchanges below the mainline.

• Get the right person to do the merge. Te burdenof change propagation can be lightened by assigningthe responsibility to the engineer best prepared toresolve file conflicts. Changes can be propagatedby (a) the owner of the target files, (b) the person

who made the original changes, or (c) someone else.Either (a) or (b) will do a better job than (c).




5. Builds A build is the business of constructing usable softwarefrom original source files. Builds are more manageableand less prone to problems when a few key practices areobserved:

• Source + tools = product. Te only ingredientsin a build should be source files and the tools to which they are input. Memorized procedures andyellow stickies have no place in this equation.Given the same source files and build tools, theresulting product should always be the same. If youhave manual setup procedures, automate them inscripts. If you have manual setup steps, documentthem in build instructions. And document alltool specifications, including OS, compilers,include files, link libraries, make programs, virtualmachines, and executable paths. Version all of theresulting materials.

• Version all original source. When software can’tbe reliably reproduced from the same ingredients,chances are the ingredient list is incomplete.Frequently overlooked ingredients are makefiles,setup scripts, build scripts, build instructions, andtool specifications. All of these are the source youbuild with. Remember: source + tools = product

• Separate built objects from original source. Organize your builds so that the directoriescontaining original source files are not polluted bybuilt objects. Original source files are those youcreate “from an original thought process” with a

text editor, an application generator, or any otherinteractive tool. Built objects are all the files thatget created during your build process, includinggenerated source files. Built objects should notgo into your source code directories. Instead,build them into a directory tree of their own. Tisseparation allows you to limit the scope of softwareversion management-managed directories to thosethat contain only source. It also groups the filesthat tend to be large and/or expendable into onelocation, and simplifies disk space management forbuilds.

• Use common build environments. Developers, test

engineers, and release engineers should all use orhave access to identical build environments. Muchtime is wasted when a developer cannot reproducea problem found in testing, or when the releasedproduct varies from what was tested. Remember:source + tools = product.

• Build often. Continuous integration (CI) withautomated regression testing safeguards your

development through continuous response to thegreatest source of defects: new check-ins. You will

1. detect build failures created by check-ins(don’t break the build)

2. find product functional problems createdby check-ins (don’t break the product.)

A beneficial side effect is that continuous integration andtesting produces link libraries and other built objectsthat can be used by developers. Every codeline shouldbe subject to regular, frequent, and complete builds andregression testing, even when product release is in thedistant future.

• Keep build logs and build outputs. For any builtobject you produce, you should be able to look upthe exact operations (e.g., complete compiler flagand link command text) that produced the lastknown good version of it. Archive build outputsand logs, including source file versions (e.g., a

label), tool and OS version info, virtual machineconfiguration, compiler outputs, intermediate files,built objects, and test results, for future reference. As large software projects evolve, components arehanded off from one group to another, and thereceiving group may not be in a position to beginbuilds of new components immediately. When theydo begin to build new components, they will needaccess to previous build logs in order to diagnosethe integration problems they encounter.

6. ProcessIt would take an entire paper, or several papers, toexplore the full scope of software version managementprocess design and implementation, and many suchpapers have already been written. Furthermore, yourshop has specific objectives and requirements that willbe reflected in the process you implement, and we donot presume to know what those are. In our experience,however, some process concepts are key to any versionmanagement implementation:

• Track change sets. Even though each file in acodeline has its revision history, each revision inits history is only useful in the context of a set of

related files. Te question “What other source files were changed along with this particular change tofoo.c?” can’t be answered unless you track changesets, or sets of files related by a logical change.Change sets, not individual file changes, are thevisible manifestation of software development.

• Track the ow of change. One clear benefit oftracking change sets is that it becomes very easyto propagate logical changes (e.g., bug fixes) from




one codeline to another. However, it’s not enoughto simply propagate change sets across codelines;you must keep track of which change sets havebeen propagated, which propagations are pending,and which codeline branches are likely donors orrecipients of propagations. Otherwise you’ll never

be able to answer the question “Is the fix for bug Xin the release Y codeline?” You should never have toresort to “diffing” files to figure out if a change sethas been propagated between codelines.

• Distinguish change requests from change sets. “What to do” and “what was done” are differentdata entities. For example, a bug report is a “what todo” entity and a bug fix is a “what was done” entity. Your software version management process shoulddistinguish between the two, because in fact therecan be a one-to-many relationship between changerequests and change sets.

• Give everything an owner. Every process, policy,

document, product, component, codeline, branch,and task in your software version managementsystem should have an owner. Owners give life tothese entities by representing them; an entity withan owner can grow and mature. Ownerless entitiesare like obstacles in an ant trail—the ants simplymarch around them as if they weren’t there.

• Use living documents. Te policies and proceduresyou implement should be described in livingdocuments; that is, your process documentationshould be as readily available and as subject toupdate as your managed source code. Documents

that aren’t accessible are useless; documents thataren’t updateable are nearly so. Process documentsshould be accessible from all of your developmentenvironments: at your own workstation, at someoneelse’s workstation, and from your machine at home. And process documents should be easily updateable,and updates should be immediately available.

• Learn and use integrations to supporting tools. Software version management often sits withina dizzying array of supporting tools in an ALM(Application Lifecycle Management) stack. Learnall the options for integrating your software versionmanagement system into tool sets that manage

requirements, features, projects, tests, releases, andmore.

As a software version management engineer, it’s alsoessential to master how your system integrates with Agile development tools and processes. CI (ContinuousIntegration), automated testing, work flow, defecttracking, and code review tools are all likely integrationsinto a software version management system.

Remember, software version management is often onepart of a larger ALM and/or Agile environment. Be sureyou understand all the integrations available for howyour software version management tool can support thislarger environment.

7. ConclusionBest practices in version management, like best practicesanywhere, always seem obvious once you’ve used them.Te practices discussed in this paper have worked wellfor us, but we recognize that no single, short documentcan contain them all. So we have presented the practicesthat offer the greatest return and yet seem to be violatedmore often than not.

We welcome the opportunity to improve this document,and solicit both challenges to the above practices as wellas the additions of new ones.

8. ReferencesBerczuk, Steve. Configuration Management Patterns ,1997.

Compton, Stephen B, Configuration Management for Software , VNR Computer Library, Van NostrandReinhold, 1993.

Continuus Software Corp., “Work Area Management”,Continuus/CM: Change Management for SoftwareDevelopment [PDF]

Dart, Susan, Spectrum of Functionality in Configuration Management Systems [PDF], Software EngineeringInstitute, 1990.

Jameson, Kevin, Multi Platform Code Management ,O’Reilly & Associates, 1994

Linenbach, erris, Programmers’ Canvas: A Pattern forSource code Management 1996.

Lyon, David D, Practical CM , Raven Publishing, 1997

McConnell, Steve, Best Practices: Daily Build and Smokeest , IEEE Software, Vol. 13, No. 4, July 1996

Van der Hoek, Andre, Hall, Richard S., Heimbigner,Dennis, and Wolf, Alexander L., Software Release Management ,Proceedings of the 6th European SoftwareEngineering Conference, Zurich, Switzerland, 1997.

Wingerd, Laura, Te Flow of Change [PDF], PerforceSoftware, Inc., 2005

Wingerd, Laura, Practical Perforce , O’Reilly & Associates,2005



North AmericaPerforce Software Inc.2320 Blanding AveAlameda, CA 94501USAPhone: +1 [email protected]

EuropePerforce Software UK Ltd.West Forest GateWellington RoadWokinghamBerkshire RG40 2ATUKPhone: +44 (0) 845 345 [email protected]

AustraliaPerforce Software Pty. Ltd.Suite 3, Level 10221 Miller StreetNorth SydneyNSW 2060AUSTRALIAPhone: +61 (0)2 [email protected]

Copyright © 2012 Perforce Software Inc. All rights reserved.

All trademarks or registered trademarks used herein are property of their respective owners.

p e r f o r c e . c o m

Updated January 2012, April 2012, James Creasy

Change propagationStreams graph, visual tools, defaultcodeline policies, enforcement of rules of

Flow of Change

Agile Flow of Change:info.perforce.com/Agile-Flow-of-Change-WP-Offer.html

Builds

Integrations with ALM:perforce.com/product/integrations/thirdparty_software_integrations

Agile tools:perforce.com/product/product_features/always_agile

Process

Software Version Management:perforce.com/products/perforce

WorkspacesStream views, P4Sandbox, DVCSconnectors

Use local task branching

Perforce Sandbox:perforce.com/sites/default/files/pdf/

p4sandbox-product-brief.pdf

Git as a Perforce Client:perforce.com/blog/120113/git-perforce-

client

Git-p4 feature of Git:kb.perforce.com/article/1417/git-p4

Codelines

Streams implement codelinesPerforce Streams:perforce.com/product/product_features/

perforce_streams

Child Codelines

Streams graph, default codeline policies,tofu scale

Use the ofu Scale:oreilly.com/catalog/practicalperforce/chapter/ch07.pdf

More about how Perforce supports these best practices

High Level Perforce Best Practices

Documents

Transcript of High Level Perforce Best Practices