CERN - IT Department CH-1211 Genève 23 Switzerland t SVN Pilot: CVS Replacement Manuel Guijarro...

CERN - IT DepartmentCH-1211 Genève 23

Switzerlandwww.cern.ch/it

SVN Pilot: CVS Replacement

Manuel Guijarro

Jonatan Hugo Hugosson

Artur Wiecek

David Horat

Jonathan Brugge

Michel Manent

September 2008



2

Outline

• Introduction• Motivation• Subversion• Objectives• Performance Tests• Security• Implementation• Questions



Version Control Systems

• Maintain current and historical versions of files and data (source code)

• There are many commercial and Open Source VC Systems:– (Centralised) CVS/SVN

– (Distributed) GIT, Bazaar, Darcs, GNU arch, Mercurial, Monotone, etc

– But subversion seems to be the most popular one (used by GCC, Phyton, PuTTY, Apache, GNOME, KDE, etc)

• Physics User Community: (IN2P3, ROOT, Totem..)

3



CERN Central CVS Service

• Hosts over 330 Software Projects– 29 for Atlas– 46 for CMS– 8 for LHCb,…..

• Over 3000 developers registered• Over 90 GBytes of source code• Creates 250 Remedy tickets per year• Over 100000 commits per month

4



CERN Central CVS Service



Central CVS service features

• High Availability and Load Balancing• Web interface to repositories• Usage Statistics• Repository Remote Replication + Mirroring• Daily archive of Repositories and DR• Developers Mailing list• Pre/Post Commit Actions (such us e-mail

notification, etc)• Various access method (ssh/kerberos)• Role split (CVS Admin/Librarian/Developer)

6



Motivation for SVN Pilot

• Originally designed to host less than 100 projects

• Requests to provide a central SVN service:– From CMS– From ATLAS (case study in 2006)– And from many others

• CVS is over 20 years old while SVN is this millennium technology

• Requests for Read Access control



SVN vs. CVS

8

Feature SVN CVS

Speed Faster Slower

Permission Full Limited

File types All Limited

Off line operations Yes No

Repository format Database File system

Locks No Yes

Atomic commits Yes No



New Features (SVN 1.5)

• Automatic update of working copy• Merge tracking

– Subversion keeps track of what changes have been merged where

• Sparse checkouts• Interactive conflict resolution

9



Pilot Objectives

• Provide current CVS service features• Add new features (available with SVN)

– Control Read access per path (module)– Authenticated Web access– Binary files handling

• Ease CVS to SVN migration• Improved usage statistics (SVN Stats)• Handling of first line support via the Help Desk• Delegate administrative tasks to Software

Librarians of each project• Prevent uncontrolled setup of SVN servers• Manpower: 1.2 FTE project



Timetable

11



SVN Pilot study

• Access methods– https

– ssh

• Shared storage– NFS 3/4

– AFS

• Securing service– Restricted Shell

– Chrooted hooks (commit scripts)

• Infrastructure: – Librarian tools, Statistics, Web Interface,…

12



Performance Tests

• SVN check out of a 110 Mb project• Parameters

– AFS/NFS3/NFS4– HTTPS/SSH

13



AFS vs NFS3 (1 server)

14



AFS vs NFS4 (1 server)

15



AFS vs NFS4 (3 servers)

16



Preliminary Conclusions

• AFS much faster than NFS• SSH much faster than https• SSH scales very well with high load• … • New tests ongoing (with mixture of read and

write operations)

17



Security

• Project Isolation• Windows/Linux clients • Worldwide access• Shared file system independent• Hooks executed on servers

• Librarians may put any script into the hooks

• Librarians might need file system level access to repository – being studied

18

Security risk!!



Hooks (scripts)

Client Server

19

Svn commit

Pre-commit hook is executed

Post commit hookis executed

SVN: Commit OKEmail notificationrecieved



Hook scripts chrooted:

Server

20

svnserer

hooks/post-commit hook

RepositoriesSystem files

Usr-hooks/post-commit hook

Librarian hooks: jailed

Repository (1)



Architecture

• svn.cern.ch (rw)– Secured subversion

server (only ssh)– Read and write

access to repository

• svnweb.cern.ch (ro)– User documentation– Project request– SVN web interface– Usage statistics

21



Pilot Implementation Summary

• SSH access for SVN clients– Restricted shell for all SVN clients– Hooks chrooted

• SVN web (ro)– Web interfaces: websvn, trac– SSO Authenticated access

• Administration delegated to librarian– Access rights– Hooks– Admin tools

22



Conclusions

• Secure service• This will replace CVS by end of 2009• The service is supported (pre-production)• Pilot setup may differ from final setup

– Access method, Web interface, shared file system, etc.

– Changes will be transparent to the users

23



Support

http://cern.ch/svn

–Try the pilot

–Documentation

[email protected]

24



Questions?

Thanks For Listening….

M. Guijarro, A. Wiecek, David Horat, Jonathan Bugge, M. Manent, H. Hugosson

25

CERN - IT Department CH-1211 Genève 23 Switzerland t SVN Pilot: CVS Replacement Manuel Guijarro...

Documents

Transcript of CERN - IT Department CH-1211 Genève 23 Switzerland t SVN Pilot: CVS Replacement Manuel Guijarro...