Post on 17-Oct-2020
Welcome to the
Introduction WorkshopWLAN:id: osbcpassword: osbc2015
Workshop: Bareos Introduction
Agenda1. Setup a virtual machine2. Introduction of Hosts and Attendees3. Install Bareos4. Install BareosWebUI5. Introduce Bareos Architecture6. Introduce Bareos Configuration7. Addressing, Passwords, Names8. Working with Bareos9. Working with BareosWebUI10. Exercises11. Open Talk
Setup Virtual Machinefirst stepsMake sure you have VirtualBox installedConnect your laptop to the networkget DHCP address
Download and unpack Appliance ofyour choice
download from (Wired)openSUSE 13.2Debian 8CentOS 7
Import into virtualbox
http://192.168.1.1/
Import appliance1. Start Virtualbox2. File .. Import Appliance3. Choose .ovf
Introduction of the hostsFrank Berkgkempermain programmer of bareos WebUI
Philipp Storzfounder of bareosprogramming and coordination
Introduction of attendeesPlease tell us yournameorganizationexperience with bareos or bacula
Create teamstwo personscan solve tasks together
login informationUsername: root / bareosPassword: bareos
configure keyboard
configure individual hostnameedit /etc/hostnameedit /etc/hostsreboot
Take a snapshotTake a snapshot of the vmIf things go wrong, you can always go back
configure/check Network:bridged Network to eth0/wlan0
Installation of Bareos
2: Installing Bareos*Use Use Database of your choice
http://doc.bareos.org
http://download.bareos.org/bareos/release/15.2/
Installation: PackagesAdd RepositoryInstall Packages
Installation: Prepare Databasecreate_bareos_databasemake_bareos_tablesgrant_bareos_privileges
Installation: Start the Daemonssystemctl start bareosdirsystemctl start bareossdsystemctl start bareosfd
Installation: Basic Testsstart bconsolestatus directorstatus clientstatus storage
Installation: SUCCESSWhat did we do?added the repositoryinstalled the bareos softwarestarted the daemonschecked daemons are running
Installation BareosWebui
https://github.com/bareos/bareoswebui/blob/bareos15.2/doc/INSTALL.md
BareosWebui: restricted consoleStep 1 is already doneCentOS: yum install epelreleaseStep 2: Configure restricted consoles
BareosWebui: Apache Configuration
BareosWebui: Webui configuration
Webui: First Login
BareosWebui: Watch out!CentOS: Needs EPEL RepoCentOS: Selinux needssetsebool P httpd_can_network_connect on
Bareos Architecture
Communication via TCP/IPdefined ports are usedcommunication can use TLS
FileDaemonRuns on Client Computerread, write, verify filesread, write ACLs, attributesmake VSS snapshotschecksum calculationcompression/encryptionrun scripts
Storage Daemondevice access (disk, tape)media changer controlread barcodes labelswrite logical labelsrun copy and migration jobshandle media errors
Directorhandle catalogmedia and pool handlingschedulingdetermine what to backupbackup leveldoes message, statistics and reportsrun scripts
Catalogstore info about all files, media, jobsPostgreSQL/MySQL/SQLite DB
bconsole
UI for restoresquery statuscatalog queriesrun jobs
Bareos WebUIResource InformationBackupRestoreStatisticsErrorsNO DB connection
Bareos Architecture
Architecture Testwhich bareos daemon schedules the backups?where is the administrative data stored?which bareos daemon reads and writes files on the client?which bareos daemon handles media?
Bareos ConfigurationConfiguration is done in config filesEach daemon has its own config fileusually in /etc/bareosbareosdir.confbareosfd.confbareossd.confbconsole.conf
Bareos configuration syntaxConfiguration files consist ofresourcesdirectives
resources can have subresources resourcename { directive = value directive = value SubResource { directive = value directive = value } }
Most important Director Resources:DirectorFilesetScheduleClientJob
Director Resource: Definition ofDirectors' properties
Director { # define myself Name = bareosdir QueryFile = "/usr/lib/bareos/scripts/query.sql" Maximum Concurrent Jobs = 1 Password = "UbCeWuuamTN/FNQhNH7rbdmx8X+ra9j0UkMMrSdUmkO4" # Console password Messages = Daemon
# remove comment in next line to load plugins from specified directory # Plugin Directory = /usr/lib64/bareos/plugins }
FileSet: Definition what to backup FileSet { Name = "Full Set" Include { Options { signature = MD5 } File = /usr/sbin }
Schedule: Definition when to run abackup
Schedule { Name = "WeeklyCycle" Run = Full 1st sun at 23:05 Run = Differential 2nd5th sun at 23:05 Run = Incremental monsat at 23:05 }
Client: Definition of a Client Client { Name = bareosfd Address = bareos Password = "lecCqzgBjxgM0J3+1adiuLzhy0cPGIHrdYMdtGHMbvKX" # password for FileDaemon File Retention = 30 days # 30 days Job Retention = 6 months # six months AutoPrune = no # Prune expired Jobs/Files }
Job: Definition of a Jobcombines the other resources to a runnable backup job
Job { [...] Client = bareosfd # what client to backup? FileSet = "Full Set" # which files to backup? Schedule = "WeeklyCycle" # when to backup? Storage = File # where to backup? Messages = Standard # where to send messages? Pool = File # what target pool? }
Configuration Testwhich resource configures when a job is scheduled?which resource configures what files are backed up?which resource configures what client to backup?which resource combines the other resources?
Addresses, Passwords, NamesTricky for beginnersDirector is the bossNeeds to know how to connectNeeds to authenticate
Addresses, Passwords,Names
Addressing via1. Address2. Port (usually default value is usedAuthentication via1. Name2. Password
Addresses, Passwords, NamesDirector knows of each daemon (Client, Storage)connection infoAddressPort
authentication infoNamePassword
Client { Name = bareosfd Address = bareos Password = "GfYPqgmav" [FDPort]}
Addresses, Passwords, NamesEach Daemon (Client/Storage) knowsauthentication infoDirector NameDirector Password
Director { Name = bareosdir Password = "GfYPqgmav" }
Names and Passwords
Director FileDaemon
StorageDaemonClient { Name = CLINAME Password = CLIPW}
Director { Name = DIRNAME}
Storage { Name = STORNAME Password = STORPW}
Director { Name = DIRNAME PASS = CLIPW}
Director { Name = DIRNAME PASS = STORPW}
FileDaemon { Name = CLINAME}
Storage { Name = STORNAME}
connection dir to fd1. Lookup client's address in client resource2. Connect to that address and port3. Use name and password to authenticate
Example: director to fd connection:Director Client
Client { Name = client-fd Address = client.example.com FD Port = 9102 Password = "+hAtR..."}
consolestatus client =client-fd
lookup client-fd resource
network connect toclient.example.com:9102
hello director bareos-dir calling
pw hash=xxxx
Director { Name = director-dir...
lookup baros-dir resource
Director { Name = director-dir Password = "+hAtR..."
check if password hashis correctaccept connection
What about the Data Channel?
Director FileDaemon
StorageDaemonClient { Name = CLINAME Password = CLIPW}
Director { Name = DIRNAME}
Storage { Name = STORNAME Password = STORPW}
?Storage { Name = STORNAME Password = STORPW}
Client { Name = CLINAME Password = CLIPW}
Data is transferred directly betweenFiledaemon and Storage Demon
BUT: No Storage Resource ClientNo Client Resource in Storage
active Client
Director FileDaemon
StorageDaemon
Director { Name = DIRNAME}
generateAuthorization key
Auth Key: ACDD-DAFAD-ZUHZ
Authorization keyClient Name
Address = STORADDR Auth Key = ACDD-DAFAD-ZUHZ
Storage { Name = STORNAME Password = STORPW Address= STORADDR}
Auth Key: ACDD-DAFAD-ZUHZClient Name: BackupClient1
Authorizati
on key
Storage Address
1
23
start console Connecting to Director localhost:9101 1000 OK: localhostdir Version: 15.2.1 (24 August 2015) Enter a period to cancel a command. *
execute help
estimate command*estimateThe defined Job resources are: 1: BackupClient1 2: BackupCatalog 3: RestoreFiles Select Job resource (13): 1 Using Catalog "MyCatalog" Connecting to Client localhostfd at localhost:9102 2000 OK estimate files=576 bytes=53,164,275
estimate listing *estimate listing [..] Using Catalog "MyCatalog" Connecting to Client bareosfd at bareos:9102 rwxrxrx 1 root root 13973 20140917 15:15:46 /usr/sbin/grubinstall.unsupported rwrr 1 root root 987 20140917 15:15:46 /usr/sbin/envvars lrwxrwxrwx 1 root root 24 20140918 14:22:29 /usr/sbin/httpd2 > /usr/sbin/httpd2prefork [..] rwxrxrx 1 root root 14768 20140917 15:15:46 /usr/sbin/switch_root rwxrxrx 1 root root 1593 20140917 15:15:46 /usr/sbin/grubonce drwxrxrx 2 root root 12288 20140918 14:22:29 /usr/sbin 2000 OK estimate files=310 bytes=18,987,521
run job BackupClient1 *run Automatically selected Catalog: MyCatalog Using Catalog "MyCatalog" A job name must be specified. The defined Job resources are: 1: BackupClient1 2: CopyToTape 3: BackupCatalog 4: RestoreFiles Select Job resource (14): 1 Run Backup job JobName: BackupClient1 Level: Incremental Client: bareosfd Format: Native FileSet: Full Set Pool: File (From Job resource) Storage: File (From Job resource)
check status by looking for messages*messages18Sep 17:16 bareosdir JobId 1: No prior Full backup Job record found.18Sep 17:16 bareosdir JobId 1: No prior or suitable Full backup found in catalog. Doing FULL backup.18Sep 17:16 bareosdir JobId 1: Start Backup JobId 1, Job=BackupClient1.20140918_17.16.49_0818Sep 17:16 bareosdir JobId 1: Created new Volume "File0001" in catalog.18Sep 17:16 bareosdir JobId 1: Using Device "FileStorage" to write.18Sep 17:16 bareossd JobId 1: Labeled new Volume "File0001" on device "FileStorage" (/var/lib/bareos/storage).18Sep 17:16 bareossd JobId 1: Wrote label to prelabeled Volume "File0001" on device "FileStorage" (/var/lib/bareos/storage)18Sep 17:17 bareossd JobId 1: Elapsed time=00:00:15, Transfer rate=1.267 M Bytes/second18Sep 17:17 bareosdir JobId 1: Bareos bareosdir 14.3.0 (21Aug14): Build OS: x86_64suselinuxgnu suse openSUSE 13.1 (Bottle) (x86_64) JobId: 1 Job: BackupClient1.20140918_17.16.49_08 Backup Level: Full (upgraded from Incremental) Client: "bareosfd" 14.3.0 (21Aug14) x86_64suselinuxgnu,suse,openSUSE 13.1 (Bottle) (x86_64) FileSet: "Full Set" 20140918 17:16:52 Pool: "File" (From Job resource) Catalog: "MyCatalog" (From Client resource)
restore files*restoreTo select the JobIds, you have the following choices: 1: List last 20 Jobs run 2: List Jobs where a given File is saved 3: Enter list of comma separated JobIds to select 4: Enter SQL list command 5: Select the most recent backup for a client[..]5[...]You have selected the following JobId: 1
Building directory tree for JobId(s) 1 ... ++++++++++++++++++++++++++++++++++++++++++++309 files inserted into the tree.cwd is: /
$ find *[...]
check restore*mess18Sep 17:29 bareosdir JobId 2: Start Restore Job RestoreFiles.20140918_17.29.41_0918Sep 17:29 bareosdir JobId 2: Using Device "FileStorage" to read.18Sep 17:29 bareossd JobId 2: Ready to read from volume "File0001" on device "FileStorage" (/var/lib/bareos/storage).18Sep 17:29 bareossd JobId 2: Forward spacing Volume "File0001" to file:block 0:196.18Sep 17:29 bareosdir JobId 2: Bareos bareosdir 14.3.0 (21Aug14): Build OS: x86_64suselinuxgnu suse openSUSE 13.1 (Bottle) (x86_64) JobId: 2 Job: RestoreFiles.20140918_17.29.41_09 Restore Client: bareosfd Start time: 18Sep2014 17:29:43 End time: 18Sep2014 17:29:45 Elapsed time: 2 secs Files Expected: 1 Files Restored: 1 Bytes Restored: 49,296 Rate: 24.6 KB/s FD Errors: 0
check in system:$ find /tmp/bareosrestores//tmp/bareosrestores//tmp/bareosrestores/usr/tmp/bareosrestores/usr/sbin/tmp/bareosrestores/usr/sbin/mtx
bconsole testWhat command shows possible available commands?What command is used to show how much data a backup willcontain, and how can the exact files be shown?What command is used to run a backup?What command is used to do a recover files?
Exercise 1BackupClient1 should additionally backup /etcHint: reload command can be used to update dirconfiguration without new start
Solution for Exercise 11. Add line "File = /etc" to FileSet "Full Set" in /etc/bareosdir.conf2. open bconsole and type "reload"3. run "estimate listing" to see if /etc would be backed up4. alternatively, run job "BackupClient1"
status commandshows status of system components
status directorshows next scheduled jobs
shows running jobs
show terminated jobs
Scheduled Jobs:Level Type Pri Scheduled Name Volume===================================================================================Incremental Backup 10 19Sep14 23:05 BackupClient1 File0001Incremental Copy 10 19Sep14 23:05 CopyToTape Full Backup 11 19Sep14 23:10 BackupCatalog File0001
Running Jobs:Console connected at 19Sep14 13:51No Jobs running.
Terminated Jobs: JobId Level Files Bytes Status Finished Name ==================================================================== 1 Full 310 18.98 M OK 18Sep14 17:17 BackupClient1 2 1 49.29 K OK 18Sep14 17:29 RestoreFiles
status clientshows runnig jobs on client
shows terminated jobs on client Terminated Jobs:
info comes from local status filecan differ from director view as other director can also connectthis client
Running Jobs:Director connected at: 19Sep14 13:54No Jobs running.====
JobId Level Files Bytes Status Finished Name ====================================================================== 1 Full 310 18.98 M OK 18Sep14 17:17 BackupClient1 2 1 49.29 K OK 18Sep14 17:29 RestoreFiles 3 Incr 0 0 OK 18Sep14 23:05 BackupClient1====
status storageshows running jobsshows waiting jobsshows terminated jobsshows device status Autochanger "LTOChanger" withdevices:
Device "FileStorage" (/var/lib/bareos/storage) is not open.Device "Drive1" (/dev/nst0) is not open.
shows volume status
"Drive1" (/dev/nst0)
Drive 0 is not loaded.
Used Volume status:====
====
status schedulershows what jobs are triggered by which schedule
shows a preview for 7 days
Schedule Jobs Triggered===========================================================WeeklyCycle BackupClient1 CopyToTape
WeeklyCycleAfterBackup BackupCatalog
Scheduler Preview for 7 days:
Date Schedule Overrides==============================================================Fri 19Sep2014 23:05 WeeklyCycle Level=IncrementalFri 19Sep2014 23:10 WeeklyCycleAfterBackup Level=FullSat 20Sep2014 23:05 WeeklyCycle Level=IncrementalSat 20Sep2014 23:10 WeeklyCycleAfterBackup Level=Full[...]
Exercise 2Full Backup of Job BackupClient1 should be scheduled in 5minutes
Solution for Exercise 21. Add line "Run = Full at 15:05" to Schedule "WeeklyCycle" in
/etc/bareosdir.conf2. open bconsole and type "reload"3. run "status schedule schedule=WeeklyCycle" and check if
schedule is updated4. wait 5 minutes and check if backup starts
Exercise 3: backup partner's clientPart 1: Configure link between director and client
Solution for Exercise 3 p.11. Add a client ressource to your director configuration pointing to
your partner's client2. Add a director ressource to your client's config pointing to your
partner's director3. Restart director and filedaemon4. run status client to see if your partner's client is accessible
Exercise 3: backup partner's clientPart 2: configure a backup job for partner's client
Solution for Exercise 3 p.21. Create a fileset to backup your partner's client2. Create a job ressource to backup partner's client3. Run backup of your partner's client4. Run restore to your partner's client
Open talkyour questions?other ideas:Disaster recoveryusage of bls/bextractcheck copy to tape setup in vm
Philipp Storz, Bareos GmbH & Co. KG Open Source Backup Conference 2015