Dedicated Storage Daemon Best Practices
Note: We will continue to update this documentation as time goes on and we work with customers on their specific environments. If you have any questions about your specific environment, please do not hesitate to contact us. Thank you!
Many of customers have dedicated storage daemon nodes that recieve thousands of jobs daily for multiple clients. In order to optimize these environments, we have a few recommendations that are particularly important in these scenarios.
1. Make sure you hardware that is powerful enough and a filesystem (often forgotten) that is up to the job. XFS is excellent with handling large files, although ext4 is more conveinent and a definite improvement over ext3. You can read more about this here: http://community.bac...ecommendations/
2. Make sure your pool volume is not too large. Generally a 50G volume size works well for most partition sizes under a few TB usable. Once you surpass this, the pool volume size can be increased to 100G safely.
3. Enable the disk autochanger to allow bacula to process jobs concurrently. As of 6/28/12, this involves removing the existing device configuration on your SD (the config is located at /opt/bacula/etc/bacula-sd.conf, backup first!) and using the following template to configure the autochanger with its devices. Once complete, please raise a ticket with support to enable autochanging on your SD. We are working towards having an auto-installer for an autochanger SD, however, in this meantime we have seen a significant performance boost by adding it manually
Our support team is more than happy to configure the autochanger for you, however, we will need root access to your SD. Alternatively, if you wish to configure the autochanger, please use the template config below, and open a ticket for support to enable autochanging on your SD.
As of 9/1/12, the autochanger config is now installed and enabled automatically by the Bacula4Hosts SD installer. Therefore, it is no longer necessary to manually configurer the autochanger after SD installation.
Autochanger & Devices config for SD
NOTE: hostname.com should match the hostname for your host. Furthermore, the archive device and spool directory make change on your SD depending on where your backup partition is mounted.
SD config (remove old device config/edit accordingly - this is based on hostname.com-sd):
Autochanger {
Name = hostname.com-sd
Changer Device = /dev/null
Changer Command = ""
Device = hostname.com-sd-0, hostname.com-sd-1
Device = hostname.com-sd-2, hostname.com-sd-3
Device = hostname.com-sd-4, hostname.com-sd-5
Device = hostname.com-sd-6, hostname.com-sd-7
Device = hostname.com-sd-8, hostname.com-sd-9
Device = hostname.com-sd-10, hostname.com-sd-11
Device = hostname.com-sd-12, hostname.com-sd-13
Device = hostname.com-sd-14
}
Device {
Name = hostname.com-sd-0 # Add a hyphen to SD/autochanger name & match with drive index
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 0
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-1
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 1
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-2
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 2
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-3
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 3
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-4
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 4
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-5
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 5
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-6
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 6
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-7
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 7
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-8
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 8
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-9
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 9
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-10
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 10
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-11
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 11
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-12
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 12
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-13
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 13
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
}
Device {
Name = hostname.com-sd-14
Device Type = File
Media Type = storage1 #unique to each archive device path, different path, different mediatype
Archive Device = /backup/storage1
AutomaticMount = yes
AlwaysOpen = yes
RemovableMedia = yes
Autochanger = yes
Drive Index = 14
Maximum Concurrent Jobs = 5
Volume Poll Interval = 15
LabelMedia = yes
Spool Directory = /backup/spool
Related Articles
Storage Daemon (SD) Recommended Hardware Recommendations
Note: We will continue to update this documentation as time goes on and we work with customers on their specific environments. If you have any questions about your specific environment, please do not hesitate to contact us. Thank you! Hardware CPU: ...
Backup Fails with error: Failed to authenticate Storage daemon
Problem: The password used for authentication in SD config does not match that listed in GUI Common Solution: When this occurs you should make sure your SD is updated with the latest release. Please login to your account and perform this ...
Error: Failed to connect to Storage daemon: yourSD IP:9103
Problem: The FD cannot connect to the SD IP/port Common Solution: Please make sure you can telnet to the IP and port from the FD to the SD along with telnet from SD to FD port 9102(check any routes and firewall configs). If LAN IPs are in use, make ...
How Storage Devices, Storage Pools and Volumes work in ClusterLogics
Storage Devices (SD) These are the devices which store the backup data (WAN or LAN attached) Storage Pools Storage pools are created on storage devices (see above) The are comprised of volumes and are resized up or down based on adding/removing a ...
Multiple Directors - Sharing Storage (SDs)
When running an install with multiple directors, it is possible to share storage (SD role) across multiple directors. This means that jobs running on director1 and director2 can both use the same SD if so desired. Often times, this is done with WAN ...