Issue
Space in a Backup Exec Deduplication Folder is not being automatically reclaimed fast enough.
Error
v-79-57344-922 - 0x039A The backup failed because the Backup Exec PureDisk Filesystem Service did not respond. Stop the Backup Exec services, stop the PureDisk Filesystem service if it is running, and restart the Backup Exec Services. Then retry the backup.
Solution
Note: This process is time consuming and will just run a process that normally happens every 12 hours anyway. It will not resolve a problem where too much data is being backed up to a too small deduplication folder. On the other hand, it will not help in cases where an unrelated deduplication folder problem is happening and there is already plenty of free space in the deduplication folder. Before going through this procedure, it is recommended to run the crstats --convert-size command from a command line window and examine the Storage Pool Available Space value. If this is already large enough to accommodate your backup operations, performing the rest of these steps is not likely to resolve the issue.
Reclaiming space in a deduplication folder is not as
easy as it is in a Backup-to-Disk folder. An erase of some media in
the deduplication folder will not immediately reflect additional free
space on disk.
In order to free space in a deduplication folder, perform the following steps.
Note: If a substantial amount of
space is being reclaimed, media server performance will degrade. Where
possible, stop other backup operations that are using the deduplication
folder during the process.
Reclaiming space in a deduplication folder
1. Overwrite or Erase media in the deduplication folder. Do not attempt to erase IMG media.
Note:Erase media is not applicable in Backup Exec 2012 and cannot be performed.
Note: When media gets erased or overwritten, Backup
Exec deletes the file associated with the media from the deduplication
folder. However, the deduplication folder services do not immediately
process the request. The deduplication folder services batch process
all of the deduplication folder requests into the database twice a day
(12:20 AM and 12:20 PM). These times are not user schedulable.
2. To force the deduplication folder to process its
queue, open a command prompt (Start -> Run -> cmd) and use the
following commands on the media server where the deduplication folder is
located.
Default Installation Directory: C:\Program Files\Symantec\Backup Exec
cd "\Program Files\Symantec\Backup Exec"
crcontrol --processqueue
crcontrol --processqueueinfo
Continue to perform the crcontrol --processqueueinfo command periodically until it reports a "No" status for Busy and Pending.
Then repeat the process.
crcontrol --processqueue
crcontrol --processqueueinfo
Then perform garbage collection using the following command:
crcollect -v -m +1,+2
Then make sure the garbage collection commands get processed.
Then repeat the process.
crcontrol --processqueue
crcontrol --processqueueinfo
Continue to perform the crcontrol --processqueueinfo command periodically until it reports a "No" status for Busy and Pending.
3. Compact the space in the deduplication folder.
crcontrol --compactstart
crcontrol --compactstate
Continue to perform the crcontrol --compactstate command periodically until it reports a "No" status for Compaction busy.
4. Verify that space has been made available in the
deduplication folder using the following command in the same command
box that is already open.
crstats --convert-size
Note: Checking the free space available on the file system will not necessarily show any space has been freed up. This happens because the deduplication folder service does not delete the files it has allocated, but instead will use the free space available within those files.
Even with all of this, one has to remember that the entire purpose of a
deduplication folder is to try to keep only one copy of data in the
folder no matter how many times it has been backed up.
Example: If there are 5 pieces of media that have full backups of a particular system and the above procedure to delete 4 of those 5 media, very little space will be reclaimed.
This is because all 5 pieces of media reference one copy of the data. A particular chunk of data can't be erased and the space freed until all references to it have been deleted first.
Note: There are rare circumstances in which some objects cannot be released during the garbage collection phase. In these cases, it is necessary to stop and restart the Backup Exec Deduplication Services. In order to restart the BE Deduplication services on the media server, use the following procedure:
Stop the Backup Exec services.
Using the Windows Service Control manager, stop the following services in order. Keep in mind that these services may be in the middle of processing operations. If the service does not shutdown, wait until it is not busy and try again.
- Backup Exec PureDisk FileSystem Service
- Backup Exec Deduplication Engine
- Backup Exec Deduplication Manager
- postgresql-8.3
Then restart the services in the reverse order, and start the Backup Exec services.