VMware Backup Solutions: What To Do When Your Backup Fails

VMware is a software company that provides virtualization solutions, allowing businesses to create virtual computers (virtual machines) that run on a single physical computer. Think of it like creating multiple separate rooms (VMs) inside one large building (physical server), where each room can run its own operating system and applications independently. For those exploring different virtualization options, understanding the differences between platforms like VMware and VirtualBox can be crucial.

Backups in VMware typically rely on two critical technologies:

• Changed Block Tracking (CBT): A mechanism that identifies and tracks only the data blocks that have changed since the last backup
• Volume Shadow Copy Service (VSS): A Windows technology that allows backup applications to create consistent snapshots of data, even while applications are running

‍

Even with its robust qualities, VMware backups are not immune to failure, which can lead to data loss. If your backup is corrupted or inaccessible, our guide on troubleshooting common VMware issues can help you recover your information.

Troubleshoot: Changed Block Tracking (CBT) data corruption

In VMware vSphere versions 6.5 and 6.7, a severe backup corruption vulnerability can render your backups completely unusable. This problem occurs when three specific conditions are simultaneously present:

1. The virtual machine is hosted on vSphere 6.5 or 6.7.
2. Changed Block Tracking (CBT) was enabled for the VM.
3. A snapshot existed when CBT was enabled.

Solution and prevention

Step 1: Remove VM snapshots

1. Open VMware vSphere Client.
2. Navigate to the Virtual Machine inventory.
3. Identify VMs with existing snapshots.
4. Right-click on the VM.
5. Select "Snapshots" > "Manage Snapshots".
6. Review existing snapshots.
7. If multiple snapshots exist:
   
   • Select the most recent snapshot.
   • Click "Revert To" to consolidate snapshots.
   • Once reverted, delete all old snapshots.

Step 2: Reset Changed Block Tracking (CBT)

1. Open VMware PowerCLI.
2. Connect to your vSphere environment.
3. Use the following command to reset CBT:

# Connect to vCenter

Connect-VIServer -Server vcenter.yourdomain.com

# Reset CBT for a specific VM

$vm = Get-VM "VMName"

$vmView = $vm | Get-View

$vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec

$vmConfigSpec.ChangeTrackingEnabled = $false

$vmView.ReconfigVM($vmConfigSpec)

# Re-enable CBT

$vmConfigSpec.ChangeTrackingEnabled = $true $vmView.ReconfigVM($vmConfigSpec)

Step 3: Perform a full backup

1. Select the "Full Backup" option.
2. Verify backup software compatibility with your VMware version.
3. Confirm all VM disks are included.
4. Allocate sufficient time (full backups take longer).
5. Monitor backup job logs.
6. Verify backup completion status.

Troubleshoot: Virtual hard disk corruption

Virtual Hard Disk (VHD) corruption can severely impact virtual environments, rendering data inaccessible. Common causes include improper shutdowns, hardware failures, or software bugs. The CHKDSK feature can help you check and fix VHD corruption.

CHKDSK is a built-in Windows tool that scans a disk's file system (NTFS, FAT32) for integrity issues and attempts to fix any logical filesystem errors it finds. It can also check for bad sectors on the disk and mark them to prevent future use.

Solution and prevention

Step 1: Unmount the VHD 

1. Open Disk Management by pressing Win + X and selecting Disk Management.
2. Click on Action > Attach VHD and select the corrupted VHD file.
3. Ensure the VHD is unmounted before running CHKDSK.

Step 2: Open the command prompt as an administrator

1. Press the Windows Key.
2. Type "Command Prompt".
3. Right-click on Command Prompt and select "Run as administrator".

Step 3: Run the CHKDSK command

Use the following command structure to run CHKDSK:

chkdsk [drive letter]: /f /r

• Replace [drive letter] with the letter assigned to the drive you want to check (e.g., C: or D:).
• The /f switch tells CHKDSK to fix any errors it finds.
• The /r switch locates bad sectors and recovers readable information.

Step 4: Review results

After running the command, CHKDSK will provide a summary of its findings:

• It will display how many files were processed, any bad sectors found, and whether it could fix any issues.
• Pay attention to messages indicating that bad sectors were found; this may suggest potential data loss for files stored in those sectors.

Troubleshoot: Incomplete backup chains

Incomplete backup chains occur when a backup process is interrupted or fails, leading to missing data points. 

Solution and prevention

When your VMware backup is incomplete and you need an old file version, contacting a VM data recovery service provider is the best option to retrieve the files.

You can prevent incomplete VMware backups by following these tips:

• Set up alerts for failed backups.
• Regularly check logs for inconsistencies.
• Use multiple backup methods (e.g., full and incremental).
• Maintain a clear backup schedule.
• Test restorations are performed periodically to verify backup integrity.
• Monitor network stability during backup transfers.

Troubleshoot: Security software blocking backup processes

Security software can interfere with backup processes by blocking file access or consuming excessive system resources.

Consequences

• Slow backup performance
• Failed backups due to file access issues

Solution and prevention

Step 1: Identify interfering software

1. Monitor performance during backups to identify problematic applications.

Step 2: Adjust settings

1. Temporarily disable security software during backups.
2. Configure exceptions for backup applications.

Step 3: Regular updates

1. Keep security software updated to minimize compatibility issues.

Troubleshoot: Resource overload failures

Resource overload failures occur when insufficient memory or processing power hinders backup operations, especially in virtualized environments.

Solution and prevention

Step 1: Assess resource requirements

1. Analyze each virtual machine's (VM) specific workloads to determine its CPU, memory, and storage needs.
2. Establish baseline performance metrics for each VM to understand typical resource usage and peak demands.

Step 2: Configure resource allocation settings

1. Implement resource pools to aggregate physical resources. This allows for more efficient sharing among VMs.
2. Specify minimum resource reservations for critical VMs to guarantee availability during high-demand periods. For example, if a VM requires a minimum of 1GHz CPU, set this as the reservation.
3. Establish upper limits on resource allocation to prevent a single VM from monopolizing resources.

Step 3: Implement dynamic resource allocation

1. Use features such as VMware's dynamic memory allocation or Hyper-V's burstable memory settings to adjust resources based on current demand rather than fixed allocations.
2. Consider implementing auto-scaling solutions that automatically adjust resources according to workload requirements, ensuring optimal performance without manual intervention.

Step 4: Educate stakeholders

1. Ensure that IT staff are trained on best practices for resource management and that documentation is available for reference. This promotes adherence to established policies and procedures.

Troubleshoot: Unsupported OS versions

Using unsupported operating system versions can lead to compatibility issues with backup software, resulting in failed backups.

Solution and prevention

Step 1. Upgrade OS versions

1. Regularly update operating systems to supported versions.

Step 2. Compatibility checks

1. Verify compatibility of backup solutions with current OS versions before implementation.

While effectively troubleshooting VMware backup failures is essential, achieving true data protection requires a more comprehensive and proactive strategy. This involves consistent monitoring of your systems, meticulous configuration of both your VMware environment and backup solutions, and ensuring regular software updates for optimal compatibility. The top three things we advise you to always keep in mind are to: prioritize the security and immutability of your backup data, continually optimize backup performance, and understand when to engage professional data recovery experts. Implementing these integrated measures will effectively safeguard your critical information against unforeseen challenges.