LinHES Forums http://forum.linhes.org/ |
|
What a cool app NCDU http://forum.linhes.org/viewtopic.php?f=24&t=24047 |
Page 1 of 1 |
Author: | mattbatt [ Mon Sep 01, 2014 10:11 pm ] |
Post subject: | What a cool app NCDU |
I got a warning that my Harddrive was 100% full! Since my storage Directories say that I'm only about 50% full I was confused. I was able to install NCDU and figure out that my redundant database backup folder on a 15 GB partition had grown out of control. NCDU was awesome at pointing me to the largest directories. Thanks admins for putting it in the Repo. |
Author: | bgrant3406 [ Wed Sep 03, 2014 6:50 am ] |
Post subject: | Re: What a cool app NCDU |
I had noticed that while my main backup directory was getting pruned, my remote backup directory wasn't, not a huge issue since it is on a 1TB drive, but knowing me, I would forget about it and in a few year run into the same issue you ran into. My MBE has two Harddrives, with the second drive setup via the LinHES setup menu->advanced settings->screen 2, remote backup enabled as: Code: /data/storage/disk1/backup/system_backups I wanted 2 main things, for my remote backup directory to be updated the same as my main backup directory and for there to be a message in "myth_mtc.log" indicating if files had been removed. It would be nice if the log message was towards the end of the "myth_mtc.log" file, rather than somewhere in the middle. So being a total hack at programming, I made the following changes to: Code: /usr/LH/bin/lh_system_backup_job I replaced the entire Code: function remove_old_backups() with the following: Code: function remove_old_backups(){ #remove old backups NumBackups=`ls $BACKUPDIR/backup*.tgz|wc -l` if [[ $NumBackups -gt $KeepBackups ]]; then numdel=$(($NumBackups-$KeepBackups)) echo echo " Removed Oldest Backup File from $BACKUPDIR: " rm -f `ls $BACKUPDIR/backup*.tgz -tr1|head -$numdel` else echo echo " No Backups removed, less than "$KeepBackups" backup files in directory. " fi if [ x$RemoteBackup = x1 ] then localRemoteCheck=`echo $RemoteBackupDir | cut -d: -f1` if [ x$localRemoteCheck = xdir ] then localRemotedir=`echo $RemoteBackupDir | cut -d: -f2` RemoteNumBackups=`ls $localRemotedir|wc -l` if [[ $RemoteNumBackups -gt $KeepBackups ]]; then remnumdel=$(($RemoteNumBackups-$KeepBackups)) echo echo "Removed Oldest Remote Backup File: " rm -f `ls $localRemotedir/backup*.tgz -tr1|head -$remnumdel` else echo echo " No Backups removed, less than "$KeepBackups" backup files in directory. " fi else /usr/bin/func ${RemoteBackupDir} ping| grep -q "FAILED" rc=$? if [ $rc = 0 ] then #this is here to mark failed removal of the backup. echo " Remote drive failed, Drive appears to be off-line. " else echo " Reminder to manually cleanup older backups from /usr/bin/func ${RemoteBackupDir} " fi fi fi } To move the message to the end of the log, I needed to move when the removal function was run. Since the script above had a check to make sure there is a minimum number of backups before it will remove any, I felt it was pretty safe to move the "remove_old_backups" call. I commented out the original location as shown: Code: #------------------------------------ lock_myth if [ "x$1" = "x" ] then #remove_old_backups backup update_backup_status remote_backup add_link remove_old_backups else #this is where we attempt to transfer again as part of the cronjob if [ -f $BACKUPDIR/remote_backup_failed.txt ] then mv -f $BACKUPDIR/remote_backup_failed.txt /tmp while read line do echo $line remote_transfer $line done < /tmp/remote_backup_failed.txt rm -f /tmp/remote_backup_failed.txt fi remove_link fi unlock_myth echo $backup_status > /var/run/systembackup.rc exit $backup_status The main issue with the moving "remove_old_backups" is that you will end up with 13 backups in rotation instead of 14. To change the number to something other than 13, locate the following line (should be about the 6th line from the top of the file) and change the 13 to what ever number you wish, I changed mine to 14 to maintain two weeks worth of backup files: Code: KeepBackups=13 The nice thing about the way the original pruning works is it will remove any files over the number specified in "KeepBackups=", so if you have had a bunch of files built up in your remote backup drive, after the change is made to the file and "myth_mtc.py" runs, all of the excess files should be removed. Hopefully if someone has a similar issue and needs a starting point on where to look/what to change, this will give them a quick idea of what to look for/at. These changes have only been tested on my specific setup, since I don't want to potentially mess up my system, I have no way to test the other scenario's or extend the scripts so that it will correctly prune either a remote network drive, or the secondary backup location when remote backup is not enabled via LinHES setup Advanced menu option. HTH anyone wanting to automate cleaning up the remote backup directory. Bruce. |
Author: | brfransen [ Wed Sep 03, 2014 9:08 am ] |
Post subject: | Re: What a cool app NCDU |
Bruce, Nice job figuring out a solution to your problem. A couple of things to note. First, be aware that the next time the LinHES-system package is updated/reinstalled all your changes to lh_system_backup_job will be overwritten. Second, there is no need for the reminder to manually clean up files in SECBACKUPLINK as it should be a straight clone (using rsync) of the BACKUPDIR. |
Author: | bgrant3406 [ Wed Sep 03, 2014 10:14 am ] |
Post subject: | Re: What a cool app NCDU |
brfransen wrote: Bruce, Nice job figuring out a solution to your problem. A couple of things to note. First, be aware that the next time the LinHES-system package is updated/reinstalled all your changes to lh_system_backup_job will be overwritten. Second, there is no need for the reminder to manually clean up files in SECBACKUPLINK as it should be a straight clone (using rsync) of the BACKUPDIR. Thanks for pointing out the SECBACKUPLINK, I've removed it from code above (and my system). Also great reminder about losing the file on an update, I have a separate directory where I keep all files I have modified, so when I do update I can just compare old/new file and see if I do a direct replacement, or if I need to modify the new file. Thanks again! Bruce. |
Author: | mattbatt [ Wed Sep 03, 2014 3:23 pm ] |
Post subject: | Re: What a cool app NCDU |
I only had 10 backups so I think the pruning process is working. |
Page 1 of 1 | All times are UTC - 6 hours |
Powered by phpBB® Forum Software © phpBB Group http://www.phpbb.com/ |