Author |
Message |
nicom
|
Posted: Mon Apr 13, 2009 10:47 pm |
|
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location:
Melbourne, Australia
|
My machine still suffers from the occasional freeze but I find I can minimise them if I regularly run optimize_db.sh. What would be ideal if this type of maintenance could be run automatically. Reading posts in the forum I see some people do this as a cron job however others do not recommend it as it may happen at an inopportune time. I run my machine (F/E + B/E) only as required so that it starts-up by ACPI for recording and shuts-down from Mythwelcome when not recording. The problem with a cron job for me is the machine has to be up and running at the specified time for it to work. Could the Mythwelcome shutdown procedure be used to initiate optimize_db.sh? Mythwelcome has done all the work of finding put whether the machine is needed and will only proceed in shutting down if safe to do so.
I would imagine this might involve creating a script like
optimize_db.sh
/sbin/poweroff
and changing the "Command to Shutdown" in mythwelcome --setup from /sbin/poweroff with the new script name. There would probably be some permissions issues so could I just add optimize_db.sh to the sudoers list?
I know I could just go and try this but with my limited knowledge of linux I would be interested to know if I am barking up the wrong tree or if there is there a better way?
Going one step further, could this approach be applied to automating backup which will also repair the db? The problem here I see is that backup would occur every time the machine is used which is probably too frequent. Perhaps the shutdown script could be modified to run backup say every 5 shutdowns.
_________________ Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest
|
|
Top |
|
![](images/spacer.gif) |
nmcaullay
|
Posted: Mon Apr 13, 2009 11:09 pm |
|
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location:
Perth, Australia
|
This is a very good idea, something i have been thinking about, but never had time to look into... getting the shutdown command to execute the optimize script strikes me as the best way, as it shuts down the sql and mythtv backend..
I might try it tonight if i get the chance...
Create a script that does
1) The optimise, then
2) the shutdown
Shouldnt be too hard i would think...
If you get there first, post what you did, and i'll do likewise if i extract my digit before you..
Nathan
_________________ LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector
|
|
Top |
|
![](images/spacer.gif) |
cecil
|
Posted: Tue Apr 14, 2009 2:21 am |
|
![Offline Offline](./styles/just_enjoy_LH/imageset/en/icon_user_offline.gif) |
Site Admin |
Joined: Fri Sep 19, 2003 6:37 pm
Posts: 2659
Location:
Whittier, Ca
|
nicom wrote: and changing the "Command to Shutdown" in mythwelcome --setup from /sbin/poweroff with the new script name. There would probably be some permissions issues so could I just add optimize_db.sh to the sudoers list? Write your script and put that script in sudoers... It would be a good idea to include a counter and have it kick off like every 10th time...
|
|
Top |
|
![](images/spacer.gif) |
nicom
|
Posted: Sun Apr 19, 2009 12:56 am |
|
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location:
Melbourne, Australia
|
Well I have had some success with my idea but I believe there could be some improvements. Firstly I created a script /home/mythtv/sd-opt/shutdown_opt.sh as follows Code: #!/bin/bash # # Script to run the optimize_db.sh script regularly # Called from Mythwelcome at shutdown # Counter included to run optimize_db.sh on every fifth shutdown # # Extract shutdown number from file and increment counter=`cat /home/mythtv/sd-opt/sdcount` max=5 let "counter += 1"
# At fifth shutdown run optimize_bd script and reset counter if [ $counter -ge $max ]; then counter=0 bash optimize_db.sh echo optimize_db.sh was last run at `date` > /home/mythtv/sd-opt/optimize_db.log fi
# Write new count number back to file echo $counter >/home/mythtv/sd-opt/sdcount # # Shutdown as per normal /sbin/poweroff
I located it and the associated files in a sub-directory of /home/mythtv so that it would survive upgrades but I am not sure if it is recommended. In Mythwelcome --setup I changed the "Command for Shutdown" from /sbin/poweroff to /home/mythtv/sd-opt/shutdown_opt.sh. It counts 5 shutdowns before running optimize_db.sh as per Cecil's suggestion. No changes to permissions or sudoers were required. It creates a log file (optimize_db.log) telling when optimize_db.sh was last run at shutdown.
When I run it works but when it performs the optimize_db.sh on shutdown I get a message in Mythwelcome saying it cannot connect to the backend but it does shutdown after another 15 seconds or so.
Improvements I think that could be made:
1. Make it part of the Mythwelcome-set-alarm.sh script. That way it would be upgrade-proof and I think it would be done at a better time in the shutdown sequence. Unfortunately this script is in perl and I struggle enough with bash so I was unable to try it.
2. Change my script to include most of the optimize_db.sh script but exclude the restart database and backend. I tied this briefly but the machine went to frantic hard disk activity and did not shutdown so I gave up and resorted to my normal method.
3. How about using this method to run backup regularly.
Any other thoughts? As I am very new to bash scripting any criticism is welcome.
_________________ Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest
|
|
Top |
|
![](images/spacer.gif) |
nmcaullay
|
Posted: Sun Apr 19, 2009 7:01 pm |
|
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location:
Perth, Australia
|
Very cool, looks like you have got your head around the scripting beast!
I like you idea about incorporating this into the set-alarm-wakeup script. I use mythwelcome to powerup/powerdown, but in R6 I could not locate the KM/LH script to do it, so i reverted to the method in the wiki (mythtv/knoppmyth i cant remember which one).
Basically, i have a sctipt that has only a few commands in it, but could cut-n-paste your stuff into it, to run at that part of the sequence, rather than at shutdown... not sure if it would work as there may be more "database" things that happen "after" the set-wakeup-alarm script is called..
Nathan
_________________ LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector
|
|
Top |
|
![](images/spacer.gif) |
tjc
|
Posted: Mon Apr 20, 2009 9:34 am |
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
Just to toss in a couple good scripting hygiene suggestions...
- You should almost always use double quotes around variable expansions, _especially_ variable expansions in tests. This prevents all sorts of nastiness. Even when you "know" it's not necessary it's a good habit to cultivate.
- I also generally recommend using the new POSIX $() expansion rather than `` since it's nesting behavior and interaction with other types of quoting is so much cleaner.
- You shouldn't need to specifically use bash to invoke optimize_db.sh if it has the right permissions (chmod +x) and #! magic.
Keep up the good work!
|
|
Top |
|
![](images/spacer.gif) |
nicom
|
Posted: Sun Mar 14, 2010 9:04 pm |
|
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location:
Melbourne, Australia
|
I finally got around to developing this further. My shutdown script now does three maintenance tasks on a cycle of 10 shutdowns. On the first shutdown it runs optimize_db.sh, on the second it runs mythbackup and on the third it defrags my /myth partition (xfs). All functions seem to work fine and I plan to monitor it for an extended period to make sure the defrag is keeping pace because my script only calls for a single defrag pass to minimise the time it runs.
Now I am wondering do I need to do the optimize_db step at all because mythbackup includes "myisamchk -f *.MYI" in its script. Reading from the MySQL Reference Manual the -f option performs the same function as the --recover option used in the optimize_db.sh. However, optimize_db.sh also includes a step with the --analyze option and a possible --safe-recover option if required, but I don't know how useful they are.
Is it still worth regularly running optimize_db.sh if mythbackup is being run just as frequently?
If anyone is interested I attach the script Code: #!/bin/bash # # shutdown_maint.sh # # Script to perform routine maintenance and housekeeping # The script is invoked by mythwelcome on shutdown and is listed in mythwelcome -setup # The script will perform the following on a cycle of 10 shutdowns # Shutdown number 1 - run the optimize_db.sh script # Shutdown number 2 - mythbackup # Shutdown number 3 - defragment the /myth partition (xfs) # Shutdown number 10 - reset counter to 0 # Extract shutdown number from file and increment counter=`cat /home/mythtv/shutdown_maint/sdcount` max=10 let "counter += 1"
# At first shutdown run optimize_bd script if [ $counter -eq 1 ]; then # /usr/local/bin/optimize_db.sh echo `date` : Started optimize_db >> /home/mythtv/shutdown_maint/sd-maint.log echo `/usr/local/bin/optimize_db.sh` > /home/mythtv/shutdown_maint/opt.log echo `date` : Completed optimize_db.sh on shutdown >> /home/mythtv/shutdown_maint/sd-maint.log fi
# At second shutdown run mythbackup if [ $counter -eq 2 ]; then # /usr/local/bin/mythbackup echo `date` : Started mythbackup >> /home/mythtv/shutdown_maint/sd-maint.log echo `/usr/local/bin/mythbackup` > /home/mythtv/shutdown_maint/backup.log echo `date` : Completed mythbackup on shutdown >> /home/mythtv/shutdown_maint/sd-maint.log fi
# At third shutdown defragment /myth partition if [ $counter -eq 3 ]; then # xfs_fsr -p 1 echo `date` : Started xfs defrag >> /home/mythtv/shutdown_maint/sd-maint.log echo `xfs_fsr -p 1` > /home/mythtv/shutdown_maint/defrag.log echo `date` : Completed Defrag on shutdown `xfs_db -c frag -r /dev/sda3` >> /home/mythtv/shutdown_maint/sd-maint.log fi
# At tenth shutdown reset counter if [ $counter -ge $max ]; then counter=0 fi
# Write new count number back to file echo $counter >/home/mythtv/shutdown_maint/sdcount # Shutdown as per normal /sbin/poweroff
The sd-maint.log keeps a running log of the activities and each task writes to its own log file. For some reason the xfs_fsr step is not writing to the log file but it is performing the defrag. More investigation needed.
I am still running R5.5.
_________________ Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest
|
|
Top |
|
![](images/spacer.gif) |
nmcaullay
|
Posted: Sun Mar 14, 2010 10:21 pm |
|
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location:
Perth, Australia
|
Brilliant stuff.... exactly the sort of thing that i required for a mythwelcome-enabled device..
The only thing remaining is the non-xfs disk-check that happens on boot.
Has anyone had any luck in the implementation of that?
Thanks for the script, i'll be implementing it tonight
Nathan
_________________ LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector
|
|
Top |
|
![](images/spacer.gif) |
nmcaullay
|
Posted: Mon May 10, 2010 7:44 pm |
|
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location:
Perth, Australia
|
Hi All,
I havent implemented this yet, just creating the script to try tonight..
I have modified the script above to my requirements, only have really changed the log file location, and the wakealarm stuff added at the bottom.
Code: #!/bin/bash # # shutdown_maint.sh # # Script to perform routine maintenance and housekeeping # The script is invoked by mythwelcome on shutdown and is listed in mythwelcome -setup # The script will perform the following on a cycle of 10 shutdowns # Shutdown number 1 - run the optimize_db.sh script # Shutdown number 2 - mythbackup # Shutdown number 3 - defragment the /myth partition (xfs) # Shutdown number 10 - reset counter to 0
# Extract shutdown number from file and increment counter=`cat /var/log/mythtv/macdoodles-shutdowncount` max=10 let "counter += 1"
# At first shutdown run optimize_bd script if [ $counter -eq 1 ]; then echo `date` : Started optimize_db >> /var/log/mythtv/macdoodles-shutdown.log echo `/usr/LH/optimize_mythdb.py` > /var/log/mythtv/macdoodles-optimise.log echo `date` : Completed optimize_mythdb.py on shutdown >> /var/log/mythtv/macdoodles-shutdown.log fi
# At second shutdown run mythbackup if [ $counter -eq 2 ]; then # /usr/local/bin/mythbackup echo `date` : Started mythbackup >> /var/log/mythtv/macdoodles-shutdown.log echo `/usr/LH/mythbackup` > /var/log/mythtv/macdoodles-backup.log echo `date` : Completed mythbackup on shutdown >> /var/log/mythtv/macdoodles-shutdown.log fi
# At third shutdown defragment /myth partition if [ $counter -eq 3 ]; then # xfs_fsr -p 1 echo `date` : Started xfs defrag >> /var/log/mythtv/macdoodles-shutdown.log echo `xfs_fsr -p 1` > /var/log/mythtv/macdoodles-xfsdefrag.log echo `date` : Completed Defrag on shutdown `xfs_fsr -p 1` >> /var/log/mythtv/macdoodles-shutdown.log fi
# At tenth shutdown reset counter if [ $counter -ge $max ]; then counter=0 fi
# Write new count number back to file echo $counter >/var/log/mythtv/macdoodles-shutdowncount
# Clear the RTC wakealarm echo 0 > /sys/class/rtc/rtc0/wakealarm #this clears your alarm. # Write out the wakeup time into RTC echo $1 > /sys/class/rtc/rtc0/wakealarm #this writes your alarm # Write the RTC wakeup time into the log echo [`date`] "Set Wakeup Time (`cat /sys/class/rtc/rtc0/wakealarm`)" >> /var/log/mythtv/macdoodles-setwakeup.log
# Shutdown as per normal #/sbin/poweroff
Additionally, I have located some information allowing the non-xfs disck check to happen on shutdown, rather than on bootup (which can be tedious when your 3yrold want to watch fireman sam!) http://bbs.archlinux.org/viewtopic.php?id=15496http://www.xpmediacentre.com.au/community/mythtv-backend-only/41160-ext3-filesystem-check-boot.htmlBasically, add a short code block to /etc/rc.shutdown (at the correct spot - after the filesystems have been remounted as RO) Code: # CUSTOM STUFF TO DO CHECKDISK day=`date +"%d"` runtoday=${day: -1} if [ "$runtoday" = 2 -o "$runtoday" = 7 ]; then if [ "$RUNLEVEL" = "0" ]; then stat_busy "Checking Filesystems" /sbin/fsck -A -T -C -a -- -f stat_done fi fi Here is my current file, with the above code in the required spot. Code: #!/bin/bash # # /etc/rc.shutdown #
. /etc/rc.conf . /etc/rc.d/functions
# avoid staircase effect /bin/stty onlcr
echo " " printhl "Initiating Shutdown..." echo " "
# avoid NIS hanging syslog-ng on shutdown by unsetting the domainname if [ -x /bin/domainname ]; then /bin/domainname "" fi
if [ -x /etc/rc.local.shutdown ]; then /etc/rc.local.shutdown fi
#unmount network filesystems for runit #if [ -e /var/service/netfs ] #then /etc/rc.d/netfs stop #fi
if [ "$PREVLEVEL" = "3" -o "$PREVLEVEL" = "5" ]; then # Shutdown daemons let i=${#DAEMONS[@]} while [ $i -ge 0 ]; do if [ "${DAEMONS[$i]:0:1}" != '!' ]; then ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@} fi let i=i-1 done # find any leftover daemons and shut them down in reverse order if [ -d /var/run/daemons ]; then for daemon in $(/bin/ls -1t /var/run/daemons); do stop_daemon $daemon done fi fi
# Terminate all processes stat_busy "Sending SIGTERM To Processes" /sbin/killall5 -15 &> /dev/null /bin/sleep 5 stat_done
stat_busy "Sending SIGKILL To Processes" /sbin/killall5 -9 &> /dev/null /bin/sleep 1 stat_done
stat_busy "Saving Random Seed" /bin/dd if=/dev/urandom of=/var/run/random-seed count=1 bs=512 2> /dev/null stat_done
stat_busy "Saving System Clock" if [ "$TIMEZONE" != "" -a -e "/usr/share/zoneinfo/$TIMEZONE" ]; then /bin/rm -f /etc/localtime /bin/cp "/usr/share/zoneinfo/$TIMEZONE" /etc/localtime fi
HWCLOCK_PARAMS="--systohc" if [ "$HARDWARECLOCK" = "UTC" ]; then HWCLOCK_PARAMS="$HWCLOCK_PARAMS --utc" else HWCLOCK_PARAMS="$HWCLOCK_PARAMS --localtime" fi if [ "$USEDIRECTISA" = "yes" -o "$USEDIRECTISA" = "YES" ]; then HWCLOCK_PARAMS="$HWCLOCK_PARAMS --directisa" fi /sbin/hwclock $HWCLOCK_PARAMS stat_done
# removing psmouse module to fix some reboot issues on newer laptops /sbin/modprobe -r psmouse >/dev/null 2>&1
# Write to wtmp file before unmounting /sbin/halt -w
stat_busy "Deactivating Swap" /sbin/swapoff -a stat_done
stat_busy "Unmounting Filesystems" /bin/umount -a -r -t noramfs,notmpfs,nosysfs,noproc stat_done
# Kill non-root encrypted partition mappings if [ -f /etc/crypttab -a -n "$(/bin/grep -v ^# /etc/crypttab | /bin/grep -v ^$)" ]; then stat_busy "Deactivating encrypted volumes:" CS=/sbin/cryptsetup.static do_uncrypt() { if [ $# -ge 3 ]; then stat_append "${1}.." $CS remove $1 >/dev/null 2>&1 if [ $? -ne 0 ]; then stat_append "failed " else stat_append "ok " fi fi } while read line; do eval do_uncrypt "$line" CS=/sbin/cryptsetup.static do_uncrypt() { if [ $# -ge 3 ]; then stat_append "${1}.." $CS remove $1 >/dev/null 2>&1 if [ $? -ne 0 ]; then stat_append "failed " else stat_append "ok " fi fi } while read line; do eval do_uncrypt "$line" done </etc/crypttab stat_done fi
if [ "$USELVM" = "yes" -o "$USELVM" = "YES" ]; then if [ -x /sbin/lvm -a -d /sys/block ]; then stat_busy "Deactivating LVM2 groups" /sbin/lvm vgchange --ignorelockingfailure -an >/dev/null 2>&1 stat_done fi fi
stat_busy "Remounting Root Filesystem Read-only" /bin/mount -n -o remount,ro / stat_done
# CUSTOM STUFF TO DO CHECKDISK day=`date +"%d"` runtoday=${day: -1} if [ "$runtoday" = 2 -o "$runtoday" = 7 ]; then if [ "$RUNLEVEL" = "0" ]; then stat_busy "Checking Filesystems" /sbin/fsck -A -T -C -a -- -f stat_done fi fi
# Power off or reboot if [ "$RUNLEVEL" = "0" ]; then printsep printhl "${C_H2}POWER OFF" /sbin/poweroff -d -f -h -i else printsep printhl "${C_H2}REBOOTING" # if kexec is installed and a kernel is loaded, use it [ -x /sbin/kexec ] && /sbin/kexec -e > /dev/null 2>&1 /sbin/reboot -d -f -i fi
# End of file # vim: set ts=2 sw=2 noet: Also, I have had to add the following line to /usr/LH/bin/mythbackup to enable me to pull the backups to work from my KM at home. since the security is tied down to a single user to log in (mythtv), I need the mythbackup files to be readable by that user (mythtv) when using pscp (putty secure copy). Not sure if this is something that could be rolled into KM, as it is quite specific. Code: # If you can't read this you've got no business restoring from it anyway. $CHOWN root:root $BACKUP_TAR* $BACKUP_SQL* $CHMOD go-rwx $BACKUP_TAR* $BACKUP_SQL* $CHMOD o+r $BACKUP_TAR* $BACKUP_SQL*
if this works, it means that i have my mythbox regularily backed up, optimised, disk checked, and mythwelcome-wakeup-enabled. Perfect.
Cheers,
Nathan
_________________ LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector
Last edited by nmcaullay on Mon May 10, 2010 10:22 pm, edited 1 time in total.
|
|
Top |
|
![](images/spacer.gif) |
nicom
|
Posted: Mon May 10, 2010 9:40 pm |
|
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location:
Melbourne, Australia
|
That looks very impressive. I only wish my limited knowledge would allow me to understand more than 5% of it.
I am in the process of planning the upgrade to R6 so your mods to my script are interesting and I have a couple of questions.
1. Is it better to keep logs in the /var/log/mythtv directory or is it just preference? Would it explain why my defrag.log refuses to work?
2. Why is it necessary to clear and re-write the the wake-up time to /sys/class/rtc/rtc0/wakealarm?
For you new script I gather that most of it is involved in safely unmounting the partition to allow you to run fsck. I assume since you included the defrag part of the previous script that your /myth partition is xfs. Why then do you need to force the fsck? The boot time increase for fsck of the / partition is trivial so I leave that to the normal 30 mountings.
By the way, my script has been running successfully for 2 months now and the single pass of defrag keeps the fragmentation around 1%. Maybe I could reduce the frequency a bit but I don't think I will bother.
_________________ Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest
|
|
Top |
|
![](images/spacer.gif) |
nmcaullay
|
Posted: Mon May 10, 2010 9:59 pm |
|
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location:
Perth, Australia
|
Its all very nice in theory... i have to try it out tonight
Actually, if your script was based on R5.5, i'll need to check that the binaries referenced in the script are still in the same place.
1. I use /var/log/mythtv as a preference. The other myth stuff seems to get logged there. I check every now and then, and delete the odd file, so dont know much about the rotation etc.
2. The clear and rewrite of RTC is something that was suggested somewhere along the way. It works, so i'm not going to try to figure it out
I have a second drive that is XFS, so your script will nicely keep that unfragmented. My first drive is a EXT3? which has LH on it, and it is subject to the disk check timesink on boot. So, without using your script (and the rc.shutdown mods) i have the following
* an EXT3 drive has a 1-in30 disk check that always happens at an inappropriate time, and
* an XFS drive that busily gets itself fragmented.
BTW, the "new" script is an existing Arch one (i think), with about 5 lines inserted by me. I dont understand what is going on in there either ![Smile :)](./images/smilies/icon_smile.gif)
_________________ LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector
|
|
Top |
|
![](images/spacer.gif) |
nmcaullay
|
Posted: Mon May 10, 2010 10:23 pm |
|
Joined: Sun Sep 04, 2005 7:48 pm
Posts: 264
Location:
Perth, Australia
|
Updated the script i posted to point to the correct optimise and mythbackup locations. Wouldnt have worked before on a R6.
also, another question.
1) what does "xfs_fsr -p 1" do. doing a man page doesnt show any "-p" options?
Cheers,
Nathan
_________________ LinHES: R6 | MB: Asus M3N-H/HDMI | CPU: AMD ??Mhz
Capture: 2xHDHR DVB-T
Graphics: Onboard 8300|PSU: Corsair vx450w
Cooling: Zalman cu?,
Display: Benq xx projector
|
|
Top |
|
![](images/spacer.gif) |
nicom
|
Posted: Mon May 10, 2010 11:08 pm |
|
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location:
Melbourne, Australia
|
You are right that to "-p" option is not mentioned in the man page but I found it in xfs_fsr --help. From my understanding the -p option is for the number of passes over the partition, so my setting of "-p 1" is the minimum of a single pass. For my 500GB disk it takes about 25 minutes if it has some work to do but can be very quick if the disk is not fragmented. I thought that would be better than setting a fixed time.
I would be interested to hear from others if my understanding is correct.
_________________ Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest
|
|
Top |
|
![](images/spacer.gif) |
alien
|
Posted: Tue May 11, 2010 1:28 am |
|
Joined: Mon Jun 21, 2004 5:28 am
Posts: 700
Location:
Germany
|
Just saw this thread and I wanted to add a couple of comments about using cron/mythwelcome.
I recommend a daily wakeup with mythwelcome, even if you don't use the cron method. Otherwise, if you ever shutdown with no scheduled recordings, it will not wakeup again by itself. This can be a problem as no schedules will be updated. If you watch TV every day then this may not be an issue.
As for the cron job possibly colliding with other backend jobs (ex. recording). It can be fixed by checking the status first. For example:
Code: #!/usr/bin/perl
while (($status=system("/usr/bin/mythshutdown -s")/256) != 64) { # Wait until status is 64 # (Idle, in daily wakeup and not about to record) sleep(20); }
system("/usr/bin/mythshutdown -l");
system("/usr/LH/optimize_mythdb.py");
$day=`date '+%a'`; chomp($day); if($day eq "Tue") { # do weekly stuff.... }
system("/usr/bin/mythshutdown -u");
_________________ ASUS AT3N7A-I (Atom 330) TBS 8922 PCI (DVB-S2)
|
|
Top |
|
![](images/spacer.gif) |
nicom
|
Posted: Thu Apr 28, 2011 9:16 pm |
|
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location:
Melbourne, Australia
|
Some developments since my previous posts. Earlier this year I had a hard disk failure. The disk failed totally (click of death) and I lost / and /myth including my backups which were all in the /myth partition. My second drive (/myth2) only had some of the recorded programs.
I bought a new drive, reinstalled and repaired the database to what recordings I still had. I also had to reload my music, videos and photos. It was not a small task so I decided to do something smarter this time. I modified my maintenance script and added the facility to create a secondary backup on the /myth2 partition. It backs it up 5 deep which is a bit excessive and I may cut it down a bit. I also included a step to copy videos, music and photos to a backup directory on /myth2. I now have a total of 3 TB storage so I don't even bother compressing. If I have another disk failure (haven't I had my share?) I should only lose some of the recordings. Anything valuable I want to keep I will transcode and put in videos.
I had also modified the defrag steps so that it does one disc at a time and I also went back to a time limit of 20 minutes rather than the passes option.
The script now looks like Code: #!/bin/bash # # shutdown_maint.sh # # Script to perform routine maintenance and housekeeping # The script is invoked by mythwelcome on shutdown and is listed in mythwelcome -setup # The script will perform the following on a cycle of 10 shutdowns # Shutdown number 1 - run the optimize_db.sh script # Shutdown number 2 - mythbackup # Shutdown number 3 - defragment the /myth partition (xfs) # Shutdown number 4 - defragment the /myth2 partition (xfs) # Shutdown number 5 - backup videos, music and photos # Shutdown number 10 - reset counter to 0 # Extract shutdown number from file and increment counter=`cat /home/mythtv/sd-maint/sdcount` max=10 let "counter += 1"
# At first shutdown run optimize_bd script if [ $counter -eq 1 ]; then # /usr/share/mythtv/contrib/maintenance/optimize_mythdb.pl echo `date` : Started optimize_db >> /home/mythtv/sd-maint/sd-maint.log echo `/usr/share/mythtv/contrib/maintenance/optimize_mythdb.pl` > /home/mythtv/sd-maint/opt.log echo `date` : Completed optimize_db.sh on shutdown >> /home/mythtv/sd-maint/sd-maint.log fi
# At second shutdown run mythbackup if [ $counter -eq 2 ]; then # /usr/LH/bin/mythbackup echo `date` : Started mythbackup >> /home/mythtv/sd-maint/sd-maint.log echo `/usr/LH/bin/mythbackup` > /home/mythtv/sd-maint/backup.log echo `date` : Completed mythbackup >> /home/mythtv/sd-maint/sd-maint.log
# Rollover secondary (myth2) backup files - 5 deep mv -f /myth2/sec-backup/mythconverg.sql.gz.4 /myth2/sec-backup/mythconverg.sql.gz.5 mv -f /myth2/sec-backup/mythconverg.sql.gz.3 /myth2/sec-backup/mythconverg.sql.gz.4 mv -f /myth2/sec-backup/mythconverg.sql.gz.2 /myth2/sec-backup/mythconverg.sql.gz.3 mv -f /myth2/sec-backup/mythconverg.sql.gz.1 /myth2/sec-backup/mythconverg.sql.gz.2 mv -f /myth2/sec-backup/mythconverg.sql.gz /myth2/sec-backup/mythconverg.sql.gz.1
mv -f /myth2/sec-backup/savedfiles.tar.gz.4 /myth2/sec-backup/savedfiles.tar.gz.5 mv -f /myth2/sec-backup/savedfiles.tar.gz.3 /myth2/sec-backup/savedfiles.tar.gz.4 mv -f /myth2/sec-backup/savedfiles.tar.gz.2 /myth2/sec-backup/savedfiles.tar.gz.3 mv -f /myth2/sec-backup/savedfiles.tar.gz.1 /myth2/sec-backup/savedfiles.tar.gz.2 mv -f /myth2/sec-backup/savedfiles.tar.gz /myth2/sec-backup/savedfiles.tar.gz.1
# Copy latest backup files to secondary backup (myth2) sudo cp -f /myth/backup/mythconverg.sql.gz /myth2/sec-backup/mythconverg.sql.gz sudo cp -f /myth/backup/savedfiles.tar.gz /myth2/sec-backup/savedfiles.tar.gz
echo `date` : Completed secondary backup >> /home/mythtv/sd-maint/sd-maint.log fi
# At third shutdown defragment /myth partition if [ $counter -eq 3 ]; then echo `date` : Started xfs defrag of /myth >> /home/mythtv/sd-maint/sd-maint.log echo `sudo xfs_fsr -v -t 1200 -m /home/mythtv/sd-maint/mtab_sda3` > /home/mythtv/sd-maint/defrag.log echo `date` : Completed Defrag of sba3 on shutdown `xfs_db -c frag -r /dev/sda3` >> /home/mythtv/sd-maint/sd-maint.log fi
# At fourth shutdown defragment /myth2 partition if [ $counter -eq 4 ]; then echo `date` : Started xfs defrag of myth2 >> /home/mythtv/sd-maint/sd-maint.log echo `sudo xfs_fsr -v -t 1200 -m /home/mythtv/sd-maint/mtab_sdb1` > /home/mythtv/sd-maint/defrag.log echo `date` : Completed Defrag of sdb1 on shutdown `xfs_db -c frag -r /dev/sdb1` >> /home/mythtv/sd-maint/sd-maint.log fi
# At fifth shutdown backup videos, music and photos if [ $counter -eq 5 ]; then echo `date` : Started media backup >> /home/mythtv/sd-maint/sd-maint.log echo `date` > /home/mythtv/sd-maint/media-backup.log echo `rsync -vrt --delete /myth/video /myth2/media-backup` >> /home/mythtv/sd-maint/media-backup.log echo `rsync -vrt --delete /myth/music /myth2/media-backup` >> /home/mythtv/sd-maint/media-backup.log echo `rsync -vrt --delete /myth/gallery /myth2/media-backup` >> /home/mythtv/sd-maint/media-backup.log echo `date` : Completed media backup >> /home/mythtv/sd-maint/sd-maint.log fi
# At tenth shutdown reset counter if [ $counter -ge $max ]; then counter=0 fi
# Write new count number back to file echo $counter >/home/mythtv/sd-maint/sdcount # Shutdown as per normal /sbin/poweroff
_________________ Intel DG965WH, Dvico DVB-T Lite x2, HDHR, Gigabyte GT220, KingstonSSD, WD20EARS version=latest
|
|
Top |
|
![](images/spacer.gif) |