| Author |
Message |
|
mihanson
|
 Posted: Mon Feb 05, 2007 4:19 pm |
|
|
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location:
Los Angeles
|
Hey all,
Been struggling with an issue when I run mythbackup as a cron job. Everyday at 5am I run mythbackup from root's crontab. Here's the command:
Code: 00 05 * * * /usr/local/bin/mythbackup However, mysql does not restart after it is done. I have not changed anything is the mythbackup or the backupcommon scripts, so I'm not sure what's going on. If I run mythbackup as root from the commandline, everything turns up roses.  Here's my syslog from the crontab backup operation: Code: Feb 5 05:00:01 mythbox /USR/SBIN/CRON[31259]: (root) CMD (/usr/local/bin/mythbackup)
Feb 5 05:00:01 mythbox /USR/SBIN/CRON[31260]: (root) CMD (/usr/local/bin/run_rrd)
Feb 5 05:00:01 mythbox mysqld[2929]: 070205 5:00:01 [Note] /usr/sbin/mysqld: Normal shutdown
Feb 5 05:00:01 mythbox mysqld[2929]:
Feb 5 05:00:03 mythbox mysqld[2929]: 070205 5:00:03 [Note] /usr/sbin/mysqld: Shutdown complete
Feb 5 05:00:03 mythbox mysqld[2929]:
Feb 5 05:00:03 mythbox mysqld_safe[31370]: ended
That's the only record I can find of the process. Since the backend went down and mysql went down to perform the backup, I have no info in syslog, mythbackend.log, kern.log and messages. Any ideas as to why it would not be restarting mysql? (Which in turn prevents mythbackend from starting) _________________ Mike
My Hardware Profile
|
|
| Top |
|
 |
|
tjc
|
Posted: Mon Feb 05, 2007 7:11 pm |
|
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
Possibly because I was sloppy and didn't use full paths for everything. I can tell you how to debug this, but it'll take a fair amount of time and initiative on your part, and I can't offer you a lot of support at the moment. 
The first thing to do is go through the script and make sure that everything which isn't a local shell function or a bash builtin is invoked using the full pathname. Next at the top of the script, after the #! magic edit in a line that says " set -vx". The third step is to run the script under env with a stripped environment. ( env -i /usr/local/bin/mythbackup) and see where it has trouble.
BTW - I've put together a version with full paths which I'll put somewhere convenient (like the wiki), but you'll still have to debug it, for issues with the mysql init script and the like.
|
|
| Top |
|
|
|
mihanson
|
Posted: Tue Feb 06, 2007 11:29 am |
|
|
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location:
Los Angeles
|
tjc,
Ok, I think I got all the full pathnames in there and the "set -vx" under the magic line in both backupcommon and mythbackup. I can run
Code: /usr/bin/env -i /usr/local/bin/mythbackup from the command line and everything goes smooth. However, I am unable to envoke the above command from cron. If I remove the Code: /usr/bin/env -i and comment out the "set -vx" I can run Code: /usr/local/bin/mythbackup from cron, but it bombs out right away. I hear Cecil saying "Initiating Backup," "Testing Backup," "Backup failed." one right after another. Is there a way I can see a verbose output of what is happening when cron is running the job? My son just woke up from his nap, so I'll try to look into this more later today. _________________ Mike
My Hardware Profile
|
|
| Top |
|
|
|
tjc
|
Posted: Tue Feb 06, 2007 9:41 pm |
|
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
mihanson wrote: Is there a way I can see a verbose output of what is happening when cron is running the job?
That's what set -vx was for. To run it under cron you may need to redirect the output to a log file since cron doesn't like programs with lots of output. Use the exec trick for that, add a line before the set -vx that says:
Code: exec >/tmp/backup.log 2>&1
Also you don't need to use the env -i wrapper under cron. That was only intended to let you test interactively in a "worst case" stripped environment. The intent is to remove all the environment settings, which should expose any dependencies on stuff in your interactive environment.
BTW - Who are you running the cron job as? It should be root.
|
|
| Top |
|
|
|
mihanson
|
Posted: Wed Feb 07, 2007 10:45 am |
|
|
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location:
Los Angeles
|
I am running mythbackup in root's crontab. Ok, this is WILD. If I add:
Code: exec >/var/log/mythbackup.log 2>&1
set -vx just under the #! magic in both backupcommon and mythbackup, everything works when executed from root's crontab. If I comment those two lines out or delete them all together it fails. 
Hey, whatever works, right? I guess I'll leave them in there, but I'd like to understand why that's happening. _________________ Mike
My Hardware Profile
|
|
| Top |
|
|
|
tjc
|
Posted: Wed Feb 07, 2007 7:07 pm |
|
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
mihanson wrote: I guess I'll leave them in there, but I'd like to understand why that's happening.
You and me both. That's pretty odd behavior... I guess it's time for me to set up a crontab entry and see what we can see...
Oh... oh crud. A smart alecky brain cell that was apparently sleeping in the back of the room just raised it's hand and asked what happens if you only comment out the set -vx. Sigh... Remember what I said about cron not liking programs with a lot of output? Redirecting it to a log file avoids that... You can also do the redirection outside the script in the cron command line.
Darn, I need more sleep or something...
|
|
| Top |
|
|
|
mihanson
|
Posted: Thu Feb 08, 2007 11:05 am |
|
|
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location:
Los Angeles
|
Ok, brain cell, here's what I found. 
If I comment out set -vx, but leave the logging to file in there, the cron job runs fine. No problems. If I comment out the logging to file as well, it fails (As we knew, but I was double checking). A brain cell in the back of my head woke up and I took the testing a bit further.
With no logging to file or set -vx turned on in mythbackup or backupcommon and run as a cron job in root's crontab:
In the mythbackup script, if I turn verbosity down on myisamchk with the -s switch, it still fails. If I remove that switch from myisamchk as well as the -v switch in the tar command, it still fails. If I add the -s switch to myisamchk and remove the -v switch in the tar command I have SUCCESS. So, I am successful with these new lines in mythbackup:
Line 27
Code: /usr/bin/myisamchk -s -f *.MYI Line 43 Code: /bin/tar cf $BACKUP_TAR $BACKUP_LIST
I'd say your theory about cron not liking verbose output is spot on based on my tests.  _________________ Mike
My Hardware Profile
|
|
| Top |
|
|
|
tjc
|
Posted: Thu Feb 08, 2007 6:59 pm |
|
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
I'd definitely recommend leaving the messages intact and just redirect them to a log file. That way you'll have something to check it if any other issues come up. Redirecting everything to /dev/null is a sure way to miss a recurring failure.
Having just spent a bunch of time battling with some code that was trapping exceptions, loosing all the detail and just passing back "a database error occured", I can tell you that debugging when the details are missing stinks like a bloated week old skunk carcass under the back porch 
|
|
| Top |
|
|
|
mihanson
|
Posted: Thu Feb 08, 2007 7:22 pm |
|
|
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location:
Los Angeles
|
|
| Top |
|
|
|
spideyk21
|
Posted: Tue May 22, 2007 9:26 am |
|
|
Joined: Tue Sep 12, 2006 6:03 am
Posts: 210
Location:
Roseville, MI
|
|
Could you post your modified backup files and cron job line. I would like to set this up also but haven't much of a clue where to really start.
I am also wondering how hard it would be for the system to be checked prior to running the cron (or have the check be part of the backup). If the system is recording a program the backup should be delayed/skipped.
Thanks in advance
_________________ -Roseville, Michigan USA LinHES R8: FE/BE, FE (x2)
|
|
| Top |
|
|
|
mihanson
|
Posted: Tue May 22, 2007 1:49 pm |
|
|
Joined: Sun Sep 25, 2005 3:50 pm
Posts: 1013
Location:
Los Angeles
|
spideyk21 wrote: Could you post your modified backup files and cron job line. I would like to set this up also but haven't much of a clue where to really start.
I am also wondering how hard it would be for the system to be checked prior to running the cron (or have the check be part of the backup). If the system is recording a program the backup should be delayed/skipped.
Don't know about the last part of your question, but here's the relevant part of my modified mythbackup & backupcommon:
Code: #!/bin/bash
exec >>/var/log/mythtv/mythbackup.log 2>&1
#set -vx
That's it. Only lines 1, 2 and 3 for each script. The exec line send the output to a log. The commented out set -vx line is for debugging the script. Uncomment it for a (very) verbose output. Here's my root crontab entry. Rather straightforward. Runs at 5 am each day: Code: 00 05 * * * /usr/local/bin/mythbackup _________________ Mike
My Hardware Profile
|
|
| Top |
|
|
|
tjc
|
Posted: Tue May 22, 2007 5:58 pm |
|
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
|
BTW - I recommend doing the logging outside of the main script if possible. That way it will continue to work normally in other circumstances. You can even do the redirection right in the crontab entry...
|
|
| Top |
|
|
|
spideyk21
|
Posted: Tue May 22, 2007 7:01 pm |
|
|
Joined: Tue Sep 12, 2006 6:03 am
Posts: 210
Location:
Roseville, MI
|
Would this do the trick in the crontab?
Code: 00 05 * * * /usr/local/bin/mythbackup>>/var/log/mythtv/mythbackup.log 2>&1
This still doesn't check the tuners for recording, will have to write a small script that is called from the cron to check the tuner cards, then if none are recording, set up the log file and call the backup script.
If the tuner cards are in use the log file will say so, if not the backup file will be run and the log will show the results. _________________ -Roseville, Michigan USA LinHES R8: FE/BE, FE (x2)
|
|
| Top |
|
|
|
tjc
|
Posted: Tue May 22, 2007 8:39 pm |
|
|
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location:
Arlington, MA
|
Yes, a wrapper script to do the checking and the redirection is a really good idea. I'd also take the opportunity to do some kind of simple logfile rotation rather than continually appending to the same file. Something like this fragment would work:
Code: LOG_FILE="/var/log/mythtv/mythbackup.log"
mv -f $LOG_FILE $LOG_FILE.old
exec >$LOG_FILE 2>&1
You could even do it by day of the week using something like: "$(date +%A)"
Oh, and when you do the check don't forget upcoming recordings. It'd stink to miss something you really wanted to record because the backup started 5 minutes before it was scheduled and took 8 minutes to finish...
|
|
| Top |
|
|
|
spideyk21
|
Posted: Thu May 24, 2007 8:58 pm |
|
|
Joined: Tue Sep 12, 2006 6:03 am
Posts: 210
Location:
Roseville, MI
|
This is what I have so far, creates the log file and everything. Just need to add the date and see if it is run from the cron tomorrow morning at 5am.
Code: #!/bin/sh
# --------------------------------------------------------------------
# Very simple script to execute mythbackup only if there are no
# scheduled recordings currently being recorded.
#---------------------------------------------------------------------
# initalize variables
script='MB:'
db='mythconverg'
mb='mythbackup'
#Setup Log File
LOG_FILE="/var/log/mythtv/mythbackup.log"
mv -f $LOG_FILE $LOG_FILE.old
exec >$LOG_FILE 2>&1
# substitute the username you want to you use to connect to mySQL
user='root'
# password variable is here, but I haven't used it
password=''
# run SQL
data1=`mysql $db -u $user -e "select count(*) from recorded where timediff(now(),endtime)<'00:00:00' and recgroup!='LiveTV';" -sN`
data2=`mysql $db -u $user -e "select count(*) from record where inactive=0 and (type!=0 or type!=8) and timediff(now(),concat(startdate,' ',starttime))<'00:30:00';" -sN`
if [ $data1 -ge 1 -o $data2 -ge 1 ]; then
echo "$script MythTV is/will be recording a scheduled program. Please run mythbackup at another time."
else
echo "$script MythTV will now run $mb"
$mb
fi
exit 0
I will still be following this thread ( http://www.mysettopbox.tv/phpBB2/viewtopic.php?t=15322&start=0&postdays=0&postorder=asc&highlight=) for debugging that is under way. I saved the file in /usr/local/bin under the file name scheduled_mythbackup. added the following to cronjob Code: 00 05 * * * /usr/local/bin/schedule_mythbackup _________________ -Roseville, Michigan USA LinHES R8: FE/BE, FE (x2)
|
|
| Top |
|
|
View unanswered posts
View active topics
Print view
Previous topic Next topic 
LinHES Home
Board index
FAQ
Search
Login 
Register
