View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 71 posts ] 
Go to page Previous  1, 2, 3, 4, 5  Next

Print view Previous topic   Next topic  
Author Message
Search for:
 Post subject:
PostPosted: Tue Oct 04, 2005 9:20 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
Mike:

One more piece of info that would be useful to confirm we aren't dealing with permission problems. The output of the following three commands would be useful:

1. ls -ld /myth/tv

2. whoami

3. df -h /myth/tv

Thanks!

marc


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 9:33 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

Don't think it is a permission issue, I must have had a fat fingered something and when I copied it back and forth cleared the error.

To create the last piece for testing. just make a piece of dead wood.. I don't know if something like
echo " " > /myth/tv/2051_20051004185400_20051004190200.nuv or if it needs to be something left after the db entry was cleared.

root@mythtv-20:/myth/tools# ls -ld /myth/tv
drwxr-xr-x 2 mythtv mythtv 4096 Oct 4 23:16 /myth/tv

root@mythtv-20:/myth/tools# whoami
root

root@mythtv-20:/myth/tools# df -h /myth/tv
Filesystem Size Used Avail Use% Mounted on
/dev/hda4 14G 6.1G 7.3G 46% /myth

I will keep an eye on it for a day or so and let you know of any discoveries. As far as I am concerned this is a big gift from you! Thank you.
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 11:28 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
Mike:

Sounds like we are done with Scenario C. Scenario D is interesting -- I had no idea this could happen, but I did a check and found 5 ".nuv" files in my /myth/tv directory that did not exist in the myth data base. Is this a common problem?

In any event, I want to mull this over a bit before I try to script it up. I did a bit of experimenting, and one approach that looks like it will work is as follows:

Code:
#! /bin/sh
ls -Li /myth/pretty/*.mpg | sort > /tmp/mpg-files
ls -Li /myth/tv/*.nuv | sort > /tmp/nuv-files
join -v 2 /tmp/mpg-files /tmp/nuv-files | cut -d " " -f 2 > /tmp/dead-files
cat /tmp/dead-files | sed -e "s/^/rm -f /" > /tmp/rm-dead-files


This will create a file called "/tmp/rm-dead-files" that contains an "rm -f" command for each nuv file that needs to be deleted. However this approach has a hole. Any recording that has been created but that has not yet had a pretty link created for it will also be deleted. This is obviously undesirable behavior, so I need to find a way to limit it to acting only on nuv files that are not "too recent". ie: At least a day old. There are other approachs that may work. Let me give it some thought.

Feel free to try this script, as it will only generate an output script and not actually remove anything. Let me know if it properly identified what needs to be removed...

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 05, 2005 6:25 am 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Good Morning Marc,
The "dead wood" does accumulate more than you think. You just don't know because other than wasting space it causes no issues, sort of just lurks. I discovered these two items because of the heavy record cycles keeping x number of days with the oldest being over written. The first thing I see is pretty not updating and that was when I found the leftovers from previous weeks.

If the clean up script runs at the end of pretty, then what ever is left is not linked to the database and therefore unneeded

I'll check out the lastest tweak as soon as I can. (PM)
Thanks
Mike
Update, just added it the bottom of my mythlink script (without the //bin/sh) and it picked the piece of dead wood that I had made last night. I think this was to easy for you :) I will test more later as it is still a labor day.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 05, 2005 7:50 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
Mike:

I understand your thinking, but there is still a risk of blowing away a valid nuv file. The scenario involves a recording starting after "mythlink.sh" takes a snapshop of the data base, but before the clean up command run. In the scenario, it will look like an "orphaned" nuv file. I've tested the script below -- it should work fine, although you might want to comment out the "find" on the next-to-last line while you are testing, so that no actual deleting occurs.

This script does not depend on also running pretty.

Marc

Code:
#! /bin/bash
#
# This scropt will look for orphased ".nuv" files in /myth/tv
# directory and delete them. To protect your system
# from "run away" deletions, this script uses the following
# logic:
#
# 1. It will not do anything to a "nuv" file that is referenced in the
#    "recorded shows" listing of the myth data base.
# 2. It will not do anything to a "nuv" file that is less than 1-day old.
# 3. If it detects more than 8 "nuv" files that appear to be orphaned,
#    it will exit without taking any action.
# 4. Files that pass the previous 3 tests are not deleted. Instead they are
#    moved to the directory /myth/tv/TRASH. If they remain untouched in this
#    directory for 7 days, then they are automatically deleted.
#
#
TV="/myth/tv"
TRASH=$TV/TRASH
AGE="7"
MAX=8
#
# Verify existance of input directories
#
if [ ! -e "$TV" ]
then
    echo "$TV does not exist -- script halted."
    exit 1
fi
mkdir -p $TRASH
if [ ! -e "$TRASH" ]
then
    echo "Unable to created directory $TRASH"
    exit 1
fi
#
# If a file exists in $TV, has not been modified for at least 1 day,
# and is not referenced in the database, then move it to the trash can
#
mysql -uroot  mythconverg -B --exec \
      "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s') \
      from recorded;" >/tmp/mythclean.db
cat /tmp/mythclean.db | grep -v "^chanid" | sed -e "s/\t/_/g" | \
      sed -e "s/$/.nuv/" | sed -e "s#^#$TV/#" | sort > /tmp/nuv-db-list
find $TV -maxdepth 1 -name \*.nuv -mtime +1 -print | sort > /tmp/nuv-files
join -v 2 /tmp/nuv-db-list /tmp/nuv-files > /tmp/dead-files
list=(`cat /tmp/dead-files`)
if [ ${#list[@]} -gt $MAX ]
then
    echo ${#list[@]} nuv files were identified for removal, but
    echo script is configured to limit removals to $MAX files.
    echo No action taken.
    exit 1
fi
for f in ${list[@]};
  do
  echo Moving $f to the trash can
  touch $f
  mv -f $f $TRASH
done
#
# Empty the trash can of all files that are older the $AGE days.
#
echo ""
echo "Checking $TRASH for files more than $AGE days old."
find $TRASH -maxdepth 1 -name \*.nuv -mtime "+$AGE" -print | \
     sed -e "s/^/  Removing file /"
find $TRASH -maxdepth 1 -name \*.nuv -mtime "+$AGE" -exec rm -f \{\} \;
exit 0



Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 05, 2005 8:48 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

Nice piece of work and I do like the leaning on the side of caution. Since I have two boxes running, I am going to run your latest version on one machine as a cron job daily.

The other I have tacked on to the bottom of my mythlink script so when it runs it will have married anything to the db already checked and should be valid for removal. The first little script recovered 21 gig that I didn't even know was missing :) Some of the "orphan" files went back to July.

I suppose one of us should let the mythtv folks know that there is a slight flaw to allow this to occur. Mean time until it is fixed this will work well to keep our KnoppMyth boxes running clean. Even if it is fixed down the road, nothing has to change with mythlink (as tweaked) , it just won't have to do any fixing. I think that is referred to as wearing a belt & suspenders. :)

Question, how do I get the first script to execute the rm command instead of dumping the info into a file? As far I see it, if it isn't in the database, it doesn't exist and I want my 21 gig back automaticly..

I probably won't have a follow up for a while as it is not like it is a major flaw, however over a period time it does eat away at the resources. What causes it, I don't know, but it definetly happens.

Thanks again for the clean up tool.
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 4:15 am 
Offline
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location: Brisbane, Queensland, Australia
I am naturally a lazy bastard and thus I hate a computerized process where you have to do something yourself that could be done through code. As such I did a little bit of research and went through the php code to see how mythweb deletes a recording.

Becuase why should you have to create a 1 byte nuv file if it only exists as a symlink and in the DB, why can't you just delete both. I have looked through the PHP code from MythWeb to see how it is done, but it seems a bit conveluted. I will have to do some more investigation into how this is done.

Also my suggestion to clean up the code is to use If Else statements instead of do unless. The reason I suggest this is that the script does not have to goes through the lines of code if doesn't have too. I suppose it does that now, but it does not seem to follow logic and would not be easy to extend to include other checks.

Please note: This code is simply my opinion and in no way critises the wonderful work that Marc and Mike have done

Code:
#!/bin/sh
# mythlink.sh - Symlinks mythtv files to more readable version in /tv/pretty
# by Dale Gass
# modified by Dave Misevich for slightly prettier names and doesn't destroy/recreate valid links.
# (I found a windows box watching a recording would terminate if the link was removed)
# Bug fixed caused by programs that don't have a subtitle. Use start time/date to create one.
# modified to add .mpg to end of symlink. Thanks mike at the mysettopbox.tv forum for the code.
# 6th Ocotober 2005 by Garth Kirkwood
# - change structure from do to if for easier logic (my thoughts)
# - cleanlinks run first to remove dud links from pretty
# Currently UNTESTED

# Remove dud links
cd /myth/pretty
cleanlinks

mysql -uroot  mythconverg -B --exec "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s'),title,subtitle from recorded;" >/tmp/mythlink.$$
perl -w -e '

        my $mythpath= "/myth/tv";
        my $altpath= "/myth/pretty";
        my $dfile= "";
        my %month = ( "01","Jan", "02","Feb", "03","Mar", "04","Apr", "05","May", "06","Jun",
                      "07","Jul", "08","Aug", "09","Sep", "10","Oct", "11","Nov", "12","Dec");
        if (!-d $altpath) {
               mkdir $altpath or die "Failed to make directory: $altpath\n";
        }
        opendir(DIR, $altpath) or die "cannot opendir $altpath: $!";
        while (defined($file = readdir(DIR))) {
                next if $file =~ /^\.\.?$/;     # skip . and ..
                @dirlist = (@dirlist,$file);
            }
            closedir(DIR);
        <>;
        while (<>) {
              chomp;
              my ($chanid,$start,$end,$title,$subtitle) = split /\t/;
              $subtitle = "" if(!defined $subtitle);
              my $ofn = "${chanid}_${start}_${end}.nuv";
              my $nfn = $title."-".$subtitle;

              #define name of new file
              $start =~ /^....(........)/;
              if ($subtitle eq "") {
                 # if no subtitle then build one from start time.
                 $subtitle = $month{substr($start,4,2)} . substr($start,6,2);
                 $subtitle = $subtitle . "_" . substr($start,0,4) . "_" . substr($start,8,4);
              }
              $nfn =~ s/&/+/g;
              $nfn =~ s/\047//g;
              $nfn =~ s/[^+0-9a-zA-Z_-]+/_/g;
              $nfn =~ s/_$//g;
             
               # create dummy if the nuv doesnt exist
      if ( !-e "$mythpath/$ofn") {
         
                   #print "Creating missing file $mythpath/$ofn\n";
                   open ($dfile,">$mythpath/$ofn") || die("Could not open txt file. $!");
                   print $dfile "Creating dummy file for $title $subtitle \n";
                   close ($dfile);
                  }
               # otherwise create the symlink
      else {
                   #print "\$ofn = $ofn $title \$nfn = $nfn\n";
                   # make a list of existing nuv filenames for use later.
                   $goodfile{$nfn.".mpg"} = 1;
                   if (!-s "$altpath/$nfn".".mpg"){
                      #  print "Creating $altpath/$nfn\n";
                       symlink "$mythpath/$ofn", "$altpath/$nfn".".mpg" or die "Failed to create symlink $altpath/$nfn".".mpg".": $!";
                   }
                }
        }

' /tmp/mythlink.$$
rm /tmp/mythlink.$$


[Edit] Correct grammer and change code a little.

_________________
Girkers


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 6:16 am 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi girkers,

Marc did all the work, I just held the light over the problem. I wouldn't go to nuts over making it efficient or fancy as the great crew at mythtv will eventually fix the flaw. This is just a simple way to help keep our boxes running clean for a long time.

Be sure to add the code from Marc to fix the error, and his second script shows you what is dead wood. Really great stuff I think. The WAF had to go up a notch because when she deletes something, (may take twice) but it will go away. :)

Look for the script that has this first line of "my" as it is the tweaked one.
my $dfile= "";

and
#do { print "Skipping $mythpath/$ofn\n"; next } unless -e "$mythpath/$ofn";

#
do {
print "Creating missing file $mythpath/$ofn\n";
open ($dfile,">$mythpath/$ofn") || die("Could not open txt file. $!");
print $dfile "Creating dummy file for $title $subtitle \l";
close ($dfile);
# next;
} unless -e "$mythpath/$ofn";
#

Thanks again and have a good day.
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 8:26 am 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
Quote:
Question, how do I get the first script to execute the rm command instead of dumping the info into a file? As far I see it, if it isn't in the database, it doesn't exist and I want my 21 gig back automaticly..


Mike, I would caution against using that first script to automatically delete files. The appeal is the it is very compact, but it will eventually delete an nuv file on you that it wasn't supposed to.

The new script does "age" orphaned files for 7 days before they are deleted -- if you want to reduce that duration, simply change the "AGE=7" statement at the top of the script to the duration you want. It is measured in days.

Having said all this, if you still want to use the old script at the bottom of pretty, you can try adding this as the last line of what I provided:

Code:
source /tmp/rm-dead-files


This is untested, but if the "rm" commands in the file look good on visual inspection, it should work file when "sourced". The newer script I posted is tested, and I am now running it daily on my machine.

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 8:35 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

I haven't tried your 7 day version yet. A couple of things bother me about your procedure.

First you are making another directory which is using resources that could be needed for recordings.

Second is then you are using bus space to move the large recorded files which may affect a recording or playback trying to stuff too much on the ide bus.

Third I would think that you could eliminate the first two potential issues just by doing a rename to say .mpg? That way you could easily preview the dead wood to see if it might be something you want to keep and also they would stand out in the tv directory for easy removal.

One other small item is the matching dead_wood.png and image_cache/dead_wood.png should also be cleaned out.

I plan to just wipe out all png's in the image_cache if it starts to build up.

Anyway, my thought for the day :)
Thanks
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 8:46 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi girkers,

Almost forgot your tweak, I will try it also however, as I see it, it may be just a hair quicker. Hey, at least I got you guys thinking :) Maybe we should have a log generated of what it does for us in the background other wise who will know it did it's work?

Thanks for all the ideas
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 9:37 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
Mike:

OK, sounds like you feel more comfortable staying with the other script. To address your 3 specific concerns with the new script:

1. An additional directory does not use a meaningful quantity of disk space.

2. Moving a file between directories that are on the same partition of the same disk drive has no more overhead than renaming the file. The system just updates pointers in the directory structures -- the content of the file is not moved. Try an "mv" by hand and you will see that the mv completes very quickly irrespective of file size. (Be sure that the two directories are on the same partition -- this will always be the case if you create a subdirectory and move it there.)

3. Easy viewing can still be done by looking in /myth/tv/TRASH. Having said this, there actually isn't any reason to look in TRASH under normal circumstances, as the empty-after-7-days is automatic. The only reason to look in TRASH is if you suddenly find a bunch of recordings are missing.

In terms of the ".png" file -- this also dawned on me and I have updated the script to deal with the ".png" file. I've also modified it so that it will also automatically generate dummy ".nuv" files when the data base has one of those "ghost" entries -- the condition that started this thread.

In any event, below is the latest, and likely final version.

Marc

Code:
#! /bin/bash
#
# This scropt will look for orphased ".nuv" files in /myth/tv
# directory and delete them. It will also check the data base
# for references to non-existant ".nuv" files, and create a
# dummy ".nuv" file where necessary. To protect your system
# from "run away" deletions, this script uses the following
# logic:
#
# 1. It will not do anything to a "nuv" file that is referenced in the
#    "recorded shows" listing of the myth data base.
# 2. It will not do anything to a "nuv" file that is less than 1-day old.
# 3. If it detects more than 8 "nuv" files that appear to be orphaned,
#    it will exit without taking any action.
# 4. Files that pass the previous 3 tests are not deleted. Instead they are
#    moved to the directory /myth/tv/TRASH. If they remain untouched in this
#    directory for 7 days, then they are automatically deleted.
#
#
TV="/myth/tv"
TRASH=$TV/TRASH
AGE="7"
MAX=8
#
# Verify existance of input directories
#
if [ ! -e "$TV" ]
then
    echo "$TV does not exist -- script halted."
    exit 1
fi
mkdir -p $TRASH
if [ ! -e "$TRASH" ]
then
    echo "Unable to created directory $TRASH"
    exit 1
fi
#
# Create a list of files that exist in $TV that aren't in the DB
#    and a list of files that exist in the DB that aren't in $TV.
#
mysql -uroot  mythconverg -B --exec \
      "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s') \
      from recorded;" >/tmp/mythclean.db
cat /tmp/mythclean.db | grep -v "^chanid" | sed -e "s/\t/_/g" | \
      sed -e "s/$/.nuv/" | sed -e "s#^#$TV/#" | sort > /tmp/nuv-db-list
find $TV -maxdepth 1 -name \*.nuv -mtime +1 -print | sort > /tmp/nuv-files
orphaned_files=(`join -v 2 /tmp/nuv-db-list /tmp/nuv-files`)
missing_files=(`join -v 1 /tmp/nuv-db-list /tmp/nuv-files`)
#
# If a file exists in $TV, has not been modified for at least 1 day,
# and is not referenced in the database, then move it to the trash can
#
if [ ${#orphaned_files[@]} -gt $MAX ]
then
    echo ${#orphaned_files[@]} nuv files were identified for removal, but
    echo script is configured to limit removals to $MAX files.
    echo No action taken.
    exit 1
fi
for f in ${orphaned_files[@]};
  do
  echo Moving $f to the trash can
  touch $f
  mv -f $f $TRASH
  if [ -e $f.png ]
  then
      echo Moving $f.png to the trash can
      touch $f.png
      mv -f $f.png $TRASH
  fi
done
#
# Empty the trash can of all files that are older the $AGE days.
#
echo ""
echo "Checking $TRASH for files more than $AGE days old."
find $TRASH -maxdepth 1 -mtime "+$AGE" -a \( -name \*.nuv -o -name \*.png \) \
     -print -exec rm -f \{\} \; | sed -e "s/^/  Removing file /"
#
# Create a dummy "nuv" file for any DB entry that does not point at
# a vaild file in $TV.
#
sleep 1
echo ""
echo "Checking for missing nuv files"
for f in ${missing_files[@]};
do
    if [ ! -e "$f" ]
    then
        echo "  Creating missing file $f"
        echo "Dummy" > $f
    fi
done
exit 0


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 06, 2005 10:11 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,
Great work!

Sorry about the senior moment as you are correct about the mv within the same partiton. I flashed on the cp which would have been a bit moving process. I guess I need a couple of those special glasses of fruit juice too. Medicimal purposes of course :)

I am not sure how the image_cache is purged so that may be one to bear watching just for the "cleanliness" as It would take a lot of png's to be an issue.

I have your new tweaks loaded on my "production" machine with a crontab for a daily check.
#8 Check for dead wood
58 23 * * * sh /myth/tools/dead_wood

Now maybe a log in the /home/mythtv ? In a perfect world, it would stay empty.

I will use the mythlink tweaks on my test box as I am constantly dinking with ideas so things have gotten messy once in a while :)
Thanks and have a good evening
Mike

I posted another idea here in General that may be a real challenge for you :idea:


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 18, 2005 11:14 pm 
Offline
Joined: Thu Sep 30, 2004 11:29 am
Posts: 2419
Location: Mechanicsburg, PA
Hi! Do you guys have an official version of mythlink.sh now?

_________________
KnoppMyth R5.5
MythiC Dragon v2.0
Join the KnoppMyth Frappr!


Top
 Profile  
 
 Post subject:
PostPosted: Wed Oct 19, 2005 6:26 am 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Human,

I don't know about how "official" the script is however, it works and seems to about complete. I just came up with the issues, Marc did all the work.

There are actually two independant tools, pretty which is enhanced to pick up missing files, and then the deadwood filter. When I was a kid, (long ago) I would be constantly cleaning a small woods near the barn so the cows didn't hurt them selves, hence we are picking up orphan files and doing a cleanup of the area so that KnoppMyth is hurt by stuff laying around :)

Here is his house keeping tool. I call it "dead_wood"
nano dead_wood

Code:
#! /bin/bash
#
# This scropt will look for orphased ".nuv" files in /myth/tv
# directory and delete them. It will also check the data base
# for references to non-existant ".nuv" files, and create a
# dummy ".nuv" file where necessary. To protect your system
# from "run away" deletions, this script uses the following
# logic:
#
# 1. It will not do anything to a "nuv" file that is referenced in the
#    "recorded shows" listing of the myth data base.
# 2. It will not do anything to a "nuv" file that is less than 1-day old.
# 3. If it detects more than 8 "nuv" files that appear to be orphaned,
#    it will exit without taking any action.
# 4. Files that pass the previous 3 tests are not deleted. Instead they are
#    moved to the directory /myth/tv/TRASH. If they remain untouched in this
#    directory for 7 days, then they are automatically deleted.
#
#
TV="/myth/tv"
TRASH=$TV/TRASH
AGE="7"
MAX=8
#
# Verify existance of input directories
#
if [ ! -e "$TV" ]
then
    echo "$TV does not exist -- script halted."
    exit 1
fi
mkdir -p $TRASH
if [ ! -e "$TRASH" ]
then
    echo "Unable to created directory $TRASH"
    exit 1
fi
#
# Create a list of files that exist in $TV that aren't in the DB
#    and a list of files that exist in the DB that aren't in $TV.
#
mysql -uroot  mythconverg -B --exec \
      "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s') \
      from recorded;" >/tmp/mythclean.db
cat /tmp/mythclean.db | grep -v "^chanid" | sed -e "s/\t/_/g" | \
      sed -e "s/$/.nuv/" | sed -e "s#^#$TV/#" | sort > /tmp/nuv-db-list
find $TV -maxdepth 1 -name \*.nuv -mtime +1 -print | sort > /tmp/nuv-files
orphaned_files=(`join -v 2 /tmp/nuv-db-list /tmp/nuv-files`)
missing_files=(`join -v 1 /tmp/nuv-db-list /tmp/nuv-files`)
#

# If a file exists in $TV, has not been modified for at least 1 day,
# and is not referenced in the database, then move it to the trash can
#
if [ ${#orphaned_files[@]} -gt $MAX ]
then
    echo ${#orphaned_files[@]} nuv files were identified for removal, but
    echo script is configured to limit removals to $MAX files.
    echo No action taken.
    exit 1
fi
for f in ${orphaned_files[@]};
  do
  echo Moving $f to the trash can
  touch $f
  mv -f $f $TRASH
  if [ -e $f.png ]
  then
      echo Moving $f.png to the trash can
      touch $f.png
      mv -f $f.png $TRASH
  fi
done
#
# Empty the trash can of all files that are older the $AGE days.
#
echo ""
echo "Checking $TRASH for files more than $AGE days old."
find $TRASH -maxdepth 1 -mtime "+$AGE" -a \( -name \*.nuv -o -name \*.png \) \
     -print -exec rm -f \{\} \; | sed -e "s/^/  Removing file /"
#
# Create a dummy "nuv" file for any DB entry that does not point at
# a vaild file in $TV.
#
sleep 1
echo ""
echo "Checking for missing nuv files"
for f in ${missing_files[@]};
do
    if [ ! -e "$f" ]
    then
        echo "  Creating missing file $f"
        echo "Dummy" > $f
    fi
done
exit 0


Here is the enhaced pretty
nano mythlink.sh
Code:

#!/bin/sh
# mythlink.sh - Symlinks mythtv files to more readable version in /tv/pretty
# by Dale Gass
# modified by Dave Misevich for slightly prettier names and doesn't destroy/recreate valid links.
# (I found a windows box watching a recording would terminate if the link was removed)
# Bug fixed caused by programs that don't have a subtitle. Use start time/date to create one.
# modified to add .mpg to end of symlink. Thanks mike at the mysettopbox.tv forum for the code.

mysql -uroot  mythconverg -B --exec "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s'),titl
e,subtitle from recorded;" >/tmp/mythlink.$$
perl -w -e '

       my $dfile= "";

        my $mythpath= "/myth/tv";
        my $altpath= "/myth/pretty";
        my %month = ( "01","Jan", "02","Feb", "03","Mar", "04","Apr", "05","May", "06","Jun",
                      "07","Jul", "08","Aug", "09","Sep", "10","Oct", "11","Nov", "12","Dec");
        if (!-d $altpath) {
               mkdir $altpath or die "Failed to make directory: $altpath\n";
        }
        opendir(DIR, $altpath) or die "cannot opendir $altpath: $!";
        while (defined($file = readdir(DIR))) {
                next if $file =~ /^\.\.?$/;     # skip . and ..
                @dirlist = (@dirlist,$file);
            }
            closedir(DIR);
        <>;
        while (<>) {
                chomp;
                my ($chanid,$start,$end,$title,$subtitle) = split /\t/;
                $subtitle = "" if(!defined $subtitle);
                my $ofn = "${chanid}_${start}_${end}.nuv";
                # skip if the nuv doesnt exist
                #do { print "Skipping $mythpath/$ofn\n"; next } unless -e "$mythpath/$ofn";

                #
                do {
                       print "Creating missing file $mythpath/$ofn\n";
                       open ($dfile,">$mythpath/$ofn") || die("Could not open txt file. $!");
                       print $dfile "Creating dummy file for $title $subtitle \l";
                       close ($dfile);
                      # next;
                } unless -e "$mythpath/$ofn";
                #

                $start =~ /^....(........)/;
                if ($subtitle eq "") {
                 # if no subtitle then build one from start time.
                  $subtitle = $month{substr($start,4,2)} . substr($start,6,2);
                  $subtitle = $subtitle . "_" . substr($start,0,4) . "_" . substr($start,8,4);
                }
                #my $nfn = "${title}-${subtitle}";
                #better but not perfect yet
                my $nfn = $title."-".$subtitle;
                $nfn =~ s/&/+/g;
                $nfn =~ s/\047//g;
                $nfn =~ s/[^+0-9a-zA-Z_-]+/_/g;
                $nfn =~ s/_$//g;
                #print "\$ofn = $ofn $title \$nfn = $nfn\n";
                # make a list of existing nuv filenames for use later.
                $goodfile{$nfn.".mpg"} = 1;
                if (!-s "$altpath/$nfn".".mpg"){
                #  print "Creating $altpath/$nfn\n";
                  symlink "$mythpath/$ofn", "$altpath/$nfn".".mpg" or die "Failed to create symlink $altpath/$nfn".".mpg".": $!";
                }
        }
        # remove symlinks that are not pointed at valid files in the database
        foreach $fname (@dirlist) {
            unlink "$altpath/$fname" unless $goodfile{$fname};
            #print "I want to unlink $altpath/$fname\n" unless $goodfile{$fname};
            }

' /tmp/mythlink.$$
rm /tmp/mythlink.$$

ls -Li /myth/pretty/*.mpg | sort > /tmp/mpg-files
ls -Li /myth/tv/*.nuv | sort > /tmp/nuv-files
join -v 2 /tmp/mpg-files /tmp/nuv-files | cut -d " " -f 2 > /tmp/dead-files
cat /tmp/dead-files | sed -e "s/^/rm -f /" > /tmp/rm-dead-files


They seem to work well
Mike

edit..I use crontab to execute the scripts
#min 0-59 * hr 0-23 * day 1-31 * month 1-12 * day of week 0-6(Su-Sa) * /cmd

#1 Tell me the time
# 0,30 * * * * /usr/bin/saytime

# 2 run mythlink.sh to make pretty tv links
5,35 * * * * /usr/local/bin/mythlink.sh


#8 Check for dead wood
58 23 * * * sh /myth/tools/dead_wood

edited 17Nov to remove a line wrap error


Last edited by mjl on Thu Nov 17, 2005 6:52 am, edited 1 time in total.


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 71 posts ] 
Go to page Previous  1, 2, 3, 4, 5  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 24 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu