View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 71 posts ] 
Go to page Previous  1, 2, 3, 4, 5  Next

Print view Previous topic   Next topic  
Author Message
Search for:
 Post subject:
PostPosted: Mon Oct 03, 2005 12:19 am 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

As I've re-read the various posts, I realize that I mis-understood the problem. I thought the mythlink was somehow blowing up when the physical file was missing. As I re-read everything, it sounds like

1. You end up with is a symbolic link in /myth/pretty that points at a non-existant ".nuv" file.

2. You would like a way to automatically create an empty ".nuv" file for each symbolic link of this nature, so that you can properly delete things in myth.

Is this correct? If so, executing the following linux command will do this:

Code:
find /myth/pretty -type l \! -xtype f -name \*.mpg -exec touch \{\} \;

Note that the character after "-type " above is a lower-case letter "L", not the number one.


This will locate all files in /myth/pretty that are symbolic links pointing at non-existant files and will execute a "touch" on that symbolic link. This results in the creation of the file that the symbolic link is pointing at.

If I did not properly understand the situation, then it is probably best if you re-summerize things for me, as I may be mis-interperting the thread, above.

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 7:18 am 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

I appreciate the help and look forward to trying the new line. Just have to figure out where to squeeze it in.

Your discription is petty close. The script doesn't crash or anything major, and if run from a term or console, it outputs basicly I what I sent as samples. It does seem that it would no longer process files once the missing file was encountered.
Quote:
1. You end up with is a symbolic link in /myth/pretty that points at a non-existant ".nuv" file.
The link is not created unless a valid nuv is present.

Quote:
2. You would like a way to automatically create an empty ".nuv" file for each symbolic link of this nature, so that you can properly delete things in myth.
Correct

Quote:
This will locate all files in /myth/pretty that are symbolic links pointing at non-existant files and will execute a "touch" on that symbolic link. This results in the creation of the file that the symbolic link is pointing at.
Almost, the links are not created unless the .nuv is real. The pretty link is not the problem, the missing .nuv is and that has to be there for house cleaning.

Just make a short "junk" recording and check that it shows up in a browser/recordings. Then remove the nuv from /myth/tv. If you run the mythlink.sh it should report that it is skipping "something" and at this point you know that you can not delete the entry with any of the myth tools. If mythlink.sh were to create the missing file instead of just reporting it, it would be deleteable since it is now real once again.

I have noticed that when run as a cron job that once the missing file is discovered, it does not make links for any recordings that were more recent. I grab short recordings daily of weather scenes and over a period of time entries will be get deleted from the /myth/tv directory however not from the listing of recordings. Probably when the auto expire runs something interupts the cycle before it completes which may cause the anomolly, just guessing on the how / why.

Hope this clears any confusion I may have left for you.

Have a good day!
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 9:26 am 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike, please ignore this message. It is a repeat of my previous msg. Somehow when I looked this morning I didn't see my previous post and thought it was somehow lost. I now see it is there. I will get back to you on this in the next day or so.

Marc

Mike:

I've re-read some of the threads above, and I now believe I misunderstood the problem that needs to be solved. Here is my current understanding of problem:

1. Sometimes a show will be left in the database even though the ".nuv" file no longer exists. When this happens, an attempt to delete the episode in myth returns an error without deleting the show.

2. Under these circumstances, there is usually still a sym link left in /myth/pretty pointing at the non-existant ".nuv" file

3. Creating an empty or "dummy" .nuv file will enable you to delete the episode in myth without getting an error.

Is this essentially correct? If so, then typing the following unix command will automatically identify each symbolic link that points at a non-existant file and create the corresponding ".nuv" file:

Code:
find /myth/pretty -type l \! -xtype f -name \*.mpg -exec touch  \{\} \;

Note that the letter after "-type " is a lower-case "L", not the digit "one".


Another approach is to simply type:

Code:
touch /myth/pretty/*.mpg


The second approach has the (mild) downside of updating the date/time stamp of the ".nuv" files that exist properly.

If I have mis-understood the problem that we are trying to solve, it owuld be best if you could summerize it for me, thanks.


Marc[/code]


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 10:02 am 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

Edit mythlink.sh and look for the code segment that looks like this:

Code:
        while (<>) {
                chomp;
                my ($chanid,$start,$end,$title,$subtitle,$seriesid,$programid,$originalairdate,$description) = split /\t/;
                $subtitle = "" if(!defined $subtitle);
                my $ofn = "${chanid}_${start}_${end}.nuv";
                # skip if the nuv doesnt exist
                do { print "Skipping $mythpath/$ofn\n"; next } unless -e "$mythpath/$ofn";


The word "chomp" only occurs once in the file, so it should be easy to find. Replace the last two lines of this code segment (the "#" comment and the "do" statement) with the following:

Code:
                # Create dummy file if the nuv doesn't exist
                do {
                       print "Creating missing file $mythpath/$ofn\n";
                       open ($dfile,">$mythpath/$ofn") || die("Could not open txt file. $!");
                       print $dfile "Creating dummy file for $title $subtitle \l";
                       close ($dfile);
                } unless -e "$mythpath/$ofn";


Also, you need to add the following line towards the top of the file where you see the other "my" statements:

Code:
        my $dfile= "";


I have tested that this code will not cause problems if the ".nuv" file exists. I have not yet tested that it properly creates the ".nuv" file when the ".nuv" is missing. Having said this, I have tested that code in another context, so it should be OK.

Marc


Last edited by marc.aronson on Mon Oct 03, 2005 9:12 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 7:17 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

I think we are getting closer however it seems the file can not be a null length. If I do a touch to create the file I still get an error. If I do echo " " > to create the file, all is well.

As far as I can tell, it didn't break anything :)

Mike


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 8:01 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

Sigh... The posting of untested code shold be outlawed. Anyway, if you remove the "next;" statement that is the next-to-last line of the code snipet in my previous posting, the problem will be solved. Once you have done this there is not any reason to use the find command anymore. This revised version of mythlink will create the ".nuv" file with a single line in it that says "Creating dummy file for <title> <subtitle>".

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 8:10 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
marc.aronson wrote:
Sigh... The posting of untested code shold be outlawed.
;-) Why do you think most of the shell commands and examples I post are cut and pasted directly from a shell window where I've just executed them? Oh, and the ones that aren't are almost always labelled as "this is completely untested". ;-)


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 9:04 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Gentlemen,

I am willing to bet that there are more than the three of us that are following the process of making this work. The ~4300 members and probably ten times that of visiting guests with as many levels of expertise, at least ten or six of them are looking for something simular or may even already been there and done it.

I really want to thank Marc for his taking the time and sharing his knowledge with this addition to mythlink.sh. The fringe benifit will be hidden from view of those that have it running that when an error occurs, it will be self cleaning. Currently, if someone has a recording they can't delete, the dance of trying to find out what is broke and what to do and how to do it begins.

This tweak, even though we are in the testing "fine tuning" stage can still be consided educational. Once it is completed, it should also help with the WAF as they seem to expect things to work properly, to be honest, so do I. You select delete, the recording should go away.

Anyway, I will make the change, break a couple recordings and let you know how it works out :)
Update, didn't work. I deleted a recording, ran the script and it complained however did not create the file. Sorry for the sad news :(
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Mon Oct 03, 2005 10:48 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

This is strange. I tested it here by blowing away a .nuv file and it properly created the dummy file for me. Did the sym link in /myth/pretty get created? Might be a good idea if you paste me a copy of your modified mythlink script. Perhaps we are working from different bases...

Update on Tuesday, 10/4. I ran another round of tests, this time blowing away 2 ".nuv" files. It properly created the dummy ".nuv" files. I confirmed it worked regardless of weather the /myth/pretty sym links exist or don't exist when the script is run. Finally, I confirmed that I could delete the episose from mythweb with the dummy file in place.

Possible explantions for our different experiences include:

1. I have made other modifications to my mythlink script. Perhaps there is sufficient difference between our scripts that my changes won't merge properly into yours. Once you send me a copy of your mythlink script, I will check for possible problems.

2. Are you running mythlink from an account that has permission to write to "/myth/tv"? The original pretty script never wrote to "/myth/tv", so this would not have been an issue before.

Finally, can u confirm that you get the message "Creating missing file /myth/tv/<numbers>.nuv" for each missing ".nuv" file when u run the script?

Thanks.

Marc


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 6:30 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

Here is my flavor of mythlink.sh before adding your mods, I like this because it works very well with that other os.

Mike

Code:
#!/bin/sh
# mythlink.sh - Symlinks mythtv files to more readable version in /tv/pretty
# by Dale Gass
# modified by Dave Misevich for slightly prettier names and doesn't destroy/recreate valid links.
# (I found a windows box watching a recording would terminate if the link was removed)
# Bug fixed caused by programs that don't have a subtitle. Use start time/date to create one.
# modified to add .mpg to end of symlink. Thanks mike at the mysettopbox.tv forum for the code.

mysql -uroot  mythconverg -B --exec "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s'),titl
e,subtitle from recorded;" >/tmp/mythlink.$$
perl -w -e '

       my $dfile= "";

        my $mythpath= "/myth/tv";
        my $altpath= "/myth/pretty";
        my %month = ( "01","Jan", "02","Feb", "03","Mar", "04","Apr", "05","May", "06","Jun",
                      "07","Jul", "08","Aug", "09","Sep", "10","Oct", "11","Nov", "12","Dec");
        if (!-d $altpath) {
               mkdir $altpath or die "Failed to make directory: $altpath\n";
        }
        opendir(DIR, $altpath) or die "cannot opendir $altpath: $!";
        while (defined($file = readdir(DIR))) {
                next if $file =~ /^\.\.?$/;     # skip . and ..
                @dirlist = (@dirlist,$file);
            }
            closedir(DIR);
        <>;
        while (<>) {
                chomp;
                my ($chanid,$start,$end,$title,$subtitle) = split /\t/;
                $subtitle = "" if(!defined $subtitle);
                my $ofn = "${chanid}_${start}_${end}.nuv";
                # skip if the nuv doesnt exist
                #do { print "Skipping $mythpath/$ofn\n"; next } unless -e "$mythpath/$ofn";

                #
                do {
                       print "Creating missing file $mythpath/$ofn\n";
                       open ($dfile,">$mythpath/$ofn") || die("Could not open txt file. $!");
                       print $dfile "Creating dummy file for $title $subtitle \l";
                       close ($dfile);
                      # next;
                } unless -e "$mythpath/$ofn";
                #

                $start =~ /^....(........)/;
                if ($subtitle eq "") {
                 # if no subtitle then build one from start time.
                  $subtitle = $month{substr($start,4,2)} . substr($start,6,2);
                  $subtitle = $subtitle . "_" . substr($start,0,4) . "_" . substr($start,8,4);
                }
                #my $nfn = "${title}-${subtitle}";
                #better but not perfect yet
                my $nfn = $title."-".$subtitle;
                $nfn =~ s/&/+/g;
                $nfn =~ s/\047//g;
                $nfn =~ s/[^+0-9a-zA-Z_-]+/_/g;
                $nfn =~ s/_$//g;
                #print "\$ofn = $ofn $title \$nfn = $nfn\n";
                # make a list of existing nuv filenames for use later.
                $goodfile{$nfn.".mpg"} = 1;
                if (!-s "$altpath/$nfn".".mpg"){
                #  print "Creating $altpath/$nfn\n";
                  symlink "$mythpath/$ofn", "$altpath/$nfn".".mpg" or die "Failed to create symlink $altpath/$nfn".".mpg".": $!";
                }
        }
        # remove symlinks that are not pointed at valid files in the database
        foreach $fname (@dirlist) {
            unlink "$altpath/$fname" unless $goodfile{$fname};
            #print "I want to unlink $altpath/$fname\n" unless $goodfile{$fname};
            }

' /tmp/mythlink.$$
rm /tmp/mythlink.$$


Edit Added testing code


Last edited by mjl on Tue Oct 04, 2005 8:47 pm, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 8:31 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

I should have been more specific in my request -- please post the version you created with my changes in it. Thanks!

marc


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 8:53 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

I've taken your copy of the script and merged my changed into it. I tried this on my system after deleting a ".nuv" file, and it displayed the proper message about creating a dummy replacement file and the file was properly created. Below is the revised script. If this doesn't solve the problem, then we need to look for other explanations. Some additional questions:

1. What username are you logged in as when you execute it? I run as "root".

2. Are you executing this on Knopix or some other distribution/OS?

3. When this revised mythlink script encounters a missing ".nuv" file, are you getting the output message "Creating missing file $mythpath/$ofn\n"?


Marc


Code:
#!/bin/sh
# mythlink.sh - Symlinks mythtv files to more readable version in /tv/pretty
# by Dale Gass
# modified by Dave Misevich for slightly prettier names and doesn't destroy/recreate valid links.
# (I found a windows box watching a recording would terminate if the link was removed)
# Bug fixed caused by programs that don't have a subtitle. Use start time/date to create one.
# modified to add .mpg to end of symlink. Thanks mike at the mysettopbox.tv forum for the code.

mysql -uroot  mythconverg -B --exec "select chanid,DATE_FORMAT(starttime,'%Y%m%d%H%i%s'),DATE_FORMAT(endtime,'%Y%m%d%H%i%s'),title,subtitle from recorded;" >/tmp/mythlink.$$
perl -w -e '

        my $mythpath= "/myth/tv";
        my $altpath= "/myth/pretty";
        my $dfile= "";
        my %month = ( "01","Jan", "02","Feb", "03","Mar", "04","Apr", "05","May", "06","Jun",
                      "07","Jul", "08","Aug", "09","Sep", "10","Oct", "11","Nov", "12","Dec");
        if (!-d $altpath) {
               mkdir $altpath or die "Failed to make directory: $altpath\n";
        }
        opendir(DIR, $altpath) or die "cannot opendir $altpath: $!";
        while (defined($file = readdir(DIR))) {
                next if $file =~ /^\.\.?$/;     # skip . and ..
                @dirlist = (@dirlist,$file);
            }
            closedir(DIR);
        <>;
        while (<>) {
                chomp;
                my ($chanid,$start,$end,$title,$subtitle) = split /\t/;
                $subtitle = "" if(!defined $subtitle);
                my $ofn = "${chanid}_${start}_${end}.nuv";
                # skip if the nuv doesnt exist
                do {
                       print "Creating missing file $mythpath/$ofn\n";
                       open ($dfile,">$mythpath/$ofn") || die("Could not open txt file. $!");
                       print $dfile "Creating dummy file for $title $subtitle \n";
                       close ($dfile);
                } unless -e "$mythpath/$ofn";
                $start =~ /^....(........)/;
                if ($subtitle eq "") {
                 # if no subtitle then build one from start time.
                  $subtitle = $month{substr($start,4,2)} . substr($start,6,2);
                  $subtitle = $subtitle . "_" . substr($start,0,4) . "_" . substr($start,8,4);
                }
                #my $nfn = "${title}-${subtitle}";
                #better but not perfect yet
                my $nfn = $title."-".$subtitle;
                $nfn =~ s/&/+/g;
                $nfn =~ s/\047//g;
                $nfn =~ s/[^+0-9a-zA-Z_-]+/_/g;
                $nfn =~ s/_$//g;
                #print "\$ofn = $ofn $title \$nfn = $nfn\n";
                # make a list of existing nuv filenames for use later.
                $goodfile{$nfn.".mpg"} = 1;
                if (!-s "$altpath/$nfn".".mpg"){
                #  print "Creating $altpath/$nfn\n";
                  symlink "$mythpath/$ofn", "$altpath/$nfn".".mpg" or die "Failed to create symlink $altpath/$nfn".".mpg".": $!";
                }
        }
        # remove symlinks that are not pointed at valid files in the database
        foreach $fname (@dirlist) {
            unlink "$altpath/$fname" unless $goodfile{$fname};
            #print "I want to unlink $altpath/$fname\n" unless $goodfile{$fname};
            }

' /tmp/mythlink.$$
rm /tmp/mythlink.$$


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 8:57 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,
Sorry about that, it is now into the previous post.

Do you follow what I am trying to do? I have found the mythlink script covers all four possiblities of data.

A & B are good
A no database entry * no recorded entry
B database entry * recorded entry

C & D are not good
C database entry * no recorded entry (echo " " > /myth/tv/missing-file-name.nuv)
D no database entry * recorded entry (delete /myth/tv/skipped-file-name.nuv)

End result, a much higher WAF and the pretty files will update properly.

Thanks
Mike


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 9:14 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1530
Location: California
Mike:

I've looked at your script and I don't see a reason any issues. Please see my previous post -- the answers to those questions may help diagnose the problem. Your conditions A, B & C should all be covered -- I have not attempted to cover condition "D", but let's worry about that after we figure out why scenario "C" isn't working.


Marc


Top
 Profile  
 
 Post subject:
PostPosted: Tue Oct 04, 2005 9:18 pm 
Offline
Joined: Sun Jun 12, 2005 10:55 pm
Posts: 3161
Location: Warwick, RI
Hi Marc,

That would be quite rude to be using the KnoppMyth board to trouble shoot something for a different system. :( Besides, the R5A16 is just simply the best. :)

I have it set in the root crontab to run at 5,35 and when manual for testing also as root.

Ok, I just put back a copy of what I sent to you and ran it. It looks good to go.. why not before ??

root@mythtv-20:/myth/tools# /usr/local/bin/mythlink.sh
Creating missing file /myth/tv/2051_20051004185400_20051004190200.nuv

root@mythtv-20:/myth/tools# cat /myth/tv/2051_20051004185400_20051004190200.nuv
Creating dummy file for Weather: PM Edition

The browser allowed the file delete as it should have.

Now, will the script catch the reverse issue?

Thanks
Mike


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 71 posts ] 
Go to page Previous  1, 2, 3, 4, 5  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu