Folks:
I began chasing a problem with MythArchive derailing at the point of thumbnail creation. I traced it back to mythtranscode generating streams that would cause tools used later in the process to quit. I've now proved to my own satisfaction that something about the specific cutlist used with each recording is causing the problem. I have instances where the same recording works fine without a cutlist, but not when a cutlist is present. And I have one pair of logs for runs in which the same recording worked with one cutlist but not with another. Both have full-verbose information for the mythtranscode run.

Unfortunately I'm not skilled enough in interpreting the detail to figure out what's going on from the logs (never mind that I know very little about digitized video), but it would seem that this is a bug in mythtranscode. Can anybody help? I'd be pleased to send logs -- they are much too long to post here.

Tb.

Still no luck. Lots of experiments with many different recordings. Upgrading FC4 -> FC6, many invocations of yum, much up-to-dateness. Still no recordings to DVD when a cutlist is present at the scene of the crime.

One trivial observation that I hope is a Telling Clue That Cracks The Case:
when it screws up, I see high values in this log line:

Using sync offset is disabled - it would be -76ms

usually in the high 70's, where with a successful attempt it's down around 16 or so. Not exactly scientific, in that my sample of successful runs is limited to one.

Just out of curiousity, did you run transcode on the files individually before using mytharchive?

Also, what version of mythtv are you using?

Mythtv is the latest out of SVN (just updated last night, in fact). I haven't run mythtranscode separately -- but it's an interesting idea, I have wondered if running a separate mythtranscode --mpeg2 without the cutlist first would "inoculate" against the problem, and I do see your point.

However, I am also reminded of somebody's quote to the effect that insanity is doing the same thing repeatedly in the hope that the outcome will be different. I am of course being pigheaded -- I'd like this to be fixed in the code because until it is mytharchive is a step backwards from the old .19 mythburn scripts. I *very much* appreciate the improvements that have been made, and I agree that cleaning up the DVD format is an important step (for one thing, I can now go back and rip myth-created DVDs for use in my portable player when I forget to make an AVI version before deleting the source file, and that's great). I've done enough software in my day to know that it is not always an orderly march straight to the goal, especially when you're volunteering your time. But not being able to cut commercials is a big disappointment.

This is a huge effort and Paul is to be warmly congratulated. I know only too well that when you're a developer on a popular freely-distributed application the entire Internet can start to look like a collection of open mouths with greed and griping attached. I admire anybody who can manage to make a sustained contribution -- and like everybody else, I want all the features I want RIGHT NOW and I deserve them because I'm the only important person in the universe.

You might also experiment with rerunning commflag. But I don't know if commflag changed when you installed the update or not.

But commflag is the piece that handles the cutlist I believe. (but I could be wrong.)

... but in these cases I went back and refined it, moving all the cut points to keyframes. Older versions of the mythburn scripts handled cut points just fine no matter where the heck you put them. The new version of mythtranscode seems to be sensitive to the location of cutpoints in ways its maker did not intend.
t.

tbruce

Have you found a way to consistently reproduce the problem? If you can give me the steps you take so I can try to reproduce it here then that would help me to get this fixed.

Alternately if you could make a 'Native Archive' of a file that you have problems with, preferably a small file, and make it available for download somewhere so that I can try to reproduce it using the same file and same cut list etc.

Paul:

I will have no trouble reproducing it -- basically, every time I try to archive anything with a cutlist this happens. I will make a native archive as you suggest. Also, what do you need to know about my configuration?

While we're on this, I found another small bug that needs attention -- really trivial, but annoying. Turns out that if a double-dash shows up in a program title dvdauthor can't cope -- it assumes that its the opening of an XML close-comment character combination (ie., the first -- of -->) and barfs because it doesn't find an angle bracket. So probably double-dashes in anything being fed to dvdauthor should be escaped.

Native archive and config information coming tonight, EDT.

t.

It's difficult to know what may be relevant. If you give me the output of running this query

in mysql should give me all I need to know.

It is certainly not my intention to de-rail this thread, but if by chance this is the same problem I am seeing here (click me!) then I will attempt to add any relevant queries/logs/etc that might help people get to the bottom of this.

If by chance this is totally unrelated, I will extract myself from this thread and let everyone else carry on and troubleshoot this issue.

=)

_________________
Currently Running:
Too lazy to update this with my current hardware, I'll redo it during my next install =)

Paul:
Files are at http://hula.law.cornell.edu/mythdrop . There are (or will be) four: two log files (main and progress), a .csv file with the mytharchive settings from the database, and a tgz'd native archive of a 75-minute show.

I say "will be" because the archive is taking some time to transfer -- should be there about 4 AM Eastern Daylight Time.

Hope this helps. I'm eager to hear.

Best,
Tb.

tbruce

Thanks for the files. I can reproduce exactly what you are seeing so long as I use your own cut list. Also now that I'm beginning to understand the problem I can, with a little tweaking of the cut list in the database, reproduce it with one of my own files.

I can also tweak your cut list in the database or edit the cut list in Myth and get the DVD to be created normally.

The problem seems to be with the very first pair of cut start/end points in the cut list. The first cut start is not actually on the first frame but appears to be on the second frame. This causes mythtranscode a problem because there is a single frame at the start of the file then a chunk that it safely removes then the start of the file proper. Changing the position of the first start cut so it is the first frame, either by tweeking its position it the DB or changing its position on Myths cut list edit, allows the DVD to be created normally.

I have a hunch that it is actually mythcommflag that is creating these incorrect start cut points because in the metadata you provided the cut list that mythcommflag created also has the first start cut point on the second frame!

When you created the cut list for this file did you start with the cut list generated by mythcommflag and then change the positions to where you wanted them?

The way to avoid this problem until a proper fix is found would be if you need to remove the start of a file is to not put a start cut point at the very start of the file but to move to where you want the end cut point to be, that is where you want the final file to start, and choose 'Delete before this frame'. That will create a correct start cut point from the first frame automatically. If the file has been comm flagged don't rely on the start cut point generated by it. Delete any cut point near to the start and create a new end cut and delete to the left as describe above so the correct start cut is created.

Does any of this tally with what you have been seeing with other files?

Paul:

That's *very* interesting. And also evidence that the edit mode of "watch recordings" lies a bit about frame positions. A couple of observations:

1) Edit mode has always done odd things at the beginning of recordings. When I first started working with MythTV I in fact did as you suggested, and was sort of surprised that it seemed to be putting in an extra cut point at the beginning -- I seemed to get 3 cut points (out, out, and in) when doing the move-to-where-you-want-to-start-and-say-delete-before technique that you recommend. But perhaps I was hallucinating, and I'll try it your way.

2) For sure edit mode is lying about frame numbers, then. Because at some point during this whole sleuthing process it occurred to me that the problem might be exactly as you describe it (based on a post I had seen in one of the forums here, or on the mythtv list). So I very self-consciously tried to set my out points on frame one, and according to the frame number indicator in edit mode, I was. But I will do it your way, and be happy, I guess.

Thanks for all of your very hard work on this. It's a huge help.

All the best,
Tb.

Experience with several recordings now shows that in fact it is the initial cut that is the problem.

I just tried to reproduce the "opening cut" problem in .20 and couldn't. But here is what I am guessing would happen under .19. My usual approach when adjusting cuts is to (once in edit mode):

a) place an initial "in" cut so as to get rid of any preliminaries (my setup starts recording 60s before the scheduled time).
b) hit "Z" to show the commercial flags
c) move through the cut points flagged by the commercial flagger, adjusting as needed one by one.

I think what used to happen under .19 was that step b) would introduce an extra cut point, maybe two, put in by the commercial flagger and usually somewhere between the first frame and the first in point I set in a). Anyway, .20 doesn't seem to do this, and I'm happy with the workaround.

I always assumed that the burner (or maybe the editor) had some kind of cutpoint-rationalizing system to fix odd bits of stuff like having overlapping cuts etc. , because otherwise my stuff would be even more of a mess than it is (grin).

All the best, and thanks again,
Tb.

Leaving the initial and final cuts as "open intervals" seems to work most reliably in my experience. Thus my cut lists always look like this (with [ representing cut after this point and ] representing cut before):

---]--------------------[------]----------------[-------]-----------------[---

I also try to keep my cuts at keyframes to minimize reencoding.