I just installed my first HDHOMERUN. RUnning LINHES 6.04.0. I did a channel scan following by a download for schedules direct. While the channel scan did find the call signs for the stations, it looks like the XMLTVID didn't get filled in for most of them as the listings data shows no match for major channels like KICU. Do I need to poke the XMLTVID's in by hand or is there an automated procedure for doing this? Thanks in advance for your help!

Marc

_________________
Marc

The views expressed are my own and do not necessarily reflect the views of my employer.

It's been awhile but I think what I did was in mythtv-setup -> channel editor I selected icon download. It then matches callsigns with xmltvids and downloads the icon. Then mythfilldatabase works. (i think)

I had to input the XMLID by hand for my WOW! service after the digital changeover. You can do this through MythWeb > Settings > TV > Channel Info tab (easiest way I know).

After you update all the XMLID's make sure to run a mythfilldatabase to propagate the listings.

_________________
-Roseville, Michigan USA
LinHES R8: FE/BE, FE (x2)

Thanks for all the suggestions. Here is what I have learned:

1. Downloading the icons does not help get the XMLTVID's set properly.

2. If you set the channel number to the logical channel number shown in schedules direct & ensure that the XMLTVID is blanked out, the next run of "mythfilldatabase --refresh-today" will fill in the XMLTVID field with the appropriate value.

_________________
Marc

The views expressed are my own and do not necessarily reflect the views of my employer.

Mostly. I think it was about 80% for Comcast here since it depends on matching the embedded channel id information with what's available in the Schedules Direct feed and some of them were either missing or enough different not to match... So depending on cable provider and line up, YMMV.

WOW cable is on rollout change 5 of 6 this month which forced me to upgrade from 5.5 to 7.03

I don't have this perfected yet, but zip2it is still your friend and has your xmltvids, channels, and even frequencies.

1. Goto tvlistings.zap2it.com and setup an account if needed to pick your provider.

2. Go to TV -> TV Listings

3. View Source and save as text file.

4. Parse the source for the goodies.

I am using the following regex expression to parse out my channel information:


From there I save the values as a CSV file:

Then in a spread sheet I will be make a formulas that takes captured regex data and makes a sql update statement:

What program are you using to process that regexp? ie, grep, sed, awk, etc.

_________________
Marc

The views expressed are my own and do not necessarily reflect the views of my employer.

I formulated the the regex and generated the csv file using a program call Expresso. Its a windows program geared at .Net.

http://www.ultrapico.com/

For the CSV I am using google docs to make <cough> sql.

Today is the final lineup change, and Zip2It is not lining up with the myth scan at all.

Hopefully the frequency in dtv_multiplex line up, then I think I have enough info to by pass SD (home of 20 golf channels, 20 tvlands, etc) and just write insert statements into the channel table.

You could probably write the sql code doing a regex replace, but have do not have any experience with complex replace regexes.

I tried running that regexp using expresso on zap2it data for my area -- it didn't product any results so I can't assess weather the zap2it data is useful in my area. I did the following:

1. Loaded the regexp by using file->Read Regular Expression...

2. Loaded the zap2it data by using file->read input data...

3. Hit tools->Run Match while in test mode

No matches. Did I do it right?

If the zap2it data seems to be good I can write a Linux script that automates the end-to-end process...

_________________
Marc

The views expressed are my own and do not necessarily reflect the views of my employer.

I think it depends on how the browser saves the html data. Last night the right click save was different than the view source source save. The right click save escaped out an ampersand which caused the previous regex not to work.

Enter this in the test tab for the regular expression:

Click the design tab and check the only the following options on the bottom:

Go back to a select the File->Read Input Data... menu item and read the zap2it html source file.

Next click the Run Match button and it should spit out a row per channel in the search results. If you highlight some of the regular expression you can click the Partial Match, and it will match only the highlighted bit of the regular expression. This is great for debugging where to see where the regex stops matching. Also clicking on search result item highlights the select text in the Input Data window. Also good for debugger. Another hint is the search results are ordered by the source file, so your channels should be listed 1 to n sequentially. I there are missing channels something needs adjusting. Note the regex is accounting for both when a channel does and does not have an icon reference.

In the Search Results, right click and select Export to comma separated values...

These are the two sql statements that worked for me. Note that I deleted all the channels and before hand and ran mythsetup. When the channel scan was complete I querried mysql after each specific add channel popup appeared. I did not click the next install all channels until all channels were mapped from the previous insert.

I could only paste like 10 statements into mysql via putty without mysql locking up.



Here is a link to Zap2it MySql XmlTvId Helper spread sheet.

Also for copy and paste, usually spread sheet to notepad, copying notepad and pasting to terminal is the best way to avoid corruption and oddities.

For each sql query column make an extra column. Select the sql formula column and select copy. Next select the blank column and perform a paste values into. This is the column you need to copy from and pastee into mysql from.[url][/url]

I offer the following and I hope won't be seen as an insult but I personally find the whole Espresso and Excel coding baffling.
As I am uncertain as to the what the final output should be, consider this script as incomplete and or wrong but I think it is close. I tried to emulate the output format:



The script assumes the page source from zap2it is created and called listings. The curl command is commented out in the script, as it results in much shorter list of channels.

The script assumes that it is in the same directory as listings and outputs two files, listings.csv and csv.formatted.





Additions or corrections welcome.

mythedoff, thanks for posting that script -- you saved me time of doing the conversion to curl, sed and awk -- very much appreciated. After cleaning up a few cut-and-paste errors it works.

Any idea why Curl only pulls down the first 103 channels?

_________________
Marc

The views expressed are my own and do not necessarily reflect the views of my employer.

Ok, so it looks like curl results differ because of the zipcode which the browser has stored in a cookie. The curl command was using 12345.

It looks as if curl can pass the the stored browser cookies which can be found in the browser source file. See: http://curl.haxx.se/docs/httpscripting.html

A better solution would be to pass the zipcode which I would think is possible.

I saved the source from my browser rather than futzing with "curl" for now, so that I could verify that the frequencies made sense. After running the script, the first bunch of lines in listings.csv contain the following:


Every channel is at a frequency of 46x,xxx,xxx. When I look at my exiting channel table, which is the result of my previous channel scans, I don't find anything on that frequency. I'm running a fresh scan now, but I also find it difficult to believe that every channel is at a frequency of 46x,xxx,xxx.

The script also seems to assume that the logical channel number and serviceid are they same, but this is rarely the case.

Is anyone successfully using this data to match scanned channel results to what the script produces?

Thanks!

When I look at the results of doing a channel scan, I don't see anything at that frequency.

_________________
Marc

The views expressed are my own and do not necessarily reflect the views of my employer.

Sorry, as I said I was uncertain as to what the final output should be. I was trying to interpret the excel results which I must have done incorrectly. So the script is clearly wrong. Perhaps you could explain, possibly by example, what the correct data should be.

Here is an alternative that could help. After entering the following and hitting the enter key, enter your zipcode.



Sample output:
<Modulation>qam256</Modulation>
<Frequency>111000000</Frequency>
<PhysicalChannel>98</PhysicalChannel>
<ProgramNumber>1</ProgramNumber>
<GuideNumber>924</GuideNumber>
<GuideName>WILLDT3</GuideName>
--
<Modulation>qam256</Modulation>
<Frequency>111000000</Frequency>
<PhysicalChannel>98</PhysicalChannel>
<ProgramNumber>2</ProgramNumber>
<GuideNumber>902</GuideNumber>
<GuideName>SMRENG</GuideName>
--