View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 14 posts ] 
Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Sat Apr 14, 2007 8:35 am 
Offline
Joined: Sun Aug 28, 2005 7:07 pm
Posts: 821
Location: Melbourne, Australia
Hi all,
I've been working on a script tonight to make the epg process a bit easier.
Assuming everyone has already moved across to the community guide at http://minnie.tuhs.org/tivo-bin/tvguide.pl, there are a few post-install things that I've automated.
Here's what you do:
1. create a file fixepg-au.sh and drop this in...

Code:
#!/bin/bash
echo "
There is a problem with using Knoppmyth in Australia.
The problem is that the xml ids used to update the epg grabber's information aren't inserted when you scan for channels, so mythfilldatabase needs a lot of knowledge to get it going.
To make this easier, I've written this script. You would have subscribed to the community epg by now. If you haven't, do it now.
You should also only run this AFTER you have scanned for channels and AFTER you have specified the name of your video source during the installation.
The script must be run by user 'mythtv'.
This script will:
1. Get and install tv_grab_au_reg if you choose this, then create the appropriate tv_grab_au_reg configuration file
2. Go get the epg xmltvids for the channels you chose at tuhs
3. Update your database with these xmlids
4. Tag all ABC channels as commercial free
5. Change the name of channel 1 so both it and 10 don't have the
same name (both are 'TEN Digital', we change 1 to '10 Digital')
The reason for this is to avoid recording The Simpsons (for example) twice when you've said 'any time on this channel' - grr!

You need to supply:
1. your TUHS username and
2. your TUHS password.
3. the name you gave to your video source
If you don't have these, they're acquired after free registration at
http://minnie.tuhs.org/twiki/bin/view/TWiki/TWikiRegistration
"
  printf "What is your TUHS username? "
  read USERNAME;
  printf "What is your TUHS password? "
  read PASSWORD;
  printf "What is the name of your video source? "
  read SOURCENAME;
  printf "Do you need to install tv_grab_au_reg (requires root password)?"
  read TV;
case $TV in
  [yY]) echo "Enter your root" && su -c 'wget -O /usr/bin/tv_grab_au http://www.cse.unsw.edu.au/~willu/xmltv/tv_grab_au_reg && chmod a+x /usr/bin/tv_grab_au' root ;;
esac
# must have a tuhs username and password to do this
TUHSUSER=$USERNAME
TUHSPASS=$PASSWORD
CONFIGFILE="/home/mythtv/.mythtv/"$SOURCENAME'.xmltv'
# this is the file we want to store the info.
# it is not deleted in case you want to look at it
THEFILE="/home/mythtv/.mythtv/xmltvidfix-au.sql"
# create the first file...
touch $THEFILE
# then the config file
echo "<?xml version=\"1.0\"?>" > $CONFIGFILE
echo "<config>" >> $CONFIGFILE
echo "<login provider=\"tvguide\" user=\"$TUHSUSER\" password=\"$TUHSPASS\" filter=\"false\"/>" >> $CONFIGFILE
echo "</config>" >> $CONFIGFILE
# then delete (in case you run it twice)
rm $THEFILE
# and re-create. I know it's not a great script
touch $THEFILE
# go get the web info using uname + passwd. Not stored anywhere just parsed
for i in `wget -q -O - wget http://$TUHSUSER:$TUHSPASS@minnie.tuhs.org/tivo-bin/tvguide.pl -|grep stationview|sed 's/^.*id="col\(.*\)"\ class.*$/\1/'`
# strip off anything before the dash separating the channel from location
do j=`echo $i |sed 's/^\(.*\)-.*$/\1/'`
# seven is tricky. It uses 7 for over air info and tuhs uses seven for the id
# this finds and changes these so it updates our db completely
if [ "$j" = Seven ] ; then
    j="7";
fi
# Add the sql to change this to our file
echo "UPDATE channel set xmltvid='$i' WHERE name LIKE '$j%';" >> $THEFILE
# and we're all done with the xmltvid stuff. Or are we?
done
# these lines modify the double naming channel ten has created (1 and 10)
# which buggers up our intentions when we "record at any time on this
# channel' - it records averything twice because of the identical names.
echo "UPDATE channel set name='10 Digital' WHERE chanid=1;" >> $THEFILE
echo "UPDATE channel set callsign='10 Digital' WHERE chanid=1;" >> $THEFILE
# set the ABC to be commercial free, but not SBS (definitely NOT SBS!)
echo "UPDATE channel set commfree=1 WHERE callsign LIKE 'ABC%';" >> $THEFILE
# set SBSWN to link to SBS Digital 2
echo "UPDATE channel set xmltvid='SBSWN' WHERE callsign = 'SBS Digital 2';" >> $THEFILE
# re-set the radio stations (next two commands)
echo "UPDATE channel set xmltvid ='' WHERE callsign LIKE '%RADIO%';" >> $THEFILE
echo "UPDATE channel set xmltvid ='' WHERE callsign LIKE 'ABC Dig%';" >> $THEFILE
# finally, go change the actual database (run the sql)
cat $THEFILE | mysql -u root mythconverg
printf "Database updated successfully. Do you want to run mythfilldatabase now?"
read MYTHRUN;
case $MYTHRUN in
  [yY]) exec mythfilldatabase;;
esac


3. chmod a+x fixepg-au.sh
4. ./fixepg-au.sh

It doesn't matter where in Aus you are. If you registered some channels with TUHS, that's what it uses. Your database will be fixed so mythfilldatabase fills your channels correctly and your KM box doesn't waste CPU by flagging the ABC.
Please tell me if there's something here that doesn't work. You will only need to run this once.

Cheers

Mike

_________________
*********************
LinHES 7.4
Australian Dragon
*********************


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 18, 2007 9:27 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
This looks just what I need, but I have had some problems which will probably highlight my ignorance of Linux scripts.

When I got to step 4 it would not run and said something like /bin/bash not found. I checked and /bin/bash is present. I did notice in your post you have only steps 1, 3 and 4. Is there a second step not shown?

I did manage to get the script working by "bash fixepg-au.sh" however it would stop after I responded to
Code:
"Do you need to install tv_grab_au_reg (requires root password)?"

no matter what I entered. I also noticed that after each of the questions relating to TUHS registration and source, the screen did not advance a line so that responses are entered over the top of the questions.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 19, 2007 6:01 am 
Offline
Joined: Sun Aug 28, 2005 7:07 pm
Posts: 821
Location: Melbourne, Australia
nicom wrote:
This looks just what I need, but I have had some problems which will probably highlight my ignorance of Linux scripts.

When I got to step 4 it would not run and said something like /bin/bash not found. I checked and /bin/bash is present. I did notice in your post you have only steps 1, 3 and 4. Is there a second step not shown?

I did manage to get the script working by "bash fixepg-au.sh" however it would stop after I responded to
Code:
"Do you need to install tv_grab_au_reg (requires root password)?"

no matter what I entered. I also noticed that after each of the questions relating to TUHS registration and source, the screen did not advance a line so that responses are entered over the top of the questions.


Hi Nicom,

No step 2 needed. I probably just forgot to re-number. Did you chmod the script? I tested it pretty thoroughly but if you still get problems, just type "bash fixepg-au.sh" and that'll just run it while ignoring the first line.

The most important bit is that you have an account with the community epg.

Tell me if it still doesn't work. Oh, it must be run as user "mythtv".

Cheers

Mike

ps If you want to fix the weather settings as well (REALLY cryptic, since it asks you for a code you could never possibly know) type:
echo 'UPDATE settings set data="ASXX0075" where value="locale"' | mysql -u root mythconverg

_________________
*********************
LinHES 7.4
Australian Dragon
*********************


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 20, 2007 6:20 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
Mike,

Still having problems. To answer your questions:
1. Yes I have registered for a TUHS account
2. I did perform the chmod script (run as root) and received no errors.
3. I tried again to make sure I was running the fixepg script as user mythtv and got the same result.

If I just enter ./fixepg-au.sh I get
Code:
sh: ./fixepg-au.sh: /bin/bash^M: bad interpreter: No such file or directory

I found that removing the # in the first line allowed the script to work.

If I run bash fixepg-au.sh on the script with the # or .fixepg-au.sh on the script without, the script starts. When prompted for TUHS username the screen does not advance a line and I am forced to enter my username over the "What is your TUHS username" prompt. Once entered, I get
Code:
: command not found25:

The same happens for the password, source and install tv_grab_au entries but with numbers 27, 29 and 31 respectively. then
Code:
'ixepg-au.sh line32: syntax error near unexpected token 'in
'iexpg-au.sh line 32: 'case $TV in

I hope you can sort me out. I am probably doing something very wrong.

As for the weather location I had already entered the ASXX0075 location code during installation so I do get the weather, for what it is worth. The weather seems to bear little resemblance to reality and tomorrow's forecast is always for some other day of the week.

Richard


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 20, 2007 8:01 pm 
Offline
Joined: Thu Mar 30, 2006 1:02 am
Posts: 22
Location: Melbourne, Australia
Richard,

Quote:
Code:
sh: ./fixepg-au.sh: /bin/bash^M: bad interpreter: No such file or directory


The ^M is causing you difficulties. Your script must have passed through a windows editor at some stage so now the operating system is looking for a shell known as "bash^M"

My machine, R5E50, has the dos2unix utility to strip all of these dos line endings out

Code:
mythingpersons@mythtv:~$ dos2unix -h
tofrodos Ver 1.7.6 Converts text files between DOS and Unix formats.
Copyright (c) 1996-2005 by Christopher Heng. All rights reserved.
Usage: dos2unix [options] [file...]
-a      Always convert (DOS to Unix: kill all CRs;
        Unix to DOS: convert all LFs to CRLFs)
-b      Make backup of original file (.bak).
-d      Convert DOS to Unix.
-e      Abort processing files on error in any file.
-f      Force: convert even if file is not writeable.
-h      Display help on usage and quit.
-l file Log most errors and verbose messages to <file>
-o      Overwrite original file (no backup).
-p      Preserve file owner and time.
-u      Convert Unix to DOS.
-v      Verbose.
-V      Show version and quit.


Point that at your script and that should solve the first issue.
Good luck on the others!


Top
 Profile  
 
 Post subject:
PostPosted: Sat Apr 21, 2007 1:40 am 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
That's solved it even though I can't see any spurious characters in the file when I look at it with nano.

Rather than use your dos2unix utility I created another copy of fixepg-au.sh untouched by Bill Gates' hands using a version of Puppy I run on the same machine. The script runs correctly now although I have other problems. I think with all my stuffing around I've broken something else so I'll do a clean install. I'm getting pretty good at reinstalling.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Apr 21, 2007 7:57 pm 
Offline
Joined: Sun Aug 28, 2005 7:07 pm
Posts: 821
Location: Melbourne, Australia
nicom wrote:
That's solved it even though I can't see any spurious characters in the file when I look at it with nano.

Rather than use your dos2unix utility I created another copy of fixepg-au.sh untouched by Bill Gates' hands using a version of Puppy I run on the same machine. The script runs correctly now although I have other problems. I think with all my stuffing around I've broken something else so I'll do a clean install. I'm getting pretty good at reinstalling.


Hi again,

If it helps, as you get better at installing, you'll also get better at fixing so you don't need to install :-)

I'll have a look at the script again later today and see what the other problem was.

Mike

_________________
*********************
LinHES 7.4
Australian Dragon
*********************


Top
 Profile  
 
 Post subject:
PostPosted: Sun Apr 22, 2007 10:54 pm 
Offline
Joined: Sun Apr 22, 2007 10:15 pm
Posts: 2
First of all, thanks for this script. It should make the task a lot simpler.
Second of all - I can't get it to work :(

I'm sure it is something I am doing wrong but for the life of me I can't work out what it is. After running the script and the runnng mythfilldatabase I can still see the error;

Code:
unknown xmltv channel identifier: ABC-NSW
skipping channel


this happens with ABC, Seven, Nine & Ten.

It looks to be connecting to TUHS and getting my channels but not the ids.
Any ideas?


Top
 Profile  
 
 Post subject:
PostPosted: Sun Apr 22, 2007 11:17 pm 
Offline
Joined: Sun Apr 22, 2007 10:15 pm
Posts: 2
Doh! Was so stupid.
I got myself confused with a previous install I had going. I hadn't done a proper channel scan so my channel database for mythtv was empty (hence your sql update was finding nothing to update).

Plugged in the freq from this thread http://mysettopbox.tv/phpBB2/viewtopic.php?t=13573&highlight=tvgrabau
and all is well.

Fantastic script mate.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 23, 2007 4:52 am 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
Mike,

You are probably right about the benefits of reinstalling but I was getting so unsure as to what I had changed I thought I'd go back to scratch.

Anyway I am back again with a problem similar to danu5. The script works successfully but I also get the
Code:
unknown xmltv channel identifier: Seven-Mel
skipping channel
message for all channels except the ABC. I only get the ABC program listing.
I have done the same as danu5 suggested, ie scanned for channels and I have a full list of channels in Channel Editor (although the names are not in the format of TUHS, eg Seven-Mel) and still it cannot recognise the commercial channels. I changed the name of 7 to Seven-Mel but still still no luck. I checked my OxTivo account I am registered for all Melbourne channels and I have selected to receive them in xmltv format.

Any ideas?

Richard


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 24, 2007 5:09 am 
Offline
Joined: Sun Aug 28, 2005 7:07 pm
Posts: 821
Location: Melbourne, Australia
nicom wrote:
Mike,

You are probably right about the benefits of reinstalling but I was getting so unsure as to what I had changed I thought I'd go back to scratch.

Anyway I am back again with a problem similar to danu5. The script works successfully but I also get the
Code:
unknown xmltv channel identifier: Seven-Mel
skipping channel
message for all channels except the ABC. I only get the ABC program listing.
I have done the same as danu5 suggested, ie scanned for channels and I have a full list of channels in Channel Editor (although the names are not in the format of TUHS, eg Seven-Mel) and still it cannot recognise the commercial channels. I changed the name of 7 to Seven-Mel but still still no luck. I checked my OxTivo account I am registered for all Melbourne channels and I have selected to receive them in xmltv format.

Any ideas?

Richard


Hi Richard,

Here's how it works:

The scan will fill mythconverg (the database) with channel information. The scan goes through the aus frequencies (7MHz increments) and finds all your local channels. As you're in Melbourne, it should be the same as mine (Torquay gets all the same channels), being 1,2,3,7,9-15,20,21,29,30,etc. 1 and 10-15 are all Channel 10. 1 and 10 have the same name, which the script has to fix.
Now, the scan does NOT grab the xml id for each channel. I'm not sure whether this is due to a problem with the DVB standard or a short-coming of MythTV, but mythfilldatabase needs to match the xml id (in the DB it's mythconverg -> channel -> xmlid). Since these aren't there, we need to manually add them. At this stage I have only written this script to match city channels (I think - it was a very long night, and I have a very short memory!). Rather than fixing or investigating why the scan doesn't do xml ids this script simply logs you in to the community guide (which is the cheapest [free] way of getting an EPG) and gets the xmltvids they use, and drops them into the database, matching up the channels as best it can.

If you are getting unknown xmltv channel identifier, it means there aren't any matching xmltvids in your DB, so the script either hasn't worked or you ran it before you scanned for channels.

I did test this very thoroughly, and it works pretty well. It can also fail when you are running through a firewall/proxy server. If this is the case, you must edit /etc/profile as root and add the following lines just *before* the line that starts "PATH":
export http_proxy='http://proxy-server:3128'

This assumes, of course that there is no proxy authentication, you are running squid on port 3128 and your proxy server is called "proxy-server"

After saving the file you should go back to your gui and do a ctrl-alt-backspace to restart the x server. This also activates the new profile settings so that any http request (such as that in the script) will now go through the proxy.

I really want you to get this working, Richard. Please tell me how it goes.

Mike

ps if you have to authenticate for a proxy, change the http_proxy bit to 'http://username:password@proxy-server:3128'

pps if you're at your wits end, you can enter these manually. Open your browser at the tuhs page, log in and have a look at your xmlids (eg Seven-Mel). In another tab, open your myth box's ip address and log in to mythweb. Select the last option along the top, then channel settings. You can from here enter the xmltvids manually.

_________________
*********************
LinHES 7.4
Australian Dragon
*********************


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 24, 2007 5:14 am 
Offline
Joined: Sun Aug 28, 2005 7:07 pm
Posts: 821
Location: Melbourne, Australia
danu5 wrote:
Doh! Was so stupid.
I got myself confused with a previous install I had going. I hadn't done a proper channel scan so my channel database for mythtv was empty (hence your sql update was finding nothing to update).

Plugged in the freq from this thread http://mysettopbox.tv/phpBB2/viewtopic.php?t=13573&highlight=tvgrabau
and all is well.

Fantastic script mate.


Thanks. It's nice to read that.

You shouldn't have to plug in the frequencies any longer. If you have selected Australia for the frequency table, you should just be able to do a complete scan and it gets through it in about a minute or so.

I used to have to enter the frequencies, but not for a while (can't remember, but certainly didn't with R5E50). It's such a pain in the arse entering those damned frequencies every time.
KM is becoming a lot more user-friendly for Aussies.

Cheers

Mike

_________________
*********************
LinHES 7.4
Australian Dragon
*********************


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 24, 2007 8:43 pm 
Offline
Joined: Mon Aug 14, 2006 8:24 pm
Posts: 320
Location: Melbourne, Australia
Success.

Thanks for your help and detailed explanation, Mike.

I ended up inputting the xmltvid names into the Mythweb listing and it worked. My mistake was that I was entering the name into the Call Sign box, not the xmltvid box. The Mythweb format is much easier to understand than the mythtv setup system.

I don't have a proxy, just a adsl router and anyway I did manage to get the ABC listing so it must have automatically named its xmltvid.

Once again thanks for your help and thanks for opening my eyes up to the wonders of mythweb.

Richard


Top
 Profile  
 
 Post subject:
PostPosted: Mon Apr 30, 2007 10:32 pm 
Offline
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location: Brisbane, Queensland, Australia
manicmike, this is a wonderful script, whilst I haven't had the opportunity to use it yet, I will be in the very near future.

You could approach Cecil about including it in the standard build.

_________________
Girkers


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 


All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 14 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu