| Author |
Message |
|
Girkers
|
 Posted: Wed Jun 07, 2006 10:42 pm |
|
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
The screen scraper version of this has been broken again and I don't have the knowledge nor the patience to fix it and it wasn't from lack of trying.
I have however found a replacement that is much easier to use as you don't have to configure the script by editing it. It can be found at:
http://immir.com/tv_grab_au
I discovered this last night and it has been discussed on the gossamer threads and is being regularly maintained.
I downloaded it, plugged it in and updated my xmltvids but alas I can not get it to work no matter what I try. I have emailed the author of the script to see if he can help me and I will let you know how it all goes.
_________________
Girkers
|
|
| Top |
|
 |
|
jbman
|
Posted: Thu Jun 08, 2006 4:25 am |
|
|
|
|
I have been using this version for a long time now, since the old one went under about a year ago.
What problems you having with it?
|
|
| Top |
|
|
|
Greg Frost
|
Posted: Thu Jun 08, 2006 5:01 am |
|
|
Joined: Mon May 10, 2004 8:08 pm
Posts: 1891
Location:
Adelaide, Australia
|
|
I am also using it. The only problem I had was that the backend runs without the $HOME environment variable which meant that when it was run automatically by the backend it failed, but when I ran it manually at the command line it worked. I fixed it by setting the HOME environment variable in /etc/init.d/mythtv-backend
|
|
| Top |
|
|
|
Girkers
|
Posted: Thu Jun 08, 2006 5:41 pm |
|
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
Here is what I get when I run it from the command line:
Code: mythtv@mythtv:~$ mythfilldatabase
2006-06-08 13:26:46.852 Using runtime prefix = /usr
2006-06-08 13:26:46.867 New DB connection, total: 1
2006-06-08 13:26:46.869 Connected to database 'mythconverg' at host: localhost
2006-06-08 13:26:46.876 mythfilldatabase: Listings Download Started
2006-06-08 13:26:46.878 New DB connection, total: 2
2006-06-08 13:26:46.879 Connected to database 'mythconverg' at host: localhost
2006-06-08 13:26:46.880 Updating source #1 (fta) with grabber tv_grab_au
2006-06-08 13:26:46.880 ----------------- Start of XMLTV output ---------------- -
2006-06-08 13:26:46.881 New DB connection, total: 3
2006-06-08 13:26:46.882 Connected to database 'mythconverg' at host: localhost
tv_grab_au 1.53: grabbing 7 days into /tmp/myth84fLT9
tv_grab_au: downloads = 0 guide pages, 0 detail pages
0 failed detail pages, 0 stop times adjusted
tv_grab_au: finished in 0 seconds
2006-06-08 13:26:47.373 ------------------ End of XMLTV output ----------------- -
2006-06-08 13:26:47.376 New DB connection, total: 4
2006-06-08 13:26:47.376 Connected to database 'mythconverg' at host: localhost
2006-06-08 13:26:47.378 Updating icons for sourceid: 1
Updated programs: 0 Unchanged programs: 0
2006-06-08 13:26:47.413 New DB connection, total: 5
2006-06-08 13:26:47.414 Connected to database 'mythconverg' at host: localhost
2006-06-08 13:26:47.416 Data fetching complete.
2006-06-08 13:26:47.416 Adjusting program database end times.
2006-06-08 13:26:47.417 0 replacements made
2006-06-08 13:26:47.420 mythfilldatabase: Listings Download Finished
2006-06-08 13:26:47.420 Marking generic episodes.
2006-06-08 13:26:47.448 Found 0
2006-06-08 13:26:47.448 Marking repeats.
2006-06-08 13:26:47.477 Found 0
2006-06-08 13:26:47.477 Unmarking new episode rebroadcast repeats.
2006-06-08 13:26:47.505 Found 0
2006-06-08 13:26:47.506
===============================================================
| Attempting to contact the master backend for rescheduling. |
| If the master is not running, rescheduling will happen when |
| the master backend is restarted. |
===============================================================
2006-06-08 13:26:47.510 Connecting to backend server: 192.168.0.100:6543 (try 1 of 5)
2006-06-08 13:26:47.518 Using protocol version 26
2006-06-08 13:26:47.570 mythfilldatabase run complete.
I firstly thought that it might be that the script does not have permissions to run as mythtv user, changed that script owner and group, same thing. Changed the permissions on the /tmp directory no good. Try running mythfilldatabase as root and it says that it is not configured, run the script with --configure option and it states that a configuration file already exists, so I overwrite that one and then try mythfilldatabase again, same result.
_________________
Girkers
|
|
| Top |
|
|
|
jbman
|
Posted: Thu Jun 08, 2006 6:25 pm |
|
|
|
|
do you make a directory for it to work in?
From memory I had to create a dir as well for it to work
/var/local/tv_grab_au i think it was
cheers
|
|
| Top |
|
|
|
Girkers
|
Posted: Thu Jun 08, 2006 11:34 pm |
|
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
This directory exists and there is a guide.xml file in it with what appears to be guide data.
I have since tried running mythfilldatabase again and now I get this
Code: mythtv@mythtv:/$ mythfilldatabase
2006-06-09 15:33:43.126 Using runtime prefix = /usr
2006-06-09 15:33:43.147 New DB connection, total: 1
2006-06-09 15:33:43.149 Connected to database 'mythconverg' at host: localhost
2006-06-09 15:33:43.158 mythfilldatabase: Listings Download Started
2006-06-09 15:33:43.159 New DB connection, total: 2
2006-06-09 15:33:43.160 Connected to database 'mythconverg' at host: localhost
2006-06-09 15:33:43.161 Updating source #1 (fta) with grabber tv_grab_au
2006-06-09 15:33:43.161 ----------------- Start of XMLTV output -----------------
2006-06-09 15:33:43.162 New DB connection, total: 3
2006-06-09 15:33:43.163 Connected to database 'mythconverg' at host: localhost
This version of tv_grab_au is not configured!
If you are using mythfilldatabase, you'll need to configure from
within mythtv-setup (or with an appropriate --config-file option),
otherwise simply run with the --configure option.
2006-06-09 15:33:43.611 ------------------ End of XMLTV output ------------------
Error in 1:1: unexpected end of file
2006-06-09 15:33:43.612 Updating icons for sourceid: 1
2006-06-09 15:33:43.613 New DB connection, total: 4
2006-06-09 15:33:43.614 Connected to database 'mythconverg' at host: localhost
Updated programs: 0 Unchanged programs: 0
2006-06-09 15:33:43.649 New DB connection, total: 5
2006-06-09 15:33:43.650 Connected to database 'mythconverg' at host: localhost
2006-06-09 15:33:43.652 Failed to fetch some program info
2006-06-09 15:33:43.655 mythfilldatabase: Failed to fetch some program info
What to do, what to do?
_________________
Girkers
|
|
| Top |
|
|
|
jbman
|
Posted: Fri Jun 09, 2006 2:02 am |
|
|
|
|
in my /var/local/tv_grab_au is the file cached.pl
which looks like it has the guide data in it.
did you run tv_grab_au --configure seperatly?
|
|
| Top |
|
|
|
Greg Frost
|
Posted: Fri Jun 09, 2006 2:53 am |
|
|
Joined: Mon May 10, 2004 8:08 pm
Posts: 1891
Location:
Adelaide, Australia
|
Also, do your xmltvids match perfectly. I set mine using a script as follows (although the Brisbane ids would be different of course).
Code: echo "
UPDATE channel SET xmltvid='abcsa.free.au' WHERE callsign='ABC TV Adelaide';
UPDATE channel SET xmltvid='abc2.abc.gov.au' WHERE callsign='ABC2';
UPDATE channel SET xmltvid='channelsevenadelaide.free.au' WHERE callsign='7 Digital';
UPDATE channel SET xmltvid='channelnineadelaide.free.au' WHERE callsign='NINE Digital';
UPDATE channel SET xmltvid='networktenadelaide.free.au' WHERE callsign='Ten Digital' AND channum='10';
UPDATE channel SET xmltvid='sbssa.free.au' WHERE callsign='SBS DIGITAL 1';
UPDATE channel SET xmltvid='news.sbs.com.au' WHERE callsign='SBS DIGITAL 2';
" | mysql -h 10.1.1.6 -u mythtv -pmythtv mythconverg
|
|
| Top |
|
|
|
Girkers
|
Posted: Sat Jun 10, 2006 1:41 am |
|
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
I have done everything everyone has suggested but no go. I reinstalled my system this morning and it still doesn't work. When I run mythfilldatabase by itself I get this:
Code: mythtv@mythtv:/tmp$ mythfilldatabase
2006-06-10 17:31:09.924 Using runtime prefix = /usr
2006-06-10 17:31:09.939 New DB connection, total: 1
2006-06-10 17:31:09.941 Connected to database 'mythconverg' at host: localhost
2006-06-10 17:31:09.948 mythfilldatabase: Listings Download Started
2006-06-10 17:31:09.949 New DB connection, total: 2
2006-06-10 17:31:09.950 Connected to database 'mythconverg' at host: localhost
2006-06-10 17:31:09.951 Updating source #1 (fta) with grabber tv_grab_au
2006-06-10 17:31:09.952 ----------------- Start of XMLTV output -----------------
2006-06-10 17:31:09.952 New DB connection, total: 3
2006-06-10 17:31:09.953 Connected to database 'mythconverg' at host: localhost
tv_grab_au 1.53: grabbing 7 days into /tmp/mythvUSrmh
tv_grab_au: downloads = 0 guide pages, 0 detail pages
0 failed detail pages, 0 stop times adjusted
tv_grab_au: finished in 0 seconds
2006-06-10 17:31:10.353 ------------------ End of XMLTV output ------------------
2006-06-10 17:31:10.356 New DB connection, total: 4
2006-06-10 17:31:10.356 Connected to database 'mythconverg' at host: localhost
2006-06-10 17:31:10.358 Updating icons for sourceid: 1
Updated programs: 0 Unchanged programs: 0
2006-06-10 17:31:10.366 New DB connection, total: 5
2006-06-10 17:31:10.367 Connected to database 'mythconverg' at host: localhost
2006-06-10 17:31:10.369 Data fetching complete.
2006-06-10 17:31:10.369 Adjusting program database end times.
2006-06-10 17:31:10.370 0 replacements made
2006-06-10 17:31:10.371 mythfilldatabase: Listings Download Finished
2006-06-10 17:31:10.372 Marking generic episodes.
2006-06-10 17:31:10.372 Found 0
2006-06-10 17:31:10.372 Marking repeats.
2006-06-10 17:31:10.374 Found 0
2006-06-10 17:31:10.374 Unmarking new episode rebroadcast repeats.
2006-06-10 17:31:10.375 Found 0
2006-06-10 17:31:10.376
===============================================================
| Attempting to contact the master backend for rescheduling. |
| If the master is not running, rescheduling will happen when |
| the master backend is restarted. |
===============================================================
2006-06-10 17:31:10.381 Connecting to backend server: 192.168.0.100:6543 (try 1 of 5)
2006-06-10 17:31:10.388 Using protocol version 26
2006-06-10 17:31:10.421 mythfilldatabase run complete.
I have check my environment variables and HOME is set to /home/mythtv
When I run the tv_grab_au manually it says it is grabbing to /tmp/[random] and when I look in this directory I find a file called xxx which contains the scrappings.
I am very frustrated, I just don't know what to do 
_________________
Girkers
|
|
| Top |
|
|
|
jbman
|
Posted: Sat Jun 10, 2006 3:35 am |
|
|
|
|
who is the owner of tv_grab_au i think i had to change mine otherwise it would not insert. run it as mythtv
|
|
| Top |
|
|
|
Greg Frost
|
Posted: Sat Jun 10, 2006 5:42 pm |
|
|
Joined: Mon May 10, 2004 8:08 pm
Posts: 1891
Location:
Adelaide, Australia
|
I just had a closer look at my post install tweaking script, and I do this:
Code: mkdir /var/local/tv_grab_au
chown mythtv:mythtv /var/local/tv_grab_au
I have a vague recollection that this directory needed to exist for things to work. Not sure how I worked it out now (it was some time ago), but I think I had exactly the same issues you had. I don;t know why, but this is the line in your post that twigged my memory Girkers wrote: I am very frustrated, I just don't know what to do
|
|
| Top |
|
|
|
silentmic
|
Posted: Sun Jun 11, 2006 7:34 am |
|
|
Joined: Fri Feb 03, 2006 11:36 pm
Posts: 63
Location:
Melbourne
|
There is also this grabber at http://www.cse.unsw.edu.au/~willu/xmltv/tv_grab_au_reg.html which gets the oztivo data. It's alot better than the standard tv_grab_au because it has descriptions and community tv channels like channel 31 in Melbourne. I couldn't get the ninemsn one to work and was worried that ninemsn would make changes to the format of the data at times to break the grabber.
|
|
| Top |
|
|
|
Girkers
|
Posted: Thu Jun 15, 2006 4:25 pm |
|
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
|
Greg,
I had thought of that and tried it, but no luck. So just to make sure I removed the directory and followed your instructions no luck.
It's funny though, cause when I run it through mythfilldatabase, it says it is going to d/l to /tmp/[insertrandomtext], but when I run the tv_grab_au script solo it d/ls to the correct location.
I am still trying to work this out, any help or pointers to work it out would be appreciated.
_________________
Girkers
|
|
| Top |
|
|
|
Girkers
|
Posted: Thu Jun 15, 2006 5:10 pm |
|
|
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location:
Brisbane, Queensland, Australia
|
I would firstly like to thank everyone for trying to help me through this and although I haven't fixed the exact problem I have found a way around it.
Code: #!/bin/sh
#This script will run the grabber manually and get the data for tv
#Then run mythfilldatabase to insert that data into mythtv
tv_grab_au --days 7
#First option: SourceID
#Second option: What to update -1 = All
#Third option: xml data file
mythfilldatabase --file 1 -1 /var/local/tv_grab_au/guide.xml
I created this script which downloads my data using the grabber and then I ran mythfilldatabase using the file the previous command created to insert the data into Myth. I have run it manually as mythtv user and it works great, I now just have to set it to run automatically.
And thanks again everyone for your help.
_________________
Girkers
|
|
| Top |
|
|
|
Ramrunner
|
Posted: Sat Jun 17, 2006 1:06 am |
|
|
Joined: Thu Mar 30, 2006 8:19 pm
Posts: 39
|
|
Guys, is it just me or are the detailed listings broken?
I used to have coloured EPG data with full titles and descriptions. Now, as it is fetching I get a lot of warnings about parsing errors - incomplete data.
I still get all the program listings, but instead of "FIFA World Cup 2006 - Australia vs Brazil", I just get "FIFA World Cup 2006" now, with no further info available. I used to get a huge spiel about everything when I moved my mouse over the program.
Not a HUGE deal, but now I have to have my paper TV guide handy to make sure I record the right games.
I this being worked on? Should it be? Or are all you guys NOT running with the --slow option?
Ramrunner.
|
|
| Top |
|
|
View unanswered posts
View active topics
Print view
Previous topic Next topic 
LinHES Home
Board index
FAQ
Search
Login 
Register
