Author |
Message |
graysky
|
Posted: Sat May 24, 2008 6:20 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
Something happened to my R5F27 box recently. Seems as though my mythbackend is really messed-up.
Symptoms: Can't connect to backend on the local box
Mythweb doesn't function
Here is the relevent section of my /var/log/mythtv/mythbackend.log:
Code: 2008-05-21 18:10:43.961 Connecting to backend server: 192.168.0.4:6543 (try 1 of 5) 2008-05-21 18:10:44.032 Using protocol version 31 2008-05-21 18:10:44.061 MainServer::HandleAnnounce Monitor 2008-05-21 18:10:44.062 adding: mythtv as a client (events: 0) 2008-05-21 18:10:44.065 MainServer::HandleAnnounce Monitor 2008-05-21 18:10:44.069 adding: mythtv as a client (events: 1) 2008-05-21 18:10:44.074 Reschedule requested for id -1. 2008-05-21 18:10:44.114 MythSocket(81a8ac8:-1): writeStringList: Error, socket went unconnected. 2008-05-21 18:10:44.249 mythfilldatabase run complete. 2008-05-21 18:10:44.483 DataDirect: Deleting temporary files 2008-05-21 18:10:45.268 Scheduled 12 items in 1.2 = 0.54 match + 0.65 place 2008-05-21 18:10:45.615 scheduler: Scheduled items: Scheduled 12 items in 1.2 = 0.54 match + 0.65 place 2008-05-21 18:16:50.516 MainServer::HandleAnnounce Monitor 2008-05-21 18:16:50.550 adding: mythtv as a client (events: 0) 2008-05-21 18:17:05.522 MainServer::HandleAnnounce Monitor 2008-05-21 18:17:05.525 adding: mythtv as a client (events: 0) 2008-05-21 18:17:14.967 MainServer::HandleAnnounce Monitor 2008-05-21 18:17:14.969 adding: mythtv as a client (events: 0) 2008-05-21 18:17:15.036 Reschedule requested for id 1189. 2008-05-21 18:17:17.067 Scheduled 13 items in 2.0 = 1.96 match + 0.07 place 2008-05-21 18:17:17.099 scheduler: Scheduled items: Scheduled 13 items in 2.0 = 1.96 match + 0.07 place 2008-05-21 18:17:22.706 MainServer::HandleAnnounce Monitor 2008-05-21 18:17:22.709 adding: mythtv as a client (events: 0) 2008-05-21 18:17:22.788 Reschedule requested for id 1190. 2008-05-21 18:17:22.929 Scheduled 14 items in 0.1 = 0.12 match + 0.02 place 2008-05-21 18:17:22.936 scheduler: Scheduled items: Scheduled 14 items in 0.1 = 0.12 match + 0.02 place 2008-05-21 18:17:30.409 MainServer::HandleAnnounce Monitor 2008-05-21 18:17:30.412 adding: mythtv as a client (events: 0) 2008-05-21 20:02:09.783 MainServer::HandleAnnounce Monitor 2008-05-21 20:02:09.806 adding: mythtv as a client (events: 0) 2008-05-21 20:02:09.808 MainServer::HandleAnnounce Monitor 2008-05-21 20:02:09.808 adding: mythtv as a client (events: 1) QSqlQuery::exec: database not open 2008-05-21 20:02:19.301 DB Error (KickDatabase): Query was: SELECT NULL; No error type from QSqlError? Strange... QSqlQuery::prepare: database not open QSqlQuery::exec: database not open 2008-05-21 20:02:19.364 DB Error (Error in JobQueue::GetJobs(), Unable to query list of Jobs in Queue.): Query was: SELECT j.id, j.chanid, j.starttime, j.inserttime, j.type, j.cmds, j.flags, j.status, j.statustime, j.hostname, j.args, j.comment, r.endtime FROM jobqueue j, recorded r WHERE j.chanid = r.chanid AND j.starttime = r.starttime ORDER BY j.inserttime, j.chanid, j.id; No error type from QSqlError? Strange... It goes on to give lines like: Code: No error type from QSqlError? Strange... 2008-05-24 07:47:12.418 Database not open while trying to load setting: UPnP/DescXmlPath 2008-05-24 07:47:12.419 Unable to connect to database! 2008-05-24 07:47:12.420 Driver error was [1/2003]: QMYSQL3: Unable to connect Database error was: Can't connect to MySQL server on '192.168.0.4' (111)
QSqlQuery::exec: database not open QSqlQuery::exec: database not open 2008-05-24 07:47:12.474 DB Error (KickDatabase): Query was: SELECT NULL; No error type from QSqlError? Strange... 2008-05-24 07:47:12.526 Database not open while trying to load setting: UPnP/SSDP/CacheInterval 2008-05-24 07:47:12.527 Unable to connect to database! 2008-05-24 07:47:12.545 Driver error was [1/2003]: QMYSQL3: Unable to connect Database error was: Can't connect to MySQL server on '192.168.0.4' (111)
The IP addy of this machine is indeed 192.168.0.4. Here is a screenshot of mythtv-setup's config page which I believe to be the right configuration since I'm running other boxes as frontends to need to connect to this backend:  I followed cesman's repairing mysql tables instructions but found that I had no errors. I also forced an fsck but no errors were detected after the reboot. Code: $ su - # cd / # touch /forcefsck # reboot After searching a bit, it seems that this "No error type from QSqlError? Strange..." could be due to the old skip-networking line in /etc/mysql/my.cnf but that has been replaced as you know by the bind-address line which is required if one is using other frontend machines which I am. Therefore my /etc/mysql/my.cnf does contain the following line: Code: bind-address = 192.168.0.4
Just for testing purposes, I did comment it out and restarted mysql but the same problem exists. I'm currently unable to connect to my own backend machine.
Suggestions are welcomed 
_________________ Retired KM user (R4 - R6.04); friend to LH users.
Last edited by graysky on Sat Jun 07, 2008 2:47 pm, edited 2 times in total.
|
|
Top |
|
 |
ceenvee703
|
Posted: Sat May 24, 2008 6:45 am |
|
Joined: Fri Apr 02, 2004 10:08 am
Posts: 1637
Location:
Virginia, USA
|
A couple of questions:
* is it only the remote frontends that can't connect? In other words, does running the frontend on the backend machine work?
* are you confident that ALL your remote frontends are running the same version of MythTV? It should warn you if your versions are out of sync, but a newer version can also update an older version and get the database all screwed up.
|
|
Top |
|
 |
graysky
|
Posted: Sat May 24, 2008 7:00 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
ceenvee703 wrote: A couple of questions:
* is it only the remote frontends that can't connect? In other words, does running the frontend on the backend machine work?
* are you confident that ALL your remote frontends are running the same version of MythTV? It should warn you if your versions are out of sync, but a newer version can also update an older version and get the database all screwed up.
No, I can't get the local machine to connect via it's frontend nor can I connect to it via mythweb. I haven't even fired out one of the remote frontend machines at all since the local frontend can't connect. So the answer to you question is: the frontend on the beckend machine doesn't connect
To answer your second bullet point: I am 100 % sure that ALL my remote frontends are running the same version. I've had them setup like this for years without problems. Again, I haven't tried one of the remote frontends since the box's frontend cannot connect to its own backend.
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
slowtolearn
|
Posted: Sat May 24, 2008 7:03 am |
|
Joined: Wed Nov 16, 2005 8:55 pm
Posts: 1381
Location:
Farmington, MI USA
|
Is MySQL actually running? See this for troubleshooting ideas.
ceenvee703 wrote: * are you confident that ALL your remote frontends are running the same version of MythTV? Yeah, been quite the rash of these lately. Any Ubuntu-based frontends in your mix?
|
|
Top |
|
 |
graysky
|
Posted: Sat May 24, 2008 7:06 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
slowtolearn wrote: Is MySQL actually running? See this for troubleshooting ideas. ceenvee703 wrote: * are you confident that ALL your remote frontends are running the same version of MythTV? Yeah, been quite the rash of these lately. Any Ubuntu-based frontends in your mix?
no Ubuntu-based frontends... all machines are KM R5F27. But, again, the backend machine sits in my living room and we also use it to watch programs via the frontend that has been able to connect to the backend (both on the same box).
Yeah, it's running...:
Code: $ netstat -ln | grep mysql unix 2 [ ACC ] STREAM LISTENING 9050 /var/run/mysqld/mysqld.sock
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
graysky
|
Posted: Sat May 24, 2008 7:34 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
If I open an xterm and try to run mythfronend while logging the output, I get the following. By the way, it take about 60-90 seconds before I see the mythfronend GUI. In the past, it runs very quickly (3-4 sec):
Code: $ mythfrontend > wtf.log 2008-05-24 09:31:02.992 Using runtime prefix = /usr 2008-05-24 09:31:03.010 XScreenSaver support enabled 2008-05-24 09:31:03.010 DPMS is disabled. 2008-05-24 09:31:03.057 New DB connection, total: 1 2008-05-24 09:32:27.955 Connected to database 'mythconverg' at host: 192.168.0.4 2008-05-24 09:32:27.959 Total desktop dim: 800x600, with 1 screen[s]. 2008-05-24 09:32:27.964 Running in a window 2008-05-24 09:32:27.965 Using screen 0, 800x600 at 0,0 2008-05-24 09:32:27.993 Current Schema Version: 1160 2008-05-24 09:32:27.996 mythfrontend version: 0.20.20070821-1 www.mythtv.org 2008-05-24 09:32:27.997 Enabled verbose msgs: important general 2008-05-24 09:32:28.750 Total desktop dim: 800x600, with 1 screen[s]. 2008-05-24 09:32:28.752 Running in a window 2008-05-24 09:32:28.752 Using screen 0, 800x600 at 0,0 2008-05-24 09:32:28.754 Switching to square mode (blootube) 2008-05-24 09:32:28.778 Using the Qt painter 2008-05-24 09:32:28.918 New DB connection, total: 2 2008-05-24 09:32:29.043 Loading from: /usr/share/mythtv/themes/blootube/base.xml 2008-05-24 09:32:29.079 Loading from: /usr/share/mythtv/themes/default/base.xml 2008-05-24 09:32:29.177 Registering Internal as a media playback plugin. 2008-05-24 09:32:29.273 Registering MythDVD DVD Media Handler as a media handler ext() 2008-05-24 09:32:29.275 Registering MythDVD VCD Media Handler as a media handler ext() 2008-05-24 09:32:29.336 Registering MythGallery Media Handler 1/2 as a media handler ext() 2008-05-24 09:32:29.338 Registering MythGallery Media Handler 2/2 as a media handler ext(gif,jpg,png) 2008-05-24 09:32:32.558 Registering MythMusic Media Handler 1/2 as a media handler ext() 2008-05-24 09:32:32.559 Registering MythMusic Media Handler 2/2 as a media handler ext(ogg,mp3,aac,flac) SIP listening on IP Address 192.168.0.4:5060 NAT address 192.168.0.4 SIP: Cannot register; proxy, username or password not set 2008-05-24 09:32:45.969 XMLParse::LoadTheme using /usr/share/mythtv/themes/blootube/ui.xml 2008-05-24 09:32:46.136 Connecting to backend server: 192.168.0.4:6543 (try 1 of 5) 2008-05-24 09:32:46.136 Connection timed out. You probably should modify the Master Server settings in the setup program and set the proper IP address. 2008-05-24 09:32:49.335 Connecting to backend server: 192.168.0.4:6543 (try 1 of 5) 2008-05-24 09:32:49.336 Connection timed out. You probably should modify the Master Server settings in the setup program and set the proper IP address. Destroying SipFsm object
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
slowtolearn
|
Posted: Sat May 24, 2008 10:43 am |
|
Joined: Wed Nov 16, 2005 8:55 pm
Posts: 1381
Location:
Farmington, MI USA
|
Do /var/log/mysql.log or /var/log/mysql.err provide any clues? And what processes do you see running with ps -ef | grep -i sql?
|
|
Top |
|
 |
warchildx
|
Posted: Sat May 24, 2008 11:08 am |
|
Joined: Sat Mar 11, 2006 4:01 pm
Posts: 61
Location:
Tobaccoville, NC, USA
|
graysky wrote: Yeah, it's running...: Code: $ netstat -ln | grep mysql unix 2 [ ACC ] STREAM LISTENING 9050 /var/run/mysqld/mysqld.sock Please post a copy of the /etc/my.cnf could be that mysql is only listening on the unix socket (/var/run/mysqld/mysqld.sock) but not on tcpip port 3306. you should have an entry under "mysqld" section that says: Code: port = 3306
Please advise. This would make it so you couldnt connect even locally via myth (since it assumes ip based connectivity instead of local socket) yet mysql would be running.
_________________ "If the facts don't fit the theory, change the facts"
--Albert Einstein
|
|
Top |
|
 |
graysky
|
Posted: Sat May 24, 2008 11:17 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
Well, it just started working without my intervention. I left the box up and did some yard work. When I came back everything was as it should be. I just power cycled the box and the frontend came up all by itself; as well, I can connect to mythweb again.
Too spooky!
Thanks to all who replied.
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
graysky
|
Posted: Sat Jun 07, 2008 2:49 pm |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
Odd, it did this behavior AGAIN today (no reason that I can tell). After leaving it to sit for about 45 min, it was fine. Again, I forced an fsck and also looked for corruption of my mysql tables. Both tests were negative.
I will gladly post any log file. I'd like to figure out what's going on 
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
cliffsjunk
|
Posted: Sat Jun 07, 2008 7:02 pm |
|
Joined: Fri Sep 15, 2006 12:16 pm
Posts: 292
|
Heat related flakeyness? Did it finally start running in the evening after
failing to run during the day? Blow all the dust out of it and see if the
problem goes away.
Out of disk space on root or on myth?
Cliff
|
|
Top |
|
 |
graysky
|
Posted: Mon Jun 09, 2008 1:30 pm |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
cliffsjunk wrote: Heat related flakeyness? Did it finally start running in the evening after failing to run during the day? Blow all the dust out of it and see if the problem goes away.
Out of disk space on root or on myth?
Cliff
Not heat related after checking my system monitors. It finally started working after 45 min or so and a reboot. I also have plenty of disk space.
??
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
graysky
|
Posted: Mon Jun 09, 2008 2:56 pm |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
Could this problem be a symptom of some sort of recovery feature in mysql or something else? We just lost power and I switched the mythtv box on and it's doing it again!
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
 |
ChapmanI
|
Posted: Tue Jun 10, 2008 7:04 am |
|
Joined: Mon May 24, 2004 10:49 pm
Posts: 112
Location:
Calgary, Alberta, Canada
|
I could be completely off base here, but could it be a network problem? How is the combined FE/BE connected to the network (wired or WiFi).
I noticed this in the snippet you posted.
Quote: 2008-05-24 09:32:46.136 Connecting to backend server: 192.168.0.4:6543 (try 1 of 5) That implies to me that your local front end is trying to go out to the network, and then back into the machine via its IP address. If there was a network adaptor, cabling, or router problem, it would make perfect sense that the FE couldn't connect to the BE via that IP routing. The flakiness of it worked, then it didn't, then it did again ... If one of the above components was failing, it could work intermittently. If a network cable wasn't quite seated properly, minor temperature difference, subtle vibration, or a pet walking behind the Myth box (or router) could cause that. A bit of corrosion on the network cable connectors would do the same. A WiFi connection could be subject to some new bit of interference, such as a neighbour with a new router on the same channel as yours. Is it possible that your router is assigning a new/different IP address to your Knoppmyth box? I don't know how Myth responds to a dynamic IP address. Is the backend fixed to the one set during "mythtv-setup," or under newer releases does that get re-assigned to the IP address of its machine when re-booted? I've always set all my machines address in the router via their MAC address. I've seen lots of cases where things stop working when the router is left to decide IP addresses on its own. Machines booted in a different order may get different IP addresses that they used to have. That's caused network printers to not print, bittorrents to stop working, webcam chats to no longer get routed to the right machine ... I know a preparer who couldn't access tax returns on his own machine when a network card went bad, because the configuration pointed to the returns as if they were across the network. I thought the connection via the machine's IP address looked odd. I know that in UTILITIES/SETUP | SETUP | GENERAL | DATABASE SERVER SETTINGS, it references "localhost". I always thought that "localhost" referred to the reserved IP address of 127.0.0.1! I notice my R5.5 release candidate version is setup that way (via the machine's actual IP address, not the standard 127.0.0.1). I just check an older, smaller, archived hard drive (read that as "sitting on a shelf in the corner of the room"), that had R5F1 on it. It does use the 127 address for the Master Backend, as I thought I had remembered. I don't have another drive with F27 on it, but I might do a temp install if it just to see what it uses. Or perhaps someone else with an F27 install can check the MBE IP address in myth-setup. Quote: Edited to add: Is there a reason that was changed between versions? Is it simply a configuration error that got mastered on the Knoppmyth CD? Or is it something that rolled down from the main Myth development team to solve some other specific problem? Perhaps one of the dev team, or testers could cast light on that.
Like I said, I could just be making gaseous emissions through the wrong orifice. I'm knowledgeable enough to understand the concepts, but not Linux skilled enough to make valuable contributions to the project. But the problem you described seems to fit with some of what I've postulated above.
|
|
Top |
|
 |