LinHES Forums
http://forum.linhes.org/

Whole system freezing after upgrade
http://forum.linhes.org/viewtopic.php?f=21&t=24569
Page 1 of 1

Author:  Big boy stan [ Thu Oct 13, 2016 12:18 pm ]
Post subject:  Whole system freezing after upgrade

I have been running the same BE/FE for a few years now and it has been rock solid. Within a day of upgrading from 8.3 to 8.4, I have started to have the whole system lockup. Remote wont respond, no VNC, no mythweb, no linhes webpage. Looking at the box, I see the number lock and scroll lock on the keyboard both flashing. The only option is a hard reboot.

The most recent lockup was lastnight at 1:07 (I can tell because the clock on the FE is fronzen). Looked at the XYMON on the linhes page does not show any high temps or full memory type stuff. Looking at the logs, I dont see anything unusual happening at that time. The only unique thing I see is about 1 minute before the lockup, the system did a smartctl on both drives. Not sure if that is relevant or not.

Considering this started within a day of the upgrade, I am thinking software. Maybe driver related. My video card is a NVIDIA GT218 GEForce 210. I have a firewire card that is no longer used that I could remove. Any thoughts?

Author:  brfransen [ Thu Oct 13, 2016 1:16 pm ]
Post subject:  Re: Whole system freezing after upgrade

I have 3 boxes with 210s and they have all been solid on R8.4 for months.

Combined with your other thread about losing a table I would suspect a possible drive issue. I had a drive go bad about a year ago. It never showed smart errors but the whole box would just lock up like you describe.

Author:  Big boy stan [ Fri Oct 14, 2016 8:31 am ]
Post subject:  Re: Whole system freezing after upgrade

Another lockup last night at 12:07AM. Still nothing to be learned from the logs. These intermittent issues are the worst. Ugh!

I agree that it smells like a hardware problem but the fact that it has been rock solid for so long and then started freezing up the same day as the upgrade make me really think that is too much of a coincidence. I am also hoping that is the case as I don't really have any funds for a new drive right now.

Looks like I am using the NVIDIA driver 340.98 which according to their website is the correct one for my card. Would you mind checking to see if that is the same driver you are using on your 3 boxes?

I think the first thing for me to try is a ISO install. I assume downgrading to 8.3 is difficult, is that true? All of the "Restore Database" files are from after the upgrade. If so, I will try a ISO 8.4 with the latest full backup and see if it makes any difference. If not, maybe a partial backup???

Thanks for any help.

Author:  brfransen [ Fri Oct 14, 2016 9:50 am ]
Post subject:  Re: Whole system freezing after upgrade

Yes I am using 340.98 on all of them. I think the R8.4 ISO has 340.96 which was the same version that was in R8.3 but rebuilt for the newer kernel in R8.4. The changelog between 340.96 and 340.98 is very short but that could be a possible issue.

Going back to R8.3 is going to be very difficult because of the mythtv .27 to .28 change unless you have a backup from R8.3 era, but then you would lose everything that has been done since that update.

Author:  Big boy stan [ Fri Oct 14, 2016 1:30 pm ]
Post subject:  Re: Whole system freezing after upgrade

So my Plan "A" will be to do a ISO install with 8.4

Any advice of if I should try a full restore of the most recent backup or a partial?

Author:  brfransen [ Fri Oct 14, 2016 1:37 pm ]
Post subject:  Re: Whole system freezing after upgrade

Do an Upgrade not an install. Upgrade will only format the first partition of the drive and reinstall, automatically restoring many of the previous settings. Upgrade doesn't touch the db, home or data partitions.

Author:  Big boy stan [ Wed Oct 19, 2016 3:05 pm ]
Post subject:  Re: Whole system freezing after upgrade

Still having lockups. Yesterday morning I found this in the frontend log at the time of the lockup. At the time, we had just started watching a show. Any idea what would cause this?

Code:
2016-10-18T05:21:32.017067-04:00 mythfrontend[29735]: E Decoder mythplayer.cpp:3483 (DecoderGetFrame) Player(0): Decoder timed out waiting for free video buffers.

Author:  thekingofspain [ Sun Oct 23, 2016 2:06 pm ]
Post subject:  Re: Whole system freezing after upgrade

Big boy stan wrote:
Still having lockups. Yesterday morning I found this in the frontend log at the time of the lockup. At the time, we had just started watching a show. Any idea what would cause this?

Code:
2016-10-18T05:21:32.017067-04:00 mythfrontend[29735]: E Decoder mythplayer.cpp:3483 (DecoderGetFrame) Player(0): Decoder timed out waiting for free video buffers.


Just upgraded last night and had a freeze as well. sv restart frontend worked for me (1 for 1). Still jumped to 100% CPU for about 35 seconds but recovered. funcd was zombied at the time of the freeze.

There are like 3 factors going on the freezes as far as I can tell that started about a year ago:
Nvidia Drivers
XOrg
Session Switching

Nvidia and XOrg seem to focused on newer features and breaking old stable features.
Nvidia has some some type of terminal feature that breaks other terminal software.
LinHES was or is using n (name is not coming to me) terminal switching software that is buggy when shutting down /restarting.

The above error is really a MythTv issue but the above service could be contributing to it. MythTv has always been really bad at handling any IO lag. I have a raid 1 for my media and have had to tweak lots of things to minimize random lag and I still get the error as you did above error. Watching video stats with the highest setting, my buffer if 99% filled 99% of the time, even when recording 3 shows with commercial detection. I still get freezes watching a single show with nothing else processing. Start experimenting the disk IO schedulers. Then move to tweaking swap such that swap is either not/used only in emergencies or used exclusively and fully populated all time. The on/off context switching of a given service is more than likely causing the stress rather that the service its self.

Page 1 of 1 All times are UTC - 6 hours
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/