View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 43 posts ] 
Go to page 1, 2, 3  Next

Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Tue Apr 26, 2011 10:01 am 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
Where I am:
Backend:
KnoppMyth R5F27
P4 2.4Ghz
754 MB RD RAM (Avg use: 57.07%)
500 GB IDE hdd with approx 102GB free (approx 347 GB of recordings 136 programs)
Brooktree Corporation Bt878 card (Took time to tweak this to work both time's I've installed/upgraded)
PVR500 card (beautifully supported with no setup difficulties
Ensoniq 5880 Sound card
ATI Radion 7200 AGP video card


Frontend:
KnoppMyth R5F27
Athlon 1.2Ghz
1 GB DDR RAM
80GB (?) IDE hdd
No tuners (frontend only)
Creative Labs SB Live! EMU10k1
nVidia Corporation NV44A [GeForce 6200] (rev a1)

Where I want to go:

Backend
LinHES 6.04.00
Same system as above, but add a SATA controller and Drive
80-120 GB IDE HDD for / (nfsroot for Frontend is here as well)
1TB SATA drive for /myth
Asus FalconII (cheap pvr150 card which is now supported)
PVR500 Card
Probably upgrade to Nvidia card for ease in Frontend config.
Eventually may also add HD3000 card that I have sitting there, but without HDTV to display, kinda pointless)

Frontend
LinHES 6.04.0 (DISKLESS from Backend)
HP DC5750 SFF Desktop ( http://www.geeks.com/details.asp?invtid ... -R&cat=SYS )
Athlon 64 x2 3800+ 2.00Ghz
1GB RAM (May upgrade this since no local swap)
Diskless
NVIDIA GeForce 8400GS 256MB DDR2 PCI Express (upgrade to Nvidia Geforce 210 when HD tv is purchased)
Built in sound

I have been testing 6.04 and the above diskless frontend with another test system, and have the frontend booting diskless over the network thanks to the diskless-legacy script.

The large issue at this point is how to transition my current backend into what I want it to become.

I'm thinking I'd like to go forward as clean as possible by doing a manual install of 6.04 onto the ide/sata drive how I would like it to end up then import my recordings/database as instructed here:

http://www.mythpvr.com/mythtv/tips/migr ... dings.html

The big question is how "clean" will this be going from r5f27 to 6.04?

Someone in the comments to that post stated that this is a horrible way to do it, is that so?

Should I "upgrade" my way through to get the database updates in line with 6.04 before doing the above? (If I do go this way, actually if I go any route I plan on using Clonezilla or the like to make a working backup of my r5f27 system that I can always roll back to) If I do the steps up, what should those steps look like, I'm assuming first would be to r5.5 then 6.01 then the in-place upgrades??

The other thought is to do a manual install of r5f27 with the drive setup I want then do the database and data migration and attack the upgrades later, but I'd rather start back with a "clean" install of the current stable and import my recordings and information.

Thoughts, suggestions?

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 26, 2011 9:32 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Wow, that's a LONG step... I suspect that the DB upgrade (done by MythTV itself) would run into trouble since we've seen issues with that even going from 5.5 to various 6.0x releases.

A fresh install and importing your recordings is probably the cleanest path, but I would not follow the mysqldump based instructions given on that page. This gets you right back into the DB upgrade issues and a partial restore makes that problem even worse.

What I recently did to move my existing recording to a new box was to use nuvexport and the "Export to .nuv and .sql" option. Recent versions are smart enough to use hard links rather than copying whenever possible so it's fast and only takes enough space for the SQL files. I also did a certain amount of scripted edits on the SQL (to fix up channel ids so they matched the new system).

The R5F27 -> R6.04 jump you're contemplating will probably require some schema related fix ups to the exported SQL as well. If you can gather data, I can probably walk you through that. There may even be an existing write up on it around here somewhere.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 27, 2011 8:22 am 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
tjc,

I was hoping you would pop in.

So, would it be better for me to make a clone image (fall back plan keep wife happy factor). Then do the backup/upgrades up through into the 6.0x series?

I'm sort of an ISO hoarder, so I already have the ISO files for most of the steps. Actually have a burned copy of R5.5 and 6.02.

Then once I'm in the same range use the "Export to .nuv and .sql" option?

I'm pretty confident with getting the recording files into place with time, so my main concern is getting the database of recorded programs and recording schedules over, I plan on changing host names etc and there are some "hacks" that I won't need (ccextractor to make .srt files on recordings so my frontend could see/use closed captions) with the new setup.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Wed Apr 27, 2011 8:27 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
TheBigRed wrote:
So, would it be better for me to make a clone image (fall back plan keep wife happy factor).

Generally a good idea on both counts.

TheBigRed wrote:
Then do the backup/upgrades up through into the 6.0x series

No. Let me rephrase that, "Merciless Cthulhu, no!" ;-)

    - Clean up your system and make a system backup. (I'd use the myth.find_orphans.pl utility and the like to make sure that there were no dead DB entries or files)

    - Do a fresh install on a new disk and copy the contents of /myth across (this will include your recordings, music, videos, the backup you just made, ....

    - Unplug the old disk and keep it as your image backup.

    - Selectively extract the data (just the INSERT statements) for certain tables from the DB backup. List to be worked out shortly. There is an existing list we worked out a couple years back which would make a good starting point.

    - Edit those INSERT statements to match the new schema. Don't worry this is actually easier than it sounds if you have even a little Linux-fu.

    - Run your edited SQL to get the database to import the same channels, videos, recordings, ...


Editing the SQL may sound scary but mostly what we're going to do is change column names. For example, here is the start of the SQL statement for the channel data:
Code:
INSERT INTO `channel` (`chanid`, `channum`, `freqid`, `sourceid`,
`callsign`, `name`, `icon`, `finetune`, `videofilters`, `xmltvid`,
`recpriority`, `contrast`, `brightness`, `colour`, `hue`, `tvformat`,
`visible`, `outputfilters`, `useonairguide`, `mplexid`, `serviceid`,
`tmoffset`, `atsc_major_chan`, `atsc_minor_chan`, `last_record`,
`default_authority`, `commmethod`)

If the DB you're inserting into doesn't contain one of the columns you can use a special surrogate name (which escapes me at the moment, I'll remember or research it later) to discard that data. This can reduce or eliminate the need to troll through the all the data values.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 28, 2011 11:30 am 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
tjc wrote:
TheBigRed wrote:
Then do the backup/upgrades up through into the 6.0x series

No. Let me rephrase that, "Merciless Cthulhu, no!" ;-)


Couldn't be clearer than that.. :lol:
tjc wrote:
    - Clean up your system and make a system backup. (I'd use the myth.find_orphans.pl utility and the like to make sure that there were no dead DB entries or files)

I've actually run this recently, and the only thing it's found out of the ordinary are my .srt closed caption files which are there so I have captions on my frontend on selected recordings.
tjc wrote:

- Do a fresh install on a new disk and copy the contents of /myth across (this will include your recordings, music, videos, the backup you just made, ....

- Unplug the old disk and keep it as your image backup.


Sweet, this saves me having to find another 500 Gig ide drive to clone to before the upgrade.
tjc wrote:
- Selectively extract the data (just the INSERT statements) for certain tables from the DB backup. List to be worked out shortly. There is an existing list we worked out a couple years back which would make a good starting point.

- Edit those INSERT statements to match the new schema. Don't worry this is actually easier than it sounds if you have even a little Linux-fu.

- Run your edited SQL to get the database to import the same channels, videos, recordings, ...

Editing the SQL may sound scary but mostly what we're going to do is change column names. For example, here is the start of the SQL statement for the channel data:
Code:
INSERT INTO `channel` (`chanid`, `channum`, `freqid`, `sourceid`,
`callsign`, `name`, `icon`, `finetune`, `videofilters`, `xmltvid`,
`recpriority`, `contrast`, `brightness`, `colour`, `hue`, `tvformat`,
`visible`, `outputfilters`, `useonairguide`, `mplexid`, `serviceid`,
`tmoffset`, `atsc_major_chan`, `atsc_minor_chan`, `last_record`,
`default_authority`, `commmethod`)

If the DB you're inserting into doesn't contain one of the columns you can use a special surrogate name (which escapes me at the moment, I'll remember or research it later) to discard that data. This can reduce or eliminate the need to troll through the all the data values.


Sounds good. What information will you need? Can you point me in the direction of the existing scripts? Looking at my upcoming recording schedule the next full day I have to tackle this without interrupting a possible recording will be Sunday May 8.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 28, 2011 11:42 am 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
Double post.. sorry..

I was just thinking.. Since my /myth is going to be on a seperate drive by itself entirely in the new system, could I use rsync to get a copy now (over the network) then is rsync smart enough that when I get to the point of needing the backup "moved" it will only need to do added/changed files?

If so, this will considerably cut down on the time needed the day I do the "upgrade" to copy data over.

I just know that moving 330+Gig of data is going to take some time. Which will take even longer over the network, but I'm thinking once the initial rsync is done each subsequent one should be lesser in that it won't have as much to copy.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 28, 2011 7:09 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Yes, rsync is smart enough, as a matter of fact that's how I do my periodic image backups of the whole drive. Plug in the removable drive, make sure it's mounted then run this script:
Code:
[root@black3 ~]# cat deepbackup.sh
#!/bin/bash

nice /usr/LH/bin/idle.sh -t 60 || {
    echo "Deep backup aborted."
    exit 1
}

echo "Cleaning up any lingering junk..."
#find / -name '*.orig' -print0 | xargs -0 rm
find / -type f -name '*~' -print0 | xargs -0 rm
find /myth/tmp -type f -print0 | xargs -0 rm

echo "Making a standard system backup..."
/usr/LH/bin/mythbackup || {
    echo "Deep backup aborted."
    exit 1
}

echo "Syncing image to backup disk...."
/usr/bin/rsync -av --delete --progress /myth /media/backup
/usr/bin/rsync -avx --delete --progress / /media/backup/root

echo "Deep backup completed."
exit 0


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 28, 2011 7:15 pm 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
My system is currently running under my basement stairs which makes it a little more difficult to attach a removable drive. I'm assuming I could do the same with an nfs mounted volume, or even over ssh by adjusting the script.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 28, 2011 7:44 pm 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
Scratch that.. I'll run a usb extension cable and then use a usb to SATA adapter when I want to do a backup.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 02, 2011 8:18 am 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
Well, r5f27 didn't like the usb adapter I had in combination with xfs on the 1TB drive, so I had to go back to the network method.

That script didn't work for me didn't have the idle.sh script, so I just used command line and started the rsync manually, had to cut it short as a recording was coming up, but started it the next day and it completed. So, I'm in a position where if I run the sync daily I should be able to keep on top of the files and get them synced.


I didn't run the mythbackup scripts yet as I just wanted to get the larger files across, and I can do the backup and sync right before I take the system down to upgrade.

So, I'm thinking I should be able to focus on this this coming weekend. What information do we need to iron out to get scripts made/edited to roll up to 6.04?

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 02, 2011 10:08 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Uhm... Mostly R5F27 the table layouts... It's been a long while since I've had an R5 system available, and the old testing machine I would use to setup one up doesn't want to power on these days.

The table schema can be extracted from a DB backup or by asking the DB server to "describe" the tables of interest. I've still got to find you the list of tables... Sigh...


Top
 Profile  
 
 Post subject:
PostPosted: Tue May 03, 2011 12:41 pm 
Offline
Joined: Tue Sep 26, 2006 9:35 am
Posts: 85
tjc wrote:
Uhm... Mostly R5F27 the table layouts... It's been a long while since I've had an R5 system available, and the old testing machine I would use to setup one up doesn't want to power on these days.

The table schema can be extracted from a DB backup or by asking the DB server to "describe" the tables of interest. I've still got to find you the list of tables... Sigh...


I think I'm still running R5F27, I'll double check tonight. I can dump the tables if that's any help, LMK.

Mark


Top
 Profile  
 
 Post subject:
PostPosted: Tue May 03, 2011 4:37 pm 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
mlbuser wrote:
tjc wrote:
Uhm... Mostly R5F27 the table layouts... It's been a long while since I've had an R5 system available, and the old testing machine I would use to setup one up doesn't want to power on these days.

The table schema can be extracted from a DB backup or by asking the DB server to "describe" the tables of interest. I've still got to find you the list of tables... Sigh...


I think I'm still running R5F27, I'll double check tonight. I can dump the tables if that's any help, LMK.

Mark


I can dump mine as well. Could even set up a spare computer to drop r5f27 on and allow you ssh access if that would help.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 13, 2011 7:56 am 
Offline
Joined: Tue Apr 18, 2006 3:42 pm
Posts: 46
Location: Central PA
Anyone who can hop in here and help?

Just had a scare.. System crashed sometime last night and this morning when I brought it up I had NO recordings. Upon further investigation I found (forget how) that the recorded tables were not opened due to being marked as corrupt. Worked some myisamchk magic and I've got my recordings and schedule back, but this just makes me ready to move forward all the more.

_________________
---------
Amos B.


Top
 Profile  
 
 Post subject:
PostPosted: Fri May 13, 2011 7:14 pm 
Offline
Joined: Thu Mar 25, 2004 11:00 am
Posts: 9551
Location: Arlington, MA
Sorry, my robotic overlords have been very demanding lately. Working on bug fixes for three releases in parallel is a good way to find your self in a seemingly endless, "eat, sleep, robots" cycle. By the end of the day I'm a bit cross eyed and the patterns on the LCD tend to turn into meaningless squiggles...

Where were we again? You need a list of tables which you're going to get me the R5F27 schema for, so I can tell you how they're different from the 6.04 tables and how to fix them up?

For starters: channel, oldprogram, oldrecorded, recorded, recordedcredits, recordedfile, recordedmarkup, recordedprogram, recordedrating, recordedseek

Channel is probably the least obvious one here, but a lot of the recording info is tied to it, so it pays to keep the same channel ids if you can.

BTW - This is the kind of info we're looking for:
Code:
mysql> describe channel;
+-------------------+-----------------------+------+-----+---------+-------+
| Field             | Type                  | Null | Key | Default | Extra |
+-------------------+-----------------------+------+-----+---------+-------+
| chanid            | int(10) unsigned      | NO   | PRI | 0       |       |
| channum           | varchar(10)           | NO   | MUL |         |       |
| freqid            | varchar(10)           | YES  |     | NULL    |       |
| sourceid          | int(10) unsigned      | YES  | MUL | NULL    |       |
| callsign          | varchar(20)           | NO   |     |         |       |
| name              | varchar(64)           | NO   |     |         |       |
| icon              | varchar(255)          | NO   |     | none    |       |
| finetune          | int(11)               | YES  |     | NULL    |       |
| videofilters      | varchar(255)          | NO   |     |         |       |
| xmltvid           | varchar(64)           | NO   |     |         |       |
| recpriority       | int(10)               | NO   |     | 0       |       |
| contrast          | int(11)               | YES  |     | 32768   |       |
| brightness        | int(11)               | YES  |     | 32768   |       |
| colour            | int(11)               | YES  |     | 32768   |       |
| hue               | int(11)               | YES  |     | 32768   |       |
| tvformat          | varchar(10)           | NO   |     | Default |       |
| visible           | tinyint(1)            | NO   | MUL | 1       |       |
| outputfilters     | varchar(255)          | NO   |     |         |       |
| useonairguide     | tinyint(1)            | YES  |     | 0       |       |
| mplexid           | smallint(6)           | YES  |     | NULL    |       |
| serviceid         | mediumint(8) unsigned | YES  |     | NULL    |       |
| tmoffset          | int(11)               | NO   |     | 0       |       |
| atsc_major_chan   | int(10) unsigned      | NO   |     | 0       |       |
| atsc_minor_chan   | int(10) unsigned      | NO   |     | 0       |       |
| last_record       | datetime              | NO   |     | NULL    |       |
| default_authority | varchar(32)           | NO   |     |         |       |
| commmethod        | int(11)               | NO   |     | -1      |       |
+-------------------+-----------------------+------+-----+---------+-------+
27 rows in set (0.00 sec)


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 43 posts ] 
Go to page 1, 2, 3  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 27 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu