View unanswered posts    View active topics

All times are UTC - 6 hours





Post new topic Reply to topic  [ 22 posts ] 
Go to page 1, 2  Next

Print view Previous topic   Next topic  
Author Message
Search for:
PostPosted: Sun Oct 15, 2006 10:58 pm 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
I just bought a new harddrive just to store recorded videos on and I need to know what filesystem to format it to.

I have a PVR-500 and two shows being recorded at the same time is normal. After a year and a half, I think the drive is pretty darn fragmented because it seems that the hard drive often has a hard time keeping up for playback when stuff is being deleted/recorded.

I was thinking about using the maximum block (cluster/allocation unit/whatever) size possible to reduce the amount of fragmentation that can occur. But I don't know what type of filesystem would be fastest for only holding large files, or how to set the block size.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 8:44 am 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
Bumpy.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 10:34 am 
Offline
Joined: Mon Jun 21, 2004 11:25 am
Posts: 291
Location: Ontario, CA
I used to make my video filesystems FAT or FAT32 so they could be moved to external cases and easily have large amounts of files dumped to them from other win or mac machines. A lot of my video rips are over 2 gigs each since I like AC3 soundtracks and high quality video. Now I use large NAS arrays with 2+TB of storage and network mount the videos over NFS or SAMBA.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 2:20 pm 
Offline
Joined: Tue Aug 08, 2006 7:08 pm
Posts: 561
Location: UK
You could use a similar line as used in a manual install, see the Phamplet of Knoppmyth .
Code:
# mke2fs -j -O sparse_super -m0 -i8000000 -L tv -M /myth/tv /dev/hdX1

Where hdX1 is whatever your drive is.

_________________
Updated 2019/10/26: AthlonII X2 265 Gigabyte GA-970A-DS3P
16Gb PC 1866 DDR3, 500GB+2TB+4TB SATA HDD,
SATA DVD-RW Asus DRW-24D5MT , NVIDIA GeForce GT1080
Hauppauage Nova-T 500, Nova-T LinHes R8.6.1


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 4:56 pm 
Offline
Joined: Tue Mar 22, 2005 9:18 pm
Posts: 1422
Location: Brisbane, Queensland, Australia
There was some discussion some time ago in regards to which filesystem to use, if you do a search on Filesystem I believe you should come across it.

_________________
Girkers


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 5:25 pm 
Offline
Joined: Thu Dec 01, 2005 4:14 pm
Posts: 78
Location: Ann Arbor, MI
xfs provides the fastest delete time of any of the filesystems (jfs is a rather close second). For small files this is not much of an issue, but for the size of files that we are using for recordings, this can hang the backend for several seconds if an improper format method is used. It seems to be most problematic with a vanilla-formatted ext3 volume. Using any of the methods described in the Pamphlet of Knoppmyth (including the default installation) or in the MythTV documentation will serve you well, but I have been very happy with xfs for the volume that houses my recordings.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 6:49 pm 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
Girkers wrote:
if you do a search on Filesystem I believe you should come across it.
I had originally searched back a year and couldn't find anything, so I posted. Maybe you have better search terms for me to use?

To clarify, I have one hard drive that has the linux/MythTV install on it, database, and all the program files. Then I have another hard drive with a single partition that has the Myth mount. All it really has on it is TV recordings.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Oct 26, 2006 8:53 pm 
Offline
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location: California
I've been using XFS for several months for recordings -- works well. Below is a link to a benchmark that you may find useful...

Marc

http://www.namesys.com/benchmarks/v4marks.html


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 27, 2006 12:36 am 
Offline
Joined: Thu Dec 01, 2005 4:14 pm
Posts: 78
Location: Ann Arbor, MI
I believe the discussion to which Girkers is referring may be the one that I'm thinking of, which is referenced in the MythTV documentation proper (i.e. on the MythTV web site). I'll quote from 24.2:

Quote:
Fortunately, there are published tests ( http://aurora.zemris.fer.hr/filesystems/big.html) that provide insight into filesystem performance under conditions relevant to MythTV usage. In addition, some limited testing (archived at http://www.gossamer-threads.com/lists/m ... sers/52672) with very large files (10 gigabytes) was reported in the MythTV Users mailing list.


The first link appears to no longer work (hasn't for as long as I can remember) but the discussion link in Gossamer Threads very much works and has some excellent insight.

The benchmarks that I performed myself led me to believe that xfs would work best in my scenario, which matches what most others have found.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Oct 27, 2006 3:54 pm 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
That is an interesting read. It's to bad there isn't a definitive answer as to one certainly being better than the other. I wonder if anything has changed in the 1.7 years since that discussion took place?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Dec 28, 2006 10:31 pm 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
I'm going to be doing some testing with a script, but I need more information to get a more accurate report. Here is my script:
Code:
#!/bin/bash

mkdir /mnt/test

for fstype in ext3 reiser xfs jfs; do
{
   echo stats for $fstype fs:
   echo time to format $fstype
   time mkfs.$fstype /dev/hda4
   mount /dev/hda4 /mnt/test

   for filesize in 1024 2048 4096 8192; do
   {
      echo time to write $filesize MB in 1 MB chunks.
      time dd bs=1024K count=$filesize if=/dev/zero of=/mnt/test/$filesize
      echo time to flush disk cache
      time sync
      echo time to erase file
      time rm /mnt/test/$filesize
      }; done

umount /mnt/test
echo
}; done


There are some things that I want to test. For instance, the "bs" option for dd tells what increments to read/write in. How do I find out what the typical read/write increment is from MythTV? While it is getting an A/V stream from an input, does it write the stream to a file every 1MB, 5MB, is it random?

Are there different block sizes I should try formatting with?

Anything else wrong with my script? It seems that ReiserFS isn't available by default?

I am currently running KnoppMyth R5E50.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Dec 29, 2006 11:47 am 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
How do I get other filesystems installed/recognized?


Top
 Profile  
 
 Post subject:
PostPosted: Sun Dec 31, 2006 9:22 pm 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
I installed the JFSutils using Aptitude, and corrected the spelling to "reiserfs". I then converted the partition to LVM so that I could use JFS on it. Okay, really my brother, who knows what he's doing, did all of that. The script now reads:
Code:
#!/bin/bash

mkdir /mnt/test

for fstype in "ext3 -F -q" "reiserfs -f -q" "xfs -f -q" "jfs -q"; do
{
   echo stats for $fstype fs:
   dd if=/dev/zero of=/dev/VG00/repo count=5000
   echo time to format $fstype
   time mkfs.$fstype /dev/VG00/repo
   mount /dev/VG00/repo /mnt/test

   for filesize in 1024 2048 4096 8192; do
   #for filesize in 1024; do
   {
      echo time to write $filesize MB in 1 MB chunks.
      time dd bs=1024K count=$filesize if=/dev/zero of=/mnt/test/$filesize
      echo time to flush disk cache
      time sync
      echo time to read $filesize from disk
      time dd bs=1024K if=/mnt/test/$filesize of=/dev/null
      sync
      echo time to erase file
      time rm /mnt/test/$filesize
      }; done

umount /mnt/test
echo
}; done


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 01, 2007 3:03 am 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
Here are the results of running the script three times. The time listed is the average time required per Gigabyte to complete the task. Lower times are better as they indicate less time required to perform the task. Times are in mm:ss.000
Writing:
Image
Reading:
Image
Deleting
Image

Here is a Zip file that contains the raw output of the script, and a formatted Excel spreadsheet with all of the numbers.


Last edited by Atamido on Mon Jan 01, 2007 3:08 am, edited 1 time in total.


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jan 01, 2007 3:06 am 
Offline
Joined: Wed Jun 15, 2005 9:14 pm
Posts: 72
I have some concerns about the delete speed in the JFS results. Almost all of the delete tests return a 12ms user time for the delete. But that is essentially impossible as that is about the time required just for the head to randomly seek to where is needed on the platter. Some type of caching must be happening, but I don't know what.

Does anyone have some insight?


Top
 Profile  
 

Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 22 posts ] 
Go to page 1, 2  Next



All times are UTC - 6 hours




Who is online

Users browsing this forum: No registered users and 28 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group

Theme Created By ceyhansuyu