LinHES Forums
http://forum.linhes.org/

What filesystem to store videos?
http://forum.linhes.org/viewtopic.php?f=6&t=12182
Page 1 of 2

Author:  Atamido [ Sun Oct 15, 2006 10:58 pm ]
Post subject:  What filesystem to store videos?

I just bought a new harddrive just to store recorded videos on and I need to know what filesystem to format it to.

I have a PVR-500 and two shows being recorded at the same time is normal. After a year and a half, I think the drive is pretty darn fragmented because it seems that the hard drive often has a hard time keeping up for playback when stuff is being deleted/recorded.

I was thinking about using the maximum block (cluster/allocation unit/whatever) size possible to reduce the amount of fragmentation that can occur. But I don't know what type of filesystem would be fastest for only holding large files, or how to set the block size.

Author:  Atamido [ Thu Oct 26, 2006 8:44 am ]
Post subject: 

Bumpy.

Author:  wififun [ Thu Oct 26, 2006 10:34 am ]
Post subject: 

I used to make my video filesystems FAT or FAT32 so they could be moved to external cases and easily have large amounts of files dumped to them from other win or mac machines. A lot of my video rips are over 2 gigs each since I like AC3 soundtracks and high quality video. Now I use large NAS arrays with 2+TB of storage and network mount the videos over NFS or SAMBA.

Author:  bruce_s01 [ Thu Oct 26, 2006 2:20 pm ]
Post subject: 

You could use a similar line as used in a manual install, see the Phamplet of Knoppmyth .
Code:
# mke2fs -j -O sparse_super -m0 -i8000000 -L tv -M /myth/tv /dev/hdX1

Where hdX1 is whatever your drive is.

Author:  Girkers [ Thu Oct 26, 2006 4:56 pm ]
Post subject: 

There was some discussion some time ago in regards to which filesystem to use, if you do a search on Filesystem I believe you should come across it.

Author:  akulcsar [ Thu Oct 26, 2006 5:25 pm ]
Post subject: 

xfs provides the fastest delete time of any of the filesystems (jfs is a rather close second). For small files this is not much of an issue, but for the size of files that we are using for recordings, this can hang the backend for several seconds if an improper format method is used. It seems to be most problematic with a vanilla-formatted ext3 volume. Using any of the methods described in the Pamphlet of Knoppmyth (including the default installation) or in the MythTV documentation will serve you well, but I have been very happy with xfs for the volume that houses my recordings.

Author:  Atamido [ Thu Oct 26, 2006 6:49 pm ]
Post subject: 

Girkers wrote:
if you do a search on Filesystem I believe you should come across it.
I had originally searched back a year and couldn't find anything, so I posted. Maybe you have better search terms for me to use?

To clarify, I have one hard drive that has the linux/MythTV install on it, database, and all the program files. Then I have another hard drive with a single partition that has the Myth mount. All it really has on it is TV recordings.

Author:  marc.aronson [ Thu Oct 26, 2006 8:53 pm ]
Post subject: 

I've been using XFS for several months for recordings -- works well. Below is a link to a benchmark that you may find useful...

Marc

http://www.namesys.com/benchmarks/v4marks.html

Author:  akulcsar [ Fri Oct 27, 2006 12:36 am ]
Post subject: 

I believe the discussion to which Girkers is referring may be the one that I'm thinking of, which is referenced in the MythTV documentation proper (i.e. on the MythTV web site). I'll quote from 24.2:

Quote:
Fortunately, there are published tests ( http://aurora.zemris.fer.hr/filesystems/big.html) that provide insight into filesystem performance under conditions relevant to MythTV usage. In addition, some limited testing (archived at http://www.gossamer-threads.com/lists/m ... sers/52672) with very large files (10 gigabytes) was reported in the MythTV Users mailing list.


The first link appears to no longer work (hasn't for as long as I can remember) but the discussion link in Gossamer Threads very much works and has some excellent insight.

The benchmarks that I performed myself led me to believe that xfs would work best in my scenario, which matches what most others have found.

Author:  Atamido [ Fri Oct 27, 2006 3:54 pm ]
Post subject: 

That is an interesting read. It's to bad there isn't a definitive answer as to one certainly being better than the other. I wonder if anything has changed in the 1.7 years since that discussion took place?

Author:  Atamido [ Thu Dec 28, 2006 10:31 pm ]
Post subject: 

I'm going to be doing some testing with a script, but I need more information to get a more accurate report. Here is my script:
Code:
#!/bin/bash

mkdir /mnt/test

for fstype in ext3 reiser xfs jfs; do
{
   echo stats for $fstype fs:
   echo time to format $fstype
   time mkfs.$fstype /dev/hda4
   mount /dev/hda4 /mnt/test

   for filesize in 1024 2048 4096 8192; do
   {
      echo time to write $filesize MB in 1 MB chunks.
      time dd bs=1024K count=$filesize if=/dev/zero of=/mnt/test/$filesize
      echo time to flush disk cache
      time sync
      echo time to erase file
      time rm /mnt/test/$filesize
      }; done

umount /mnt/test
echo
}; done


There are some things that I want to test. For instance, the "bs" option for dd tells what increments to read/write in. How do I find out what the typical read/write increment is from MythTV? While it is getting an A/V stream from an input, does it write the stream to a file every 1MB, 5MB, is it random?

Are there different block sizes I should try formatting with?

Anything else wrong with my script? It seems that ReiserFS isn't available by default?

I am currently running KnoppMyth R5E50.

Author:  Atamido [ Fri Dec 29, 2006 11:47 am ]
Post subject: 

How do I get other filesystems installed/recognized?

Author:  Atamido [ Sun Dec 31, 2006 9:22 pm ]
Post subject: 

I installed the JFSutils using Aptitude, and corrected the spelling to "reiserfs". I then converted the partition to LVM so that I could use JFS on it. Okay, really my brother, who knows what he's doing, did all of that. The script now reads:
Code:
#!/bin/bash

mkdir /mnt/test

for fstype in "ext3 -F -q" "reiserfs -f -q" "xfs -f -q" "jfs -q"; do
{
   echo stats for $fstype fs:
   dd if=/dev/zero of=/dev/VG00/repo count=5000
   echo time to format $fstype
   time mkfs.$fstype /dev/VG00/repo
   mount /dev/VG00/repo /mnt/test

   for filesize in 1024 2048 4096 8192; do
   #for filesize in 1024; do
   {
      echo time to write $filesize MB in 1 MB chunks.
      time dd bs=1024K count=$filesize if=/dev/zero of=/mnt/test/$filesize
      echo time to flush disk cache
      time sync
      echo time to read $filesize from disk
      time dd bs=1024K if=/mnt/test/$filesize of=/dev/null
      sync
      echo time to erase file
      time rm /mnt/test/$filesize
      }; done

umount /mnt/test
echo
}; done

Author:  Atamido [ Mon Jan 01, 2007 3:03 am ]
Post subject: 

Here are the results of running the script three times. The time listed is the average time required per Gigabyte to complete the task. Lower times are better as they indicate less time required to perform the task. Times are in mm:ss.000
Writing:
Image
Reading:
Image
Deleting
Image

Here is a Zip file that contains the raw output of the script, and a formatted Excel spreadsheet with all of the numbers.

Author:  Atamido [ Mon Jan 01, 2007 3:06 am ]
Post subject: 

I have some concerns about the delete speed in the JFS results. Almost all of the delete tests return a 12ms user time for the delete. But that is essentially impossible as that is about the time required just for the head to randomly seek to where is needed on the platter. Some type of caching must be happening, but I don't know what.

Does anyone have some insight?

Page 1 of 2 All times are UTC - 6 hours
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/