Author |
Message |
akulcsar
|
Posted: Tue Sep 30, 2008 12:43 am |
|
Joined: Thu Dec 01, 2005 4:14 pm
Posts: 78
Location:
Ann Arbor, MI
|
Just throwing this one out there as a brainstorm:
Would it be possible to create a run level - like, say, 7 - have it run the defragmentation in an init script, and then when it is done to return to run level 5? Something like this:
- cron (intelligently, somehow) schedules a job to drop to init level X.
- The init scripts in level X mimic level 1.
- The last init script in level X is a custom script that runs the xfs defrag routine for a predetermined amount of time, and then switches back to run level 5 when done.
Does that sound...sane?
|
|
Top |
|
|
opel70
|
Posted: Tue Oct 14, 2008 8:59 am |
|
Joined: Tue Apr 11, 2006 7:44 am
Posts: 287
Location:
Los Angeles, CA
|
graysky wrote: Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV.
Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size?
_________________ Tim
LinHES 8.4 HDHR3 BioStar A770, AMD X2 4050e, 2GB RAM GigaByte GeForce 8400, Chaintech AV710 USB-UIRT
|
|
Top |
|
|
graysky
|
Posted: Tue Oct 14, 2008 12:57 pm |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
opel70 wrote: graysky wrote: Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV. Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size?
Dunno man, capping 4-6 gig SDTV files here. 512 does a great job for me. I just used it based on the suggestion on the wiki I cited.
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
|
RacerX
|
Posted: Wed Oct 15, 2008 10:08 am |
|
Joined: Sun Sep 05, 2004 7:06 pm
Posts: 690
|
Thanks Graysky for the tip. After I scaled out my setup with multiple drives and storage groups I didn't really realize how fragramented the drives really were. On my old box it was like 52% and 63% and the new box was 97%. I ran the tool for 1.5 hours and it went down to something like 10% or less on the old box and 1% on the new box.
NICE WORK!!!
|
|
Top |
|
|
graysky
|
Posted: Wed Oct 15, 2008 3:13 pm |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
Cool man, glad to hear people are getting mileage out of this thread.
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
|
tatuirovka
|
Posted: Sat Oct 25, 2008 7:57 am |
|
Joined: Mon Oct 09, 2006 12:18 pm
Posts: 29
|
Wondering if someone can help me with a little problem I'm having with the output of xfs_db after running xfs_fsr... for some reason xfs_db gives the same numbers before and after... if I reboot then they update.
Code: # xfs_db -c frag -r /dev/sdb1 # actual 170, ideal 150, fragmentation factor 11.76% # xfs_fsr -v -t 60 # xfs_db -c frag -r /dev/sdb1 # actual 170, ideal 150, fragmentation factor 11.76% # reboot # xfs_db -c frag -r /dev/sdb1 # actual 156, ideal 149, fragmentation factor 4.49%
|
|
Top |
|
|
graysky
|
Posted: Sat Oct 25, 2008 8:15 am |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
I've seen this before and can't explain it. You can reset the stats by umounting then remounting the filesystem, i.e.
Code: # umount /myth # mount /myth
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
|
marc.aronson
|
Posted: Sat Oct 25, 2008 11:22 pm |
|
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location:
California
|
opel70 wrote: graysky wrote: Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV. Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size?
Each recording also creates a ".png" file in /myth/tv -- this means that each of those files will use 1GB of disk. Not a show-stopper -- just something you should be aware of.
I have my machine defrag for 10 minutes once a day, just before it shuts down. My fragmentation has remained very low.
Marc
|
|
Top |
|
|
adoute
|
Posted: Fri Oct 31, 2008 6:39 am |
|
Joined: Tue Mar 29, 2005 6:15 pm
Posts: 79
Location:
Plymouth, MI
|
I'm running R5F27, and recently replaced my media drives with larger ones. During the replacement, I also switched from ext3 to xfs.
I thought I'd see what my fragmentation levels were, and defrag if necessary. Both media drives are showing EXTREMELY high frag levels (98.80% and 92.39%), so I thought I'd defrag them while the box was going to be idle for several hours. Unfortunately, I do not have xfs_fsr on my system... The only xfs_* files available are:
Code: xfs_admin xfs_copy xfs_growfs xfs_logprint xfs_quota xfs_bmap xfs_db xfs_info xfs_mkfile xfs_repair xfs_check xfs_freeze xfs_io xfs_ncheck xfs_rtcp
Can anyone offer any insight as to why I would have xfs_db and not xfs_fsr?
And more importantly, is there a way that I can get xfs_fsr so I can defrag my drives without making large changes to the core OS?
Thank you in advance for any advice you can offer.
_________________ Running Knoppmyth R5F27 since 09/12/07
Hardware Specs
|
|
Top |
|
|
marc.aronson
|
Posted: Fri Oct 31, 2008 8:04 am |
|
Joined: Tue Jan 18, 2005 2:07 am
Posts: 1532
Location:
California
|
adoute wrote: And more importantly, is there a way that I can get xfs_fsr so I can defrag my drives without making large changes to the core OS?
Thank you in advance for any advice you can offer.
Look here
Marc
|
|
Top |
|
|
adoute
|
Posted: Fri Oct 31, 2008 8:53 am |
|
Joined: Tue Mar 29, 2005 6:15 pm
Posts: 79
Location:
Plymouth, MI
|
Thank you for such a quick response. The required software installed perfectly. Now I just need to find a nice block of time to start defragging...
FWIW, I did try searching - I can't believe I missed the post that you linked to. Thanks again.
_________________ Running Knoppmyth R5F27 since 09/12/07
Hardware Specs
|
|
Top |
|
|
Grant_Edwards
|
Posted: Fri Oct 31, 2008 10:41 am |
|
Joined: Fri Feb 08, 2008 9:19 pm
Posts: 70
|
My XFS /myth partition was 99% fragmented. I never noticed any
performance issues, but after all the dire warnings in this
thread about fragmentation, I degragmented it anyway (which
took 15-20 hours). I also added the option "allocsize=256m" to
fstab and re-mounted the partition.
After doing the defrag and setting the allocsize, playback now
pauses and stutters when a recording is being deleted. It's
not as bad as my old machine that used ext3.
AFAICT, defragging and using a larger allocsize has caused this
stuttering as files are deleted. I don't suppose there's a
re-fragement utility I can use to get back to the previous
state? I guess if I decrease the allocsize it will eventually
return to "normal", right?
--
Grant
_________________ Grant
|
|
Top |
|
|
graysky
|
Posted: Fri Oct 31, 2008 1:20 pm |
|
Joined: Wed Dec 10, 2003 8:31 pm
Posts: 1996
Location:
/dev/null
|
@grant - I never heard of that before.... I have been using my XFS system like this (with the 512meg thing in the fstab) and it's been just fine, in fact, it's noticibly faster when deleting 4-5 gig files (usually takes 1-2 sec). I wonder if you have an unrelated problem? Have you tried checking the consistency of the partition for troubles? See this thread for a step-by-step walk through.
Failing that, and if you suspect it's the allocsize statement, do this: take out the allocsize statement in the fstab, umount, remount, THEN try running the defrag util.
Let us know in either case
_________________ Retired KM user (R4 - R6.04); friend to LH users.
|
|
Top |
|
|
Grant_Edwards
|
Posted: Sat Nov 01, 2008 2:56 pm |
|
Joined: Fri Feb 08, 2008 9:19 pm
Posts: 70
|
graysky wrote: @grant - I never heard of that before.... I have been using my XFS system like this (with the 512meg thing in the fstab) and it's been just fine, in fact, it's noticibly faster when deleting 4-5 gig files (usually takes 1-2 sec). I wonder if you have an unrelated problem? It's possible it was just a coincidence -- I haven't really done anything to troubleshoot it since it only stutters for a second or two and then returns to normal. My ext3 based system (a different box) used to stutter for 30+ seconds while a file was being deleted (occasionally the player would even exit spontaneously). Quote: Have you tried checking the consistency of the partition for troubles? I ran the XFS file system checker both before and after I defragged the partition, but I haven't run in since then. I'll run it again to make sure nothing's gone bad. Quote: Failing that, and if you suspect it's the allocsize statement, do this: take out the allocsize statement in the fstab, umount, remount, THEN try running the defrag util.
IIRC, I didn't add the allocsize statement until after I had
defragged the partition. I assumed the allocsize statement
would only affect new files that were being created.
_________________ Grant
|
|
Top |
|
|
knappster
|
Posted: Wed Nov 12, 2008 12:15 am |
|
Joined: Thu Sep 27, 2007 5:44 pm
Posts: 580
|
I've tried add a cron job for this and I have been unable to get xfs_fsr to output to a log file. xfs_db does no problem. The weird thing is that if I run the command as a cronjob it runs and the output ends up in /var/log/messages. If I copy and paste the same command into a shell, it runs and puts the output into the proper log file. Any thoughts on what I might be doing wrong?
Code: 02 00 * * 3 (/usr/local/bin/idle.sh && /usr/sbin/xfs_fsr -v -t 600) > /var/log/defrag.log 2>&1 as a cron job (root), results in: Quote: Checking what MythTV is doing now or plans within 20 minutes...
mythshutdown returned 0 schemalock 0 running jobs 0 inuse programs 0 potential recordings 0 planned recordings 0
System is idle
in /var/log/defrag.log andQuote: Nov 12 00:02:02 mythtv fsr[9145]: Found 1 mounted, writable, XFS filesystems Nov 12 00:02:02 mythtv fsr[9145]: xfs_fsr -m /etc/mtab -t 600 -f /var/tmp/.fsrlast_xfs ... Nov 12 00:02:02 mythtv fsr[9145]: START: pass=0 ino=0 /dev/sda3 /myth Nov 12 00:02:02 mythtv fsr[9146]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9147]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9148]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9149]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9150]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9151]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9152]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9153]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9154]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9154]: ino=1124970593 Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:03 mythtv fsr[9154]: ino=1124970594 Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:03 mythtv fsr[9154]: ino=1124970596 Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:03 mythtv fsr[9155]: /myth start inode=0 Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970593 Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970594 Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970596 Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:03 mythtv fsr[9145]: Completed all 10 passes
in /var/log/messages If I run Code: (/usr/local/bin/idle.sh && /usr/sbin/xfs_fsr -v -t 600) > /var/log/defrag.log 2>&1 in a shell as root, I get: Quote: Checking what MythTV is doing now or plans within 20 minutes...
mythshutdown returned 0 schemalock 0 running jobs 0 inuse programs 0 potential recordings 0 planned recordings 0
System is idle Found 1 mounted, writable, XFS filesystems xfs_fsr -m /etc/mtab -t 600 -f /var/tmp/.fsrlast_xfs ... START: pass=0 ino=0 /dev/sda3 /myth /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 Completed all 10 passes with nothing new in /var/log/messages
What gives? Shouldn't I get the same log results whether a command is run in a shell or a cron job? If it makes any difference I am running R5F27.
|
|
Top |
|
|