LinHES Forums http://forum.linhes.org/ |
|
XFS Speed Hints-Measure % frag and defrag your XFS partition http://forum.linhes.org/viewtopic.php?f=3&t=18595 |
Page 2 of 3 |
Author: | akulcsar [ Tue Sep 30, 2008 12:43 am ] |
Post subject: | |
Just throwing this one out there as a brainstorm: Would it be possible to create a run level - like, say, 7 - have it run the defragmentation in an init script, and then when it is done to return to run level 5? Something like this:
Does that sound...sane? |
Author: | opel70 [ Tue Oct 14, 2008 8:59 am ] |
Post subject: | |
graysky wrote: Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV.
Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size? |
Author: | graysky [ Tue Oct 14, 2008 12:57 pm ] |
Post subject: | |
opel70 wrote: graysky wrote: Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV. Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size? Dunno man, capping 4-6 gig SDTV files here. 512 does a great job for me. I just used it based on the suggestion on the wiki I cited. |
Author: | RacerX [ Wed Oct 15, 2008 10:08 am ] |
Post subject: | |
Thanks Graysky for the tip. After I scaled out my setup with multiple drives and storage groups I didn't really realize how fragramented the drives really were. On my old box it was like 52% and 63% and the new box was 97%. I ran the tool for 1.5 hours and it went down to something like 10% or less on the old box and 1% on the new box. NICE WORK!!! |
Author: | graysky [ Wed Oct 15, 2008 3:13 pm ] |
Post subject: | |
Cool man, glad to hear people are getting mileage out of this thread. |
Author: | tatuirovka [ Sat Oct 25, 2008 7:57 am ] |
Post subject: | |
Wondering if someone can help me with a little problem I'm having with the output of xfs_db after running xfs_fsr... for some reason xfs_db gives the same numbers before and after... if I reboot then they update. Code: # xfs_db -c frag -r /dev/sdb1
# actual 170, ideal 150, fragmentation factor 11.76% # xfs_fsr -v -t 60 # xfs_db -c frag -r /dev/sdb1 # actual 170, ideal 150, fragmentation factor 11.76% # reboot # xfs_db -c frag -r /dev/sdb1 # actual 156, ideal 149, fragmentation factor 4.49% |
Author: | graysky [ Sat Oct 25, 2008 8:15 am ] |
Post subject: | |
I've seen this before and can't explain it. You can reset the stats by umounting then remounting the filesystem, i.e. Code: # umount /myth
# mount /myth |
Author: | marc.aronson [ Sat Oct 25, 2008 11:22 pm ] |
Post subject: | |
opel70 wrote: graysky wrote: Did you add the allocsize=512m to your /etc/fstab? It made a difference for my machine capturing SDTV. Any reason not to use the max of allocsize-1024m? Even my small 1/2 hour shows are almost about 900MB, while the vast majority of what I record are multiple GB. I have been running with this for the past few months with no noticeable adverse affects, but am I missing some problem with this size? Each recording also creates a ".png" file in /myth/tv -- this means that each of those files will use 1GB of disk. Not a show-stopper -- just something you should be aware of. I have my machine defrag for 10 minutes once a day, just before it shuts down. My fragmentation has remained very low. Marc |
Author: | adoute [ Fri Oct 31, 2008 6:39 am ] |
Post subject: | |
I'm running R5F27, and recently replaced my media drives with larger ones. During the replacement, I also switched from ext3 to xfs. I thought I'd see what my fragmentation levels were, and defrag if necessary. Both media drives are showing EXTREMELY high frag levels (98.80% and 92.39%), so I thought I'd defrag them while the box was going to be idle for several hours. Unfortunately, I do not have xfs_fsr on my system... The only xfs_* files available are: Code: xfs_admin xfs_copy xfs_growfs xfs_logprint xfs_quota
xfs_bmap xfs_db xfs_info xfs_mkfile xfs_repair xfs_check xfs_freeze xfs_io xfs_ncheck xfs_rtcp Can anyone offer any insight as to why I would have xfs_db and not xfs_fsr? And more importantly, is there a way that I can get xfs_fsr so I can defrag my drives without making large changes to the core OS? Thank you in advance for any advice you can offer. |
Author: | marc.aronson [ Fri Oct 31, 2008 8:04 am ] |
Post subject: | |
adoute wrote: And more importantly, is there a way that I can get xfs_fsr so I can defrag my drives without making large changes to the core OS?
Thank you in advance for any advice you can offer. Look here Marc |
Author: | adoute [ Fri Oct 31, 2008 8:53 am ] |
Post subject: | |
Thank you for such a quick response. The required software installed perfectly. Now I just need to find a nice block of time to start defragging... FWIW, I did try searching - I can't believe I missed the post that you linked to. Thanks again. |
Author: | Grant_Edwards [ Fri Oct 31, 2008 10:41 am ] |
Post subject: | |
My XFS /myth partition was 99% fragmented. I never noticed any performance issues, but after all the dire warnings in this thread about fragmentation, I degragmented it anyway (which took 15-20 hours). I also added the option "allocsize=256m" to fstab and re-mounted the partition. After doing the defrag and setting the allocsize, playback now pauses and stutters when a recording is being deleted. It's not as bad as my old machine that used ext3. AFAICT, defragging and using a larger allocsize has caused this stuttering as files are deleted. I don't suppose there's a re-fragement utility I can use to get back to the previous state? I guess if I decrease the allocsize it will eventually return to "normal", right? -- Grant |
Author: | graysky [ Fri Oct 31, 2008 1:20 pm ] |
Post subject: | |
@grant - I never heard of that before.... I have been using my XFS system like this (with the 512meg thing in the fstab) and it's been just fine, in fact, it's noticibly faster when deleting 4-5 gig files (usually takes 1-2 sec). I wonder if you have an unrelated problem? Have you tried checking the consistency of the partition for troubles? See this thread for a step-by-step walk through. Failing that, and if you suspect it's the allocsize statement, do this: take out the allocsize statement in the fstab, umount, remount, THEN try running the defrag util. Let us know in either case |
Author: | Grant_Edwards [ Sat Nov 01, 2008 2:56 pm ] |
Post subject: | |
graysky wrote: @grant - I never heard of that before.... I have been using my XFS system like this (with the 512meg thing in the fstab) and it's been just fine, in fact, it's noticibly faster when deleting 4-5 gig files (usually takes 1-2 sec). I wonder if you have an unrelated problem? It's possible it was just a coincidence -- I haven't really done anything to troubleshoot it since it only stutters for a second or two and then returns to normal. My ext3 based system (a different box) used to stutter for 30+ seconds while a file was being deleted (occasionally the player would even exit spontaneously). Quote: Have you tried checking the consistency of the partition for troubles? I ran the XFS file system checker both before and after I defragged the partition, but I haven't run in since then. I'll run it again to make sure nothing's gone bad. Quote: Failing that, and if you suspect it's the allocsize statement,
do this: take out the allocsize statement in the fstab, umount, remount, THEN try running the defrag util. IIRC, I didn't add the allocsize statement until after I had defragged the partition. I assumed the allocsize statement would only affect new files that were being created. |
Author: | knappster [ Wed Nov 12, 2008 12:15 am ] |
Post subject: | |
I've tried add a cron job for this and I have been unable to get xfs_fsr to output to a log file. xfs_db does no problem. The weird thing is that if I run the command as a cronjob it runs and the output ends up in /var/log/messages. If I copy and paste the same command into a shell, it runs and puts the output into the proper log file. Any thoughts on what I might be doing wrong? Code: 02 00 * * 3 (/usr/local/bin/idle.sh && /usr/sbin/xfs_fsr -v -t 600) > /var/log/defrag.log 2>&1 as a cron job (root), results in:Quote: Checking what MythTV is doing now or plans within 20 minutes... in /var/log/defrag.logmythshutdown returned 0 schemalock 0 running jobs 0 inuse programs 0 potential recordings 0 planned recordings 0 System is idle and Quote: Nov 12 00:02:02 mythtv fsr[9145]: Found 1 mounted, writable, XFS filesystems in /var/log/messagesNov 12 00:02:02 mythtv fsr[9145]: xfs_fsr -m /etc/mtab -t 600 -f /var/tmp/.fsrlast_xfs ... Nov 12 00:02:02 mythtv fsr[9145]: START: pass=0 ino=0 /dev/sda3 /myth Nov 12 00:02:02 mythtv fsr[9146]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9146]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9146]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9147]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9147]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9147]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9148]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9148]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9148]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9149]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9149]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9149]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9150]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9150]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9150]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9151]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9151]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9151]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9152]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9152]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9152]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9153]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970593 Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970594 Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:02 mythtv fsr[9153]: ino=1124970596 Nov 12 00:02:02 mythtv fsr[9153]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:02 mythtv fsr[9154]: /myth start inode=0 Nov 12 00:02:02 mythtv fsr[9154]: ino=1124970593 Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:03 mythtv fsr[9154]: ino=1124970594 Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:03 mythtv fsr[9154]: ino=1124970596 Nov 12 00:02:03 mythtv fsr[9154]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:03 mythtv fsr[9155]: /myth start inode=0 Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970593 Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970593 Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970594 Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970594 Nov 12 00:02:03 mythtv fsr[9155]: ino=1124970596 Nov 12 00:02:03 mythtv fsr[9155]: No improvement will be made (skipping): ino=1124970596 Nov 12 00:02:03 mythtv fsr[9145]: Completed all 10 passes If I run Code: (/usr/local/bin/idle.sh && /usr/sbin/xfs_fsr -v -t 600) > /var/log/defrag.log 2>&1 in a shell as root, I get:Quote: Checking what MythTV is doing now or plans within 20 minutes... with nothing new in /var/log/messages
mythshutdown returned 0 schemalock 0 running jobs 0 inuse programs 0 potential recordings 0 planned recordings 0 System is idle Found 1 mounted, writable, XFS filesystems xfs_fsr -m /etc/mtab -t 600 -f /var/tmp/.fsrlast_xfs ... START: pass=0 ino=0 /dev/sda3 /myth /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 /myth start inode=0 ino=1124970593 No improvement will be made (skipping): ino=1124970593 ino=1124970594 No improvement will be made (skipping): ino=1124970594 ino=1124970596 No improvement will be made (skipping): ino=1124970596 Completed all 10 passes What gives? Shouldn't I get the same log results whether a command is run in a shell or a cron job? If it makes any difference I am running R5F27. |
Page 2 of 3 | All times are UTC - 6 hours |
Powered by phpBB® Forum Software © phpBB Group http://www.phpbb.com/ |