Author |
Message |
drhood
|
Posted: Sun Oct 06, 2019 10:31 am |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
Hello, I have been struggling tracking what is taking up 100% of disk space on my /dev/sda1 disk. Nothing I do seems to help. Space on disks/partitions (note: /dev/sda is a SSD and /deb/sdb and sdc are regular/old type 2.5" platter drives) Code: df -h Filesystem Size Used Avail Use% Mounted on dev 3.9G 0 3.9G 0% /dev run 3.9G 928K 3.9G 1% /run /dev/sda1 4.5G 4.3G 0 100% / shm 3.9G 12M 3.9G 1% /dev/shm /dev/sda7 94G 14G 80G 16% /data/storage/disk0 /dev/sda5 4.5G 1.1G 3.2G 26% /home /dev/sda6 1.9G 596M 1.2G 35% /data/srv/mysql /dev/sdb4 392G 321G 52G 87% /mnt/428GB /dev/sdb1 481G 392G 65G 86% /mnt/524GB_1 /dev/sdb3 481G 414G 43G 91% /mnt/524GB_2 /dev/sdb2 481G 388G 69G 85% /mnt/524GB_3 /dev/sdc1 688G 591G 62G 91% /mnt/750GB
Command to list largest directories in / (note: bold means directory is stored on /dev/sda1) Code: sudo du -h --max-depth=1 / [b]20M /etc[/b] [b]16K /lost+found[/b] [b][color=#FF0000]2.7G[/color] /usr[/b] [b]1.3M /root[/b] [b]72K /tmp[/b] 2.1T /mnt [b]928K /run[/b] 1.1G /home [b]200M /opt[/b] [b]47M /boot[/b] [b]0 /sys[/b] du: cannot read directory '/proc/1797/task/1797/net': Invalid argument du: cannot read directory '/proc/1797/net': Invalid argument du: cannot read directory '/proc/15297/task/15297/net': Invalid argument du: cannot read directory '/proc/15297/net': Invalid argument du: cannot access '/proc/17266/task/17266/fd/3': No such file or directory du: cannot access '/proc/17266/task/17266/fdinfo/3': No such file or directory du: cannot access '/proc/17266/fd/4': No such file or directory du: cannot access '/proc/17266/fdinfo/4': No such file or directory [b]0 /proc[/b] [b]65M /var[/b] [b]12K /media[/b] [b]4.0K /service[/b] [b]12K /srv[/b] 29G /data [b]0 /dev[/b] [b]59M /mythbuntu*[/b] 2.1T /
I removed four orphaned packages with the command that did not free up much disk space (<50 Mb). Code: sudo pacman -Qtdq I verified that my log files are small size. Code: sudo du -h --max-depth=1 /var 4.0K /var/games 4.0K /var/empty 104K /var/spool 76K /var/tmp 5.4M /var/cache 4.0K /var/opt 27M /var/lib 34M /var/log 4.0K /var/local 24K /var/db 66M /var
The only other odd about the system is the large number (and size) of backup files in /data/storage/disk0/backup/system_backups/errored_backups (note: I deleted all of the backups from September). Code: ls /data/storage/disk0/backup/system_backups/errored_backups backup.2019-10-01_03-14.tgz backup.2019-10-01_12-14.tgz backup.2019-10-03_01-25.tgz backup.2019-10-03_17-05.tgz backup.2019-10-05_02-57.tgz backup.2019-10-01_04-14.tgz backup.2019-10-01_13-14.tgz backup.2019-10-03_01-35.tgz backup.2019-10-03_17-35.tgz backup.2019-10-05_03-57.tgz backup.2019-10-01_05-14.tgz backup.2019-10-02_02-35.tgz backup.2019-10-03_02-35.tgz backup.2019-10-03_19-55.tgz backup.2019-10-05_04-57.tgz backup.2019-10-01_06-14.tgz backup.2019-10-02_03-35.tgz backup.2019-10-03_03-35.tgz backup.2019-10-03_20-35.tgz backup.2019-10-06_09-58.tgz backup.2019-10-01_10-44.tgz backup.2019-10-02_04-35.tgz backup.2019-10-03_04-35.tgz backup.2019-10-03_21-35.tgz backup.2019-10-01_11-14.tgz backup.2019-10-02_05-35.tgz backup.2019-10-03_05-35.tgz backup.2019-10-05_02-37.tgz Because /data/storage/disk0/backup/system_backups/ is stored on a different partition (sda7), I don't see how all of the space on the main partition (/ of sda1) can be gone. Any ideas of what can be done to determine what might be taking up all the space on /sda1 (/) will be greatly appreciated. With kind regards
Last edited by drhood on Sat Dec 07, 2019 7:29 am, edited 2 times in total.
|
|
Top |
|
|
drhood
|
Posted: Sun Oct 06, 2019 10:39 am |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
One thing is that I have a Ceton Initivtv4 PCI tuner in the system that is no longer being used (i.e., no cable card is present and tuner is not connected to a cable wire). Could this be causing the loss of disk space?
|
|
Top |
|
|
welner
|
Posted: Thu Oct 10, 2019 12:55 pm |
|
Joined: Sat Jan 06, 2007 7:08 pm
Posts: 125
|
i wish i had a better answer for you, but after comparing disk usage on my system and comparing it to yours, there isn't much difference other than the partition size. Code: df -h Filesystem Size Used Avail Use% Mounted on dev 7.9G 0 7.9G 0% /dev run 7.9G 512K 7.9G 1% /run /dev/sdb1 23G 15G 7.2G 67% / shm 7.9G 0 7.9G 0% /dev/shm /dev/sdb7 7.6G 1.9G 5.7G 25% /data/storage/disk0 /dev/sdb5 14G 8.5G 4.6G 65% /home /dev/sdb6 9.1G 419M 8.2G 5% /data/srv/mysql /dev/sda1 5.5T 2.7T 2.9T 49% /data/storage/WDCWD6002FRYZ-01WD5B1_K1KJ5NDD Code: du -h --max-depth=1 / 16K /media 156K /tmp 7.4T /diskstation 4.0K /service 3.7G /var 0 /dev 40M /root 4.0K /mnt 3.9G /usr 2.7T /data 16K /lost+found 4.0K /movies 0 /proc 44M /boot 512K /run 12K /srv 20M /etc 8.0K /library 0 /sys 6.9G /opt 8.4G /home 11T / Similar to you, i am using a 64gig ssd drive (sdb in my case). when i did the install, and based on past experience of running out of room, i made the root partition as large as the installer would allow. One thing i did note to save on space is to reset logverbose from 6 to 0 in the xinit startup command. occasionally, xinit would go crazy spewing edid data at an extremely high rate, though nothing was wrong. this can be accomplished by editing /usr/LH/bin/LinHES-start and changing logverbose from 6 to 0. this script occasionally gets overwritten when an update is applied, so logvoerbose gets reset back to 6. while looking at this question, i noticed that i have been running logverbose set to 6 since last april, and i have an old log of 77meg and a current log of 36meg. not in the gig range, but it is something to look at. when xinit did loose its mind and start spewing to the log, my disk would fill up within a matter of minutes. related to the previous post you had concerning ncid, it appears that you can disable the ncid software by going into service menu -> linhes settings -> advanced. my suspicion is that "run callerid" is checked on. if so, please try unchecking it. i have never used this software, and do not have it installed, so it is probably safe to remove it via pacman
_________________ DH87MC i7-4770 16GB ram Xonar Essence ST geforce 710 LinHes 8.6
|
|
Top |
|
|
mattbatt
|
Posted: Fri Oct 11, 2019 9:29 am |
|
Joined: Tue Aug 15, 2006 11:14 am
Posts: 1343
Location:
Orlando FL
|
I see a couple of things going on here. When posting terminal outputs on the forum it looks better to use the "code" box instead of the "quote" box. It keeps the formatting correct. I tried to manually correct it your data but it's still askew but it's more legible. Code: df -h Filesystem Size Used Avail Use% Mounted on dev 3.9G 0 3.9G 0% /dev run 3.9G 928K 3.9G 1% /run /dev/sda1 4.5G 4.3G 0 100% / shm 3.9G 12M 3.9G 1% /dev/shm /dev/sda7 94G 14G 80G 16% /data/storage/disk0 /dev/sda5 4.5G 1.1G 3.2G 26% /home /dev/sda6 1.9G 596M 1.2G 35% /data/srv/mysql /dev/sdb4 392G 321G 52G 87% /mnt/428GB /dev/sdb1 481G 392G 65G 86% /mnt/524GB_1 /dev/sdb3 481G 414G 43G 91% /mnt/524GB_2 /dev/sdb2 481G 388G 69G 85% /mnt/524GB_3 /dev/sdc1 688G 591G 62G 91% /mnt/750GB
This is the output of Code: sudo df -h Filesystem Size Used Avail Use% Mounted on dev 3.8G 0 3.8G 0% /dev run 3.8G 468K 3.8G 1% /run /dev/sda1 19G 3.8G 14G 22% / shm 3.8G 0 3.8G 0% /dev/shm /dev/sda7 70G 35G 35G 50% /data/storage/disk0 /dev/sda5 8.4G 2.3G 5.7G 29% /home /dev/sda6 5.4G 337M 4.8G 7% /data/srv/mysql /dev/sdb1 932G 719G 213G 78% /data/storage/WDCWD10JPVX-75JC3T0_WX61A8691H4H
and Code: sudo du -h --max-depth=1 du: cannot read directory './proc/1691/task/1691/net': Invalid argument du: cannot read directory './proc/1691/net': Invalid argument du: cannot read directory './proc/16160/task/16160/net': Invalid argument du: cannot read directory './proc/16160/net': Invalid argument du: cannot access './proc/16260/task/16260/fd/3': No such file or directory du: cannot access './proc/16260/task/16260/fdinfo/3': No such file or directory du: cannot access './proc/16260/fd/4': No such file or directory du: cannot access './proc/16260/fdinfo/4': No such file or directory 0 ./proc 76M ./var 3.2G ./usr 12K ./srv 148K ./tmp 787G ./data 468K ./run 0 ./sys 0 ./dev 19M ./etc 329M ./opt 4.0K ./service 47M ./boot 12K ./media 4.0K ./mnt 36M ./root 16K ./lost+found 2.3G ./home 792G .
The default installer makes the SDA1 partition way too small. I manually made mine 19gb. Why did you chop up your SDB drive into 4 partitions of roughly 500GB each? It looks like you have 80gb available on your sda7 partition. If I were in your situation I would mount the drive in a full Linux OS or even a live disk that has GParted and adjust those partitions to make sda1 at least 20GB and take it from sda7. This is a little dangerous because you can accidentally wipe your drive so back up your drive first if you can.
_________________ My System
|
|
Top |
|
|
drhood
|
Posted: Sat Oct 19, 2019 10:24 am |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
welner wrote: related to the previous post you had concerning ncid, it appears that you can disable the ncid software by going into service menu -> linhes settings -> advanced. my suspicion is that "run callerid" is checked on. if so, please try unchecking it. i have never used this software, and do not have it installed, so it is probably safe to remove it via pacman Thank you for the tips! Regarding the ncid, I discovered that callerid was enbabled in the LinHES menu as you suspected. I disabled this feature and will see if it resolves that issue. I will also remove the callerid feature via pacman as you suggested. I discovered that my recordedseek table in the mysql database gets corrupted daily. I can fix it with the optimize command; however, I will monitor now that I have disabled callerid change. Thanks again for these tips!
|
|
Top |
|
|
drhood
|
Posted: Sat Oct 19, 2019 10:27 am |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
mattbatt wrote: The default installer makes the SDA1 partition way too small. I manually made mine 19gb. Why did you chop up your SDB drive into 4 partitions of roughly 500GB each? It looks like you have 80gb available on your sda7 partition. If I were in your situation I would mount the drive in a full Linux OS or even a live disk that has GParted and adjust those partitions to make sda1 at least 20GB and take it from sda7. This is a little dangerous because you can accidentally wipe your drive so back up your drive first if you can.
I edited the orignal post to use the code and not quote for proper formatting. Also, thanks for the storage suggestions. I will have to reconfigure my storage as you suggested. I am thinking about upgrading to much larger hard drives; however, this will be a project for the near future. Appreciate the helpful comments.
|
|
Top |
|
|
welner
|
Posted: Sat Oct 19, 2019 2:42 pm |
|
Joined: Sat Jan 06, 2007 7:08 pm
Posts: 125
|
drhood wrote: I discovered that my recordedseek table in the mysql database gets corrupted daily. I can fix it with the optimize command; however, I will monitor now that I have disabled callerid change.
if your database is getting corrupted daily, this may also hint at running out of disk space on the slice holding your database (though it doesn't appear to be the case based your df's). its best to attack the too small partitions first
_________________ DH87MC i7-4770 16GB ram Xonar Essence ST geforce 710 LinHes 8.6
|
|
Top |
|
|
drhood
|
Posted: Wed Oct 23, 2019 6:15 pm |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
welner wrote: drhood wrote: I discovered that my recordedseek table in the mysql database gets corrupted daily. I can fix it with the optimize command; however, I will monitor now that I have disabled callerid change.
if your database is getting corrupted daily, this may also hint at running out of disk space on the slice holding your database (though it doesn't appear to be the case based your df's). its best to attack the too small partitions first Thanks for the reply! We do get database corruptions daily and it is super annoying. The database table that gets corrupted in the recordedseek. There are two commands I use that report different results. Code: $ df -h Filesystem Size Used Avail Use% Mounted on dev 3.9G 0 3.9G 0% /dev run 3.9G 868K 3.9G 1% /run /dev/sda1 4.5G 4.5G 0 100% / shm 3.9G 2.9M 3.9G 1% /dev/shm /dev/sda7 94G 39G 55G 42% /data/storage/disk0 /dev/sda5 4.5G 1.1G 3.2G 25% /home /dev/sda6 1.9G 731M 1019M 42% /data/srv/mysql /dev/sdb4 392G 321G 52G 87% /mnt/428GB /dev/sdb1 481G 384G 73G 85% /mnt/524GB_1 /dev/sdb3 481G 414G 43G 91% /mnt/524GB_2 /dev/sdb2 481G 380G 77G 84% /mnt/524GB_3 /dev/sdc1 688G 548G 106G 84% /mnt/750GB
and Code: $ sudo du -hx --max-depth=1 / 20M /etc 16K /lost+found 2.7G /usr 1.3M /root 5.2M /tmp 4.0K /mnt 200M /opt 47M /boot 171M /var 12K /media 4.0K /service 12K /srv 9.7M /data 59M /mythbuntu* 3.1G /
When recording stop, the routine is the following: Delete log files or directories in /var/log larger than 100Mb, which is typically a backup directory that is created daily (e.g., 2019-10-24). Execute the database optimize command... Code: sudo optimize_mythdb.py Reboot the machine. If I don't reboot, then the df-h command will remain at zero (0%) whereas the 'du -hx --max-depth=1 /' will report more space ; however, rebooting seems to get the two commands to report similar output. I have searched and cannot find any other space that could be freed up. Is there a way to recover orphaned space? I don't think enlarging partitions will be helpful if my issue is something wrong with the system. Any thoughts will be appreciated.
|
|
Top |
|
|
welner
|
Posted: Thu Oct 24, 2019 1:51 pm |
|
Joined: Sat Jan 06, 2007 7:08 pm
Posts: 125
|
in your original post, you indicate that you clean up "orphaned" packages via the command: Code: sudo pacman -Qtdq i am not sure what your definition of "orphaned" is. be that as it may, the way i use pacman to delete older out of date packages is: Code: pacman -Sc i don't know if there is much of a difference between the two. you can also look at what packages pacman is storing in this directory: Code: /data/storage/disk0/pacman/pkg considering that the pacman pkg directory is located on a different slice, this will not free up space on your root partition. you could also look for unused packages that have been installed. for example, do you have kodi or plex installed? you can search for "extra" packages via this command Code: pacman -Ss | grep extra | grep installed This might free up some space on the root partition. your root partition is too small. based on the "df" info, it really doesn't look like you are going to be able to free up much more space on the root partition i would fix the partition sizes before looking at the database table corruption
_________________ DH87MC i7-4770 16GB ram Xonar Essence ST geforce 710 LinHes 8.6
|
|
Top |
|
|
drhood
|
Posted: Sat Dec 07, 2019 7:28 am |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
welner wrote: in your original post, you indicate that you clean up "orphaned" packages via the command: Code: sudo pacman -Qtdq i am not sure what your definition of "orphaned" is. be that as it may, the way i use pacman to delete older out of date packages is: Code: pacman -Sc i don't know if there is much of a difference between the two. you can also look at what packages pacman is storing in this directory: Code: /data/storage/disk0/pacman/pkg considering that the pacman pkg directory is located on a different slice, this will not free up space on your root partition. you could also look for unused packages that have been installed. for example, do you have kodi or plex installed? you can search for "extra" packages via this command Code: pacman -Ss | grep extra | grep installed This might free up some space on the root partition. your root partition is too small. based on the "df" info, it really doesn't look like you are going to be able to free up much more space on the root partition i would fix the partition sizes before looking at the database table corruption Thank you for this suggestion, very helpful. I will be replacing my storage with larger drives. Thanks for replying to this message. Cheers!
|
|
Top |
|
|
drhood
|
Posted: Sun Dec 08, 2019 6:40 am |
|
Joined: Mon Jun 05, 2017 8:23 am
Posts: 57
|
welner wrote: i am not sure what your definition of "orphaned" is. be that as it may, the way i use pacman to delete older out of date packages is: Code: pacman -Sc I discovered the following pacman command that appears to identified "orphaned" packages. Code: pacman -Rns $(pacman -Qtdq) I tried it once on a manjaro desktop system and it seemed to remove packages without harming the system. Interested in others opinion about this command who have more experience with it. Cheers!
|
|
Top |
|
|