LinHES Forums
http://forum.linhes.org/

Many files in /home/xymon/var/hostdata/mythtv
http://forum.linhes.org/viewtopic.php?f=21&t=30916
Page 1 of 1

Author:  winnpeny [ Sat Jan 23, 2021 4:27 pm ]
Post subject:  Many files in /home/xymon/var/hostdata/mythtv

I just discovered I have 15,670 files in /home/xymon/var/hostdata/mythtv, going back to January 2016. Here are the most recent files:
Code:
-rw-r--r-- 1 nobody nobody  57153 Jan 21 23:35 1611293439
-rw-r--r-- 1 nobody nobody  56348 Jan 22 00:00 1611294943
-rw-r--r-- 1 nobody nobody  55651 Jan 22 08:45 1611326461
-rw-r--r-- 1 nobody nobody  58528 Jan 22 17:16 1611357131
-rw-r--r-- 1 nobody nobody  55626 Jan 23 00:03 1611381486
-rw-r--r-- 1 nobody nobody  56903 Jan 23 02:47 1611391412
-rw-r--r-- 1 nobody nobody  56701 Jan 23 12:48 1611427497

Should some process be deleting the oldest files or is that my job?

Thanks,
Ken

Author:  welner [ Sun Jan 24, 2021 7:20 am ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

Looking at a system backup (i recently migrated to a new boot disk), i found that i had 6929 of these files spanning from 2013 to 2020. they take up about 500meg. So, i have similar history as you.

there is a xymon command called trimhistory

Code:
/home/xymon/server/bin/trimhistory


After a quick grep in /etc i did not find any jobs that were calling that command for scheduled trimming.

I can't state categorically that it is your job to do the cleanup, but it appears that way to me.

It may be difficult to set up a trim job for all circumstances. How far back to trim may be machine/user dependent.

Author:  winnpeny [ Thu Jan 28, 2021 5:22 pm ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

I think I've solved this problem by putting the following file in /etc/cron.daily:
Code:
[mythtv@mythtv cron.daily]$ cat xymontrim
#!/bin/sh

/usr/bin/chpst -u nobody /usr/bin/find /home/xymon/var/hostdata -type f -mtime +28 -delete
/usr/bin/chpst -u nobody /home/xymon/server/bin/trimhistory --cutoff=$(date +%s --date="today -28 days") --drop --droplogs --dropsvcs --env=/home/xymon/server/etc/xymonserver.cfg


This is derived from this post: https://lists.xymon.com/archive/2017-November/045069.html

My system backups have gone from 2.2GB to 75MB.

Author:  welner [ Thu Jan 28, 2021 5:37 pm ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

glad you got it working. I have never used the trimhistory command, so I am not speaking from experience.

be that as it may, you may wish to do the delete after you do the trim, as xymon is keeping track of what files/data it has.

also trimhistory had an option to do the delete for you

Code:
      --droplogs
              Process the XYMONHISTLOGS directory also, and delete status-logs from events prior to  the
              cut-off  time.  Note  that this can dramatically increase the processing time, since there
              are often lots and lots of files to process.

Author:  winnpeny [ Fri Jan 29, 2021 8:01 am ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

Thanks for the feedback. As did the example I referenced, I already had --droplogs included. So, I've commented the find...delete line out and will let it go for a couple of days to see if the logs really are getting cleaned up. If I discover I need the find...delete, I'll put it after the trimhistory line.

Author:  welner [ Sun Jan 31, 2021 8:03 am ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

Just another thought for anyone that may be reading this post in the future...

Sometimes it is beneficial to have some history so you can better see trends year on year.

Assuming that /home space is not an issue, just backup size, one could do a trim job for a longer time period, and simply exclude the xymon directory from the backup.

the backup exclude list is located in /home/mythtv/backup_config/backup_exclude.txt

Author:  winnpeny [ Sun Jan 31, 2021 5:24 pm ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

I did need the find...delete command to remove those files.

And, I like your idea about excluding xymon from the daily backups, so I think I'll do that also.

Thanks for your help.

Author:  nicom [ Thu Aug 12, 2021 7:18 pm ]
Post subject:  Re: Many files in /home/xymon/var/hostdata/mythtv

I too find myself with thousands of files in the xymon/histlog directory dating back 5 years. The trimhistory command seems the perfect way to solve it, however when I try to use it in the command line it fails to find the history directory.
Code:
[mythtv@mymachine ~]$ sudo /usr/bin/chpst -u nobody /home/xymon/server/bin/trimhistory --cutoff=$(date +%s --date="today -28 days") --drop --droplogs
2021-08-13 11:12:37.268570 Cannot cd to history directory: No such file or directory
Any suggestions?

Page 1 of 1 All times are UTC - 6 hours
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/