Automatic File Purge on /scratch
The /scratch filesystem is provided for temporary file storage. To maintain optimal performance of the scratch filesystem we must delete unused files from the /scratch filesystem. This approach is consistent with how most HPC centers manage their temporary file systems. Automated purge has long been documented on the system description page. We held off implementation as long as we could, but recently we are unable to trim filesystem utilization to acceptable levels through appeals to the users using scratch as long term storage.
We will be enabling automated file purging March 1, 2016. Files that have not been read or written to in 28 days or more are eligible for automated purge. We plan to start deleting files with no access > 1 year first, then work our way closer to the 28-day age as needed to trim space utilization to return /scratch to optimal performance and stability.
- ensure you have copies of critical data in the /mss filesystem. Request additional quota via email to hpc-help if you find you need more space in /mss.
- plan migration of critical files to /mss into your workflow so that you won't need to worry about the automated process deleting something you need.
- look through your /scratch directory and delete un-needed files and directories after copying critical data. For example: temporary files, intermediate results, and checkpoint/restart files can often be deleted once a job has completed.