#!/bin/ksh ## ## SCRIPT: diruse_subdirs_LOCAL_onelevel_bygui ## ## Where: in $FEDIR/scripts where $FEDIR=/apps/nns_com/fea ## ########################################################################### ## PURPOSE: Shows the disk usage (in 1K blocks) in 'LOCAL' sub-directories ## of a user-specified directory -- ONE LEVEL DOWN (no more). ## ## Shows the sub-directories SIZE-SORTED. ## ########################################################################### ## METHOD: ## This script calls 'diruse_subdirs_LOCAL_onelevel_2stdout' ## to execute the 'du -ks' command for multiple sub-directories of the ## user-specified directory (in a 'for' loop) on a local/remote host. ## ## See the notes in that script. ## ########################################################################### ## LOCAL-DIRECTORY DETECTION METHOD: ## This script and the '_2stdout' script may use the 10th line ## of the SGI 'stat' command ## to figure out if the directory is 'nfs' (remote). Examples: ## ## $ stat -q /local/scratch | head -10 |tail -1 ## xfs ## ## $ stat -q /usr/people | head -10 |tail -1 ## nfs ## ########################################################################### ## CALLED BY: 'spacetools' script in $FEDIR/scripts ## actually, by 'spacetools.chestdef' script in $FEDIR/scripts ########################################################################### ## MAINTENANCE HISTORY: ## Written by: Blaise Montandon 03apr2003 Based on the script ## 'diruse_subdirs_onelevel_bygui' ## in $FEDIR/scripts. ## ## Updated by: Blaise Montandon 29apr2003 Replace Tk-GUI prompt script ## 'enter_hostid_and_dir.tk' by ## 'enter_hostid_and_dir_toghelp-scroll.tk'. ## ## Updated by: Blaise Montandon 03sep2003 Make the WIN_INFO and report trailer ## a little more clear & accurate. ## ## Updated by: Blaise Montandon 04sep2003 Load HOST_ID & DIRNAME vars from ## script args $1 & $2. If not loaded ## (i.e. = "") then use the previous ## defaults. ## ## Updated by: Blaise Montandon 08apr2004 To avoid a bug in new SGI 'sort' ## in IRIX 6.5.22, changed several cases ## of 'sort -k1nr' to ## 'sort +0 -1nr'. ########################################################################### ############################################################################# ## SET A UTILITY SCRIPTS PATHNAME in case this toolchest is not started from ## the HandyTools toolchest or the nnsFEAmenu toolchest --- and a ## $FEDIR utility script (or help file) is needed. ############################################################################# if test "$FEDIR" = "" then FEDIR=/apps/nns_com/fea export FEDIR fi ############################################################################## ## SET query hostname and directory from script arguments 1 & 2 ## --- if not already set. ############################################################################## if test "$HOST_ID" = "" then HOST_ID="$1" fi if test "$HOST_ID" = "" then DIRNAME="$2" fi ############################################################################## ## SET LOCAL HOST NAME FOR MESSAGES & REPORTS & LOCAL-HOST-CHECKS. ############################################################################## THISHOST=`hostname` SERVER="engprd00" FEA_SERVER="engfea00" VIZ_SERVER="engvis00" ######################################################################## ## PREP 'CONSTANT' PARAMETERS of the GUI that prompts for hostname and ## directory. ## Also set a DEFAULT HOSTNAME and DIRECTORY NAME. ######################################################################## WINTITLE="(LOCAL)SubDirSizes@OneLevel_for_given_Host:Dirname" export WINTITLE WIN_INFO="\ Enter 1) the name of an SGI (Unix) host on the site network and 2) a directory name. Examples: $SERVER and $HOME $SERVER and /data//cae/$USER $THISHOST and /local/scratch/$USER $THISHOST and /local/scratch $FEA_SERVER and /local/scratch/$USER $VIZ_SERVER and /division/.../... This utility lists all the SUB-DIRECTORY NAMES, at *ONE LEVEL* under the specified directory --- ALONG WITH THE SUB-DIRECTORY SIZES. The resulting sub-directories list is sorted by SIZE --- BIGGEST SUB-DIRECTORIES AT THE TOP. ------- METHOD: This utility gets the sub-directory names for the specified directory, and, FOR THE SUB-DIRECTORIES THAT ARE 'LOCAL' (NOT NFS-mounts) to the specified host, this utility runs the 'du -ks' command, for each LOCAL sub-directory, in the form du -ks / at the specified host. The result is a list of sub-directory names WITH TOTAL-SIZES, at ONE level under the specified directory, . For a heavily populated sub-directory, the 'du' will take MINUTES to run. ------------ PERFORMANCE: For moderate sized ('shallow') directories, this report is typically generated in 3 to 45 seconds. If there are many 1,000's of sub-directories and/or files under one or more of the sub-directories of the specified directory --- i.e. if the specified directory is a 'deep' directory, it may take MUCH MORE THAN A MINUTE. For '/data' directories, you should try the 'Top 500 /data SUBDIRS (from weekly snapshot)' option of the HandyTools-SpaceTools toolchest, first. If the 'local directory tree', of the directory specified, is not huge, performance is not usually an issue, but ... For FAST PERFORMANCE (and for a non-empty list from this 'local-subdirectories-only' utility), SPECIFY THE HOST AT WHICH THE DIRECTORY'S DISK IS LOCALLY ATTACHED --- so that the 'du -s' directory-usage-summary-command is executed on the host where the directory 'resides', rather than having the sub-directories rejected because they are mounted 'over the network'. Local/remote directories references: The 'ManyDirs' button on this GUI --- or the 'df -m' command, to show local and remote file-systems on the host of interest --- or the 'Show FILE-SYSTEM-SIZES @AnyHost (%-USED SORT)' drawer in the HandyTools -> SpaceTools toolchest. ---------- THE OUTPUT (and its use): This utility shows the DISK USAGE in THE 'LOCAL-to-the-specified-host' SUB-DIRECTORIES --- *ONE* LEVEL BELOW the specified directory, no further. This report can be especially helpful, when (1) a FILE SYSTEM IS NEAR-FULL or (2) A HOME DIRECTORY HAS REACHED ITS QUOTA, to locate sub-directories which will yield the most pay-back if they could be cleaned up (files DELETED, COMPRESSED, or ARCHIVED). ------- THE GUI: Note that you can expand this window to enter-and-see the full directory name. You can do queries on multiple hosts/directories and bring the report windows side-by-side for comparison. --- This GUI is presented by the 'wrapper' script $0 This 'wrapper' script uses a utility script $FEDIR/scripts/diruse_subdirs_LOCAL_onelevel_2stdout to perform the du-processing-loop for report generation. This 'wrapper' script then shows the report." export WIN_INFO ## WIN_GEOM="+40+40" ## export WIN_GEOM ## ## WIN_COLOR="#AAAAAA" ## export WIN_COLOR ###################################### ## Set default HOST & DIRECTORY NAMES. ###################################### ## HOSTS_LIST="`hostname` xxxxx yyyyy zzzzz" ## export HOSTS_LIST if test "$HOST_ID" = "" then HOST_ID="$SERVER" # HOST_ID="$THISHOST" fi export HOST_ID if test "$DIRNAME" = "" then DIRNAME="$HOME" # DIRNAME="/local/scratch" # DIRNAME="/local/scratch/$USER" fi export DIRNAME ######################################################################## ## START OF PROMPTING LOOP, FOR HOST & DIRECTORY NAMES. (Not implemented.) ######################################################################## ## # while true ## while : ## do ##################################### ## PROMPT for HOST & DIRECTORY NAMES. ##################################### ## FOR TESTING: # set -x # TEMP=`$FEDIR/tkGUIs/enter_hostid_and_dir.tk` TEMP=`$FEDIR/tkGUIs/enter_hostid_and_dir_toghelp-scroll.tk` eval "$TEMP" ## FOR TESTING: # set - ## FOR TESTING: # echo "$TEMP" if test "$HOST_ID" = "" then exit fi if test "$DIRNAME" = "" then exit fi ############################################################ ## IF $HOST_ID = $SERVER, AND ## IF THE DIRECTORY SPECIFIED IS '/' or '/usr' or '/usr/people', ## POP A MSG AND EXIT. ############################################################ FONT_XCONFIRM="-*-courier-bold-r-normal-*-14-*-*-*-*-*-*-*" if test \( "$HOST_ID" = "$SERVER" \) -a \ \( "$DIRNAME" = "/" -o "$DIRNAME" = "/usr" -o \ "$DIRNAME" = "/usr/" -o "$DIRNAME" = "/usr/people" -o \ "$DIRNAME" = "/usr/people/" \) then JUNK=`xconfirm -c -header "(LOCAL)SubdirSizes,One-Level: EXITING" \ -B DISMISS \ -font "$FONT_XCONFIRM" \ -t "The host specified was server $SERVER --- AND" \ -t "the directory specified was " \ -t "" \ -t " $HOST_ID : $DIRNAME" \ -t "" \ -t "-- one of the 'BIG-BAD' directories: '/' or '/usr' or '/usr/people'." \ -t "The user 'home' directories --- and other huge 'directory-trees'" \ -t "--- are under these directories." \ -t "There are MANY THOUSANDS of files there --- whose sizes would have" \ -t "to be added up to answer your query. EXITING ..." \ -t "" \ -t "Please change your query to a more specific SERVER directory, like" \ -t " /var /local /tmp /etc /bin /sbin /lib /lib32 /lib64" \ -t " /opt /Desktop /dev --- OR a specific '/usr' sub-directory, like" \ -t " /usr/adm /usr/var /usr/tmp /usr/spool /usr/sysadm" \ -t " /usr/bin /usr/sbin /usr/bsd /usr/etc /usr/webdocs" \ -t " /usr/lib /usr/lib32 /usr/lib64 /usr/gfx /usr/include" \ -t " /usr/people/$USER etc." \ -t "If the root directory allocation on the Server is nearly full," \ -t "per 'df' command, the most likely 'culprits' are '/var' or '/local'." \ -icon warning` exit fi ############################################################ ## WARN THE USER ABOUT HUGE DIRECTORIES. GIVE THEM A CANCEL ## A CANCEL OPTION. ############################################################ FONT_XCONFIRM="-*-courier-bold-r-normal-*-14-*-*-*-*-*-*-*" GO_NOGO=`xconfirm -c -header "SubdirSizes,One-Level: BIG_DIRECTORY_warning" \ -b CANCEL -B GO \ -font "$FONT_XCONFIRM" \ -t "*IF* the directory specified" \ -t " $HOST_ID : $DIRNAME" \ -t "is a directory that contains MANY, MANY THOUSANDS of (LOCAL)" \ -t "sub-directories and/or files, i.e. extensive 'sub-directory trees'," \ -t "it could take a long time to generate the" \ -t "SUB-DIRECTORY-SIZES (ONE-LEVEL) report." \ -t "" \ -t "One option is to try a lower-level directory" \ -t "that is likely to have fewer than MANY, MANY THOUSANDS of" \ -t "(LOCAL) sub-directories and/or files." \ -t "" \ -t "Another (fast-response) option, if you want to query the size" \ -t "of a /data or /apps directory, is to use the 'spacetools' option/drawer" \ -t "-- '/data,/apps SUBDIRS SIZE QUERY Utility (weekly snapshot)'." \ -t "" \ -t "If this query takes a long time, you can use a command like 'ps -fu $USER'" \ -t "on the host -- $HOST_ID -- to check that the query is running." \ -t "" \ -t "Cancel or Go?" \ -icon warning` # -font $CONFIRM_FONT \ if test "$GO_NOGO" = "CANCEL" then exit fi ##################################################################### ## CHECK THAT THE HOST_ID EXISTS, if not THISHOST. ##################################################################### ## FOR TESTING: # set -x if test ! "$HOST_ID" = "$THISHOST" then HOSTCHECK=`rsh $HOST_ID cd 2>&1 | grep 'Unknown host'` if test ! "$HOSTCHECK" = "" then CONFIRM_TEXT="\ Specified Host: $HOST_ID is Unknown. " export CONFIRM_TEXT CONFIRM_GEOM="+080+080" CONFIRM_MINSIZE="350 150" CONFIRM_TITLE="$WINTITLE" export CONFIRM_TITLE CONFIRM_MINSIZE CONFIRM_GEOM CONFIRM_COLOR="#3DADFF" export CONFIRM_COLOR CONFIRM_FONT="-adobe-helvetica-bold-r-normal--*-100-*-*-p-*-*-*" export CONFIRM_FONT JUNK=`$FEDIR/tkGUIs/confirm.tk DISMISS` & exit ## continue fi ## END OF if test ! "$HOSTCHECK" = "" fi ## END OF if test ! "$HOST_ID" = "$THISHOST" ##################################################################### ## DIRECTORY CHECKs. ##################################################################### ## CHECK THAT THE DIRNAME IS ACCESSIBLE/EXISTS ## -- on $HOST_ID (do local or remote check). ##################################################################### ## For a slightly different technique of handling 'stdout & stderr' ## (with 2>&1), see $FEDIR/scripts/find_big_or_old_files4dir_bygui ##################################################################### ## FOR TESTING: # set -x if test "$HOST_ID" = "$THISHOST" then DIRCHECK=`ls -d $DIRNAME 2> /dev/null` else . $FEDIR/scripts/mak_rhosts DIRCHECK=`rsh $HOST_ID ls -d $DIRNAME 2> /dev/null` fi if test "$DIRCHECK" = "" then CONFIRM_TEXT="\ Specified Directory: $DIRNAME Not found or does not exist, according to $HOST_ID. Exiting. " export CONFIRM_TEXT CONFIRM_GEOM="+080+080" CONFIRM_MINSIZE="350 150" CONFIRM_TITLE="$WINTITLE" export CONFIRM_TITLE CONFIRM_MINSIZE CONFIRM_GEOM CONFIRM_COLOR="#3DADFF" export CONFIRM_COLOR CONFIRM_FONT="-adobe-helvetica-bold-r-normal--*-100-*-*-p-*-*-*" export CONFIRM_FONT JUNK=`$FEDIR/tkGUIs/confirm.tk DISMISS` & exit ## continue fi ########################################################### ## CHECK THAT THE DIRNAME IS LOCAL (not remote/NFS) ## -- on $HOST_ID (do local or remote check). ########################################################### ## FOR TESTING: # set -x if test "$HOST_ID" = "$THISHOST" then DIR_TYPE=`stat -q $DIRNAME | head -10 | tail -1` else . $FEDIR/scripts/mak_rhosts DIR_TYPE=`rsh $HOST_ID stat -q $DIRNAME | head -10 | tail -1` fi if test "$DIR_TYPE" = "nfs" then CONFIRM_TEXT="\ Specified Directory: $DIRNAME is NOT a LOCAL directory, local to $HOST_ID. Exiting. " export CONFIRM_TEXT CONFIRM_GEOM="+080+080" CONFIRM_MINSIZE="350 150" CONFIRM_TITLE="$WINTITLE" export CONFIRM_TITLE CONFIRM_MINSIZE CONFIRM_GEOM CONFIRM_COLOR="#3DADFF" export CONFIRM_COLOR CONFIRM_FONT="-adobe-helvetica-bold-r-normal--*-100-*-*-p-*-*-*" export CONFIRM_FONT JUNK=`$FEDIR/tkGUIs/confirm.tk DISMISS` & exit ## continue fi ########################################################################## ## SET OUTPUT FILENAME, local if possible. ########################################################################## . $FEDIR/scripts/set_localoutlist OUTLIST2="${OUTLIST}_2" ########################################################################## ## SET REPORT HEADING on size of sub-directories.. ## NOTE: Output is to stdout. ########################################################################## echo "\ ......................... `date '+%Y %b %d %a %T%p'` ........................ DISK USAGE OF 'LOCAL' SUB-DIRECTORIES OF DIRECTORY: ${HOST_ID}:$DIRNAME ONE-DIRECTORY-LEVEL DOWN (no further). SORTED BY *SIZE* --- BIGGEST SUB-DIRECTORIES AT THE TOP. This report was generated by running the 'du' command on $HOST_ID . SIZE-SORT ************* Disk usage (Megabytes) Subdirectory name ------------- ------------------------ TerGigMeg.Kil | | | |" > $OUTLIST ########################################################################## ## GENERATE REPORT CONTENTS. (from Local or Remote host) ########################################################################## ## A SEPARATE SCRIPT IS USED BECAUSE MULTIPLE COMMANDS (AND A for-LOOP) ## ARE USED TO CREATE A SUBDIRECTORY LIST FOR THE LOCAL/REMOTE DIRECTORY ## AND LOOP THROUGH THE SUBDIRECTORIES TO GET DISK USAGE SUMMARY OF EACH. ########################################################################## ## MIGHT WANT TO MOVE THE REPORT TITLE & BOTTOM echo-STATEMENTS FROM ## THAT SCRIPT TO THIS ONE. ########################################################################## ## THISHOST=`hostname` ## SET ABOVE. if test "$HOST_ID" = "$THISHOST" then ## FOR TESTING: # set -x $FEDIR/scripts/diruse_subdirs_LOCAL_onelevel_2stdout \ "$DIRNAME" > $OUTLIST2 ## FOR TESTING: # set - else ## FOR TESTING: # set -x rsh $HOST_ID \ $FEDIR/scripts/diruse_subdirs_LOCAL_onelevel_2stdout "$DIRNAME" \ > $OUTLIST2 ## FOR TESTING: # set - fi ######################################################################### ## SORT AND FORMAT THE OUTPUT for the directory and its subdirectories. ## NOTE: Output is to stdout. ######################################################################### # sort -k1nr $OUTLIST2 | \ sort +0 -1nr $OUTLIST2 | \ awk '{printf ("%13.3f %s\n", $1/1000, $2 )}' >> $OUTLIST rm $OUTLIST2 ######################################################################## ## Add TRAILER to report. ######################################################################## echo "\ | | | | TerGigMeg.Kil ------------- ------------------------ (Megabytes) Subdirectory name Disk usage ************* SIZE-SORT ......................... `date '+%Y %b %d %a %T%p'` ........................ The output above was generated by the script $0 which ran the 'du' command on host $HOST_ID . The script was initiated on host $THISHOST . ----------------- PROCESSING METHOD: 1) This utility first BUILDS A SUB-DIRECTORY LIST --- of the 'LOCAL' sub-directories one level below the specified directory: $HOST_ID : $DIRNAME It uses the 'stat' command to determine which sub-directories are NOT NFS-mounted (i.e. which are 'local') to the host $HOST_ID . 2) Then, for each sub-directory, the utility uses the 'du' command du -ks $DIRNAME/ to get the size of each subdirectory. This can be the time-consuming part, if there are any 'deep', 'heavily-populated' sub-directories. 3) Then, on the listing of results, the script uses a 'pipe' of commands (sort and awk) sort +0 -1nr | awk '{printf ( ... )}' to sort the subdirectories by size, in descending order, AND to format the output (right-justify the size figures). If a more efficient way to do this is found or proffered, it can easily be implemented. ------------- FEATURES NOTE: This utility provides sorting and formatting that is not available by using only the 'du' command. In addition, this utility provides logic to query directory-tree space for only those sub-directories 'local' to specified host $HOST_ID . ---------------------- IMPLEMENTATION METHODS: This 'LOCAL-subdir-sizes, one level, size-sorted' utility script can be accessed via a drawer in a 'spacetools' toolchest command-utility --- via a drawer name like 'LOCAL SUBDIR-SIZES 4aDir@AnyHost (ONE level,SIZE-SORT)'. You can get to this drawer via the site SGI Toolchest: SGI Toolchest -> HandyTools -> AnyHost - SpaceTools (Files) -> LOCAL SUBDIR-SIZES 4aDir@AnyHost (ONE level,SIZE-SORT) Or you can implement the script (or 'spacetools') as - a command alias, via your $HOME/.profile file; - a desktop icon, via the 'Find, File QuickFind' tool drawers; - a drawer in the SGI toolchest, via your $HOME/.auxchestrc file. ........................................................................ " >> $OUTLIST ######################################################################## ## SHOW REPORT, with option to print. ######################################################################## . $FEDIR/scripts/shofil $OUTLIST ## export HOST_ID DIRNAME ## ## done ######################################################################## ## END OF prompting loop, for directory name. (Not implemented.) ######################################################################## ## RETURN TO DIRECTORY PROMPT. ########################################################################