#!/bin/ksh ## ## SCRIPT: diruse_subdirs_onelevel_sizesort_bygui ## ## Where: in $FEDIR/scripts where $FEDIR=/apps/nns_com/fea ## ## (Renamed from diruse_subdirs_onelevel_bygui, 4sep2003.) ## ########################################################################### ## PURPOSE: Shows the disk usage (in 1K blocks) in subdirectories of ## a user-specified directory -- ONE LEVEL DOWN (no more). ## ## Shows the subdirectories SIZE-SORTED. ## ########################################################################### ## METHOD: ## This script calls 'diruse_subdirs_onelevel_2stdout' to execute ## the 'du -ks' command for multiple sub-directories of the ## user-specified directory (in a 'for' loop) on a local/remote host. ## See the notes in that script. ########################################################################### ## CALLED BY: 'spacetools' script in $FEDIR/scripts ## actually, by 'spacetools.chestdef' script in $FEDIR/scripts ########################################################################### ## MAINTENANCE HISTORY: ## Written by: B. Montandon O06 15Mar2000 Based on 'duseanydir_onelevel' ## and 'diruse_subdirs_all_bygui' ## and 'find_big_or_old_files4dir_bygui' ## in $FEDIR/scripts. ## Updated by: B. Montandon O06 6Sep2000 Add 'Unknown host' check. ## Updated by: B. Montandon O06 7Sep2000 Add local/remote DIRCHECK routine. ## Updated by: Blaise Montandon 6apr2001 Chg from calling utility script ## 'diruse_subdirs_onelevel' to using ## 'diruse_subdirs_onelevel_2stdout'. ## Updated by: Blaise Montandon 13dec2001 Add 'BIG_DIRECTORY_warning' xconfirm. ## Updated by: Blaise Montandon 24apr2002 Exit with popup msg if DIRNAME is ## '/', '/usr', or '/usr/people'. ## ## Updated by: Blaise Montandon 29apr2003 Replace Tk-GUI prompt script ## 'enter_hostid_and_dir.tk' by ## 'enter_hostid_and_dir_toghelp-scroll.tk'. ## ## Updated by: Blaise Montandon 08apr2004 To avoid a bug in new SGI 'sort' ## in IRIX 6.5.22, changed several cases ## of 'sort -k1nr' to ## 'sort +0 -1nr'. ########################################################################### ############################################################################# ## SET A UTILITY SCRIPTS PATHNAME in case this toolchest is not started from ## the HandyTools toolchest or the nnsFEAmenu toolchest --- and a ## $FEDIR utility script (or help file) is needed. ############################################################################# if test "$FEDIR" = "" then FEDIR=/apps/nns_com/fea export FEDIR fi ############################################################################## ## SET query hostname and directory from script arguments 1 & 2 ## --- if not already set. ############################################################################## if test "$HOST_ID" = "" then HOST_ID="$1" fi if test "$HOST_ID" = "" then DIRNAME="$2" fi ############################################################################## ## SET LOCAL HOST NAME FOR MESSAGES & REPORTS & LOCAL-HOST-CHECKS. ############################################################################## THISHOST=`hostname` SERVER="engprd00" FEA_SERVER="engfea00" VIZ_SERVER="engvis00" ######################################################################## ## SET GUI VARIABLES, including default HOSTNAME and DIRECTORY NAME. ######################################################################## WINTITLE="SubDirSizes@OneLevel_for_given_Host:Dirname" export WINTITLE WIN_INFO="\ Enter 1) the name of an SGI (Unix) host on the site network and 2) a directory name. Examples: $SERVER and $HOME $SERVER and /data//cae/$USER $THISHOST and /local/scratch/$USER $THISHOST and /local/scratch $FEA_SERVER and /local/scratch/$USER $VIZ_SERVER and /division/.../... This utility lists all the SUB-DIRECTORY NAMES, at *ONE LEVEL* under the specified directory --- ALONG WITH THE SUB-DIRECTORY SIZES. The resulting sub-directories list is sorted by SIZE --- BIGGEST SUB-DIRECTORIES AT THE TOP. --- For moderate sized ('shallow') directories, this report is typically generated in 3 to 45 seconds. If there are many 1,000's of sub-directories and/or files under one or more of the sub-directories of the specified directory --- i.e. if the specified directory is a 'deep' directory, it may take MUCH MORE THAN A MINUTE. For '/data' directories, you should try the 'Top 500 /data SUBDIRS (from weekly snapshot)' option of the HandyTools-SpaceTools toolchest, first. ---------------------- METHOD and PERFORMANCE: This utility gets the sub-directory names at the first-level under the specified directory, and uses the 'du -ks' command, for each sub-directory, in the form du -ks / This command is run at the specified host, for each sub-directory, to list all the sub-directory names, at ONE level under , with their sizes. Note: The size of a sub-directory includes all files in all of its sub-directories --- through all levels, to the lowest level. Also, no attempt is made in this utility to 'screen out' sub-directories that are not 'local to' the specified host. Hence this query CAN TAKE A LOT OF TIME GATHERING THE SIZE DATA for the sub-directories at the one level --- if any of the sub-directories are 'deep' --- and if any of the 'deep' ones are not 'local to' the specified host. ---- If the 'directory tree' of the directory specified is NOT huge, performance is not usually an issue, but ... For FAST PERFORMANCE, on a really 'DEEP' directory tree, it is BEST TO SPECIFY THE HOST AT WHICH THE DIRECTORY'S DISK IS LOCALLY ATTACHED --- so that the 'du -s' summary-commands are executed on the host where the top-level directory 'resides', rather than accessing the top-level directory 'over the network'. Local/remote directories references: The 'ManyDirs' button --- or the 'df -m' command on the host --- or the 'Show FILE-SYSTEM-SIZES @AnyHost (%-USED SORT)' drawer in the HandyTools-SpaceTools toolchest. ---------- THE OUTPUT (and its use): This utility shows the DISK USAGE in THE SUB-DIRECTORIES *ONE* LEVEL BELOW the specified directory, no further. The resulting one-level sub-directories list is sorted by SIZE --- BIGGEST SUB-DIRECTORIES AT THE TOP. This report can be especially helpful, when 1) a FILE SYSTEM IS NEAR-FULL or 2) A HOME DIRECTORY HAS REACHED ITS QUOTA, to locate sub-directories which will yield the most pay-back if they could be cleaned up --- files DELETED, COMPRESSED, or ARCHIVED. ------- THE GUI: Note that you can expand this window to enter-and-see the full directory name. You can do queries on multiple hosts/directories and bring the report windows side-by-side for comparison. ---- This GUI is presented by the 'wrapper' script $0 This 'wrapper' script also performs the report generation --- using a du-processing-loop indicated above, with output piped to 'sort' and 'awk'. The 'wrapper' script also shows the sorted, formatted report." export WIN_INFO ## WIN_GEOM="+40+40" ## export WIN_GEOM ## ## WIN_COLOR="#AAAAAA" ## export WIN_COLOR ######################################################################## ## Sset default HOST & DIRECTORY NAMES. ######################################################################## ## HOSTS_LIST="`hostname` xxxxx yyyyy zzzzz" ## export HOSTS_LIST if test "$HOST_ID" = "" then HOST_ID="$SERVER" # HOST_ID="$THISHOST" fi export HOST_ID if test "$DIRNAME" = "" then DIRNAME="$HOME" # DIRNAME="/local/scratch" # DIRNAME="/local/scratch/$USER" fi export DIRNAME ######################################################################## ## START OF PROMPTING LOOP, FOR HOST & DIRECTORY NAMES. ######################################################################## ## # while true ## while : ## do ################################ ## PROMPT FOR HOST & DIR NAMES. ################################ ## FOR TESTING: # set -x # TEMP=`$FEDIR/tkGUIs/enter_hostid_and_dir.tk` TEMP=`$FEDIR/tkGUIs/enter_hostid_and_dir_toghelp-scroll.tk` eval "$TEMP" ## FOR TESTING: # set - ## FOR TESTING: # echo "$TEMP" if test "$HOST_ID" = "" then exit fi if test "$DIRNAME" = "" then exit fi ############################################################ ## IF THE DIRECTORY SPECIFIED IS '/' or '/usr' or '/usr/people', ## POP A MSG AND EXIT. ############################################################ FONT_XCONFIRM="-*-courier-bold-r-normal-*-14-*-*-*-*-*-*-*" if test \( "$DIRNAME" = "/" -o "$DIRNAME" = "/usr" -o \ "$DIRNAME" = "/usr/" -o "$DIRNAME" = "/usr/people" -o \ "$DIRNAME" = "/usr/people/" \) then JUNK=`xconfirm -c -header "SubdirSizes,One-Level: EXITING" \ -B DISMISS \ -font "$FONT_XCONFIRM" \ -t "The directory specified was " \ -t "" \ -t " $HOST_ID : $DIRNAME" \ -t "" \ -t "-- one of the 'BIG-BAD' directories: '/' or '/usr' or '/usr/people'." \ -t "" \ -t "The user 'home' directories are under these directories." \ -t "" \ -t "There are MANY THOUSANDS of files there --- whose sizes would have" \ -t "to be added up to answer your query. EXITING ..." \ -t "" \ -t "Please change your query to a more specific directory, like" \ -t " /var /local /tmp /etc /bin /sbin /lib /lib32 /lib64" \ -t " /opt /Desktop /dev --- OR a specific '/usr' sub-directory, like" \ -t " /usr/adm /usr/var /usr/tmp /usr/spool /usr/sysadm" \ -t " /usr/bin /usr/sbin /usr/bsd /usr/etc /usr/webdocs" \ -t " /usr/lib /usr/lib32 /usr/lib64 /usr/gfx /usr/include" \ -t " /usr/people/$USER etc." \ -t "If the root directory allocation on the local machine is nearly full," \ -t "per 'df' command, the most likely 'culprits' are '/var' or '/local'." \ -icon warning` exit fi ############################################################ ## WARN THE USER ABOUT HUGE DIRECTORIES. GIVE THEM A CANCEL ## A CANCEL OPTION. ############################################################ GO_NOGO=`xconfirm -c -header "SubdirSizes,One-Level: BIG_DIRECTORY_warning" \ -b CANCEL -B GO \ -font "$FONT_XCONFIRM" \ -t "IF the directory specified" \ -t " $HOST_ID : $DIRNAME" \ -t "is a directory that contains MANY THOUSANDS of" \ -t "sub-directories (& even more files), it could take a long time" \ -t "to generate the SUB-DIRECTORY-SIZES (ONE-LEVEL) report." \ -t "" \ -t "One option is to try a lower-level directory" \ -t "that is likely to have MANY HUNDREDS of" \ -t "sub-directories (or less), rather than MANY THOUSANDS." \ -t "" \ -t "Another (fast-response) option, if you want to query the size" \ -t "of a /data or /apps directory, is to use the 'spacetools' option/drawer" \ -t "-- '/data,/apps SUBDIRS SIZE QUERY Utility (weekly snapshot)'." \ -t "" \ -t "If this query takes a long time, you can use a command like 'ps -fu $USER'" \ -t "on the host -- $HOST_ID -- to check that the query is running." \ -t "" \ -t "Cancel or Go?" \ -icon warning` # -font $CONFIRM_FONT \ if test "$GO_NOGO" = "CANCEL" then exit fi ##################################################################### ## CHECK THAT THE HOST_ID EXISTS, if not THISHOST. ##################################################################### ## FOR TESTING: # set -x if test ! "$HOST_ID" = "$THISHOST" then HOSTCHECK=`rsh $HOST_ID cd 2>&1 | grep 'Unknown host'` if test ! "$HOSTCHECK" = "" then CONFIRM_TEXT="\ Specified Host: $HOST_ID is Unknown. " export CONFIRM_TEXT CONFIRM_GEOM="+080+080" CONFIRM_MINSIZE="350 150" CONFIRM_TITLE="$WINTITLE" export CONFIRM_TITLE CONFIRM_MINSIZE CONFIRM_GEOM CONFIRM_COLOR="#3DADFF" export CONFIRM_COLOR CONFIRM_FONT="-adobe-helvetica-bold-r-normal--*-100-*-*-p-*-*-*" export CONFIRM_FONT JUNK=`$FEDIR/tkGUIs/confirm.tk DISMISS` & exit ## continue fi ## END OF if test ! "$HOSTCHECK" = "" fi ## END OF if test ! "$HOST_ID" = "$THISHOST" ##################################################################### ## DIRECTORY CHECKs. ##################################################################### ## CHECK THAT THE DIRNAME IS ACCESSIBLE/EXISTS ## -- on $HOST_ID (do local or remote check). ##################################################################### ## For a slightly different technique of handling 'stdout & stderr' ## (with 2>&1), see $FEDIR/scripts/find_big_or_old_files4dir_bygui ##################################################################### ## FOR TESTING: # set -x if test "$HOST_ID" = "$THISHOST" then DIRCHECK=`ls -d $DIRNAME 2> /dev/null` else . $FEDIR/scripts/mak_rhosts DIRCHECK=`rsh $HOST_ID ls -d $DIRNAME 2> /dev/null` fi if test "$DIRCHECK" = "" then CONFIRM_TEXT="\ Specified Directory: $DIRNAME Not found or does not exist, according to $HOST_ID. Exiting. " export CONFIRM_TEXT CONFIRM_GEOM="+080+080" CONFIRM_MINSIZE="350 150" CONFIRM_TITLE="$WINTITLE" export CONFIRM_TITLE CONFIRM_MINSIZE CONFIRM_GEOM CONFIRM_COLOR="#3DADFF" export CONFIRM_COLOR CONFIRM_FONT="-adobe-helvetica-bold-r-normal--*-100-*-*-p-*-*-*" export CONFIRM_FONT JUNK=`$FEDIR/tkGUIs/confirm.tk DISMISS` & exit ## continue fi ########################################################### ## EXIT IF USER SPECIFIES '/' for the directory. ########################################################### ## if test "$DIRNAME" = "/" ## then ## echo " ## Try a lower level directory than the root directory (/). ## ## *** Press Enter to continue." ## read NNS_JUNK ## ## continue ## # exit ## fi ########################################################### ## EXIT IF USER SPECIFIES '/' for the directory. ########################################################### ########################################################################## ## SET OUTPUT FILENAME, local if possible. ########################################################################## . $FEDIR/scripts/set_localoutlist OUTLIST2="${OUTLIST}_2" ########################################################################## ## SET REPORT HEADING on size of sub-directories.. ## NOTE: Output is to stdout. ########################################################################## echo "\ ......................... `date '+%Y %b %d %a %T%p'` ........................ DISK USAGE OF SUB-DIRECTORIES OF DIRECTORY: ${HOST_ID}:$DIRNAME ONE-DIRECTORY-LEVEL DOWN (no further). SORTED BY *SIZE* --- BIGGEST SUB-DIRECTORIES AT THE TOP. This report was generated by running the 'du' command on $HOST_ID . Total usage is shown on first line, in the top-level directory. SIZE-SORT ************* Disk usage (Megabytes) Subdirectory name ------------- ------------------------ TerGigMeg.Kil | | | |" > $OUTLIST ########################################################################## ## GENERATE REPORT CONTENTS. (from Local or Remote host) ########################################################################## ## A SEPARATE SCRIPT IS USED BECAUSE MULTIPLE COMMANDS (AND A for-LOOP) ## ARE USED TO CREATE A SUBDIRECTORY LIST FOR THE LOCAL/REMOTE DIRECTORY ## AND LOOP THROUGH THE SUBDIRECTORIES TO GET DISK USAGE SUMMARY OF EACH. ########################################################################## ## MIGHT WANT TO MOVE THE REPORT TITLE & BOTTOM echo-STATEMENTS FROM ## THAT SCRIPT TO THIS ONE. ########################################################################## ## THISHOST=`hostname` ## SET ABOVE. if test "$HOST_ID" = "$THISHOST" then ## FOR TESTING: # set -x $FEDIR/scripts/diruse_subdirs_onelevel_2stdout "$DIRNAME" > $OUTLIST2 ## FOR TESTING: # set - else ## FOR TESTING: # set -x rsh $HOST_ID $FEDIR/scripts/diruse_subdirs_onelevel_2stdout "$DIRNAME" \ > $OUTLIST2 ## FOR TESTING: # set - fi ######################################################################### ## SORT AND FORMAT THE OUTPUT for the directory and its subdirectories. ## NOTE: Output is to stdout. ######################################################################### # sort -k1nr $OUTLIST2 | \ sort +0 -1nr $OUTLIST2 | \ awk '{printf ("%13.3f %s\n", $1/1000, $2 )}' >> $OUTLIST rm $OUTLIST2 ######################################################################## ## Add TRAILER to report. ######################################################################## echo "\ | | | | TerGigMeg.Kil ------------- ------------------------ (Megabytes) Subdirectory name Disk usage ************* SIZE-SORT ......................... `date '+%Y %b %d %a %T%p'` ........................ The output above was generated by the script $0 which ran the 'du' command on host $HOST_ID . The script was initiated on host $THISHOST . ----------------- PROCESSING METHOD: 1) The script first BUILDS A SUBDIRECTORY LIST --- of the sub-directories one level below the specified directory: $DIRNAME 2) Then, for each sub-directory, the script uses the 'du' command du -ks $DIRNAME/ to get the size of each subdirectory. 3) Then, on the listing of results, the script uses a pipe' of commands (sort and awk) sort +0 -1nr | awk '{printf ( ... )}' to sort the subdirectories by size, in descending order, AND to format the output (right-justify the size figures). If a more efficient way to do this is found or proffered, it can easily be implemented. ------------ FEATURE NOTE: This utility provides size-sorting and columnar-formatting that is not available by using only the 'du' command. ---------------------- IMPLEMENTATION METHODS: This 'subdir-sizes, one level, size-sorted' utility script can be accessed via a drawer in a 'spacetools' toolchest command-utility --- via a drawer name like 'Show SUBDIR-SIZES 4aDir@AnyHost (ONE level,SIZE-SORT)'. You can get to this drawer via the site SGI toolchest-drawers: SGI Toolchest -> HandyTools -> AnyHost - SpaceTools (Files) -> Show SUBDIR-SIZES 4aDir@AnyHost (ONE level,SIZE-SORT) OR you can implement the script (or 'spacetools') as - a command alias, via your $HOME/.profile file; - a desktop icon, via the 'Find, File QuickFind' tool drawers; - a drawer in the SGI toolchest, via your $HOME/.auxchestrc file. ........................................................................ " >> $OUTLIST ######################################################################## ## SHOW REPORT, with option to print. ######################################################################## . $FEDIR/scripts/shofil $OUTLIST ## export HOST_ID DIRNAME ## ## done ######################################################################## ## END OF prompting loop, for directory name. ######################################################################## ## RETURN TO DIRECTORY PROMPT. ########################################################################