#!/bin/ksh
##
## SCRIPT:  space_topNsubdirs_apps_bygui
##
## Where: in $FEDIR/scripts   where $FEDIR=/apps/nns_com/fea
##
##############################################################################
## PURPOSE: For DIR_TOP="/apps", this script
##          shows the top N sub-directories (size-wise)
##          of the engprd00:$DIR_TOP directory
##          --- where N can passed in var $1; otherwise defaulted to 500.
##
##############################################################################
## FEATURES OF THIS SCRIPT:
##
##       -  This script uses 'grep' and 'head' commands to extract
##          the top N data-lines from a 'raw' weekly cron-report file
## 
##              $DIR_REPORTS/spac_apps_dirs_siz.lis
##          or
##              $DIR_REPORTS/spac_apps_dirs_siz.lis_prev
##          or
##              $DIR_REPORTS/spac_apps_dirs_siz.lis_prev2
##           
##          where DIR_REPORTS="/usr/people/ideasadm/cleandir".
##
##          The report files contain two columns of data --- subdir-names
##          and sizes.  The 'raw' listings are generated by a separate (cron)
##          script that uses the 'du' command.
##
##      -   The size-numbers in the 'raw' listing (1K blocks) are
##          re-formatted to Gigabytes, by this script, for readability.
##
##      -   This script gives the option of sorting the data-lines by
##          subdir-sizes or subdir-names. 
##
##      -   This script puts the report into a temp-file
##          and shows the temp-file with the site 'shofil' Tk GUI
##          (the site 'xpg' utility).
##
##############################################################################
## DATA SOURCE:
##
##    The report file of 'du' data for all sub-directories of
##    directory $DIR_TOP, namely
##
##           $DIR_REPORTS/spac_apps_dirs_siz.lis
##
##    is generated --- on the SGI File Server engprd00 ---
##    by a script (run in a weekly cron job):
##
##                             /apps/ideas/cron/creat_datadirsiz_lists .
##
##    Previous reports are moved to '_prev' and '_prev2' files.
##
##    See that script for details.
## 
##           The cron script is meant to be called by a 'ideasadm'
##           crontab job on 'engvis00' --- say, weekly.
##
##    Also, see the crontab file
##                       /apps/ideas/cron/CRONTAB_IDEASADM_engprd00 
##
##############################################################################
## CRON-REPORT DATA CONTENT:
##
##          The cron-script makes 'header' & 'trailer' lines of
##          the cron-report with a '#' character in column 1.   So ...
##
##          The data can be separated from the 'header' & 'trailer' lines
##          (and the null lines) using 
##
##                  grep -v "^#" $FILEIN | grep -v "^$"
##
##############################################################################
## HEADING OF THE CRON REPORT INPUT FILE (25 lines):
##
# ..................... 2001 Dec 06  Thu  00:34:40AM ......................
# 
# DISK USAGE (in 1K blocks) in SUB-DIRECTORIES of directory /engprod/apps
#                                              mounted from  engprd00
# 
# Uses commands like:
# 
#            du -kl /engprod/apps    AND    sort -k1nr
#
#           Sorts by sub-directory size.
#           Largest sub-directories at the top.
# 
#           Total usage (top directory) is shown on first line.
#
#           Because of the many subdirectories of /engprod/apps, this report is
#           generated --- at the server engprd00 --- in a weekly cron job:
#
#                             /apps/ideas/cron/creat_datadirsiz_lists .
#
#           See that script for details.
#
# Disk
# usage     Subdirectory
# (1K blks)    name
# --------  ---------
##############################################################################
## CALLED BY:
##            the script   'spacetools'  in  $FEDIR/tkGUIs
##
##            actually, by the file 'spacetools.chestdef'  in  $FEDIR/tkGUIs
## 
##  The toolchest-drawer hierarchy is something like
##
##    SGI Toolchest ->
##        HandyTools ->
##          AnyHost - SpaceTools (Files)
##
##############################################################################
## MAINTENANCE HISTORY:
## Written by: Blaise Montandon 07dec2001 Based on script
##                                              'space_topNsubdirs_data_bygui'
##                                                in $FEDIR/scripts.
##
## Updated by: Blaise Montandon 13oct2003 Add an 'xconfirm' prompt, to
##                                        allow for using the '_prev' or
##                                        '_prev2' reports.
##                                        
## Updated by: Blaise Montandon 14oct2003 Add xconfirm-prompt to allow for
##                                      sort by dirname instead of dirsize.
##                                        
##############################################################################

#############################################################################
## Set names (hostnames and dirnames) for reports.
#############################################################################

# THISHOST=`hostname`

MAIN_SGI_SERVER="engprd00"

DIR_TOP="/apps"

ADMIN_ID="ideasadm"

#############################################################################
##  Set input files directory --- the directory where the cron job
##  keeps the latest, 'prev', and 'prev2'  directory-sizes 'du' data files.      
#############################################################################

DIR_REPORTS="/usr/people/ideasadm/cleandir"

#############################################################################
## Set scripts pathname in case this toolchest is not started from
## the nnsFEAmenu system --- and a $FEDIR utility script (or help) is needed.
#############################################################################

if test "$FEDIR" = ""
then
   FEDIR="/apps/nns_com/fea"
   export FEDIR
fi


############################################################
##  Set NRECS var from $1 -- if $1 is not null.
############################################################

NRECS=500

if test ! "$1" = ""
then
  NRECS="$1"
fi


#############################################################################
## PROMPT for a file to use (latest, previous, or previous2)
## --- in the var FILEIN.
#############################################################################

  FILEIN_1="spac_apps_dirs_siz.lis"
  FILEIN_2="spac_apps_dirs_siz.lis_prev"
  FILEIN_3="spac_apps_dirs_siz.lis_prev2"

  FULLFILEIN_1="$DIR_REPORTS/$FILEIN_1"
  FULLFILEIN_2="$DIR_REPORTS/$FILEIN_2"
  FULLFILEIN_3="$DIR_REPORTS/$FILEIN_3"


   WHICH_FILE=`xconfirm -c \
   -header "Biggest $NRECS  ${MAIN_SGI_SERVER}:$DIR_TOP  directories, via batch 'du' run" \
           -B Cancel  -b Prev2  -b Prev -b Latest  \
   -t "FROM WHICH REPORT, on sub-directory sizes of directory $DIR_TOP ," \
   -t "do you want to extract and format the top $NRECS lines:" \
   -t "" \
   -t "  - Latest      (ordinarily from last Wed. night run)" \
   -t "or" \
   -t "  - Prev        (ordinarily from a week earlier)" \
   -t "or" \
   -t "  - Prev2       (ordinarily from 2 weeks earlier)" \
   -t "" \
   -t "  Latest,   Prev,   Prev2,  or  Cancel?" \
   -t "-------------------------------------------------" \
   -t "In other words, which file do you want to use?" \
   -t "  $FILEIN_1" \
   -t "or" \
   -t "  $FILEIN_2" \
   -t "or" \
   -t "  $FILEIN_3" \
   -t "" \
   -t "from directory $DIR_REPORTS"`

   if test "$WHICH_FILE" = "Cancel"
   then
      exit
   fi

   if test "$WHICH_FILE" = "Latest"
   then
      FILEIN="$FULLFILEIN_1"
   fi

   if test "$WHICH_FILE" = "Prev"
   then
      FILEIN="$FULLFILEIN_2"
   fi

   if test "$WHICH_FILE" = "Prev2"
   then
      FILEIN="$FULLFILEIN_3"
   fi


##############################################################################
## PROMPT USER WHETHER SORT should be by dirname or dirsize.
##############################################################################

SORT_TYPE=`xconfirm -c \
     -header "SubDir Size Report Utility - ${MAIN_SGI_SERVER}:$DIR_TOP" \
     -b NAME -B SIZE \
     -t "Do you want to see the lines of the SubDir-Sizes Report sorted by" \
     -t "subdir-SIZE or by subdir-NAME?" \
     -t "" \
     -t "            SIZE or NAME" \
     -t "" \
     -t "[Suggestion: Try SIZE first.  After seeing biggest, try NAME.]" \
     -icon warning`

# SORTCMD=""
# if test "$SORT_TYPE" = "NAME"
# then
#      SORTCMD=" | sort -k2"
# fi


###################################################################
## PREPARE REPORT FILENAME -- in $OUTLIST.
###################################################################

. $FEDIR/scripts/set_localoutlist


###################################################################
## PUT TOP (time-stamp) LINE OF CRON REPORT HEADING -- in $OUTLIST.
###################################################################

head -1 $FILEIN > $OUTLIST

###################################################################
## PUT COLUMN HEADING -- in $OUTLIST. 
###################################################################

      echo "\

DISK USAGE of the LARGEST $NRECS SUB-DIRECTORIES

                                 of directory '$DIR_TOP'
                                 at SGI host $MAIN_SGI_SERVER
SORTED BY:  $SORT_TYPE

The following data was gathered by the command:

    du -kl /engprod$DIR_TOP

in a separate, many-minute job.


         Because of the many sub-directories of $DIR_TOP (over 20,000)
         and the fact that it takes MINUTES for 'du' to accumulate the sum
         of disk space used by the near-million files in these directories,
         this report is generated --- at the SGI File Server --- in a
         weekly cron job.

         Total usage (top directory) is shown on first line.

         (See further info at bottom of this report.)


Line   Disk usage      Name of
Number (GigaBytes)     SubDirectory
-----  -------------   ---------------------------------------------------
       TerGig.MegKil
         |  |   |  |" >>  $OUTLIST


## Actual commands used in the cron script:
##
##    du -kl $DIR_TOP    AND    sort -k1nr
##


###################################################################
## PUT REPORT HEADING -- in $OUTLIST.
###################################################################

   ## FOR TESTING:
   #  set -x 

   if test "$SORT_TYPE" = "SIZE"
   then
      grep -v "^#" $FILEIN | grep -v "^$" | head -$NRECS  | nl | \
      awk '{printf ("%5d  %13.6f    %s\n", $1, $2/1000000,  $3 )}' \
      >> $OUTLIST
   else
      grep -v "^#" $FILEIN | grep -v "^$" | head -$NRECS | sort -k2 | nl | \
      awk '{printf ("%5d  %13.6f    %s\n", $1, $2/1000000,  $3 )}' \
      >> $OUTLIST
   fi

   ## FOR TESTING:
   #   set -

   ###########################################
   ## ADD A TRAILER TO THE REPORT-FILE.
   ###########################################

      echo "\
         |  |   |  |
       TerGig.MegKil
-----  -------------   ---------------------------------------------------
Line   Disk usage      Name of
Number (GigaBytes)     SubDirectory

..........................................................................

     The output above was generated by the script

                $0

     which extracted the top $NRECS data-lines from the report file in

           $FILEIN

     The data-lines are shown sorted by $SORT_TYPE.

     The sizes (1K blocks) are re-formatted to Gigabytes for readability.

----------------------------------------------------------------------------

NOTE1: DATE OF THIS DATA-SNAPSHOT

       The date-time-stamp at the top of this report is from the
       cron report.  In other words, that date-time is the time
       that the cron job started.

----------------------------------------------------------------------------

NOTE2: DATA EXTRACT OPTIONS

       This (nicely-formatted) list of $NRECS sub-directories --- 
       can be BROWSED  and  SEARCHED  and  EXTRACTED  [and PRINTED]
       with site textfile-browse utility 'xpg' ---
                                  or browsed & searched with 'nedit -read'.

       To look at the 'raw' (unformatted) list, with ALL the sub-directories,
       you can paste the command

  xpg  $FILEIN      

       at a command prompt.

----

       With 'xpg', you can BROWSE (this list or the 'raw' list) by 

          - 'scrolling' with the MOUSE and the scroll button

       or by

          - 'paging' and 'scrolling' with KEYS
                     (Page-Down, Page-Up, End, Home, down-arrow, up-arrow).

----

       AND, more importantly,
       you can EXTRACT by entering a search string, like '$DIR_TOP/ABAQUS',
       and clicking the  'Show All Matches' button.

----------------------------------------------------------------------------

NOTE3: TOUCHING UP THIS REPORT

       If this extract-and-sort is not quite to your liking,
       you can EDIT this report file (or a copy of it) --- removing lines
       and re-grouping remaining lines and inserting blank lines.

       Then you can (browse and) PRINT the result with 'xpg'.

----------------------------------------------------------------------------

An example path to this query via the site toolchests-drawers:

      SGI Toolchest ->
          HandyTools ->
            AnyHost - SpaceTools (Files) ->
               Top $NRECS '$DIR_TOP' SUBDIRS (from weekly 'du' snapshot)

----------------------------------------------------------------------------
" >>  $OUTLIST


# NOTE4: DATA ACCURACY
#
#        Because the weekly cron report is run as userid '$ADMIN_ID'
#        and '$ADMIN_ID' does not have permission to all $DIR_TOP
#        directories,
#
#        some of these directory-space totals may be
#        under-reported. (?)
#        
#        AND/OR
#        
#        some of the sub-directory totals may be un-reported. (?)
#
#        But most of the line-figures are 95% to 100% of actual directory
#        space totals.


###################################################################
## SHOW the 'top N subdirs' report.
###################################################################

$FEDIR/scripts/shofil  $OUTLIST