#!/bin/ksh ## ## SCRIPT: host2host_diff_two_dirs_showBYgui ## ## Where: in $FEDIR/scripts where $FEDIR=/apps/nns_com/fea ## ############################################################################## ## PURPOSE: ## For a given pair of Host & Directory names --- Host1:RefDir1 ## and Host2:RefDir2 ( passed in vars $1, $2, $3, $4 ), ## this script creates a REPORT of ## DIFFERENCES BETWEEN FILE-SIZES AND FILE-NAMES ## in directory1: Host1:RefDir1 ## and directory2: Host2:RefDir2. ## ## This utility uses the 'sdiff' command. (or could use 'diff') ## ----- ## This utility is meant mainly as the 2nd of 3 comparison/difference ## utilities to look for differences in two huge directory ## hierarchies. ## ## The three utilities ## - 'FilCnt,AllSubdirs' ## - 'Diff-Listboxes' ## - 'Diff-Two-Files' ## are briefly described in the help document of the 'xrcp' utility ## /apps/nns_com/fea/helps/host2host_fileman.hlp ## ## This utility ## - 'Diff-Listboxes' ## is meant to be used after the 'first step' utility ## - 'FilCnt,AllSubdirs' ## which reports summary file counts for ALL sub-directories at ## ALL LEVELS under a given directory. That 'FilCnt,AllSubdirs' ## report utility is actually run twice -- for two ## similar directories. ## ## If differences in sub-directory file-counts are found, in that ## 'first step' comparison, ## then one can 'zero in' on specific sub-directories where there ## are differences in counts, with this utility --- to look at ## differences in two lists ## of file-names & file-sizes, for the 2 differing sub-directories. ## ## If files that should be equal in size are different in size, one ## can 'zero in' further, if necessary, doing a 'diff' on the two ## files. ## ############################################################################## ## PROCESSING LOGIC: ## 1) The command 'ls -lAp ' is run for the two ## Host:Directory pairs. Output is capture in vars LS_OUT1 and ## LS_OUT2. ## ## 2) The LS_OUT1 & LS_OUT2 vars are echoed into a grep-tail-awk ## pipe and the results are put in two temporary files, on ## 'this host', the host running this script. ## ## The 'results' are two formatted columns: ## file-sizes & relative-file-names. ## ## 3) A report headter is built in a third temporary 'report' file. ## ## 4) An 'sdiff' of the two temporary files is concated to the ## 'report' file. ## ## 5) The report file is shown with 'xpg', running on 'this host'. ## ##--------------------------------------------------------------------------- ## REMOTE HOST NOTE: ## If either 'Host1' or 'Host2' is a remote host, the corresponding ## LS_OUT# var is created by using 'rsh ' to run the ## 'ls -lAp ' command. ## ## The 'ls' output to stdout is captured by 'command substitution' ## in the LS_OUT# var. ##--------------------------------------------------------------------------- ## ## This script puts the report (stdout from the 'find' command) ## into a temp-file (on the host running this script) ## and shows the report file with shofil_ver2.tk (the 'xpg' utility). ## ## This script checks for 4 arguments and checks that the pair ## host#:directory# exist. If any of these conditions are not satisfied, ## this script pops an 'xconfirm' error msg and exits. ## ############################################################################## ## CALLED BY: ## host2host_fileman.tk in $FEDIR/tkGUIs ############################################################################## ## ## CALL FORMAT (example): ## ## $FEDIR/scripts/host2host_diff_two_dirs_showBYgui \ ## engprd00 /apps/autobatch \ ## engprd00 /apps/division_2000i2_aix/autobatch ## ############################################################################## ## MAINTENANCE HISTORY: ## Written by: Blaise Montandon 05dec2001 Based on files-info script ## 'host2host_filecnt_4subdirs_alllevs_locrmt_showBYgui' ## in $FEDIR/scripts ## (includes rmt-dir-existence check) ## and test script ## 'test_diff_two_dirs' ## in $FEDIR/Dscrtest. ## Updated by: Blaise Montandon 05dec2001 ############################################################################## THISHOST=`hostname` ############################################################################# ## Set scripts pathname in case this toolchest is not started from ## the nnsFEAmenu system --- and a $FEDIR utility script (or help) is needed. ############################################################################# if test "$FEDIR" = "" then FEDIR=/apps/nns_com/fea export FEDIR fi ############################################################ ## Assure that the DISPLAY variable is set. ## Set to do X-displays on 'this host'. ############################################################ # # if test "${DISPLAY}" = "" # then # DISPLAY="$THISHOST:0" # export DISPLAY # fi ############################################################ ## If this is a remote login, ## set X-display to the 'remote host'. ############################################################ # if test ! "$REMOTEHOST" = "" # then # REMOTEHOST1=`echo $REMOTEHOST | cut -d. -f1` # DISPLAY="$REMOTEHOST1:0" # export DISPLAY # fi ############################################################ ## Set dirHOST & DIRNAME vars using $1 & $2 -- and check ## that they are not null. ############################################################ ## FOR TESTING: # set -x dirHOST1="$1" DIRNAME1="$2" dirHOST2="$3" DIRNAME2="$4" if test "$dirHOST1" = "" then xconfirm -c -header "NO_INPUT_warning" \ -B Dismiss \ -t "The script" \ -t " $0" \ -t "needs 4 inputs:" \ -t " hostname1 dirname1 hostname2 dirname2 ." \ -t "" \ -t "hostname1 was NOT specified. Exiting..." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit fi if test "$DIRNAME1" = "" then xconfirm -c -header "NO_INPUT_warning" \ -B Dismiss \ -t "The script" \ -t " $0" \ -t "needs 4 inputs:" \ -t " hostname1 dirname1 hostname2 dirname2 ." \ -t "" \ -t "dirname1 was NOT specified. Exiting..." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit fi if test "$dirHOST2" = "" then xconfirm -c -header "NO_INPUT_warning" \ -B Dismiss \ -t "The script" \ -t " $0" \ -t "needs 4 inputs:" \ -t " hostname1 dirname1 hostname2 dirname2 ." \ -t "" \ -t "hostname2 was NOT specified. Exiting..." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit fi if test "$DIRNAME2" = "" then xconfirm -c -header "NO_INPUT_warning" \ -B Dismiss \ -t "The script" \ -t " $0" \ -t "needs 4 inputs:" \ -t " hostname1 dirname1 hostname2 dirname2 ." \ -t "" \ -t "dirname2 was NOT specified. Exiting..." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit fi ############################################################ ## Check that host1:dir1 exists. ## If not, exit with msg. ############################################################ ## Directory-Existence-Check is broken into two ## mutually-exclusive cases: ## 1) 'specified-host' is 'this-host' ## 2) 'specified-host' is NOT 'this-host' (the case of ## a 'remote-host' directory) ############################################################ ############################################################ ## 1) 'specified-host' is 'this-host' ############################################################ if test "$dirHOST1" = "$THISHOST" then if test ! -d $DIRNAME1 then xconfirm -c -header "DIRECTORY_NOT_FOUND_warning" \ -B Dismiss \ -t "The directory $DIRNAME1 " \ -t "is NOT KNOWN TO host $dirHOST1." \ -t "" \ -t "*EXITING*." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit # break # continue fi ## END OF if test ! -d $DIRNAME1 fi ## END OF if test "$dirHOST1" = "$THISHOST" ############################################################ ## 2) 'specified-host' is NOT 'this-host' ############################################################ if test ! "$dirHOST1" = "$THISHOST" then RETcode=`rsh $dirHOST1 \ "if test -d $DIRNAME1 ; then echo 1 ; else echo 0 ; fi"` if test $RETcode = 0 then xconfirm -c -header "DIRECTORY_NOT_FOUND_warning" \ -B Dismiss \ -t "The directory $DIRNAME1 " \ -t "is NOT KNOWN TO host $dirHOST1." \ -t "" \ -t "*EXITING*." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit # break # continue fi ## END OF if test $RETcode = 0 fi ## END OF if test ! "$dirHOST1" = "$THISHOST" ############################################################ ## Check that host2:dir2 exists. ## If not, exit with msg. ############################################################ ## Directory-Existence-Check is broken into two ## mutually-exclusive cases: ## 1) 'specified-host' is 'this-host' ## 2) 'specified-host' is NOT 'this-host' (the case of ## a 'remote-host' directory) ############################################################ ############################################################ ## 1) 'specified-host' is 'this-host' ############################################################ if test "$dirHOST2" = "$THISHOST" then if test ! -d $DIRNAME2 then xconfirm -c -header "DIRECTORY_NOT_FOUND_warning" \ -B Dismiss \ -t "The directory $DIRNAME2 " \ -t "is NOT KNOWN TO host $dirHOST2." \ -t "" \ -t "*EXITING*." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit # break # continue fi ## END OF if test ! -d $DIRNAME2 fi ## END OF if test "$dirHOST2" = "$THISHOST" ############################################################ ## 2) 'specified-host' is NOT 'this-host' ############################################################ if test ! "$dirHOST2" = "$THISHOST" then RETcode=`rsh $dirHOST2 \ "if test -d $DIRNAME2 ; then echo 1 ; else echo 0 ; fi"` if test $RETcode = 0 then xconfirm -c -header "DIRECTORY_NOT_FOUND_warning" \ -B Dismiss \ -t "The directory $DIRNAME2 " \ -t "is NOT KNOWN TO host $dirHOST2." \ -t "" \ -t "*EXITING*." \ -icon warning > /dev/null # -font $CONFIRM_FONT \ exit # break # continue fi ## END OF if test $RETcode = 0 fi ## END OF if test ! "$dirHOST2" = "$THISHOST" ################################################################### ## PREPARE REPORT FILENAME -- in $OUTLIST. ################################################################### . $FEDIR/scripts/set_localoutlist OUTLIST1="${OUTLIST}_1" OUTLIST2="${OUTLIST}_2" ################################################################### ## PUT 'ls -lAp' output for Host1:Dir1 in var LS_OUT1. ################################################################### if test "$dirHOST1" = "$THISHOST" then ## FOR TESTING: # set -x LS_OUT1=`ls -lAp $DIRNAME1` ## FOR TESTING: # set - else ## FOR TESTING: # set -x LS_OUT1=`rsh $dirHOST1 ls -lAp $DIRNAME1` ## FOR TESTING: # set - fi ## END OF if test "$dirHOST1" = "$THISHOST" ################################################################### ## PUT 'ls -lAp' output for Host2:Dir2 in var LS_OUT2. ################################################################### if test "$dirHOST2" = "$THISHOST" then ## FOR TESTING: # set -x LS_OUT2=`ls -lAp $DIRNAME2` ## FOR TESTING: # set - else ## FOR TESTING: # set -x LS_OUT2=`rsh $dirHOST2 ls -lAp $DIRNAME2` ## FOR TESTING: # set - fi ## END OF if test "$dirHOST2" = "$THISHOST" ################################################################### ## PUT LS_OUT1 & LS_OUT2 output (after re-formatting) into ## temp files $OUTLIST1 & $OUTLIST2. ################################################################### echo "$LS_OUT1" | grep -v '/$' | tail +2 | \ awk '{printf ("%15.6f %s\n", $5/1000000, $9 )}' \ > $OUTLIST1 echo "$LS_OUT2" | grep -v '/$' | tail +2 | \ awk '{printf ("%15.6f %s\n", $5/1000000, $9 )}' \ > $OUTLIST2 ########################################################################## ########################################################################## ## PREPARE THE REPORT -- by running the 'sdiff' command on 'this host' ## with stdout directed to $OUTLIST. But first, prepare a report heading. ########################################################################## ########################################################################## COLWIDTH=93 FILE_LEN=`expr $COLWIDTH - 19` ####################################################### ## PREPARE THE 'DIFF-TWO-DIRS' REPORT HEADING. ####################################################### echo " ********************* `date '+%Y %b %d %a %T%p %Z'` ****************** DIFFERENCE BETWEEN FILE-SIZES AND FILE-NAMES in Directory1: ${dirHOST1}:$DIRNAME1 and Directory2: ${dirHOST2}:$DIRNAME2 for 'NON-DIRECTORY' FILES. (I.e. we do NOT show SUB-DIRECTORIES differences, for sub-directories immedidately under these two directories.) (File-sizes are in Megabytes -- to 6 decimal places, i.e. down to the byte.) File differences are indicated by '<' or '>' or '|' characters between the directory lists. Filenames will be truncated after about $FILE_LEN characters. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- " > $OUTLIST ############################################# ## ADD THE DIRECTORY NAMES JUST ABOVE THE COLUMNS. ############################################# echo "Directory1======>${dirHOST1}:$DIRNAME1 \ Directory2======>${dirHOST2}:$DIRNAME2" | \ awk '{printf ("%-93s %-93s\n", $1, $2 )}' \ >> $OUTLIST echo " --FILE-SIZE-1-- --------------FILE-NAME-1--------------------------------------------------- --FILE-SIZE-2-- --------------FILE-NAME-2-------------------------------------------------- " >> $OUTLIST ############################################# ## RUN THE 'sdiff' COMMAND -- 'LOCALLY' ## --- with stdout directed to $OUTLIST. ############################################# # diff $OUTLIST1 $OUTLIST2 >> $OUTLIST TOTWIDTH=`expr $COLWIDTH + $COLWIDTH + 4` # sdiff -s -w$TOTWIDTH $OUTLIST1 $OUTLIST2 >> $OUTLIST sdiff -w$TOTWIDTH $OUTLIST1 $OUTLIST2 >> $OUTLIST ########################################### ## ADD A TRAILER TO THE REPORT-FILE. ########################################### echo " --FILE-SIZE-1-- --------------FILE-NAME-1--------------------------------------------------- --FILE-SIZE-2-- --------------FILE-NAME-2-------------------------------------------------- ..................... `date '+%Y %b %d %a %T%p'` ............................ The output above was generated by the script $0 ------------- REPORT USAGE: This utility is meant mainly as the 2nd of 3 comparison/difference utilities to look for differences in two huge directory hierarchies. The three utilities - 'FilCnt,AllSubdirs' - 'Diff-Listboxes' - 'Diff-Two-Files' are briefly described in the help document of the 'xrcp' utility /apps/nns_com/fea/helps/host2host_fileman.hlp This utility - 'Diff-Listboxes' is meant to be used after the 'first step' utility - 'FilCnt,AllSubdirs' which reports summary file counts for ALL sub-directories at ALL LEVELS under a given directory. That 'FilCnt,AllSubdirs' report utility is actually run twice -- for two similar directories. If differences in sub-directory file-counts are found, in that 'first step' comparison, then one can 'zero in' on specific sub-directories where there are differences in counts, with this utility --- to look at differences in two lists of file-names & file-sizes, for the 2 differing sub-directories. If files that should be equal in size are different in size, one can 'zero in' further, if necessary, doing a 'diff' on the two files. Or, proceed to 're-equalize' the directory structure. ----------------------------- SELECTING LINES OF THE REPORT -- via a string-pattern: When you browse the report with the 'xpg' utility, you may want to extract/see just those lines that contain a certain string. To do that, you can use the 'ShowAllMatches' button with the plus-or-minus N lines option, with N set to zero (0). Example: To see all files containing the string '123', put the string '123' (without the quotes) in the String entry field of the 'xpg' GUI. Set N to 0 and click the 'ShowAllMatches' button. ----------------- PROCESSING METHOD: This report was generated by capturing the output of the Unix command 'ls -lAp' for directories ${dirHOST1}:$DIRNAME1 and ${dirHOST2}:$DIRNAME2 That output was re-formatted, extracting just the file-sizes and relative-file-names, into temporary files on $THISHOST . The 'sdiff' command was run on the two temporary files, and the report was assembled-on and presented-from $THISHOST . ---------------------------------------------------------------------------- " >> $OUTLIST ################################################################### ## SHOW the FILE-COUNTS report. ################################################################### $FEDIR/scripts/shofil $OUTLIST