In-place Hyperion Essbase Application log formatter using shell script – Law of unintended consequences

In this blog, I would be talking about designing an in-place Hyperion Essbase application log formatter using shell script. I actually wanted to try this out since it reduces the space complexity. In my previous blog, I had designed one that keeps the existing log as is and generates a second parsed log file. (http://exploitsinhyperion.blogspot.in/2017/01/essbase-log-file-parser-shell-script.html). In this one, I try to make changes to the existing log file.

This log formatter assumes that you are using a Linux/Unix box and you have some knowledge of shell scripting.
The below snapshot shows the Essbase application log file.

The file present in the backend is as shown in the below snapshot. The file is called REVENUE.log

A simple cat on this file is shown in the next snapshot.

The first thing that I do before running the parser is to stop the Essbase application. This is because the Essbase process (ESSSVR) would be holding a lock on this file and it would not be possible to manipulate the file in-place if this is open.

The application has now been stopped as shown in the next snapshot.

I now run the shell script to parse the Essbase application log in place. The shell script takes the full path of the Essbase application log file as a parameter and parses the same. The script running in debug mode is shown in the next snapshot.

The contents of the file after the script has run is as shown in the next snapshot.

I now start up the application and view the log file. Observe that the new records are not formatted in a single row. 

The previous records from the log file are however in a single row which makes it easy for other processes to manipulate the records.

A snapshot showing both parsed and non-parsed records existing in the same log file is shown below.

The next couple of snapshots show me testing the script a couple more times and each time it works as expected.

 
Understanding the law of unintended consequences:
Well, this was the unintended consequence of the script. When I finished running through the script, I tried to see if I can display the records which start at a particular start date. But it looks like the internal code that is used for date filtering is not able to read the contents in a straight line… Testing this use case is shown in the below snapshots…

The code that I have created is present below:

# Script designed by Sibin Jose for Hyperion Essbase log file parsing
# This script is designed to be in place

echo "Hyperion Essbase log parser"

if [ $# -eq 0 ]
then
 echo "Incorrect possitional parameters... Usage: log_parser.sh <PATH OF LOG FILE>"
exit 1
fi

export ESSBASE_LOG_FILE=$1
echo ${ESSBASE_LOG_FILE}

#export PARSED_ESSBASE_LOG_FILE=`echo ${ESSBASE_LOG_FILE} | sed 's/.log/_parsed.log/g'`
#echo ${PARSED_ESSBASE_LOG_FILE}


sed -i 's/^\[/#\[/g' ${ESSBASE_LOG_FILE}

tr '\n' ' ' < ${ESSBASE_LOG_FILE} > working_temp.log.txt

mv working_temp.log.txt ${ESSBASE_LOG_FILE}

sed -i 's/#\[/\n\[/g' ${ESSBASE_LOG_FILE}

#sed -i 's/###/\n/g' ${ESSBASE_LOG_FILE}

#sed -i 's/)/)\t/1' ${ESSBASE_LOG_FILE}


#tr '\n' ' ' | sed 's/###/\n/g' | sed 's/)/)\t/1'  > parsed_essbase_log.txt

echo "THe script has finished executing...Please check the folder for parsed_essbase_log.txt file"