FLIGHTSTATS – Sneak Peek

So here is something that has kept me busy for a lot of days (more than one month to be precise). And it has been sheer fun and learning since I learnt a lot of new things on the way and did things and encountered errors that would not have been possible otherwise. Well, as I learnt in my college days, the only way to learn new things is to get down in the trenches and get your hands dirty. (As a side, I remember a unique learning experience in college when I had to put printf statements in a thousand line C program to debug a segmentation fault because of array index out of bounds.)

So just to give a glimpse of the system. The system is called as FLIGHTSTATS and stores openly available data of domestic US flight travels for 21 years in a Hyperion Essbase ASO application. This is just the first cut of the application and data has been loaded only to one subset of the intersection and we have many more data snippets to load. The architecture of the system is as shown in the below snapshot.

The flat file is a combined repository of data and metadata that has been created.

• This flat file data is then pushed into a relational system.
• The relational system consists of views for building the hierarchies and views that define specific data load intersections for the system.
• The metadata and data then gets pushed into an ASO application using rule files.
• Reports scripts are built on top of the ASO application to extract data.
• Finally, visualization of the data takes place using Oracle Data Visualization Desktop.

Just to give a list of the tools and technologies used to get the system up and running is as follows:

• Oracle Data Integrator
• Hyperion Essbase
• SQL
• Oracle Data Visualization Desktop
• FDMEE

In the coming blogs, I plan to cover more of this as well as cover some unique trends and patterns which reveal themselves as we examine a large amount of data. Stay tuned for more.