Tuesday, October 03, 2006

The Tao of Data

The more I work with numbers, the more I feel awed by the power and mystic of data. The shape and form raw numbers take on the trigger of analysis, undoubtedly makes them a powerful source of competitive advantage for any entity.

It was love at first sight for me when I understood the inherent nature of numbers. The fact that any random set of data falls into similar shapes like a normal distribution gave them almost a spiritual tinge. Like the “Tao” or ‘the way’ in Chinese philosophy, which professes that everything follows ‘the way’ of nature and bliss is obtained by one which espouses ‘the way’, numbers so beautifully follow the Tao.

Whether the data is quantitative in nature or a qualitative set of data metamorphosed into quantitative set via innovative techniques, it speaks volumes if we are tuned in and are listening.

The intriguing essence of data further gets interesting when we move to the next level and watch the interaction of one set of data with another or what we technically call as ‘correlation’. This ostensibly simple phenomenon is the backbone of all advanced data analysis techniques. How one set relates to another predicts the nature of events through discriminant analysis, shapes the visage of new entities through conjoint analysis and makes it easier for us mortals to handle the huge magnitude of data by decreasing its span through Factor Analysis. The techniques are myriad, but as the Gita professes ‘One Source’ of all beings, the core of understanding and using data is ‘correlation’.

Hindu philosophy delineates that to find that connection with the ultimate One, we need to tread a long and difficult path, what is specifically named as ‘Tapas’ or ‘the heat’ so that we come out of the effort spiritually evolved. Data collection is such a path a dedicated analyst treads to understand the true nature of data. Undoubtedly, collecting data is the most difficult and biggest bottle neck in all efforts for unraveling the mystery of any phenomenon. Unfortunately, the strength of interpretation depends on the quantum of the data, making it what Goldratt would describe in his ‘Theory of Constraints’ as the one constraining process limiting the overall process. If the analyst is able to overcome this bottle neck through constant efforts, the rewards are deliciously gratifying.

In spite of the beautiful potential and inherent nature of data to define the truth behind all phenomena, most of the entities sit on piles of numbers, not realizing the leverage they can exercise to catapult into a higher orbit. Like the omnipresence of the higher powers waiting for us to make the effort of connection, using a set of numbers to discover the truth can take us to highs that only true yogi experiences on enlightenment.

0 Comments:

Post a Comment

<< Home