Organisations are today suffering from a malaise of data overflow. The developments in the transaction processing technology has given rise to a situation where the amount and rate of data capture is very high, but the processing of this data into information that can be utilised for decision making, is not developing at the same pace. Data warehousing and data mining (both data & text) provide a technology that enables the decision-maker in the corporate sector/govt. to process this huge amount of data in a reasonable amount of time, to extract intelligence/knowledge in a near real time.
The data warehouse allows the storage of data in a format that facilitates its access, but if the tools for deriving information and/or knowledge and presenting them in a format that is useful for decision making are not provided the whole rationale for the existence of the warehouse disappears. Various technologies for extracting new insight from the data warehouse have come up which we classify loosely as "Data Mining Techniques".
Our paper focuses on the need for information repositories and discovery of knowledge and thence the overview of, the so hyped, Data Warehousing and Data Mining.