Intro to Data Mining & Data Warehousing

There are many applications for data warehousing and data mining in the world today and not just limited to the business applications. This paper will go over a brief history of data mining and data warehousing, what each is, and how they are both used in a business setting to gather valuable information from raw data.

Evolution of Data Warehousing

Data warehousing became a specific type of database during the late 1980’s and early 1990’s. The need for data warehousing became apparent when reporting functions of operational databases were no longer sufficient as the amount of data held in these databases grew to hold very large amounts of data. Data warehouses differ today than when they first started becoming popular in the early 1990’s. “As technology improved (lower cost for more performance) and user requirements increased (faster data load cycle times and more features), data warehouses have evolved through several fundamental stages:

Offline Operational Databases - Data warehouses in this initial stage are developed by simply copying the database of an operational system to an off-line server where the processing load of reporting does not impact on the operational system's performance.

Offline Data Warehouse - Data warehouses in this stage of evolution are updated on a regular time cycle (usually daily, weekly or monthly) from the operational systems and the data is stored in an integrated reporting-oriented data structure.

Real Time Data Warehouse - Data warehouses at this stage are updated on a transaction or event basis, every time an operational system performs a transaction (e.g. an order or a delivery or a booking etc.)

Integrated Data Warehouse - Data warehouses at this stage are used to generate activity or transactions that are passed back into the operational systems for use in the daily activity of the organization.” (A History of Data Warehousing)

It is interesting to see how the data warehouse techniques have evolved since...