Data Warehousing

Data warehousing refers to the organization and assembly of data created from day-to-day business operations. Data warehousing enables a user to retrieve data from online transaction processing (OLTP) and online analytical processing (OLAP), and allows for the storage of that data in a format that can be read and analyzed. The integrated information, which is stored in a data warehouse, can be analyzed and queried to help management make more informed business decisions.

The idea of data warehousing dates back to the early 1980s. At that time, a popular system that utilized the concept of data warehousing was the relational database, which was run on minicomputers and used for OLTP functions. Quite often, relational database systems operated networks such as automated teller machines. As technology continued to advance, several key factors—including changing business trends, the evolution of the global economy, enterprise resource planning (ERP), business process reengineering (BPR), increased focus on customer needs, and the rise of e-business—led to the development of data warehouses in the 1990s.

Run on powerful client/server networks, not only can data warehouses read OLTP, they are equipped to translate OLAP as well. The development of data warehousing enabled companies to gather several types of information concerning business transactions, as well as important analytical data. In Contract Professional Magazine, Pam Derringer wrote that as a knowledge tool, "data warehousing restructures massive volumes of unorganized data into new formats that can be queried for answers to individual questions or sliced and diced for analytical trend reports."

Two important types of information in data warehousing are operational and informational data. Operational data—the data businesses use on a day-to-day basis—is stored, retrieved, and updated by an OLTP system. This type of data normally is stored in a relational database. Informational data is operational data that has been manipulated and summarized, and is what makes up a data warehouse. In the process of data warehousing, informational data is created from operational data and systems by using transformation or propagation tools. This process is necessary to ensure that the information can be retrieved in an easy and time-efficient manner. Multidimensional analysis, or OLAP, is the desired result of data warehousing. It allows a user to analyze large amounts of data regarding things like sales, products, time periods, and geographies. The multidimensional data structure, or data warehouse, allows for the storing and analyzing of such data.

Another component to data warehousing is meta-data, which is made up of technical data and business data. Technical data is used by system administrators and contains information about the data warehouse itself. Business data, on the other hand, is what an analyst might be searching for in order to forecast sales or predict trends. Data mining tools are then used to interpret data and find patterns within the information. For example, a retail company might use data warehousing and data mining to find relationships in purchasing patterns and to gather information about its customers.

Implementing a data warehouse structure within a company can be a costly and time-intensive process. These barriers have led to the development of data marts—smaller versions of data warehouses that are more specialized to serve a specific department and/or cover a specific topic. Traditional data warehouses are measured in gigabytes and terabytes, whereas the more compact data marts are measured in megabytes. Smaller companies, with more limited budgets, often opt for this type of data structure.

With the rise of e-commerce, data warehousing is becoming a key business component in the operation of both brick-and-mortar companies as well as dot-com ventures. The evolution of customer relationship management (CRM), the increasing popularity of the Internet, and the formation of online marketplaces and business-to-consumer companies such as Amazon.com and other e-tailers, have increased the demand for data warehousing solutions. While implementing data warehousing can be very costly, a study conducted by International Data Corp. concluded that firms utilizing data warehouse systems saw an average return on investment of nearly 400 percent over three years. Each year, as the billions of dollars spent online for products and services increases, businesses are turning to advanced data management solutions to analyze information, make forecasts, look for trends, identify shopper characteristics, and control inventory. This has increased the demand for data warehousing and, therefore, increased competition between solution-based companies. According to DM Review, the top ten business intelligence vendors—those offering e-business, CRM, and data warehousing solutions—at the advent of the twenty-first century were SAS, NCR Corp., Oracle Corp., Computer Associates, Cognos Corp., MicroStrategy Inc., Microsoft Corp., IBM, Informix Business Solutions, and Hyperion.