@GerryDales That's a rather incomplete view. Data "warehouses" have come and gone before. The change is this one is going to be in-house.
— Gabriel Malor (@gabrielmalor) February 7, 2014@GerryDales Well, it worked before.
— Gabriel Malor (@gabrielmalor) February 7, 2014Sometimes, 140 characters just isn't enough for a reply.
A cliche I hear in NFL broadcasts is that if a team has two first-string quarterbacks, then they really have none. Leaving aside the obvious counter-example of Joe Montana and Steve Young, it has more than a small grain of truth. It's overly glib, but I couldn't help but think that if the GOP has had data warehouses that have come and gone, they've really not had a good data warehouse.
Building out a robust data warehouse and the data mining operation to go along with it is a huge task involving many IT disciplines. The schema must be architected. The ETL (extraction, transformation, and load) processes must be built to populate the warehouse with data in usable form. The tools which will be used for querying must be deployed. The database must be tuned-- repeatedly, and iteratively. This is not a one or two month process. If, truly, there have been warehouses that have come and gone, then the wheel has been re-created several times over at a tremendous dollar cost and an even bigger lost mining opportunity cost.
An operation that is starting warehousing over at some cycle will fall behind another that built a robust warehouse and adapted it over time to integrate new data sources (hence a likely explanation for why what worked before has not been working recently) or to take advantage of the latest advances. That they are bringing it in-house, to give some more permanency to the warehouse and make its development an ongoing, iterative processes, shows that they have realized this and are making the right moves to rectify.
This is heartening news, as this is not the kind of race where if one falls behind, catching up is impossible. At the same time, it is not going to happen over night. While we are building over again, they are already mining while looking for ways to further enhance theirs. At least now we'll be running the race, rather than perpetually going back to the starting line.
I remain astounded, though, that this inefficiency existed for as long as it did.
