BI, DW, Analytics and Big Data isn’t it all about the Reporting Business?… An Overview from Harvard Graphics to Big Data & Dataviz Tools

I believe that good part of BI, DW, Analytics & Big Data senior consultants nowadays have never heard about HG, or even know what these two letters stand for in the history of the Reporting Business. Harvard Graphics, or HG for intimates, was one of first top-market reporting tools.

It has started still in DOS world and could afford to hold its position in the preference of the reporting professionals up to mid-90’s, as well as Lotus 1-2-3, was the preference in the spreadsheet market, by that time the reporting activities were centered in the financial departments.

The main appeal of these tools does not differ much from the today’s self-service BI tools: beauty, ease of use, agility and IT independence. Louts 1-2-3, for example, had a special menu called WYSIWYG (What You See Is What You Get), which is also how SAP calls one of its new UI development tools. But from early 90’s to the first decade of 2000’s with downsizing and ERP’s (including CRM, SCM, B2B) waves, the reporting tools were relegated to second place.

The focus of the reporting business industry became to be, the Data Warehouse. Much due to technology limitations in processing the huge amount of data generated by the business transactions of the systems that I “packed” as ERP’s, but also much due to the business schools’ drivers of managing by KPI’s.

So, we saw the emerging of MIS and EIS, then the “star schemas”, ETL tools and the Data Warehouse, but all this has been developed to be held/supported by an entangle of IT processes, and requiring enrollment of many IT high skilled professionals.

The more IT departments invested in DWs, the more business areas looked for spreadsheets and desktop DB. They were doing just fine with HG and Lotus 1-2-3, and after they were “replaced” by MS-Office Excel, PowerPoint, and eventually MS Access the business moved on.

In these 19 years, what I have heard the most from my users was one single plea: “— Give me a solution where I don’t need to call IT!”.

Ok, and now what? We do have the technology to process huge amount of data, excellent and easy-to-use data visualization tools, the users are, more than ever, capable to use and understand the “computer world”. What to do next?

It is most likely that it is not by keeping attached to the DW paradigms era, centered in hiring tool experts to desperately implement magical and sold as fast IT solutions.

Many articles already have been written on the behalf of the need for a proper agnostic/not-branded architecture to avoid the Data Lake to become a Data Swamp.

I do share this vision, but IT buzzwords (nowadays Big Data Solutions) could never enchant me as consultant/advisor. It is not that I am an “IT innovations denial one”, but in each company I step in, I always ask first: what do you need it for?

So, let’s take the Big Data Solutions, for example. In one make-to-stock company, A; let’s say that 60 to 70% of the data used to run the business comes from an ERP or ERP’s like systems, the other 30 to 40% comes from specialist systems. Of course, that we can be talking about a huge amount of Data, but 80 to 98% of this data might be structured data, but it is not enough to look for a Hadoop alike solution.A consistent and well-structured BI/DW implementation, with a powerful hardware and power-user, governed Datviz tool seems to be more effective for the business.

On the other hand, if for the specialist systems the company has decided to heavily invest in automation with sensors spread in all its producing processes, then a so-called Big Data Solution should be addressed.

My point is: An Agnostic/non-branded and flexible Informational Architecture is important? Yes. Seek IT innovations following the market trends is wrong? No. For this, high skilled IT people will be required? Yes. But all this cannot be self-centered in IT, no matter how IT-intensive is the investment, the focus of the Information Strategy must be: to bring the Business Data that the business wants/need Back to Business.