This is an exclusive guest post by prominent American computer scientist Bill Inmon. Bill Inmon is a prolific author, recognized by most data managers and scientists as the father of data warehousing. Inmon wrote the first book exploring the central concepts of data warehouses and has written over 60 more books since. Inmon brought data scientists together for the first data warehousing conference, and now offers a range of educational classes in data management, plus he also writes for a number of prominent publications.

Table of Contents

  1. Introduction

  2. What is the Hierarchy of Technical Needs? 

  3. How useful is it to understand the hierarchy of technical needs? 

  4. Venture Capital Companies: A Brief Moment in the Sun 

  5. Understanding The Data Warehouse: Prioritizing Accuracy over Speed 

  6. Why Big Data is a failure

  7. Bill Inmon’s Solution to Big Data Challenges and How Integrate.io Can Help 

Introduction

In the ever-evolving world of technology, understanding the hierarchy of technical needs is essential for success. This article will explore why Big Data fails, why some venture capital companies languish after a brief success, and how data warehouses focus on data integrity rather than automation or speed processing.

What is the Hierarchy of Technical Needs? 

The hierarchy of technical needs is a structured approach to identifying the main areas that need to be addressed to ensure successful outcomes when developing software, hardware, or managing Big Data. The first level of necessity focuses on the most basic elements such as spreadsheets, databases, or oT devices. 

The end user's ever-growing demands mean that the hierarchy of technical needs must be constantly addressed to meet those requests. The end user never seems to be satisfied. Whenever one project is complete it is just a matter of time before the end user is back at the door asking for something else. Indeed, the end user operates in a – “give me what I say I want, then I can tell you what I really want“  - mode. The end user really does operate in an exploratory mode. They don’t know what they want until they can see what the possibilities are. Then – and only then – can they articulate their needs, usually a new set of needs.

So, does this mean that the end user is being unreasonable? Is there ever an end to the end user’s insatiable appetite for something new in their computer systems? To understand this phenomenon, we have to look no further than Maslow’s hierarchy of needs.

In 1943 Abraham Maslow -a psychologist – introduced the concept of a hierarchy of needs for humans. Maslow’s hierarchy of needs looks like this:

Picture 1

In Maslow’s hierarchy of needs, it is seen that there are some basic needs for each human. Then, as the most basic needs are met, there is another completely different set of needs once the basic set of needs are met. Maslow’s hierarchy of needs is not limited to one race or religion. The hierarchy of needs applies to all of mankind, whatever race or religion, in all countries.

In order to explain why the end user is never happy with whatever they are presented with, there is another set of needs that needs to be explored. This set of needs can be called the technical hierarchy of needs. The technical hierarchy of needs looks like this:

Picture 2

The technical hierarchy of needs shows that at the most basic level, the end user is satisfied with merely getting a system put on a computer. By putting a system on a computer, the end-user has alleviated a huge manual burden. And for most people, this is very important.

But once that manual burden has been alleviated, the end user discovers that they want something more. It is not enough that a system has been automated, now the end user wants the computer to operate quickly and easily. If you don’t believe this look at your ATM. What would happen if ATMs suddenly gave a 1-minute performance rather than a 1-second performance? People would become very irritated with ATMs and probably would not use them. With an ATM you just expect a fast response time as part of what ATMs do.

So, after automation of a system comes speed of access and ease of access.

Then one day the end user discovers that there is a need for the believability of data. It is one thing to automate a system, it is another thing to make the system operate quickly. But what good is all of this if the data the system is operating on produces incorrect data? The system is then worthless. You can’t make important business decisions based on data that is inaccurate or unbelievable.

The end user demands that data have integrity. But they don’t make this discovery until they have progressed up the hierarchy.

Then one day the end user decides that data should be able to be shared across the enterprise, or even beyond the enterprise. Why should the same data element appear in one system and reappear in another system with a different set of values? The end user sees that there is great value in being able to have a single understanding of data that is vital to the corporation. As long as data is scattered across the corporation like falling leaves in the autumn, making important strategic decisions is difficult if not impossible to do.

Then one day the end user finds that data is accessible, easy to get to, reliable, and interoperable. This is the day that the end user can start to equate computerization with true business value.

So when the end user is asking for something new, all the end user is doing is progressing up the ladder of the hierarchy of needs for technology. And that progression is as inevitable as grass growing in the summertime. Or snow falling in the wintertime.

How useful is it to understand the hierarchy of technical needs? 

Extremely! 

It helps to identify the main areas that must be addressed to ensure successful outcomes when developing software, hardware, or managing Big Data. By understanding this hierarchy and addressing each level of need as it arises, developers can manage user expectations, deliver the best products possible, and avoid costly errors or delays.

The hierarchy of technical needs explains a lot of things –

Why spreadsheets are not used for every system that is made? Certainly, spreadsheets satisfy the need for quick computerization. Nothing can beat the ability to get a system on the computer faster than a spreadsheet. But when I have a spreadsheet, I can assign myself a salary of $1,000,000 a month. But that assignment is not a reflection of reality. Spreadsheets do not satisfy the need for the integrity of data.

 The turnover of technical corporations. If every decade you look at the top computer companies and you look at the list of the top computer companies ten years later, you find out that the list of top technical corporations is constantly turning over. It is why IBM – once the premier technology company in the world is also running today. Technology companies focus on and sell their existing technology. As a rule, technology companies do not evolve to a higher set of needs. Consequently, they wake up one day to find that the end user's needs have left behind their technology.

Venture Capital Companies: A Brief Moment in the Sun 

Despite their initial success, venture capital companies often fail to sustain their momentum and collapse. This is due to a lack of knowledge about Big Data technology and its potential uses. Venture capitalists may invest in Big Data solutions that are too complex for the company’s current resources and abilities or undervalue Big Data initiatives with little return on investment.

   Why some venture capital companies languish after a brilliant moment in the sun. Venture capitalists only focus on short-term investments. They want a return on their investment as soon as possible. They invest in companies that solve a problem right now. Then one day they wake up and find that the company that solves a problem today does not solve the problems of tomorrow. And the VC is stuck with companies that solve yesterday’s problems.

Understanding the Data Warehouse: Prioritizing Accuracy over Speed 

Because Big Data lacks reliability, many organizations turn to data warehouses. Unlike Big Data, data warehouses prioritize accurate information over speed processing. This allows users to avoid errors due to incomplete or incorrect data while still accessing information quickly.

The Data warehouse does not solve the problems of automation or speed of processing. Instead, a data warehouse addresses the problem of data integrity. A lot of people get excited when they can automate something. But automation is only the first step up the ladder. People think they have climbed the ladder when all they have done is gotten to the first step.

Why Big Data is a failure

Despite its potential capabilities, Big Data fails because it is often too complex for many users who lack the necessary skillset or understanding of Big Data. Big Data requires a high level of technical expertise and precise coding parameters, making it time-consuming and costly to implement.

Big Data solves the problem of storage of information, and that is basic and correct. But Big Data is very poor at solving the problems of ease of use, speed of access, or integration of data. Buying into Big Data only gets you up the first step of the ladder. Big Data also fails when the data being collected is not organized properly or does not meet certain standards for accuracy or completeness

So when the end user never seems to be satisfied, don’t blame your end user for having an insatiable appetite. Just recognize that your end user is merely progressing up the ladder of the technical hierarchy of needs. And it is both normal and expected that evolution will occur. It is inevitable as the sun rises in the east and sets in the west.

Bill Inmon’s Solution to Big Data Challenges and How Integrate.io Can Help 

Fortunately, Bill Inmon recognized Big Data’s potential as early as 1970 when he devised a framework called “The Three-Tier Big Data Architecture Model.” His solution sought to bridge the gap between Big Data and data warehouses by providing organizations with an agile Big Data solution that is both reliable and cost-effective. 

Integrate.io provides a suite of Big Data technologies that allow organizations to quickly and securely access, integrate, analyze, and report on Big Data from multiple sources.  By leveraging its expertise in Big Data integration, Integrate.io helps organizations make informed decisions based on accurate data without sacrificing speed or security. Schedule a demo today to try it yourself.

To conclude, The hierarchy of technical needs highlights the importance of understanding Big Data technology before investing in it. Big Data has many potential benefits, but it can be costly and difficult to implement properly if users lack the necessary knowledge or resources.

 To ensure success with Big Data initiatives, companies must prioritize data accuracy over speed processing when selecting Big Data solutions. Bill Inmon’s Big Data Architecture Model provides an effective solution to many Big Data challenges while Integrate.io can help organizations implement Big Data Integration with confidence and efficiency.  Understanding the hierarchy of technical needs is essential for effectively leveraging Big Data solutions in any organization. 

Bill Inmon – the father of data warehousing – has a company in Denver, Colorado. Forest Rim Technology supports textual disambiguation which reads text and turns text into a standard database. In doing so, textual disambiguation opens up corporate decision-making to the full spectrum of their data. See more at www.forestrimtech.com