What is the majority of large data items that ended in failure?

In almost everyone's mind, big data is the large and small amount of structured and unstructured data that the enterprise IT department is growing exponentially. But while big data has become a mainstream it phenomenon, most large data projects still end in failure.

The reason is that it is difficult for enterprises to find appropriate methods for large data collection, management and understanding, and ultimately from large data information to extract valuable things.

Conquering large data items, and ultimately extracting the business insights that your business needs, is a daunting task in itself. But when it comes to defining the scope of large data projects and ensuring that the relevant facilities are in place, the people in your business cannot maintain a unified pace, and the project is doomed to fail.

Analysis of the causes of failure

As follows, I have seen the main reason for the failure of large data projects:

Lack of consistency. The IT department lacks consistency with the business unit in addressing issues related to the business unit. The IT department is only looking at the problem from a technical standpoint. Similarly, the lack of real commitment from corporate stakeholders tends to make big data projects difficult to succeed.

Lack of data access rights. Access to data is often limited, and it team members do not have permission to access the relevant dataset so that they cannot find relevant data that will make the project successful.

Lack of professional knowledge. Given that many of the technologies, methodologies, and disciplines in large data areas are new, enterprise employees lack the expertise to handle data and to complete business.

Lack of consistency

The first of all these issues is lack of consistency. Is the first problem that your business must address, and the most important one. The crux of the matter is that what you are exploring and looking for in your business is a field that you are unfamiliar with, so it is important to first understand what your business unit is trying to achieve in order to succeed in a large data project.

Although it is the most important factor in the success of your large data project, it is also challenging to achieve consistency between your business and IT departments. Not only do large data have different meanings for different people, but also a series of external factors that may affect changes in business requirements, making it more important to address certain issues than it can maintain. If the IT department and the business unit cannot agree on the scope of the large data project, the project will involve too many orientations, too many people, so that the focus will be on resolving specific business issues to the management of it technology so that everyone can meet their needs.

Another challenge that affects consistency between business units and it is the reluctance to change. Many times, if a large data project suggests taking relevant actions or changes, while the business sector stakeholders do not understand the relevant actions or changes involved, they tend to take the slack approach, first silently accept the proposal, but later relegated it to a wrong process, analysis or dataset. In this regard, the analyst team may think that the business unit has agreed to and put into action, but the action they have taken the results of only a suboptimal business results.

Lack of data access rights

The second reason for the failure of large data projects-the lack of access to data can be traced back to a basic it prerequisite: silos. The Sales Department, marketing department, Human Resources department and so on have the Data Warehouse, each department's data warehouse all limits the related data the Access authority and the protection measure. The reasons for the existence of the data warehouse are well understood, but if some of the relevant data in the data warehouse required by the IT department is not available, it can be said that the employees in the IT department are doomed to not solve the problem until they try to solve some problems.

To deal with this problem, large data projects must have the right to execute relevant data from the outset. Without access to all business-related data, it is impossible to identify the relationships and patterns of business problems and solve the problems facing business units. So, the authorization for large data projects comes from the top of the enterprise, and if a business team is looking for a specific business problem that is very important to solve, the IT department will have enough access to any data they need. "If the correct data information is not available, the project will undoubtedly be stagnant for a long time."

Lack of professional knowledge

The third major flaw-lack of relevant expertise. This stems from the lack of the right skills to perform large data projects. Because large data technology is still very new to "mainstream" enterprises, it teams often lack the relevant expertise to determine how to use large data to achieve the purpose of analysis.

While recruiting data science experts is an alternative to the possibility of solving this lack of expertise, it is not feasible for many businesses. This new role requires a combination of programmer skills and research scientist thinking, the cost of setting up a job like this is very high, and the relevant skill set required is uncommon and difficult to create.

How to make your enterprise's Big Data project successful

Consider a practical solution. First, don't call it a "big Data project." Name it a similar project: "A project that helps us better understand our customers and why they like to shop in a particular store." "The project is to answer important business questions, and big data is the source of the answer." For example, there are some best practices to help your project achieve success:

Start by listing a list of business issues that you want to solve

Don't start with solving a big problem. Start with a small project, choose a specific problem that you need to address, and stick to it. Make a list of the questions you need to answer, and don't overlook your goals because you're stuck with technical problems. Ensure that the responsibility of the IT team does not become too broad or "omni-directional", so as to avoid the failure to deal with the change in the scope of the problem: that is, changes in demand from the business unit to the IT department cause the problem focus to shift. Ensure that all stakeholders objectively agree on the implementation and implementation of the project so that everyone can focus on the completion of the project.

Get an endorsement from the top of the enterprise before you start the project

Once you have identified the business issues that you are trying to resolve, you must obtain support from the business team from above for all the relevant data that you need to ensure successful completion of the project. Be sure to obtain the authority of the company's top leaders to access all relevant business data so that you can find patterns and relationships to answer business questions. That is, you must have access, control permissions.

Ensure that your team has the expertise needed to execute the project

Ideally, your team will have members who are professionally trained, equipped with the skills and mindset of data scientists who can use this data information to generate the business results that are needed. If not, you can use your existing system to solve the problem. This is a good time to take a step back and think about the business issues you need to answer. You may not need professional training or NLP to get the answers you need at this point, just give the right people access to the information inside the enterprise.

Choose a problem that creates business value and stick to it on the right path. Remember, a successful project does not have much to do with the scope involved. There is no need to eat a fat man, that will lead to greater failure. After all, the success of a small project is much better than the failure of a major project.

(Responsible editor: The good of the Legacy)