Qlikview implements the partial load function)

Problem Background:

The company spent more than N to ask for a solution made by qlikview consultant, which did not involve reload, so that the entire data model needs to be refreshed every time the data is refreshed, in my previous discussions with my colleagues in the Department, I also vowed that qlikview could only be like this, you cannot find a method to refresh the new data to the data model without loading the data already in memory again.

Fortunately, today a friend mentioned the partial load in add load, so he checked Google for an hour. After coming back to the experiment, he suddenly became very cheerful. After a year and a half, I can rewrite the company's solution again. Haha, this is the end of my dream. The theme will start from below:

 

What is partial load:

As the name suggests, partial load refers to partial data loading. For example, when the data source of dimension tables in the data warehouse has data to be inserted into the database, merge is used to control the addition of only? Or update the data, but do not process the data that has not changed in the database. For qlikview, we can imagine that because qlikview imports data into the memory, when it loads data into the memory again, is there a way to load new data without processing the unchanged data in the memory? The results of my tests today are positive. You can use the following methods.

LET vTimeStart = Now();FOR vI = 1 to 1000LOAD$(vI) as SellerId,$(vI) as SalesFromTestAdd.csv;Next;TestAddTable:Add LOADSellerId,SalesFromTestAdd.csv(txt, codepage is 936, embedded labels, delimiter is ',', msq)Wherenot Exists(SellerId);LET vTimeEnd = Now();

Note: To store the testadd.csvfile and the qvw file in a folder, you must write the full path of testadd.csv. Copy the preceding qlikview code to the qlikview script and Click Reload;

Because pieces of data are contained in the for... nextsentence loop, the testadd.csv file contains only the following data:

SellerId, Sales10000123,987654321

Therefore, after the data is loaded, preview the data in testaddtale in the data model, for example :.

Then add it to the testadd.csv file? The data in testadd.csv is changed:

SellerId,Sales10000123,98765432112332112,312677876

Create a Text object in the report. The formula is: = second (vtimeend-vtimestart). The result is about 5 seconds, which indicates that the entire reload takes 5 seconds.

 

The following exciting moments are coming: Click partial load (file-> partial RELOAD), the data is instantly loaded, and the text object on the page has changed to 0, indicates that the data loading time is almost zero. preview the data in testaddtable in the data model: the new data has been loaded, and the old data remains unchanged in the memory.

Because the filter condition of not exists is added, the first data segment 123,987654321 is not loaded into the data model again. (I will describe the role of exists in another article.) Two identical data entries will appear.

 

Prospect of using partial load in practical work:

After I found out this function, I was asked by my friends whether it is useful in real projects. I thought about it. It must be practical, but it is limited by today's solution, it may take some time for me to wait for my brother to mature his technology, and he will surely be able to write a better solution. For example, we can use this method: For monthly update data, we use the general load method. For files to be refreshed every day, we use the Add Load Method for partial load, in this way, the monthly data will not be refreshed together every time the daily file is refreshed.

 

Conclusion: today, I am also new to qlikview. Some comments are not correct.