The method of multiple filtering for a large amount of data by JS _javascript skill

Objective

The main requirement is that the front-end through Ajax to obtain a large number of data from the backend, need to filter according to some conditions, the first filtering method is this:

Class Filter {
 FilterA (s) {let
 data = This.filterdata | | this.data;
 This.filterdata = Data.filter (m => M.A = = s);
 }
 
 FilterB (s) {let
 data = This.filterdata | | this.data;
 This.filterdata = Data.filter (m => m.b = = s);
 }

Now confused, think this processing data is wrong, but do not know how to deal with.

Find problems

The problem is filtering, which can be implemented with multiple filters (called filterA() and then invoked filterB() ), but this filtering is irreversible.

If the filtration process is this:

F.filtera ("A1");
F.filterb ("B1");
F.filtera ("A2");

Originally want to press "A1" and "B1" filtered data, and then modify the first condition for "A2", but the result has become the empty set.

Solve the problem

Found the problem, on the targeted solution. Since this problem is caused by the irreversible filtration process, it this.data can solve the problem by filtering directly from the beginning, rather than this.filterData filtering from the beginning. If you want to do this, you need to record the filter criteria that you selected first.

Record filter conditions

It is certainly possible to record filter conditions with a list, but note that two filters for the same condition are mutually exclusive, only the last one, so it should be more appropriate to use HASHMAP.

Class Filter {
 constructor () {
 this.filters = {};
 }

 Set (key, filter) {
 This.filters[key] = filter;
 }

 Getfilters () {return
 Object.keys (this.filters). Map (key => This.filters[key]);
 }

In this case, the process like the above is expressed as

F.set ("A", m => M.A = = "A1");
F.set ("B", m => m.b = = "B1");
F.set ("A", m => M.A = = "A1");

Let filters = F.getfilters (); length = = 2;

The filter in the 3rd sentence above covers the one set by the 1th sentence. Now with the final filters to filter the original data this.data , you can get the correct results.

Some people will feel that the getFilters() returned list is not in the order of set-indeed, this is the HashMap characteristic of disorder. But for simple conditions, whichever comes first, the result is the same. But for some compound condition judgment, may have the influence.

If you really need to, you can use array instead of map to solve the order problem, but this will reduce the search efficiency (linear lookup). If you also want to solve the problem of search efficiency, you can use array + map to handle. There's not much to say here.

Filter

In fact, it's really slow to use one loop at a time when you're working with it getFilter() . Since data is encapsulated in filter, you can consider giving a filter() method to deliver the filter interface directly.

Class Filter {
 filter () {Let
 data = This.data;
 For (let F of This.getfilters ()) {
  data = Data.filter (f);
 }
 return data;
 }

But I don't think it's very efficient, especially when it comes to a lot of data. You may wish to take advantage of the Lodash delay process.

Using Lodash to delay processing

Filter () {Let
 chain = _ (this.data);
 For (let F of This.getfilters ()) {
 chain = Chain.filter (f);
 }
 return Chain.value ();
}

Lodash enables deferred processing when the data is greater than 200, that is, it is processed into a loop that calls each filter sequentially instead of one loop for each filter.

Deferred processing and non-delay processing can be distinguished by the following figure. Non-deferred processing will have a total of n (here n = 3) of the second large cycle, resulting in n-1 intermediate results. But the delay processing only carries on one big circulation, does not have the intermediate result to produce.

But to be honest, I don't like to load a library for a small thing, so simply make a simple implementation of myself.

Implement deferred processing yourself

Filter () {
 Const filters = this.getfilters ();
 Return Data.filter (M => {for
 -let F of filters) {
  //If a filter has filtered it out, you don't have to use the back filter to determine if
  (!f (m)) {return
  false;
  }
 }
 return true;
 });

The For loop inside can also be used Array.prototype.every to simplify:

Filter () {
 Const filters = this.getfilters ();
 Return Data.filter (M => {return
 filters.every (f => f (m));}
 );

Data filtering is not much of a complex thing, as long as the idea is clear, figure out what data is needed to retain, what data is temporary (intermediate process), what data is the final result ... Using Array.prototype the relevant methods, or tools such as Lodash, can be easily handled.

Summarize

The above is the entire content of this article, I hope the content of this article for everyone's study or work can help, if there is doubt you can message exchange.