MySQL in and limit cannot be a problem

Today in MySQL processing a data volume to reach Tens database table, to take out the data set to meet the conditions, and then deposit to the MONGO database, using JPA provided pageble to get the paging, and then use multithreading to fetch data, found that the first efficiency can be, certainly more than the single-line Chengqiang a lot, But this pageable has a problem, when the SQL statement is generated, will automatically add a limit *,10000, such a restriction, "*" represents the row from which to start fetching data, 10000 for this operation to take 10,000 data. This kind of operation will seriously occupy the server's cache, if you do this kind of operation, other people also in the visit, will cause everybody together to get a ratio.

The workaround is to use a concurrent queue to store the ID of the last piece of data for each page, but I have a problem:

This version of MySQL doesn ' t yet support ' LIMIT & in/all/any/some subquery

Limit cannot be directly hyphenated with in-scope keywords, bad SQL, and improved SQL.

Error sqlselect max (ID) from News_info where ID in (select ID from News_info where pub_date>?1 and id>?2 order By ID limit 10000)//correct sqlselect max (ID) from News_info where ID in (select ID from news_info where pub_da Te>?1 and id>?2 order by ID limit 10000) as news)

Since you can't do it directly, create a temporary table in the middle.

MySQL in and limit cannot be a problem