Ngx_lua consistent hash implementation

Ngx_lua has been out for a long time, but has never been concerned about it. Recently, an I/O-intensive project has encountered serious insufficiency in PHP performance, but it is very costly to develop and expand through C, he failed to respond to the demand in a timely manner and tried Lua. The result was very gratifying. His synchronization of non-blocking I/O and collaborative programs gave him unparalleled performance!

In the development of the project, we need to use the consistent hash to ensure the reliability of the cache. So we wrote a hash algorithm and shared it with you. This is the first version. I hope you can give more comments:

See https://github.com/qiaodandedidi/ngx_lua_consistent/wiki for more information

--[[-- consistent hash---- @author [email protected]--]]do-- virtual nodes numberlocal VIRTUAL_COUNT = 160;-- sharding vaitual nodes numberlocal CONSISTENT_BUCKETS = 1024;-- the table of virtual nodeslocal VIRTUAL_NODE = {};-- the talbe of shardinglocal BUCKETS = {};-- module of consistentlocal _M = {};-- crc32 algorithmlocal crc32 = function(arg) return math.abs(ngx.crc32_long(arg)) end--[[-- add servers and to generate the ‘BUCKETS‘---- @param {table} server all of the servers-- ]]function _M.add_server(server)for i,v in pairs(server) dofor n=1,math.floor(VIRTUAL_COUNT) dolocal hash_key = v..‘-‘..(n-1);table.insert(VIRTUAL_NODE, {v, crc32(hash_key)});endend-- sorting by ‘crc32(hash_key)‘, it means arg[2]table.sort(VIRTUAL_NODE, function (arg1, arg2) return (arg1[2] < arg2[2]);end);-- shardinglocal slice = math.floor(0xFFFFFFFF / CONSISTENT_BUCKETS);for i=1, CONSISTENT_BUCKETS dotable.insert(BUCKETS, i, hash_find(math.floor(slice * (i -1)), 1, #VIRTUAL_NODE));endend--[[-- Binary search---- @param {float} key the value of we are looking for-- @param {float} lo first index-- @param {float} hi last index-- @return {table} the node --]]function hash_find(key, lo, hi) if key <= VIRTUAL_NODE[lo][2] or key > VIRTUAL_NODE[hi][2] thenreturn VIRTUAL_NODE[lo];endlocal middle = lo + math.floor((hi - lo) / 2);if middel == 1 thenreturn VIRTUAL_NODE[middle];elseif key <=VIRTUAL_NODE[middle][2] and key > VIRTUAL_NODE[middle-1][2] thenreturn VIRTUAL_NODE[middle];elseif key > VIRTUAL_NODE[middle][2] thenreturn hash_find(key, middle+1, hi);endreturn hash_find(key, lo, middle-1);end--[[-- get consistent hash value---- @param {string} key -- @return {string} node--]]function _M.get_upstream(key) return BUCKETS[(crc32(key) % CONSISTENT_BUCKETS) + 1][1];endreturn _M;end

This article is from the "Village Chief love technology" blog, please be sure to keep this source http://weijingwu.blog.51cto.com/8376555/1559685

Ngx_lua consistent hash implementation