Twitter's distributed self-increment ID algorithm snowflake (Java edition)

Overview

In distributed systems, there are scenarios that require the use of a globally unique ID, which can be used to prevent ID collisions using a 36-bit UUID, but the UUID has some drawbacks, first he is relatively long, and the UUID is generally unordered.

There are times when we want to use a simple ID and want the ID to be generated in an orderly fashion.

The snowflake of Twitter solved this demand, and initially Twitter migrated the storage system from MySQL to Cassandra, because Cassandra had no sequential ID generation mechanism, so it developed a set of global unique ID generation services.

Structure

The structure of the snowflake is as follows (each part with-separate):

0-0000000000 0000000000 0000000000 0000000000 0-00000-00000-000000000000

The first bit is unused, the next 41 bits are millisecond (41-bit lengths can be used for 69 years), then 5-bit Datacenterid and 5-bit Workerid (10-bit lengths support deployment of up to 1024 nodes), The last 12 bits are counted in milliseconds (the 12-bit count sequence number supports 4,096 ID numbers per millisecond for each node)

Altogether add up just 64 bits, for a long type. (up to 19 length after conversion to string)

The IDs generated by the snowflake are sorted on a per-time basis, and no ID collisions (differentiated by datacenter and Workerid) are generated throughout the distributed system and are highly efficient. Tested snowflake can generate 260,000 IDs per second.

Source

(Java version of source code)

/*** The structure of twitter_snowflake<br> * snowflake is as follows (per part with-separate):<br> * 0-0000000000 0000000000 0000000000 0000000000 0-0 0000-00000-000000000000 <br> * 1-bit ID, because the long base type is signed in Java, the highest bit is the sign bit, the positive number is 0, the negative number is 1, so the ID is generally positive, the highest bit is 0<br> * 41-bit time-truncation (millisecond), note that the 41-bit time-intercept is not the time-intercept for storing the current time, but rather the difference in the time-intercept (the current time-stop-start-time-cut) * Gets the value), where the start time is usually the time that our ID generator started to use, specified by our program (the following program Idworker the StartTime property of the class below). 41-bit time-cut, can be used 69 years, Year t = (1L << x)/(1000L * x * * 365) = 69<br> * 10 bits of data machine, can be deployed on 1024 nodes, including 5-bit datacente RID and 5-bit workerid<br> * 12-bit sequence, count in milliseconds, 12-bit count sequence number supports each node per millisecond (same machine, same time intercept) to generate 4,096 ID number <br> * add up just 64 bits, for a long type. <br> * Snowflake's advantage is that the overall time-to-order, and the entire distributed system will not generate ID collisions (by the data center ID and machine ID to differentiate), and high efficiency, tested, snowflake per second can produce about 260,000 ID.*/PublicClassSnowflakeidworker {//==============================fields===========================================/**Start time Cut (2015-01-01)*/PrivateFinalLong Twepoch =1420041600000L;/**The number of bits that the machine ID occupies*/PrivateFinalLong workeridbits = 5L;/**The number of digits that the data identification ID occupies*/PrivateFinalLong datacenteridbits = 5L;/**The maximum supported machine ID, the result is 31 (this shift algorithm can quickly calculate the maximum number of decimal digits that can be represented by several binary numbers)*/PrivateFinalLong Maxworkerid = -1l ^ ( -1l <<Workeridbits);/**The maximum supported data identification ID, the result is 31*/PrivateFinalLong Maxdatacenterid = -1l ^ ( -1l <<Datacenteridbits);/**Number of digits in the ID of the sequence*/PrivateFinalLong sequencebits = 12L;/**Machine ID Move left 12 bits*/PrivateFinalLong Workeridshift =Sequencebits;/**Data ID ID shifted left 17 bits (12+5)*/PrivateFinalLong Datacenteridshift = Sequencebits +Workeridbits;/**Time Intercept left 22 bits (5+5+12)*/PrivateFinalLong Timestampleftshift = sequencebits + workeridbits +Datacenteridbits;/**Generate a mask for the sequence, here is 4095 (0b111111111111=0xfff=4095)*/PrivateFinalLong Sequencemask = -1l ^ ( -1l <<Sequencebits);/**Work Machine ID (0~31)*/PrivateLongWorkerid;/**Data center ID (0~31)*/PrivateLongDatacenterid;/**Sequence in milliseconds (0~4095)*/PrivateLong sequence = 0L;/**The last time the ID was generated is truncated*/PrivateLong Lasttimestamp = -1l;//==============================constructors=====================================/*** Constructor Function *@paramWorkerid Job ID (0~31) *@paramDatacenterid Data Center ID (0~31)*/Public Snowflakeidworker (Long Workerid,LongDatacenterid) {if (Workerid > Maxworkerid | | Workerid < 0) {ThrowNew IllegalArgumentException (String.Format ("worker Id can ' t is greater than%d or less than 0", Maxworkerid)); }if (Datacenterid > Maxdatacenterid | | Datacenterid < 0) {ThrowNew IllegalArgumentException (String.Format ("Datacenter Id can ' t is greater than%d or less than 0", Maxdatacenterid)); }This.workerid =Workerid;This.datacenterid =Datacenterid; }//==============================methods==========================================/*** Get the next ID (the method is thread-safe) *@returnSnowflakeid*/PublicSynchronizedLongNextID () {Long timestamp =Timegen ();//If the current time is less than the timestamp generated by the last ID, it should throw an exception when the system clock is rolled back.if (Timestamp <Lasttimestamp) {ThrowNewRuntimeException (String.Format ("Clock moved backwards. Refusing to generate ID for%d milliseconds ", Lasttimestamp-timestamp)); }//If it is generated at the same time, the sequence in millisecondsif (Lasttimestamp = =Timestamp) {sequence = (sequence + 1) &Sequencemask;//Sequence overflow in millisecondsif (sequence = = 0) {//Blocks to the next millisecond, obtaining a new timestamp timestamp =Tilnextmillis (Lasttimestamp); } }//Timestamp change, sequence reset in millisecondsElse{sequence = 0L; }//Time truncation of last build id Lasttimestamp =Timestamp//Shift and join or operate together to form a 64-bit IDReturn ((Timestamp-twepoch) << timestampleftshift)// | (Datacenterid << Datacenteridshift)// | (Workerid << Workeridshift)// |Sequence }/*** block to the next millisecond until a new timestamp is obtained *@paramLasttimestamp time of last generation ID *@returnCurrent time stamp*/ProtectedLong Tilnextmillis (long Lasttimestamp) { Timegen (); while (timestamp <= Lasttimestamp) {timestamp = Timegen (); } return timestamp;} /** * Returns the current time in milliseconds * @return current time (msec) */protected long Timegen () {return System.currenttimemillis ();}//==============================test============================== ===============/** Test */public static void main (string[] args) {Snowflakeidworker idworker = new Snowflakeidworker (0, 0 ); for (int i = 0; i < i++) {Long id = idworker.nextid (); SYSTEM.OUT.PRINTLN (long.tobinarystring (id)); SYSTEM.OUT.PRINTLN (ID); } }}

How to use

 Public class Main {    publicstaticvoid  main (string[] args) {        =new Snowflakeidworker (1,20);         long l= worker.nextid ();        System.out.println (l);    }}

View Code

Twitter's distributed self-increment ID algorithm snowflake (Java edition)