Migrating data from Oracle to PostgreSQL with EDB-MTK tools

Spring-data-redis is the support part of redis in the spring-data module, referred to as "SDR", which provides a high degree of encapsulation based on the jedis client API and integration with the spring container. In fact, the jedis client is already simple and light enough On the order of magnitude, spring-data-redis is suspected of "over-design".

One. The jedis client has the following deficiencies in the implementation of programming:

    1) Connection management lacks automation, and the design of connection-pool lacks the necessary container support.

    2) Data operations need to pay attention to "serialization" / "deserialization", because jedis's client API accepts string and byte data types, and requires additional support for structured data (json, xml, pojo, etc.) operations.

    3) Transaction operations are purely hard-coded

    4) The pub / sub function lacks the necessary design pattern support, and needs too much attention for developers.

    However, the integration of jedis and spring is also very simple, see "jedis connection pool example".

 

2. spring-data-redis provides the following functions for jedis:

    1. Automatic connection pool management, providing a highly encapsulated "RedisTemplate" class

    2. Encapsulate and encapsulate a large number of APIs in the jedis client, encapsulate the same type of operation as the operation interface ValueOperations: simple K-V operations SetOperations: set type data operations ZSetOperations: zset type data operations HashOperations: map type data operations

ListOperations: for list-type data operations

    3. Provides a "bound" (bound) convenient operation API for the key, you can encapsulate the specified key through the bound, and then perform a series of operations without having to "explicitly" specify the key again, namely BoundKeyOperations: BoundValueOperations BoundSetOperations BoundListOperations BoundSetOperations BoundHashOperations

  

    4. Encapsulate transaction operations and have container control.

    5. A variety of alternative strategies (RedisSerializer) are provided for the "serialization / deserialization" of data (RedisSerializer) JdkSerializationRedisSerializer: POJO object access scene, using JDK's own serialization mechanism to serialize the pojo class through ObjectInputStream / ObjectOutputStream Operation, and finally a sequence of bytes will be stored in redis-server. Is currently the most commonly used serialization strategy. StringRedisSerializer: When the Key or value is a string, the byte sequence of the data is encoded into a string according to the specified charset, which is a direct encapsulation of "new String (bytes, charset)" and "string.getBytes (charset)". It is the most lightweight and efficient strategy. JacksonJsonRedisSerializer: The jackson-json tool provides the conversion capability between javabean and json. It can serialize the pojo instance into json format and store it in redis, or convert the json format data into pojo instance. Because the Jackson tool needs to specify the Class type explicitly when serializing and deserializing, this strategy is slightly more complicated to encapsulate. [Requires jackson-mapper-asl tool support] OxmSerializer: Provides the conversion capability between javabean and xml. The currently available three-party support includes jaxb, apache-xmlbeans; the data stored by redis will be the xml tool. However, using this strategy, programming will be somewhat difficult and the most efficient; it is not recommended. [Requires spring-oxm module support]

    For "Serialization and Serialization", JdkSerializationRedisSerializer and StringRedisSerializer are the most basic strategies. In principle, we can store data in any format for application access and analysis (including applications such as app, hadoop and other tools), but It is still not recommended to directly use "JacksonJsonRedisSerializer" and "OxmSerializer" at design time, because whether it is json or xml, they are still String themselves.

    If your data needs to be parsed by a third-party tool, then the data should use StringRedisSerializer instead of JdkSerializationRedisSerializer.

    If your data format must be json or xml, then at the programming level, StringRedisSerializer is still used in the redisTemplate configuration. Before storage or after reading, use the "SerializationUtils" tool to convert to json or xml, please see the example below.

 

    6. Based on the design pattern and JMS development ideas, the pub / sub API design is encapsulated to make development more convenient.

    7. Spring-data-redis does not provide a good package for sharding. If your architecture is based on sharding, then you need to implement it yourself. This is also the only feature missing from sdr and jedis.


3. About the usage of API in BoundKeyOperations

/ **
* Need to specify the name of hashtable
*
* @param tableName
* /
public void boundHashOps (String tableName) {
System.out.println ("================== Hash ==============");
BoundHashOperations <String, String, String> ops = stringRedisTemplate
.boundHashOps (tableName);
stringRedisTemplate.delete ("student");
stringRedisTemplate.delete ("student: 1");
ops.put ("cs01", "123"); // Store data ops.putAll (maps); Store multiple data
String key1 = ops.getKey (); // The name of tableName
System.out.println ("key1:" + key1);
String key11 = ops.get ("cs01");
System.out.println ("key11:" + key11); // Get the value of key

ops.putIfAbsent ("cs02", "456");
String key2 = ops.getKey ();
System.out.println ("ops.getKey ()-key2:" + key2);
String key21 = ops.get ("cs02");
System.out.println ("ops.get (cs02) -key21:" + key21);

Map <String, String> maps = ops.entries (); // Get all key-value values
for (String key: maps.keySet ()) {
System.out.println ("map-key:" + key + "map-value:" + maps.get (key));
}
// ops.persist (); // Delete expired (if any) data.
System.out.println ("ops.getExpire ():" + ops.getExpire ()); // -1
System.out.println ("ops.expireAt (new Date ()):"
+ ops.expireAt (new Date ())); // true set survival expiration time
System.out.println ("ops.getType ():" + ops.getType ()); // Hash
System.out.println ("ops.hasKey (cs01):" + ops.hasKey ("cs01")); // true
System.out.println ("ops.hasKey (cs02):" + ops.hasKey ("cs02")); // true
System.out.println ("ops.size ():" + ops.size ()); // 2

Set <String> keys = ops.keys (); // Get all keys
for (String string: keys) {
System.out.println ("ops.keys ():" + string);
}

System.out.println ("ops.values ():" + ops.values ()); // Get all values
System.out.println ("ops.size ():" + ops.size ()); // 2 Get the number

ops.delete ("cs01"); // Delete the data whose key is cs01
}

/ **
* The name of hashtable is not specified
*
* @param tableName
* /
public void opsForHash (String tableName) {
System.out.println ("================== Hash ==============");
HashOperations <String, Object, Object> ops = stringRedisTemplate
.opsForHash ();
stringRedisTemplate.delete ("student");
stringRedisTemplate.delete ("student: 1");
ops.put (tableName, "cs01", "123"); // store data ops.putAll (maps); store multiple data
Object key11 = ops.get (tableName, "cs01");
System.out.println ("key11:" + key11); // Get the value of key

ops.putIfAbsent (tableName, "cs02", "456");
Object key21 = ops.get (tableName, "cs02");
System.out.println ("ops.get (cs02) -key21:" + key21);

Map <Object, Object> maps = ops.entries (tableName); // Get all key-value values
for (Object key: maps.keySet ()) {
System.out.println ("map-key:" + key + "map-value:" + maps.get (key));
}
// ops.persist (); // Delete expired (if any) data.
System.out.println ("ops.hasKey (cs01):"
+ ops.hasKey (tableName, "cs01")); // true
System.out.println ("ops.hasKey (cs02):"
+ ops.hasKey (tableNam
e, "cs02")); // true
System.out.println ("ops.size ():" + ops.size (tableName)); // 2

Set <Object> keys = ops.keys (tableName); // Get all keys
for (Object string: keys) {
System.out.println ("ops.keys ():" + string);
}

System.out.println ("ops.values ():"
+ ops.values (tableName)); // Get all values
System.out.println ("ops.size ():"
+ ops.size (tableName)); // 2 Get the number

ops.delete ("cs01"); // Delete the data whose key is cs01
}

/ **
* List has duplicate data
*
* @param tableName
* /
public void boundListOps (String tableName) {
System.out.println ("================== List ==============");
BoundListOperations <String, String> ops = stringRedisTemplate
.boundListOps (tableName);
ops.leftPush ("cs01"); // left push left stack, advanced first out, compared with the right, left first out
ops.leftPushIfPresent ("cs011"); // I don't know what is the difference with the above
ops.leftPush ("cs01", "cs0111"); // Push on the left side of cs01
ops.leftPushAll ("cs01111", "cs011111");

List <String> values = ops.range (0, -1);
for (String string: values) {
System.out.println ("letf push:" + string);
}

ops.rightPush ("cs02"); // right push right stack first in first out, compared with left
ops.rightPushIfPresent ("cs022");
ops.rightPush ("cs02", "cs0222"); // Push to the right of cs02
ops.rightPushAll ("cs02222", "cs022222");

ops.set (0, "cs04"); // Replace the first data with cs04
// ops.trim (0, 3); // delete from the first data to the fourth data

List <String> values1 = ops.range (0, -1); // Find all data
for (String string: values1) {
System.out.println ("right push:" + string);
}

List <String> values2 = ops.range (1, 2); // Find from the second to the third
for (String string: values2) {
System.out.println ("right push1:" + string);
}
System.out.println ("ops.index (1):" + ops.index (0)); // Get the first data
System.out.println ("ops.remove (0, cs01):"
+ ops.remove (0, "cs01")); // 1, delete "cs01"
System.out.println ("ops.leftPop ():" + ops.leftPop ()); // left stack
System.out.println ("ops.rightPop ():" + ops.rightPop ()); // pop out on the right
System.out.println ("ops.remove (0, cs01) 1:"
+ ops.remove (0, "cs01")); // 0, if "cs01" does not exist, return 0

// ops.persist (); // Delete expired (if any) data.
}

public void boundSetOps () {
System.out.println ("================== Set ==============");
String tableName2 = "caoshuai03";
BoundSetOperations <String, String> ops = stringRedisTemplate
.boundSetOps (tableName2);

String [] values = {"cs03", "cs04"};
System.out.println ("ops.add (values):"
+ ops.add (values)); // Add multiple data to the set and return the added amount

Set <String> sets1 = ops.members (); // Get all the data in the set, the order of each display may be different
for (String string: sets1) {
System.out.println ("ops.members () 1:" + string);
}

// Get a random number
System.out.println ("ops.randomMember ():" + ops.randomMember ());
// Get a random number
System.out.println ("ops.randomMembers (1):" + ops.randomMembers (1));
// Get two random numbers, the value may be the same
System.out.println ("ops.randomMembers (2):" + ops.randomMembers (2));

System.out.println ("ops.distinctRandomMembers (1):"
+ ops.distinctRandomMembers (1)); // Get a random number
System.out.println ("ops.distinctRandomMembers (2):"
+ ops.distinctRandomMembers (2)); // Get two different random numbers

System.out.println (ops.isMember ("cs04")); // Whether it contains cs04, returns true if there is
System.out.println (ops.isMember ("cs01")); // No return false

System.out.println (ops.size ()); // The amount of data in set

System.out.println (ops.getKey ()); // Get the name of the set

System.out.println (ops.getType ()); // Get type

Set <String> set7 = ops.diff ("caoshuai02"); // Get different data from another set, difference set
for (String string: set7) {
System.out.println ("ops.diff (caoshuai02):" + string);
}

// Get the same data as another set, intersection
Set <String> set8 = ops.intersect ("caoshuai02");
for (String string: set8) {
System.out.println ("ops.intersect (caoshuai02):" + string);
}

// Get all the data of another set, merge it with yourself, remove the duplicate data, and set
Set <String> set6 = ops.union ("caoshuai02");
for (String string: set6) {
System.out.println ("ops.union (caoshuai02):" + string);
}

// Get the same data as other set, intersection
List <String> keys = new ArrayList <String> ();
keys.add ("caoshuai02");
Set <String> set = ops.intersect (keys);
for (String string: set) {
System.out.println ("ops.intersect (keys):" + string);
}

// Get different data from other set, difference set
List <String> keys1 = new ArrayList <String> ();
keys1.add ("caoshuai02");
Set <String> set3 = ops.diff (keys);
for (String string: set3) {
System.out.println ("ops.diff (keys) 3:" + string);
}

// Get all the data of other set, merge it with yourself, remove duplicate data, and set
List <String> keys2 = new ArrayList <String> ();
keys2.add ("caoshuai02");
Set <String> set4 = ops.union (keys2);
for (String string: set4) {
System.out.println ("ops.union (keys2):" + string);
}

// Get data that is different from another set, difference set, and save it in another set
ops.diffAndStore ("caoshuai02", "caoshuai04");
// ops.diffAndStore (keys, destKey);
The
// Get the same data as another set, intersect, and store in another set
ops.intersectAndStore ("caoshuai02", "caoshuai05");
// ops.intersectAndStore (keys, destKey);
The
// Get all the data in another set, union and store in another set
ops.unionAndStore ("caoshuai02", "caoshuai06");
// ops.unionAndStore (keys, destKey);

// Move the specified data to another set, it will return true if the move is successful
System.out.println (ops.move ("caoshuai07", "cs03"));
// Move the specified data to another set, move failed, return false, Note: There is no cs01 in the current set
System.out.println (ops.move ("caoshuai07 "," cs01 "));

System.out.println (ops.pop ()); // Pop out a data, the first data is popped out

System.out.println (ops.remove ("cs03")); // Delete multiple data, return the number of deleted
System.out.println (ops.remove ("cs01")); // Delete data, return the number of deleted
}
The
public void boundValueOps () {
System.out.println ("================== String ==============");
String tableName2 = "LiMing01";
BoundValueOperations <String, String> ops = stringRedisTemplate
.boundValueOps (tableName2);

System.out.println (ops.append ("stu01")); // Add data, return the number of added data characters
The
// Get the existing data first, then add the data to overwrite the original
System.out.println (ops.getAndSet ("stu02"));
The
System.out.println (ops.get ()); // Get the added data
The
System.out.println (ops.get (0, 1)); // Get characters from the first start to the second end
System.out.println (ops.get (0, 5)); // Get data from the first to the sixth end, note that there are only two data
The
System.out.println (ops.size ()); // Get the number of data characters
ops.set ("stu03"); // Add data
System.out.println ("ops.set (stu03):" + ops.get ()); // Get added data
ops.set ("stu04", 0); // Add data at position 0
System.out.println ("ops.set (stu04, 0):" + ops.get ()); // Get the added data
The
// If there is data in the original string, using this method to set new data will fail and return false
System.out.println (ops.setIfAbsent ("stu04"));
System.out.println ("ops.setIfAbsent (stu04):" + ops.get ()); // Get the added data
The
stringRedisTemplate.delete (tableName2); // Delete this string
// If there is no data in the original string, using this method to set new data will succeed and return true
System.out.println (ops.setIfAbsent ("stu06"));
System.out.println ("ops.setIfAbsent (stu06):" + ops.get ()); // Get the added data
ops.set ("stu05", 30, TimeUnit.SECONDS); // Set 30 seconds to expire
}

public void boundZSetOps () {
System.out.println ("================== Zset ==============");
String tableName2 = "LiMing03";
BoundZSetOperations <String, String> ops = stringRedisTemplate
.boundZSetOps (tableName2);
The
System.out.println (ops.add ("stu01", 1)); // Add data in Zset
System.out.println (ops.add ("stu03", 1)); // Add data in Zset
The
System.out.println (ops.count (0, 1)); // Return the number of scores in the given interval
The
// Add data and score, return the size of the score, if stu04 exists, you only need to add the score
// For example: the original data stu04 score is 2.0, after the execution of the following statement becomes: stu04, score: 6.0
System.out.println (ops.incrementScore ("stu04", 4));
The
ops.intersectAndStore ("LiMing02", "LiMing04");
//ops.intersectAndStore(otherKeys, destKey);
ops.unionAndStore ("LiMing02", "LiMing05");
//ops.unionAndStore(otherKeys, destKey);
The
Set <TypedTuple <String >> sets = new HashSet <TypedTuple <String >> ();
TypedTuple <String> typedTuple = new TypedTuple <String> () {
The
@Override
public int compareTo (TypedTuple <String> o) {
// TODO Auto-generated method stub
return 0;
}
The
@Override
public String getValue () {
// TODO Auto-generated method stub
return "stu06";
}
The
@Override
public Double getScore () {
// TODO Auto-generated method stub
return 6.0;
}
};
sets.add (typedTuple);
ops.add (sets); // Add data
The
Set <String> set1 = ops.range (0, -1); // Return the elements of the specified range
for (String string: set1) {
System.out.println ("ops.range (0, 1):" + string);
}
The
Set <String> set2 = ops.rangeByScore (1, 4); // Return the elements of the specified score interval, including 1 and 4
for (String string: set2) {
System.out.println ("ops.rangeByScore (0, 4):" + string);
}
The
// Return the element at the specified position
Set <TypedTuple <String >> set3 = ops.rangeWithScores (0, 4);
for (TypedTuple <String> string: set3) {
System.out.println ("ops.rangeByScore (0, 4):" + string.getValue ()
+ "score:" + string.getScore ());
}
The
System.out.println (ops.remove ("stu01")); // Delete data, return the number of deleted data
The
Set <String> set5 = ops.range (0, -1); // Return the elements of the specified range
for (String string: set5) {
System.out.println ("ops.range (0, 1) 5:" + string);
}
The
ops.removeRangeByScore (1, 4); // The range of deleting score is the number of 1 and 4, including 1 and 4
The
Set <String> set6 = ops.range (0, -1); // Return the elements of the specified range
for (String string: set6) {
System.out.println ("ops.range (0, 1) 6:" + string);
}
The
ops.removeRange (0, 1); // Delete the data from the first start to the second end
The
Set <String> set7 = ops.range (0, -1); // Return the elements of the specified range
for (String string: set7) {
System.out.println ("ops.range (0, 1) 7:" + string);
}
}