Oracle Coherence 3.5 Reading notes 3-meet performance, scalability, and availability goals

Meet Performance goals

Algorithms and data structures are important factors that affect the execution time of an operation. In the distributed system, there is also an important factor is the network delay.

Processing delay

Developers often do not consider latency during development, as is the case with testing.

The author gives an example of a 20ms operation (typical web operation completion time) where only 0.067ms latency is performed locally, and 264ms is required for cross-country execution.

Typically, a Web request requires multiple trips to the Web server, and the latency is even more severe. Therefore, you should reduce the number of round trips, even for memory access.

The usual recommendations for "make your remote services coarse grained" or "batch multiple operations together" also mean the same thing.

Reduce bandwidth usage

According to Moore's law, bandwidth is usually not a problem.
Network delay is related to the speed of light and is not easy to improve. Bandwidth is increasing, but it does not mean that the bandwidth problem can be ignored.

In any case, you should minimize the I/O on the network, get only the necessary information, and if the amount of data is too large, devolve the processing to the data rather than the opposite.

Coherence and performance

Coherence can address latency and bandwidth issues by following several means:

• Reduce latency by reducing the data cache for the back-end Database
• and cache recently used data by Near-cache to reduce latency
• Data nodes can be executed in parallel, reducing latency and reducing bandwidth usage

Meet Scalability Goals

Scalability can be achieved in two ways, scaling up or scaling out

Scale up:
Turning small servers into large servers by adding cpu/disks/memory, etc.
The problem is balance, sometimes adding CPU to find that the memory becomes a new bottleneck, more expensive.

Scale out:
Add more clusters, and share workloads. Less expensive.
However, more complex design and management, such as the need to eliminate the single point of failure from the architecture, the need to ensure the service of non-stateful (stateless)

Stateless service does not exist

The application layer is stateless for better expansion. But the application still needs the state that this is constant.
Typically, the state is moved down to the most difficult database layer to scale.

Extending the database is difficult

The database must meet acid (atomicity, consistency, isolation, durability).

Disk systems are important: in order to increase concurrency, the disk needs to be properly distributed, and in order to ensure durability, the database is ultimately limited by disk performance.

When the amount of data increases and user access increases, the database will always reach its limit, which needs to be expanded. The scale up method is usually used, but this method is expensive and unsustainable.

At this point, you can instead consider the scale out method.

Database scale-out method

See previous: Three ways to scale-out a relational database

Back to State

Removing the state from the application greatly increases the burden on the database, and in order to reduce the burden on the database, we also need to get the state back to the application layer.
Of course it is not added to the stateless service, but a new level is introduced between the stateless application logic and the database.

As Professor Bellovin said:

Any software problem can is solved by adding another layer of indirection
The indirection here can be understood as abstraction.

This new abstraction layer requires the following capabilities:
* Manage object data, because apps can manipulate objects directly
* objects are stored in memory to improve performance and reduce I/O
* Can transparently load back-end database data into memory
* Data modifications can be persisted to the backend, usually asynchronously
* Easy to scale out, like a stateless application layer

Coherence meet all of the above requirements

Reduce database burden with Coheren

The usual database queries are based on primary keys, and transferring these queries to the application-tier cache can greatly reduce the burden on the backend database.

Coherence's distributed architecture can also do more complex work.

With coherence, master-slave replication is unnecessary. Read-only slave libraries are replaced by distributed caches, and there is no problem with the data inconsistency between the main library and the library.

The backend database still requires clustering, but coherence reduces the pressure on the backend database, the cluster can be simple, the nodes can be reduced.

In terms of sharding, the coherence has built-in distributed queries and aggregations features. Make multi-node queries simple, applications don't need to make too many changes, and data is protected by replication.

Coherence and scalability

Coherence is the ideal scalable data management solution.
By adding nodes, you can add capacity and throughput (processing power).
Of course, the application still needs a good design of the architecture, the use of a bad product too many cases.
Good product + poor design = Poor product

Meet High availability targets

To achieve high-availability targets, we need to eliminate all single points of failure in the architecture.
Overall availability = Product of the availability of all parts
as = A1 * A2 * ... *
This also means that a part is not available and the entire system is not available.
So we need to do two things:
1. Provide redundancy for each component
2. Loose coupling of the parts, but fault isolation

2nd, usually by introducing asynchronous implementations, such as the introduction of Message Queuing, file caching, and so on. Coherence can also cache updates, even if the backend database fails. Asynchrony can also improve throughput.

Add redundancy to the system

F = F1 * F2 * ... * Fn
F is the possibility of component failure
Fc = 1-ac (availability)

For example, a system has 3 parts, and the availability of each component is 0.99;
As = 0.99 * 0.99 * 0.99 = 97%
The probability of failure for each component is 1-0.99 = 0.01
If you add a redundant part to each part.
The probability of failure for each component is 0.01x0.01 = 0.0001
New as = (1-0.0001) * (1-0.0001) * (1-0.0001) = 99.97%
Usability has improved.

Redundancy alone is not enough.

Redundancy alone is not enough, and it requires the system to have enough capacity to meet peak requests and avoid system crashes.

If the system requires n servers to handle peak requests, assuming that the system can still work after the X server fails, the system needs to n+x the server. Otherwise, if the server fails at peak time, the remaining servers will not be able to process the request, resulting in reduced response times, degraded performance, or even service.

Coherence and Availability

Coherence is designed to be highly available, and any node failure will have no effect. This is achieved by adding replication to the cluster.
Each object has redundancy in the other node, and of course the update to the object increases overhead, but the overhead is much smaller than the cluster database. However, the sizing of the coherence cluster is still important.
In addition, the high availability of the coherence cluster does not mean that the entire system is highly available. Other such as load balancing, routers, switches and so on need redundancy.

Comprehensive considerations for performance and availability design

In order to achieve performance and availability objectives, it is necessary to locate single points of failure and bottlenecks in the system and eliminate them, which requires careful design of the architecture and scalability at the outset of the architecture.

We should forget about small efficiencies, say about 97% of the Time:premature optimization are the root of all evil.

A lot of people are out of context, only see the following sentence: Premature optimization is the root of all evil, thereby escaping the complex architecture design. Latency and scalability are by no means trivial and must be properly considered at the beginning of system design.

Scalability is a prerequisite to functionality, a priority-0 requirement, if ever there was one.

Randy Shoup-ebay Architect

Set performance targets at each level

User response time of not more than 2 seconds, such a goal is not enough, we need to continue to decompose, such as how much time on the network, the background processing how long, access to the database how much time and so on. So we can accurately locate the bottleneck and do the corresponding optimization.

Measurement and monitoring

Count What's countable, measure what's measurable, and what's isn't measurable, make measurable.

– Galileo

Whether the application meets the goal requires measurement. such as measuring the time of service execution, measuring the load that a single server can withstand, and the load that can be sustained after expansion, so that a reasonable planning system, and the cost of the expected expansion of the required investment.

The measurement tools that you can use during the development phase include:
* Server-side code-Yourkit (www.yourkit.com),
* Client Web Measurement tool-FireBug (getfirebug.com), YSlow (developer.yahoo.com/yslow) or page speed (code.google.com/p/page-speed)
* Pressure test Tool-HP LoadRunner, or an open source tool such as Apache JMeter (Jakarta.apache.org/jmeter), the Grinder (GRINDER.S ourceforge.net), or Pylot (www.pylot.org).

Monitoring tools in the run phase:
* JMX, or ERMA (erma.wikidot.com),
* Coherence Monitoring-JConsole
* Commercial Tools-evident ClearStone Live (www.evidentsoftware.com) or SL Rtview (www.sl.com).

Educate your team

Let the team achieve consistent performance and availability goals, and understand the issues that need to be overcome to reach the goal.

The recommended books are:
1. Scalable Internet Architectures-theo Schlossnagle
2. Building Scalable Websites-cal Henderson
3. High performance websites:essential knowledge for front-end Engineers
4. Even Faster Web sites:performance best Practices for Web Developers-steve souders

Summarize

This chapter discusses how to achieve performance, availability, and scalability goals, and how coherence can help us achieve our goals.

To improve performance, you need to reduce the amount of data that is delayed and moved. Coherence's near caching and entry processors can help us achieve this goal.

By introducing coherence into the architecture, coherence can achieve the most difficult layer of scaling-the data management layer scale-out to avoid complex database expansions such as cluster and sharding.

To achieve high availability, you must eliminate any single point of failure, although coherence is designed to be highly available, but you also need to ensure that other components are equally highly available. Redundancy can often be introduced through replication.

Finally, it needs to be emphasized that performance, scalability, and availability should be considered at the time of the system's initial design and need to be measured and monitored at runtime, rather than later remediation.

Oracle Coherence 3.5 Reading notes 3-meet performance, scalability, and availability goals