Java experience (1. Postfix mail delivery performance and effective delivery issues)

Business requirements:

Need to regularly send ads to registered members, questionnaires and other mail, daily need to complete the millions mail sent.

Server:
several postfix servers

Problems encountered:
The postfix server is easy to complete for a daily millions mail sending task, and the Java client contains the business logic processing time when it is sent, and several threads are easy to do.
However, if the sending speed is too fast, will be blacklisted by the mail service provider and reject the message, and each mail service provider to receive the message speed requirements are not the same.

Workaround:
In this way, you need to send a bulk task for each message and send it according to the speed at which each mail service provider can receive it.
For example, 163 per second 3, Sina 4 per second, and so on.

So we designed the following structure of the mail delivery system:

Use Redis to store the last send time for each domain name,

Repeat, A, b
A) obtain and lock the domain name that can be sent.
b) For each of the domains that can be sent,
1. If a pending message exists in the corresponding domain name queue, send and update the last send time of the changed domain name.
2. Unlock the domain to send records.

In this way, the non-blocking queue is used for message queue, which improves the concurrency of the system.

It seems to solve the problem, but it was blacklisted by some service providers.

The system is then analyzed as follows:
This is a typical producer-to-consumer issue in which Java client-produced messages are timed to be pushed to the postfix queue, and then postfix messages on the consumer queue are sent to the target mail server.
Produce one, send a letter, there will be no problem, but if there are multiple in the queue, then the target mail server delivery speed can not be controlled.

Then check the postfix queue:
Postqueue-p

The discovery did accumulate some mail.
Cause: During a system run, a delay in sending a message causes a delay in subsequent mail delivery.

Disposal Method:
The Java side of the transmission speed control is obviously a palliative, so the Java side of the same logic to modify the postfix in the mail sent code,
With the same configuration file, it ultimately solves the problem of controlling the sending speed of mail.

Java experience (1. Postfix mail delivery performance and effective delivery issues)