Program RPC 1726 error issue tracking

Recently encountered a headache in the customer environment, one node through the RPC connection to the other node succeeded, but when sending RPC packets, but returned a 1726 error.

error Message

Let's take a look at MSDN's explanation, "This remote call failed", this sentence message content is really too little ah, error I must know that the remote call failed AH.

rpc_s_call_failed1726 (0X6BE) The remote procedure call failed.

And then Google, for the RPC 1726 error tracking is rarely a clear statement, how to do? Customers are urging, quickly help me solve!!! I searched the company's internal case on this issue, mostly due to firewalls, and many are non-Windows self-brought firewalls, such as some of IPS,IBM's packages and so on ... However, it was found that the Windows Firewall was turned off on the client's machine and there were no suspicious third-party firewall software. So is there a firewall for the intermediate nodes of the two node communication? The client said that the middle node's firewall would not block any traffic to RPC ... Hehe, the customer said, always have to remain skeptical, the fact that they are finally proved wrong.

This time the code is reviewed, on the other hand, Google is constantly tracking the problem. Customers are also anxiously urged, this time found a Microsoft technical article described as follows:

Explanationa server connection is lost while the server is attempting to perform a remote procedure call. It is unknown whether the remote procedure call executed, or how much of it executed. The connection might has broken because of a problem with the network hardware or because a process terminated.   User actionwait a few minutes and then try the operation again. If This message is reappears, check the server or try to connect to another server. You might also has to check the integrity of the network. If the problem persists, contact the supplier of the running application.

First of all, Microsoft said, this problem is generally caused by two reasons: network hardware problems (should be counted as network firewall problems) or the process of RPC server terminated. To be sure that our RPC server process does not terminate, then we have to suspect that the network caused by the cause!

Problem Tracking

This direction is clear enough, so we use a non-mixed-mode grab on both client side and server side of RPC in our customer environment. With non-mixed mode capture, it is ensured that the crawled network packet is a designated native NIC. Set the Wireshark non-mixed mode as shown, and start grabbing the package:

Then reproduce the RPC 1726 error problem and stop the client side and server side of the grab. To begin the analysis, first set the following filter in the Wirehshark filter on the client side:

(ip.src_host==x.x.x.98 && ip.dst_host==x.x.x.207) | | ((ip.src_host==x.x.x.207 && ip.dst_host==x.x.x.98)) && Dcerpc

where "x.x.x.98" is the IP address of the client, "x.x.x.207" is the server-side address, which indicates that only RPC packets that show client and server interaction (TCP/IP-based RPC communication is used by default in our program) According to the contents of RPC request, find the corresponding request package, you can see that the selected RPC request package was sent out by the client, and did not receive the response package from the server!

Is that the problem with RPC server processing? We were on the server side, using the same Wireshark filter, to filter out the RPC packets that were communicated between the client and the server, and did not find that the RPC request was received. This network packet was swallowed by the network firewall??? Although this time has basically proved that it is not the problem of our products, but there is no more powerful evidence?

TCP Connection Reset

This is the time to look at TCP messages, first on the client side using the following filter:

(ip.src_host==x.x.x.98 && ip.dst_host==x.x.x.207) | | ((ip.src_host==x.x.x.207 && ip.dst_host==x.x.x.98))

See in, after sending No. After the request of 3372, a TCP Reset package was received!!! This is what firewalls do. This means that the client and server connections are disconnected.

But is this connection reset really server sent to client? After analyzing the server with the same Wireshark filter, it was found that the server did not send the TCP reset packet, and at the same time received the TCP Reset packet of IP source as client, The client can also be found in the client's Wireshark capture packet without sending the TCP reset message to the server. Then this time basically can be sure, the client and server in the middle of a node's firewall, shut down this connection!!!

The merit seems to be gaocheng, since the root cause is not our product, then always let their network administrator to find it. But customers always think that their firewalls are OK, he only wants our products to run properly ... Because customers in the United States, so each time is WebEx, in such a network environment is more suitable for network administrators to slowly chase the investigation.

We want their administrators to look for, they do not cooperate with the administrator, it has to let our products run normally, it can only find other feasible solutions. Finally, the feasible solution is to use RPC based on namepiped to communicate, is also a last resort.

Reference Articles

RPC Error:

Http://www.microsoft.com/technet/support/ee/transform.aspx? Prodname=windows+operating+system&prodver=5.0&evtid=1726&evtsrc=rpc

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Program RPC 1726 error issue tracking