Build and configure ngrok service for Intranet penetration tutorial, build ngrok

Build and configure ngrok service for Intranet penetration tutorial, build ngrok
Problem

With the development of the Internet ecosystem, more and more third-party services are being used in today's Web projects. Generally, these third-party services are initiated by the Client through the Server API, however, Server callback Client is also an inevitable method in many services. In such a scenario, developers may encounter the following problems:

In the development process, how does one make the development machine in the Intranet receive a callback?

Old solution 1

In traditional solutions, if there is no fixed ip address, a dynamic domain name is required first, and a port ing table from the Internet to the Intranet needs to be maintained, finally, if the Client has an operation that obtains the Host information, it also needs to respond to the Hack (this will be mentioned later ). Of course, if you do not have a public ip, you can simply give up this solution.

Solution 2

A more effective solution is to use a host with a public IP address to implement forwarding through a tunnel. As a matter of fact, I often solve this problem through SSH Tunnel to allow servers on the Intranet of the campus network to access the Internet.

# Forward the remote host 10086 to the local 3000ssh-C-f-N-g-R 10086: 127.0.0.1: 3000 user @ Tunnel_Server

Although this method is easy to use, it is not stable and will be automatically disconnected if no Tunnel is requested for a period of time. In addition, the user must have the ssh Login permission for Tunnel_Server. Every time a service is opened, the user must use a port of Tunnel_Server.

Ngrok

Just as I tried to write various Connection Scripts and daemon scripts for SSH Tunnel, I first came into contact with Ngrok. (2013) at that time, ngrok was a very cold tool, and it relied on the same Go tool. In addition to limited documentation, it did not set up this service after various attempts.

Later, tunnel.mobi maintained by the golden data team appeared in China, silently providing convenience for domestic developers for a long time. The rapid popularization of ngrok in China has benefited a lot from the influence of tunnel.mobi. However, after a year (2014.10-2015.10) of such an excellent service, the service was disabled.

Since then, various ngrok service providers in China have sprung up, and our company has a large dependence on ngrok, so I also set up an ngrok on our own server, it's almost the past year. During this period of time, we also found some inconveniences or unfriendly places. In addition, the previously built server is now overwhelmed, so I set up a new one over the weekend and made a little configuration optimization.

Server

My environment is Aliyun ECS + Ubuntu 14.04, dual NIC (intranet Nic + Internet Nic)

Source code Installation

First install the necessary tools:

sudo apt-get install build-essential golang mercurial git

Obtain ngrok source code:

git clone https://github.com/inconshreveable/ngrok.git ngrokcd ngrok

Compilation & installation:

sudo make release-serversudo cp bin/ngrokd /usr/local/bin/ngrokd

Apt-get installation (either)

If you do not need the latest version of ngrokd and are not interested in source code installation, you can be lazy.

sudo apt-get install ngrok-server

Domain Name

Select the domain name you want to use, such as yii. im, and add two domain names to your server.

Yii. im *. yii. im

Certificate

Ngrok communication relies on TLS certificates for encryption, so you need to specify your domain name and the corresponding certificate at startup.

Since the certificate is dependent, you should first have a certificate. When building the ngrok service, you can handle certificates in multiple ways:

Use the certificate issued by the CA, that is, the formal TLS certificate uses the self-signed certificate, and compile and distribute the client with the self-signed certificate using the self-signed certificate, using the general client, however, you need to add the self-signed certificate to your root certificate.

The first method is used in this article. The Domain Name Certificate is obtained through the free SSL Certificate of wotong CA.

For more information about the second method, see https://imququ.com/post/self-hosted-ngrokd.html. The popular ngrok services or tutorials on the Internet are basically based on this method.

In the third method, except for adding a root certificate, other configurations are the same as in this document.

Https support

Because ngrok works by allocating subdomains, all the domain names we actually use are yii. sub-domain name of im, such as pub. yii. to enable the https service for this subdomain name, im must support at least three points:

Ngrok supports https, which is the pub enabled by default. yii. im also needs to have a certificate or include it in a wildcard Domain Name Certificate. The browser (or other terminal) trusts pub. yii. im root certificate

Based on these three requirements, we will re-interpret the Processing Methods of the above three certificates:

First, because the free certificate is a Single Domain Name Certificate, you need to sign the certificate for a second-level domain name. Of course, if you have enough money, you can also buy a certificate containing all the second-level domain names.

Type 2: Self-signed certificates are easy to achieve the second point, but they are useless. no one except the self-compiled ngrok-client recognizes this certificate.

Third: https is supported, but the requirement that all users (including Access Users) add root certificates is slightly...

To sum up, we chose to discard https. Because https is not required in daily use, it is enough to run.

Manual funny

Startup settings

The ngrokd generated earlier is the ngrok server. You can start it by specifying the certificate, domain name, and Port:

# Obtain help information ngrokd-h # Usage of ngrokd: #-domain = "ngrok.com": Domain where the tunnels are hosted #-httpAddr = ": 80 ": public address for HTTP connections, empty string to disable #-httpsAddr = ": 443": Public address listening for HTTPS connections, emptry string to disable #-log = "stdout ": write log messages to this file. 'stdout' and 'none' have special meanings #-tlsCrt = "": Path to a TLS certificate file #-tlsKey = "": path to a TLS key file #-tunnelAddr = ": 4443": Public address listening for ngrok client # Try to start ngrokd-tlsKey = server. key-tlsCrt = server. crt-domain = yii. im-httpad DR =: 8081-httpsAddr =

In this step, the ngrok service has been running. You can view more information through the logs displayed on the screen. HttpAddr and httpsAddr are the ports used by ngrok to forward http and https services. I leave it blank because I do not need https. Ngrokd will also open port 4443 for communication with the client (you can specify it through-tunnelAddr = ": xxx"). If you have configured iptables rules, the TCP protocol on these ports must be released.

Now you can access the forwarding service provided by ngrok through http://sub.yii.im: 8081. Before the client is connected, you should see:

Tunnel sub. yii. im: 8081 not found

This indicates that everything is ready, and only the client is connected.

Port problems (optional)

The port on the url is usually not affected, and it is easy to hide it through nginx:

# ngrokd.confserver {    server_name *.yii.im;    listen 80;    location / {        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;        proxy_set_header Host $host:8081;        proxy_redirect off;        proxy_pass http://127.0.0.1:8081;    }}

But! Here is a very annoying place. ngrokd has its own Host processing, so proxy_set_header Host must carry the port listened by ngrokd. Otherwise, even if the request is forwarded to the corresponding port, ngrokd won't be handled correctly either.

Another problem occurs when you attach the port number: sub. yii. im, the Host you get in the web application is sub. yii. im: 8081. If your program contains Request Host-Based redirection, it will be redirected to sub. yii. im: Go below 8081.

To solve this port problem perfectly, you need to let ngrokd listen directly to port 80.

Generally, VPS is dual-nic (inside and out), which makes it a waste of ngrokd listening to 80 of the Internet. It is reasonable to leave this port to nginx. Therefore, the ideal method is: nginx listens to Internet 80, ngrokd listens to intranet 80, and nginx forwards the corresponding requests to intranet 80.

For example:

Intranet ip: 10.160.xx.xx Internet ip: 112.124.xx.xx

Start ngrokd:

sudo ngrokd -tlsKey=server.key -tlsCrt=server.crt -domain=yii.im -httpAddr=10.160.xx.xx:80 -httpsAddr=

Configure nginx:

# ngrokd.confserver {    listen      112.124.xx.xx:80;    server_name *.yii.im;    location / {        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;        proxy_set_header Host $host;        proxy_redirect  off;        proxy_pass      http://10.160.xx.xx:80;    }}# the_others_need_80.confserver {    listen      112.124.xx.xx:80;    #...}

If you are a single Nic, you can also solve it through docker: http://www.hteen.cn/docker/docker-ngrok.html

You can also manually add ip addresses.
[Root @ nginx conf] # ifconfig eth0: 0 192.168.3.60/24 up
[Root @ nginx conf] # ifconfig-
Eth0 Link encap: Ethernet HWaddr 00: 0C: 29: 12: 99: D4
Inet addr: 192.168.3.49 Bcast: 192.168.3.255 Mask: 255.255.255.0
Inet6 addr: fe80: 20c: 29ff: fe12: 99d4/64 Scope: Link
Up broadcast running multicast mtu: 1500 Metric: 1
RX packets: 792652 errors: 0 dropped: 0 overruns: 0 frame: 0
TX packets: 460481 errors: 0 dropped: 0 overruns: 0 carrier: 0
Collisions: 0 FIG: 1000
RX bytes: 972250327 (927.2 MiB) TX bytes: 92493014 (88.2 MiB)


Eth0: 0 Link encap: Ethernet HWaddr 00: 0C: 29: 12: 99: D4
Inet addr: 192.168.3.60 Bcast: 192.168.3.255 Mask: 255.255.255.0
Up broadcast running multicast mtu: 1500 Metric: 1

Maintenance script (optional)

Because ngrokd's startup command was often used by elders and occasionally found that it was dead and needed to be restarted, the command had been put together for half a day, so I wrote a maintenance script.

Note: Ubuntu is applicable. You need to modify Centos a bit.

Wget https://gist.githubusercontent.com/IvanChou/1be8b15b1b41bf0ce2e9d939866bbfec/raw/1a2445599fe7fd706505a6e103a9dc60b4d3a0ed/ngrokd-O ngrokd # modify the configuration vi ngrokdchomd + x ngrokdsudo mv ngrokd/etc/init. d/ngrokd in the script

TCP support-SSH etc. (optional)

Ngrok is TCP penetration, that is, as long as it is based on TCP communication, it can help us to penetrate, of course, also including SSH and mstsc.

During TCP connection, ngrok enables an additional port. If the Client does not specify a port, ngrokd will randomly open a large port. If you specify ngrokd to use port 10086, you can access the specified port of the Client through yii. im: 10086 after the connection is established.

We recommend that you allow a small number of large-caliber ports in iptables for backup.

Client download

Because the CA certificate is used, you do not need to compile the client on your own. You can download various ngrok v1.7 clients on the Internet. Theoretically, they are all available (some seem to have modified the client, you may not be able to use it because of other unknown reasons. Please skip this step)

October 2016: After the Mac is upgraded to 10.12, the old ngrok-client has a heartbeat bug, which causes frequent disconnections and the interface status remains unchanged. You need to download or compile the latest client.

Resource address routing (routing, routing) routing => http://pan.baidu.com/s/1b548fO

In MAC & Linux, you can put ngrok in/usr/local/bin/for backup.

Ngrok. yml

server_addr: "yii.im:4443"trust_host_root_certs: true

This configuration is used to specify the Server and authentication method:

The host in server_addr must be exactly the same as the certificate used by ngrokd as trust_host_root_certs whether to trust the system root certificate. If it is an ngrok client compiled with a self-signed certificate, this value should be set to false; if the CA certificate is used or the root certificate is added, set this value to true.