Build Your own Ngrok service

Reprint: http://tonybai.com/2015/03/14/selfhost-ngrok-service/

In the domestic development of the public number, enterprise number and to do the front-end development of friends must be not unfamiliar with Ngrok, at present, Ngrok is the best in-Network debugging services tunnel tools. Remember this year before the Spring Festival, Ngrok.com provides services are all normal, but after the spring festival seems to be all abnormal. Ngrok.com cannot access, Ngrok Although can connect ngrok.com to provide the service, but the end because cannot access ngrok.com, cause the message can not send to our service address, for example xxxx.ngrok.com. All this shows that the ngork was the wall. Without Ngrok tunnel, everything began to become difficult and inefficient. Intranet to external host deployment and debugging is a slow thing that people want to squalling.

Ngrok not less. Ngrok and its server ngrokd are open source, I also know that through the source code can be self-built Ngrok services. After the search engine, found that a friend in the country has built a www.tunnel.mobi Ngrok Public services, similar to ngrok.com, I also experimented.

Write a ngrok.cfg that reads as follows:

SERVER_ADDR: "tunnel.mobi:44433"
Trust_host_root_certs:true

Execute the following command with Ngrok latest client version 1.7:

$ngrok-subdomain Tonybaiexample-config=ngrok.cfg 80

A tunnel can be successfully established for this machine to externally provide "tonybaiexample.tunnel.mobi" services.

Tunnel Status Online
Version 1.7/1.7
Forwarding Http://tonybaiexample.tunnel.mobi-127.0.0.1:80
Forwarding Https://tonybaiexample.tunnel.mobi-127.0.0.1:80
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

and the domestic Ngrok service is obviously much faster than the service provided by ngrok.com, the news instant reach.

But this is the result of direct access on the public Internet. Inside the company, what I see is another result:

Tunnel Status Reconnecting
Version 1.7/
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

We can not build tunnel from the intranet, which means it is still inconvenient and inefficient, because many basic services are deployed in the intranet, and the interaction between intranet and Internet is very inconvenient. But the intranet is not connected to Tunnel.mobi is also a fact, and can not know why, because the server side of the connection error log is not visible.

So I decided to build my own Ngrok service.

First, the preparatory work

Build Ngrok service need to have a VPS in the public network, the end of last year in Amazon applied for an experience host EC2, a public network IP, this time intends to use this host as a NGROKD server.

Need a domain name of your own. If you have a domain name, you can create a subdomain to associate with the Ngrok service, which will not interfere with the service provided by the original domain name. (The way you don't use a domain name may be, but I haven't tried it.) ）

The construction of reference materials mainly from the following three:
1) Ngrok's official Selfhost guide: https://github.com/inconshreveable/ngrok/blob/master/docs/SELFHOSTING.md
2) A friend's blog abroad: http://www.svenbit.com/2014/09/run-ngrok-on-your-own-server/
3) "Shipping Blog" in an article: http://www.haiyun.me/archives/1012.html

second, the actual exercise Steps

My AWS EC2 Instance installs Ubuntu Server 14.04 x86_64 and installs Golang 1.4 (Go version go1.4 linux/amd64). Golang is necessary for compiling NGROKD and Ngrok, it is recommended to download the binary installation package of the corresponding platform directly from the Golang (domestic can be downloaded from the golangtc.com, slower).

1. Download Ngrok Source code

(gopath=~/goproj)
$ mkdir ~/goproj/src/github.com/inconshreveable
$ git clone https://github.com/inconshreveable/ngrok.git
$ Export Gopath=~/goproj/src/github.com/inconshreveable/ngrok

2. Generate a self-signed certificate

When using the Ngrok.com official service, we use the official SSL certificate. Self-built NGROKD services, we need to generate our own certificates and provide the Ngrok client that carries the certificate.

The certificate generation process requires a ngrok_base_domain. Take Ngrok official randomly generated address 693c358d.ngrok.com as an example, its ngrok_base_domain is "ngrok.com", if you want to provide the address of the service is "example.tunnel.tonybai.com", That Ngrok_base_domain should be "tunnel.tonybai.com".

Here we take ngrok_base_domain= "tunnel.tonybai.com" as an example, the command to generate the certificate is as follows:

$ CD ~/goproj/src/github.com/inconshreveable/ngrok
$ OpenSSL genrsa-out Rootca.key 2048
$ OpenSSL req-x509-new-nodes-key rootca.key-subj "/cn=tunnel.tonybai.com"-days 5000-out Rootca.pem
$ OpenSSL genrsa-out Device.key 2048
$ OpenSSL req-new-key device.key-subj "/cn=tunnel.tonybai.com"-out DEVICE.CSR
$ OpenSSL x509-req-in device.csr-ca rootca.pem-cakey rootca.key-cacreateserial-out device.crt-days 5000

After executing the above command, the Ngrok directory will be reborn into 6 files:

-rw-rw-r–1 ubuntu ubuntu 1001 Mar 02:22 DEVICE.CRT
-rw-rw-r–1 ubuntu Ubuntu 903 Mar 02:22 DEVICE.CSR
-rw-rw-r–1 ubuntu Ubuntu 1679 Mar 02:22 Device.key
-rw-rw-r–1 ubuntu Ubuntu 1679 Mar 02:21 Rootca.key
-rw-rw-r–1 ubuntu ubuntu 1119 Mar 02:21 Rootca.pem
-rw-rw-r–1 ubuntu Ubuntu 02:22 ROOTCA.SRL

Ngrok Package The Assets directory (resource file) in the Ngrok source directory into the executable (NGROKD and Ngrok) via Bindata, Assets/client/tls and assets/server/ The default certificate files for Ngrok and NGROKD are stored under TLS, and we need to replace them with our own generated: (so this step must be placed before compiling the executable file)

CP Rootca.pem ASSETS/CLIENT/TLS/NGROKROOT.CRT
CP DEVICE.CRT ASSETS/SERVER/TLS/SNAKEOIL.CRT
CP Device.key Assets/server/tls/snakeoil.key

3. Compiling NGROKD and Ngrok

In the Ngrok directory, execute the following command to compile the NGROKD:

$ make Release-server

On my AWS, however, the following error occurred:

goos= "" goarch= "" Go get github.com/jteeuwen/go-bindata/go-bindata
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/client/assets/assets_release.go \
assets/client/...
Make:bin/go-bindata:command not found
Make: * * * [client-assets] Error 127

Go-bindata was installed in the $gobin, the go compiler could not find it. The fix is to copy the $gobin/go-bindata to the current ngrok/bin.

$ cp/home/ubuntu/.bin/go14/bin/go-bindata./bin

Execute make Release-server again.

~/goproj/src/github.com/inconshreveable/ngrok$ make Release-server
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/client/assets/assets_release.go \
assets/client/...
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/server/assets/assets_release.go \
assets/server/...
Go get-tags ' release '-d-v ngrok/...
Code.google.com/p/log4go (Download)
Go:missing Mercurial command. See Http://golang.org/s/gogetcmd
Package code.google.com/p/log4go:exec: "HG": Executable file no found in $PATH
Github.com/gorilla/websocket (Download)
Github.com/inconshreveable/go-update (Download)
Github.com/kardianos/osext (Download)
Github.com/kr/binarydist (Download)
Github.com/inconshreveable/go-vhost (Download)
Github.com/inconshreveable/mousetrap (Download)
Github.com/nsf/termbox-go (Download)
Github.com/mattn/go-runewidth (Download)
Github.com/rcrowley/go-metrics (Download)
Fetching Https://gopkg.in/yaml.v1?go-get=1
Parsing meta tags from https://gopkg.in/yaml.v1?go-get=1 (status code 200)
Get "gopkg.in/yaml.v1": Found meta tag main.metaimport{prefix: "Gopkg.in/yaml.v1", VCS: "Git", Reporoot: "https:// Gopkg.in/yaml.v1 "} at Https://gopkg.in/yaml.v1?go-get=1
GOPKG.IN/YAML.V1 (Download)
Make: * * * [deps] Error 1

It's a mistake! Tip No HG was found and it turns out that HG is not installed on AWS. Install HG (sudo apt-get install mercurial), then compile.

$ make Release-server
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/client/assets/assets_release.go \
assets/client/...
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/server/assets/assets_release.go \
assets/server/...
Go get-tags ' release '-d-v ngrok/...
Code.google.com/p/log4go (Download)
Go install-tags ' release ' Ngrok/main/ngrokd

Similarly compiled Ngrok:

$ make Release-client
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/client/assets/assets_release.go \
assets/client/...
Bin/go-bindata-nomemcopy-pkg=assets-tags=release \
-debug=false \
-o=src/ngrok/server/assets/assets_release.go \
assets/server/...
Go get-tags ' release '-d-v ngrok/...
Go install-tags ' release ' Ngrok/main/ngrok

NGROKD and Ngrok on AWS are installed under $gobin.

Third, commissioning

1. Start Ngrokd

$ ngrokd-domain= "tunnel.tonybai.com"-httpaddr= ": 8080"-httpsaddr= ": 8081"
[03/14/15 04:47:24] [INFO] [Registry] [Tun] No Affinity Cache specified
[03/14/15 04:47:24] [INFO] [Metrics] Reporting every seconds
[03/14/15 04:47:24] [INFO] Listening for public HTTP connections on [::]:8080
[03/14/15 04:47:24] [INFO] Listening for public HTTPS connections on [::]:8081
[03/14/15 04:47:24] [INFO] Listening for control and proxy connections on [::]:4443
... ...

2, public network connection Ngrok D

Download the generated Ngrok to your computer.

Create a configuration file, Ngrok.cfg, as follows:

SERVER_ADDR: "tunnel.tonybai.com:4443"
Trust_host_root_certs:false

Execution Ngrok:
$ ngrok-subdomain EXAMPLE-CONFIG=NGROK.CFG 80

Tunnel Status Reconnecting
Version 1.7/
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

The connection failed. At the moment my computer is on the public web. View the log of the NGROKD, no discovery connection reached the server side. Try to ping the address locally and find the address tunnel.tonybai.com. Is there a problem with DNS settings? Previously I just set the "*.tunnel.tonybai.com" a address, not set "tunnel.tonybai.com". As a result, a record of "tunnel.tonybai.com" was added to the DNS Administration page.

After the DNS records refresh OK, start Ngrok again:

Tunnel Status Online
Version 1.7/1.7
Forwarding http://epower.tunnel.tonybai.com:8080-127.0.0.1:80
Forwarding https://epower.tunnel.tonybai.com:8080-127.0.0.1:80
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

This connection was successful!

3. Intranet Connection Ngrokd

Copy the Ngrok to a PC on the intranet, which sets up the company's agent.

Follow the same steps to start Ngrok:

$ ngrok-subdomain EXAMPLE-CONFIG=NGROK.CFG 80

Tunnel Status Reconnecting
Version 1.7/
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

Unfortunately, how failed again! From the server side, or did not receive the client connection, it is obvious that the connection did not get through the company intranet. Judging from my own squid proxy server, it seems that only 443 port requests are allowed through the company proxy server, and 4443 cannot go out.

1426301143.558 9294 10.10.126.101 tcp_miss/000 366772 CONNECT api.equinox.io:443–default_parent/proxy.xxx.com-passed the
1426301144.441 10.10.126.101 tcp_miss/000 1185 CONNECT tunnel.tonybai.com:4443–default_parent/proxy.xxx.com- Didn't seem to pass

Only the server listening port can be modified. Change-tunneladdr from 4443 to 443 (note that the firewall's port rules need to be modified on AWS, which takes effect in real time without restarting the instance):

$ sudo ngrokd-domain= "tunnel.tonybai.com"-httpaddr= ": 8080"-httpsaddr= ": 8081" -tunneladdr= ": 443"
[03/14/15 04:47:24] [INFO] [Registry] [Tun] No Affinity Cache specified
[03/14/15 04:47:24] [INFO] [Metrics] Reporting every seconds
[03/14/15 04:47:24] [INFO] Listening for public HTTP connections on [::]:8080
[03/14/15 04:47:24] [INFO] Listening for public HTTPS connections on [::]:8081
[03/14/15 04:47:24] [INFO] Listening for control and proxy connections on [::]:443
... ...

Change the address in ngrok.cfg to 443:

SERVER_ADDR: "tunnel.tonybai.com:443"

Execute the Ngrok client again:

Tunnel Status Online
Version 1.7/1.7
Forwarding http://epower.tunnel.tonybai.com:8080-127.0.0.1:80
Forwarding https://epower.tunnel.tonybai.com:8080-127.0.0.1:80
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

This time the success is connected.

4, 80 port

Is it done? When we look at the results of Ngrok, what do we feel wrong? Oh, what is the forwarding address of port 8080? Why not 80? The public number/enterprise number can only support port 80!

We also need to modify the server-side parameters to change the-httpaddr from 8080 to 80.

$ sudo ngrokd-domain= "tunnel.tonybai.com"-httpaddr= ":"-httpsaddr= ": 8081" -tunneladdr= ": 443"

This time again with Ngrok connection:
Tunnel Status Online
Version 1.7/1.7
Forwarding http://epower.tunnel.tonybai.com-127.0.0.1:80
Forwarding https://epower.tunnel.tonybai.com-127.0.0.1:80
Web Interface 127.0.0.1:4040
# Conn 0
AVG Conn Time 0.00ms

This is a match for our needs.

5. Testing

Build a simple HTTP server program on your intranet PC: Hello

Hello.go
Package Main

Import "Net/http"

Func Main () {
http. Handlefunc ("/", hello)
http. Listenandserve (":", nil)
}

Func Hello (w http. Responsewriter, R *http. Request) {
W.write ([]byte ("hello!"))
}

$ go build-o Hello Hello.go
$ sudo./hello

Access the "http://epower.tunnel.tonybai.com" address via the public web browser, if you see the browser return "hello!" Words, then your NGROKD service has been built successfully!

Iv. Matters of note

The value after server_addr in client ngrok.cfg must be exactly the same as-domain and Ngrok_base_domain in the certificate, otherwise the server side will receive the following error log:

[03/13/15 09:55:46] [INFO] [tun:15dd7522] New connection from 54.149.100.42:38252
[03/13/15 09:55:46] [DEBG] [tun:15dd7522] Waiting to read message
[03/13/15 09:55:46] [WARN] [tun:15dd7522] Failed to read Message:remote error: Bad certificate
[03/13/15 09:55:46] [DEBG] [tun:15dd7522] Closing

Build Your own Ngrok service