Peel the comparison to the original code 08: How does the original dashboard?

Author: freewind

Compared to the original project warehouse:

GitHub Address: Https://github.com/Bytom/bytom

Gitee Address: Https://gitee.com/BytomBlockc ...

In the previous articles, we have been studying how to establish a connection to a node that is more than the original and request chunk data from it. However, I soon ran into a bottleneck.

Because when I deal with the chunk data I get, I find that I have reached the core of the chain, the data structure of the blockchain and the processing of the fork. If this piece is not fully understood, there is no way to properly handle chunk data. However, it involves too much content, in a short period of time to understand it is a very difficult thing.

The way I did it was like I wanted to know a city, so I went along a road from the periphery to the center of the city. The front has been very smooth, but when the city center, found here many road miscellaneous, a little lost. In this case, I think I should suspend the study of the core, but from the other way to start, from the outside to come again. Because in the process of travel, I can gradually accumulate more knowledge, so that I am in the learning area rather than panic zone. The end of the road will also reach the core, but not in depth. In this way, after I have gone a few more roads, accumulated knowledge enough, and then study the core will not feel confused.

So this article is intended to study, when the other nodes to the chunk data sent to us, we should how to deal with, now instead of the original dashboard how to do it. Why choose this one? Because it's a very intuitive way to show the various information and functions that are available to us. In this article, we don't talk too much about the functionality above, but instead focus on how the dashboard is implemented at the code level. The function above it will be studied slowly in the future.

Our question today is "How does it come out of the original dashboard", but the problem is a bit big and not specific enough, so let's just break it up before we do it:

1. How do we enable the dashboard feature in the original?
2. What information and features are available in the dashboard?
3. How is the HTTP server implemented?
4. What kind of front-end frame does the dashboard use?
5. Dashboard the above data, in what way to get from the backstage?

Let's start with the discussion below.

How do we enable the dashboard feature in the original?

When we use bytomd node boot than the original node, do not need any configuration, it will automatically enable the dashboard feature, and will open the page in the browser, very convenient.

If this is the first run and the account has not been created, it will prompt us to create an account and the associated private key:

We can create the account alias, the key alias and the corresponding password, or click on "Restore Wallet" below to restore the previous account (if it was previously backed up):

When you click "Register", you will create a success and go to the Administration page:

Note that its address is: Http://127.0.0.1:9888/dashboard

If we look at the configuration file config.toml , you can see it in the figure:

fast_sync = truedb_backend = "leveldb"api_addr = "0.0.0.0:9888"chain_id = "solonet"[p2p]laddr = "tcp://0.0.0.0:46658"seeds = ""

Note that this api_addr is the address of dashboard and WEB-API. After startup, it takes BaseConfig.ApiAddress the corresponding value from the configuration file:

Config/config.go#l41-l85

type BaseConfig struct {    // ...    ApiAddress string `mapstructure:"api_addr"`    // ...}

The address is then used at startup, compared to the original Web API and dashboard, and dashboard is opened in the browser.

However, there is a strange problem here, that is, regardless of the value here, the browser will always open http://localhost:9888 this address. Why is it? Because it was written dead in the code.

In the code, http://localhost:9888 there are now three places, one is used to represent dashboard's access address, located node/node.go in:

Node/node.go#l33-l37

const (    webAddress               = "http://127.0.0.1:9888"    expireReservationsPeriod = time.Second    maxNewBlockChSize        = 1024)

Here webAddress , only when opening the browser from the code to display dashboard is used:

node/node.go#l153-l159

func lanchWebBroser() {    log.Info("Launching System Browser with :", webAddress)    if err := browser.Open(webAddress); err != nil {        log.Error(err.Error())        return    }}

Compared to the original through "github.com/toqueteos/webbrowser" this third-party library, you can invoke the system default browser when the node starts, and open the specified URL to facilitate the user. (Note that there are a number of typos in this code, such as, lanch broser that have been fixed in subsequent versions)

Another place that is used for bytomcli this command-line tool, but strangely it is placed util/util.go below:

Util/util.go#l26-l28

var (    coreURL = env.String("BYTOM_URL", "http://localhost:9888"))

Why do you say it belongs bytomcli to you? Because this coreURL is eventually used in util a function under the package to ClientCall(...) send the request from the code to the specified Web API and use its reply message. But this method is bytomcli used in the package in which it is located. If so, coreURL and the related function, it should be moved to the bytomcli package.

The third place, much like the second one, but tools/sendbulktx/core/util.go in, it's for another command-line tool sendbulktx :

Tools/sendbulktx/core/util.go#l26-l28

var (    coreURL = env.String("BYTOM_URL", "http://localhost:9888"))

It's exactly the same, right. In fact, not only here, there is a bunch of related methods and functions, it is exactly the same, a look at the second place with each other copied over.

With regard to the question here, I have mentioned two issue:

• The address of the dashboard and Web API is written in configuration file config.toml, but at the same time it is written in the code: it is really difficult to implement here, because in the configuration file, it is written 0.0.0.0:9998 , but when accessed from the browser or the command-line tool, You need to use a specific IP (not 0.0.0.0 ), otherwise some functionality will be unhealthy. In addition, you will see in the following code analysis that, in addition to this address in the configuration file, the address LISTEN of the corresponding address Web API will be taken precedence from the environment variable. So more research is needed to fix it correctly.
• There is a lot of duplication in the code associated with reading WEBAPI: The official explanation is that sendbulktx the tool will be separated from the Bytom project in the future, so the code is duplicated and, if so, acceptable.

What information and features are available in the dashboard?

Let's quickly go over what information and features are available from the original dashboard. Because in this article, the focus of our attention is not these specific features, so will not be scrutiny. In addition, the front just created a good account, a lot of data are not, in order to display convenience, I have done some data beforehand.

First, the key:

This shows the current number of keys, what their aliases are, and shows the primary public key. We can create multiple keys by clicking on the "New" button in the upper right corner, but no longer present here.

Account:

Assets:

By default, only BTM this asset is defined, and multiple assets can be added with the New button.

Balance:

It seems that I am still quite rich (but I can't use it).

Transaction:

Show a lot of deals, actually dug up in this machine. Since the dug-out BTM is transferred directly from the system to our account, it can also be viewed as a transaction.

To create a transaction:

We can also create our own deals like this, and transfer some of the assets we hold (like BTM) to another address.

Output not spent:

The simple understanding is that every transaction associated with me is recorded, with input and output sections, where the output may be the input to another transaction. This shows the output that has not yet been spent (it can be used to calculate how much of my current balance is left)

To view the core status:

Define access control:

Backup and restore operations:

Also below the left column of each page are the types of links (here solonet ), as well as the synchronization situation and the number of other nodes connected to the current node.

The information and functions shown here do not need to be scrutiny, but the nouns appearing here are to be noted, because they are the core concepts of the original. When we study later than the original inner blockchain core functions, it is actually around them. Each of these concepts may require one or more articles devoted to discussion.

What we're focusing on today is the technology implementation level, and we're going to start getting into the code time.

How is the HTTP server implemented?

First, let's start with the boot from the original node and always find the place to start the HTTP service:

Cmd/bytomd/main.go#l54-l57

func main() {    cmd := cli.PrepareBaseCmd(commands.RootCmd, "TM", os.ExpandEnv(config.DefaultDataDir()))    cmd.Execute()}

Cmd/bytomd/commands/run_node.go#l41-l54

func runNode(cmd *cobra.Command, args []string) error {    // Create & start node    n := node.NewNode(config)    if _, err := n.Start(); err != nil {    // ..}

node/node.go#l169-l180

func (n *Node) OnStart() error {    // ...    n.initAndstartApiServer()    // ...}

Soon found, initAndstartApiServer :

node/node.go#l161-l167

func (n *Node) initAndstartApiServer() {    // 1.    n.api = api.NewAPI(n.syncManager, n.wallet, n.txfeed, n.cpuMiner, n.miningPool, n.chain, n.config, n.accessTokens)    // 2.     listenAddr := env.String("LISTEN", n.config.ApiAddress)    env.Parse()    // 3.    n.api.StartServer(*listenAddr)}

As you can see, the method is divided into three parts:

1. Constructs an object by passing in a large number of arguments API . You'll see a lot of URL-related configurations when you go in.
2. First get LISTEN the corresponding value from the environment, if not, then use the config.toml value specified in the api_addr API service as the entry address
3. Really start the service

Since 2 is relatively simple, we will carefully analyze 1 and 3 below.

Find the corresponding method of 1 quarters first api.NewAPI :

api/api.go#l143-l157

func NewAPI(sync *netsync.SyncManager, wallet *wallet.Wallet, txfeeds *txfeed.Tracker, cpuMiner *cpuminer.CPUMiner, miningPool *miningpool.MiningPool, chain *protocol.Chain, config *cfg.Config, token *accesstoken.CredentialStore) *API {    api := &API{        sync:          sync,        wallet:        wallet,        chain:         chain,        accessTokens:  token,        txFeedTracker: txfeeds,        cpuMiner:      cpuMiner,        miningPool:    miningPool,    }    api.buildHandler()    api.initServer(config)    return api}

It is mainly to pass in the various parameters to hold, for later use. Then you api.buildHandler can configure the paths and handlers for each feature point, and use it api.initServer to initialize the service.

Enter api.buildHandler() . This method is a bit long and divides it into several parts to explain:

api/api.go#l164-l244

func (a *API) buildHandler() {    walletEnable := false    m := http.NewServeMux()

It seems that the HTTP service uses the package that comes with Go http .

Down is, when the user's wallet function is not disabled, will be configured with the wallet-related function points (such as accounts, transactions, keys, etc.):

    If A.wallet! = Nil {walletenable = True M.handle ("/create-account", Jsonhandler (A.createaccount))         M.handle ("/list-accounts", Jsonhandler (a.listaccounts)) M.handle ("/delete-account", Jsonhandler (A.deleteAccount)) M.handle ("/create-account-receiver", Jsonhandler (A.createaccountreceiver)) M.handle ("/list-addresses", JSON Handler (a.listaddresses)) M.handle ("/validate-address", Jsonhandler (a.validateaddress)) M.handle ("/create-as Set ", Jsonhandler (A.createasset)) M.handle ("/update-asset-alias ", Jsonhandler (A.updateassetalias)) M.Handle ( "/get-asset", Jsonhandler (A.getasset)) M.handle ("/list-assets", Jsonhandler (a.listassets)) M.handle ("/create -key ", Jsonhandler (A.pseudohsmcreatekey)) M.handle ("/list-keys ", Jsonhandler (A.pseudohsmlistkeys)) M.Handle ( "/delete-key", Jsonhandler (A.pseudohsmdeletekey)) M.handle ("/reset-key-password", Jsonhandler ( A.pseudohsmresetpassword)) M.handle ("/Build-transaction ", Jsonhandler (A.build)) M.handle ("/sign-transaction ", Jsonhandler (a.pseudohsmsigntemplates)) M.handle ("/submit-transaction", Jsonhandler (A.submit)) M.handle ("/estimate-transaction-gas", JsonHandler (A.estim Atetxgas)) M.handle ("/get-transaction", Jsonhandler (a.gettransaction)) M.handle ("/list-transactions", JsonHa Ndler (a.listtransactions)) M.handle ("/list-balances", Jsonhandler (a.listbalances)) M.handle ("/list-unspent-o Utputs ", Jsonhandler (a.listunspentoutputs)) M.handle ("/backup-wallet ", Jsonhandler (a.backupwalletimage)) M.H Andle ("/restore-wallet", Jsonhandler (A.restorewalletimage))} else {log. Warn ("Please enable wallet")}

The wallet feature is enabled by default and how can users disable it? The method is in the configuration file config.toml , plus this section of code:

[wallet]disable = true

In the previous code, when the feature point was configured, a lot of m.Handle("/create-account", jsonHandler(a.createAccount)) such code was used, what does it mean?

1. /create-account: The path to the function, for which the user needs to use the address in the browser or command line to http://localhost:9888/create-account access
2. a.createAccount: Used to process the user's access, such as to get the data provided by the user, after processing and then return a data to the user, will be explained below
3. jsonHandler: is a middle tier that takes the JSON data sent by the user to the go type parameter required by the 2nd step handler, or turns the go data returned by 2 into JSON to the user
4. m.Handle(path, handler): Used to correspond the function point path to the corresponding processing function.

Let's look at the code in step 3rd jsonHandler :

api/api.go#l259-l265

func jsonHandler(f interface{}) http.Handler {    h, err := httpjson.Handler(f, errorFormatter.Write)    if err != nil {        panic(err)    }    return h}

It is used inside httpjson , it is a package that is provided in the original code, located in Net/http/httpjson. Its main function is to add a layer of conversion between the HTTP access and go functions. Usually when the user interacts with the API via HTTP, the JSON data is sent and received, and we define the GO function in the handler of the 2nd step, which httpjson can be automatically converted between the two, so that when we write the Go code, There is no need to consider JSON and the HTTP protocol related issues. Accordingly, in order to work with Jsonhttp, the 2nd step of the handler in the format will have some requirements, details can be found here in the detailed note: NET/HTTP/HTTPJSON/DOC.GO#L3-L40. Because the Httpjson involved in the code is more, here is not detailed, after the opportunity to open an article.

Then we'll look at the code for the 2nd step a.createAccount :

Api/accounts.go#l16-l30

func (a *API) createAccount(ctx context.Context, ins struct {    RootXPubs []chainkd.XPub `json:"root_xpubs"`    Quorum    int            `json:"quorum"`    Alias     string         `json:"alias"`}) Response {    acc, err := a.wallet.AccountMgr.Create(ctx, ins.RootXPubs, ins.Quorum, ins.Alias)    if err != nil {        return NewErrorResponse(err)    }    annotatedAccount := account.Annotated(acc)    log.WithField("account ID", annotatedAccount.ID).Info("Created account")    return NewSuccessResponse(annotatedAccount)}

The contents of this function we do not scrutiny here, we need to pay attention to is its format, because the previous said, it needs to be jsonHandler used in conjunction with. The format requirement is probably, the first argument is Context that the second parameter is a parameter that can be converted from JSON data, the return value is a response and an error, but these four are all optional.

Let's go back api.buildHandler() and continue down:

    M.handle ("/", Alwayserror (errors. New ("not Found"))) M.handle ("/error", Jsonhandler (A.walleterror)) M.handle ("/create-access-token", Jsonhandler (a.cr Eateaccesstoken)) M.handle ("/list-access-tokens", Jsonhandler (A.listaccesstokens)) M.handle ("/delete-access-token ", Jsonhandler (A.deleteaccesstoken)) M.handle ("/check-access-token ", Jsonhandler (A.checkaccesstoken)) M.Handle ("/ Create-transaction-feed ", Jsonhandler (A.createtxfeed)) M.handle ("/get-transaction-feed ", JsonHandler (A.getTxFeed) ) M.handle ("/update-transaction-feed", Jsonhandler (A.updatetxfeed)) M.handle ("/delete-transaction-feed", Jsonhandler (A.deletetxfeed)) M.handle ("/list-transaction-feeds", Jsonhandler (a.listtxfeeds)) M.Handle ("/ Get-unconfirmed-transaction ", Jsonhandler (A.GETUNCONFIRMEDTX)) M.handle ("/list-unconfirmed-transactions ", Jsonhandler (A.LISTUNCONFIRMEDTXS)) M.handle ("/get-block-hash", Jsonhandler (A.getbestblockhash)) M.Handle ("/ Get-block-header ", Jsonhandler (A.getblockheadER)) m.handle ("/get-block", Jsonhandler (A.getblock)) M.handle ("/get-block-count", Jsonhandler (A.getblockcount)) m . Handle ("/get-difficulty", Jsonhandler (a.getdifficulty)) M.handle ("/get-hash-rate", Jsonhandler (A.gethashrate)) m. Handle ("/is-mining", Jsonhandler (a.ismining)) M.handle ("/set-mining", Jsonhandler (a.setmining)) M.handle ("/ Get-work ", Jsonhandler (A.getwork)) M.handle ("/submit-work ", Jsonhandler (A.submitwork)) M.handle ("/gas-rate ", JsonHa Ndler (a.gasrate)) M.handle ("/net-info", Jsonhandler (A.getnetinfo))

Can see the definition of various functions, mainly with the block data, mining, access control and other related functions, here is not detailed.

To continue:

    handler := latencyHandler(m, walletEnable)    handler = maxBytesHandler(handler)    handler = webAssetsHandler(handler)    handler = gzip.Handler{Handler: handler}    a.handler = handler}

Here is the function point configuration package defined previously as a handler, and then outside it wrapped layer after layer, adding more features:

1. latencyHandler: I can't tell you exactly what it's going to do, so I'll add it later.
2. maxBytesHandler: Prevents users from submitting data that is too large and is currently worth about 10MB . Valid for signer/sign-block URLs other than
3. webAssetsHandler: Provide users with dashboard related front-end page resources (such as Web pages, pictures, etc.). Perhaps for performance and convenience, front-end files are obfuscated, embedded in dashboard/dashboard.go as strings, and real code Https://github.com/Bytom/dashboard in another project. We'll take a look at the back.
4. gzip.Handler: Whether the HTTP client is supported for gzip detection, and if supported, uses GZIP compression when transferring data

Then let's go back to the main line and look at the NewAPI last call in the previous api.initServer(config) :

api/api.go#l89-l122

Func (a *api) initserver (config *cfg. Config) {//The Waithandler accepts incoming requests, but blocks until its underlying//handler are set, when the    Second phase is complete. var corehandler waithandler var handler http. Handler coreHandler.wg.Add (1) Mux: = http. Newservemux () Mux. Handle ("/", &corehandler) handler = mux if config.    Auth.disable = = False {handler = Authhandler (handler, a.accesstokens)} handler = Redirecthandler (handler) Secureheader. Defaultconfig.permitclearloopback = True Secureheader. Defaultconfig.httpsredirect = False Secureheader. Defaultconfig.next = Handler A.server = &http.        server{//Note:we should not set tlsconfig here;        We took care for TLS with the listener in Maybeusetls. Handler:secureheader. Defaultconfig, Readtimeout:httpreadtimeout, writetimeout:httpwritetimeout,//Disable HTTP/2 for N        ow until the Go implementation is more stable. Https://github.com/golang/go/issues/16450//https://github.com/golang/go/issues/17071 Tlsnextproto:map[stri Ng]func (*http. Server, *tls. Conn, http. Handler) {},} Corehandler.set (a)}

This method is not appropriate in this article, because it is more about the HTTP level of something, not the focus of this article. The interesting point is that the method created a go to provide http.Server , the front of our painstakingly configured handler plug in, everything is ready, only due to start.

Here's the start. We can finally go back to the latest initAndstartApiServer method, remember its 3rd piece of content? The main thing is to call n.api.StartServer(*listenAddr) :

api/api.go#l125-l140

func (a *API) StartServer(address string) {    // ...    listener, err := net.Listen("tcp", address)    // ...    go func() {        if err := a.server.Serve(listener); err != nil {            log.WithField("error", errors.Wrap(err, "Serve")).Error("Rpc server")        }    }()}

This piece is relatively simple, is to use go net.Listen to listen to the incoming Web API address, get the corresponding listener, and then pass it to the method we created earlier http.Server , we Serve are done.

This piece of code analysis is very painful to write, mainly because its Web API involves almost all of the features that are available from the original, very complicated. There are many things that are related to the HTTP protocol. At the same time, because exposed the interface, here is prone to security risks, so there are many code inside the user input, security checks and so on. These things are of course very important, but from the point of view of code reading is inevitably boring, unless we are to study security.

The main task of this paper is to study how to provide HTTP service, and what things are done in terms of security, and there will be a special analysis later.

What kind of front-end frame does the dashboard use?

The original front-end code is in another separate project: Https://github.com/Bytom/dash ...

In this article, we don't go into the details of the code, but just look at the front-end frames it uses, with a general impression.

By Https://github.com/Bytom/dashboard/blob/master/package.json we can probably understand that it is more used than the original front end:

1. Build tools: Direct Use npm ofScripts
2. Front-End Frame: React +Redux
3. CSS aspects:bootstrap
4. Javascript:es6
5. HTTP request:fetch-ponyfill
6. Resource Packaging:webpack
7. Test:mocha

Dashboard the above data, in what way to get from the backstage?

Take account-related code as an example:

Src/sdk/api/accounts.js#l16

const accountsAPI = (client) => {  return {    create: (params, cb) => shared.create(client, '/create-account', params, {cb, skipArray: true}),    createBatch: (params, cb) => shared.createBatch(client, '/create-account', params, {cb}),    // ...    listAddresses: (accountId) => shared.query(client, 'accounts', '/list-addresses', {account_id: accountId}),  }}

These functions are mainly through fetch-ponyfill the method provided in the library, to the previous use go to create a Web API interface to send HTTP requests, and get the corresponding reply data. And they will be called in the react component, and the returned data is used to populate the page.

In the same way, the more detailed content will not be spoken in this article.

Finally, after the analysis of this big article, I think I have some basic impression on how to make it out of the original dashboard. The rest, in the future, for the function of the detailed study.