Nodejs simple realization of Chinese and English translation

Source: Internet
Author: User
Tags foreach manual json readfile require

This article mainly introduced the Nodejs simple realization Chinese and English translation method and the example, although still has the very big question, but is also the basic can use, here recommends to everybody.

Help former colleagues to solve a demand, Chinese project translation English Project ~ ~ ~

Given the specific implementation of the problem, if smart, it is necessary to do the Chinese grammar analysis, but it is difficult to feel this.

So the final scheme is to traverse the file, the Chinese phrase to match out, and then artificial translation, the Chinese phrase to replace the content of the translation. Of course, later still need to manually test, after all, the code in Chinese, may affect the relevant procedures.

This problem, obviously involves multithreading, file reading and writing, the first time to think of is Nodejs, although Nodejs is a main thread, but asynchronous file read and write, event response mechanism, must also call the thread, in the actual programming time does not need to consider threading related issues.

The code is not complex as follows, after written, the appropriate encapsulation of the next

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26-27--28 29---30 31--32 33 34 35 36 37 38-39 40 41 42 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 5, 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 11 9 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139-140 var fs = require (' FS '); var http = require (' http '); var filePath = ' d:work_new '; var logPath = ' D:chinese.log ';   var map = {}; var num = 0;   var dictionary = (function () {var map = {}; return {logPath: ' D:chinese.log ', Set:function (key, Val) {Map[key ] = Val | | ''; }, Get:function (key) {return map[key]| | ';}, Save2file:function () {fs.writefile (This.logpath, json.stringify (map). R Eplace (/","/g, ' ", RN"), {encoding: ' UTF8 ', flag: ' W '}, Function (err) {if (err) throw err;}); }, Loadfile:function (callback) {Fs.readfile (This.logpath, {encoding: ' UTF8 '},function (err, data) {map = Json.parse (dat a); Callback (); }), Translatebygoogle:function (callback) {var index = 0; for (var key in map) {if (map[key] = = ') {index++; (function (key) {Http.get ("http://translate.google.cn/translate_a/t?client=t&hl=zh-CN&sl=zh-CN&tl=en &ie=utf-8&oe=utf-8&oc=2&otf=1&ssel=3&tsel=6&sc=2&q= "+key, function (res) { Res.setencoding (' UTF8 '); VarBODY = ""; Res.on (' Data ', function (chunk) {body+=chunk;}). On (' End ', function () {var obj = eval (' (' +body+ ') '); Map[key] = obj[0][0][0]; index--; if (index = = 0) {callback ();}}) ; ). On (' Error ', function (e) {console.log (' HTTP error '); index--if (index = = 0) {callback ();} console.log ("Got Error:" + e.message); }); }) (key); } } } } })();   function File () {var index = 0; var _readfile = function (Pathstr, Fileback, Doneback) {fs.readfile (Pathstr,{enc oding: ' UTF8 '}, function (err, data) {index--; if (err) {data = ""; Console.log (ERR,PATHSTR)//throw err; } fileback (DATA,PATHSTR); if (index = = 0) {doneback ();}}); }; var _walkdir = function (Pathstr, Fileback, Doneback) {fs.readdir (pathstr, function (err, files) {Files.foreach (function (file) {if (Fs.statsync (pathstr + '/' + file). Isdirectory ()) {_walkdir (pathstr + '/' + file, Fileback, doneback);} else {if (/. js$|. html$|. htm$|. Jsp$/.test (file) {index + +; _readfile (pathstr + '/' + file, Fileback, doneback);} RetuRn } }); }); } This.walkdir = function (Pathstr, Fileback, doneback) {index = 0; _walkdir (pathstr, Fileback, doneback);}  //First step to get Chinese Dictionary.logpath = LogPath;   New File (). Walkdir (FilePath, function (data) {if (!!) Data) {var match = Data.match (/[u4e00-u9faf]+/g); if (!! Match) {Match.foreach (function (MAT) {Dictionary.set (MAT);})}} The function () {console.log (' Get Chinese OK '); Dictionary.save2file ();    //Step two Google translation/* Dictionary.loadfile (function () {Dictionary.translatebygoogle (function () {DICTIONARY.S Ave2file (); }) }); *//Third steps Chinese replace/* dictionary.loadfile (function () {new File (). Walkdir (FilePath, function (data,pathstr) {Fs.writefile (pat Hstr, Data.replace (/[u4e00-u9faf]+/g, function (CH) {return dictionary.get (ch);}), {encoding: ' ASCII ', flag: ' W '}, function (Err) {if (err) throw err;}); The function () {console.log (' Chinese replace OK ');}) }); */

There's still some problems.

1.nodejs coding problem, in window environment for GBK coding support is not good, mainly UTF8 file processing

2. Efficiency may be further optimized by threading, this piece is not considered in depth

3. Match out, may have a single punctuation phrase and so on, need manual investigation

In fact, the file is GBK, and some files are utf8, and then when you think about it through the scripting language,

1. File coding problems, judging by search

Judge file first 3 bytes is not EF BB BF, but this is only for the UTF8 format with BOM

For UTF8 format without BOM, it is necessary to judge the byte signature (difficulty, limited energy, use the above scheme, for no BOM situation, Carry on manual investigation).

2. Because the quick multithreading is easy to program, it is always thought that multithreading is better than single-threaded efficiency. The actual situation is not the same as thinking, single-threaded faster than multithreading. It seems the main bottleneck is still on the read and write file IO.

The above mentioned is the entire content of this article, I hope you can enjoy.

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.