Capture weather forecasts from more than two thousand cities in China (Sina SAE version)-PHP source code

Capture the php code for weather forecasts in more than two thousand cities nationwide (Sina SAE version)

/* 1. use the Fetchurl, KVDB, and cron services of Sina SAE, which is fast and accurate. The Central Meteorological Observatory provides an API to query the weather in the last six days. the returned format is json. for example: http://m.weather.com.cn/data/ 101200101. html 101200101 is the City Code of the city of Wuhan, so as long as we know the city code, we can capture the city's weather. I have prepared data from all cities in the country that have been screened. these data have been tested by myself to ensure authenticity and effectiveness. the mysql database file download link http://www.kuaipan.cn/file/id_33799150646592444.html , Excel file download link http://www.kuaipan.cn/file/id_33799150646592446.html 2. at the beginning, I want to automatically load the city data into the database in the early morning, and then capture the data at AM. each script runs every minute, and each script captures 20 pieces of data, then, each time the data is captured successfully, the City record is deleted from the database. after testing, the database reads and writes too frequently, resulting in too much consumption and low efficiency. 3. after analysis, we decided to adopt KVDB, with extremely fast read/write speeds. it took about 5 seconds to load two thousand six hundred pieces of data. The code is as follows: code for loading the City ID, named KV. php: */getData ($ SQL); // retrieve all city information from the database $ num = count ($ data); $ kv = new SaeKV (); $ ret = $ kv-> init (); // initialize the KVDB service for ($ I = 0; $ iset ($ cityid, $ cityname);}?> /* Capture the data code and name it curl. php */$ kv = new SaeKV (); $ mysql = new SaeMysql (); $ mysql-> setCharset ('utf8 '); $ ret = $ kv-> init (); $ result = $ kv-> pkrget ('', 20 ); // extract 20 pieces of data from KVDB $ f = new SaeFetchurl (); foreach ($ result as $ key => $ value) {$ url =" http://m.weather.com.cn/data/ ". $ Key. ". html "; if ($ content = $ f-> fetch ($ url) {$ data = json_decode ($ content, true ); $ data = $ data ['atherinfo']; $ cityid = $ key; $ tl = array (); $ th = array (); $ wind = array (); $ fl = array (); $ fx = array (); $ weather = array (); for ($ I = 1; $ I $ th [$ I]) {$ tmp = $ th [$ I]; $ th [$ I] = $ tl [$ I]; $ tl [$ I] = $ tmp ;} if (isset ($ data ['wind '. $ I]) {$ wind [$ I] = $ data ['wind '. $ I];} else {$ wind [$ I] = "no data";} if (isset ($ data ['fl '. $ I]) {$ fl [$ I] = $ d Ata ['fl '. $ I];} else {$ fl [$ I] = "no data";} if (isset ($ data ['FX '. $ I]) {$ fx [$ I] = $ data ['FX '. $ I];} else {$ fx [$ I] = "no data";} if (isset ($ data ['weate '. $ I]) {$ weather [$ I] = $ data ['weather '. $ I];} else {$ weather [$ I] = "no data" ;}}$ cityname = $ data ['city']; $ time = time (); $ time = date ("Y-m-d", $ time); $ SQL = "insert into weather (cityid, cityname, time, tl1, tl2, tl3, tl4, tl5, tl6, th1, th1, th3, th4, th5, th6, wind1, wind2, wind3, wind4, win D5, wind6, fl1, fl2, fl3, fl4, fl5, fl6, fx1, fx2, fx3, fx4, fx5, fx6, weather1, weather2, weather3, weather4, weather5, weather6) values ('". $ mysql-> escape ($ cityid ). "','". $ mysql-> escape ($ cityname ). "','". $ mysql-> escape ($ time ). "','". $ mysql-> escape ($ tl [1]). "','". $ mysql-> escape ($ tl [2]). "','". $ mysql-> escape ($ tl [3]). "','". $ mysql-> escape ($ tl [4]). "','". $ mysql-> escape ($ tl [5] ). "','". $ Mysql-> escape ($ tl [6]). "','". $ mysql-> escape ($ th [1]). "','". $ mysql-> escape ($ th [2]). "','". $ mysql-> escape ($ th [3]). "','". $ mysql-> escape ($ th [4]). "','". $ mysql-> escape ($ th [5]). "','". $ mysql-> escape ($ th [6]). "','". $ mysql-> escape ($ wind [1]). "','". $ mysql-> escape ($ wind [2]). "','". $ mysql-> escape ($ wind [3]). "','". $ mysql-> escape ($ wind [4]). "','". $ mysql-> escape ($ wind [5]). "','". $ mysql-> escape ($ wind [6]). "','". $ mysql-> escape ($ fl [1]). "','". $ mysql-> escape ($ fl [2]). "','". $ mysql-> escape ($ fl [3]). "','". $ mysql-> escape ($ fl [4]). "','". $ mysql-> escape ($ fl [5]). "','". $ mysql-> escape ($ fl [6]). "','". $ mysql-> escape ($ fx [1]). "','". $ mysql-> escape ($ fx [2]). "','". $ mysql-> escape ($ fx [3]). "','". $ mysql-> escape ($ fx [4]). "','". $ mysql-> escape ($ fx [5]). "','". $ mysql-> escape ($ fx [6]). "','". $ mysql-> escape ($ weather [1]). "','". $ mysql-> escape ($ weather [2]). "','". $ mysql-> escape ($ weather [3]). "','". $ mysql-> escape ($ weather [4]). "','". $ mysql-> escape ($ weather [5]). "','". $ mysql-> escape ($ weather [6]). "')"; $ mysql-> runSql ($ SQL); if ($ mysql-> Errno ()! = 0) {die ("Error :". $ mysql-> errmsg ();} else {$ kv-> delete ($ key); // The record is successfully deleted from KVDB. Otherwise, the record is retained, next time you continue to capture}/* and then write the scheduled command to execute it */cron:-description: weather url: curl. php schedule: every 1 min-description: kv url: kv. php schedule: every day of month 10: 00