Recently came to a new company and started AWS, which will keep track of some of the things that AWS is doing. Today is to talk about time zone problem, it is very simple to say, there is no server international estimates of the attention of the few people.
Our EMR machine is in us, and the actual application of the server in CH, colleague parser for log to find that the actual user time and our machine time is always different, and the exclusion of client network problems are quite large. I didn't notice at first, I thought it was a network problem, I looked at it later or.
Here's how to find ideas:
--Most of the server time is 04-26select collector_tstamp:: Date,count (*) from Dev.eventsgroup by Collector_tstamp:: Date order by Count ( *) desc;2015-04-2610127692015-04-2716932-device time is mostly 04-26select Dvce_tstamp:: Date,count (*) from Dev.eventsgroup by Dvce _tstamp:: Date ORDER by Count (*) desc;2015-04-269387462015-04-25904882015-04-27174 .... (less than 100)
From two points of view will be mistaken for no problem.
Here's a look at the diff data:
Select DateDiff (M,collector_tstamp,dvce_tstamp) as Minute,count (*) from Dev.eventsgroup by DateDiff (m,collector_ Tstamp,dvce_tstamp) Order by COUNT (*) desc; Minutes cnt-480949916-47924772-48117785-4788325-4774397-4763000-4822852-4752492-4742219-4731628-4701384-4691103
After confirming the log rotation, check the EMR program and discover the localtime conversion EMR Python script when calling the Python Time library:
row[' dvce_tstamp ' = Time.strftime ('%y-%m-%d%h:%m:%s ', time.localtime (int (row[' dvce_tstamp '])/1000))
So the problem is found. Rootcause: Local code uploads AWS to do time processing requires timezone designation
Test results when not specified: Local machine:>>> print time.localtime (1430378360) time.struct_time (tm_year=2015, tm_mon=4, tm_mday=30, Tm_hour=15, tm_min=19, tm_sec=20, tm_wday=3, tm_yday=120, tm_isdst=0)
EMR Machine:>>> Print time.localtime (1430378360) time.struct_time (tm_year=2015, tm_mon=4, tm_mday=30, tm_hour=7, tm_min=19, Tm_sec=20, tm_wday=3, tm_yday=120, tm_isdst=0)
--
Pay attention to time zone issues when using AWS services