Hive cleans data using scripts: time stamp to date

import sysimport datetimefor line in sys.stdin:  line = line.strip()  userid, movieid, rating, unixtime = line.split(‘\t‘)  weekday = datetime.datetime.fromtimestamp(float(unixtime)).isoweekday()  print ‘\t‘.join([userid, movieid, rating, str(weekday)])

Use the Mapper script:

CREATE TABLE u_data_new (  userid INT,  movieid INT,  rating INT,  weekday INT)ROW FORMAT DELIMITEDFIELDS TERMINATED BY ‘\t‘;add FILE weekday_mapper.py;INSERT OVERWRITE TABLE u_data_newSELECT  TRANSFORM (userid, movieid, rating, unixtime)  USING ‘python weekday_mapper.py‘  AS (userid, movieid, rating, weekday)FROM u_data;SELECT weekday, COUNT(*)FROM u_data_newGROUP BY weekday;

1. From (
2. Map doctext using 'python wc_mapper.py 'As (word, CNT)
3. From docs
4. Cluster by word
5. )
6. Reduce word, CNT using 'python wc_performance.py ';

Hive cleans data using scripts: time stamp to date