File-based data structures: about Mapfile

Mapfile is an already sequenced sequencefile, it has an index, so you can press the key to find

1.MapFile Write operation

The Mapfile write operation is similar to the Sequencefile write operation. Create a new Mapfile.writer instance, and then call the Append () method to write the file contents in order. If you do not write sequentially, a IOException exception is thrown. The key must be an instance of the Writablecomparable type, and the value must be an instance of the writable type.

Write Mapfile, the program is as follows:

 PackageCom.lcy.hadoop.io;ImportJava.net.URI;Importorg.apache.hadoop.conf.Configuration;ImportOrg.apache.hadoop.fs.FileSystem;Importorg.apache.hadoop.io.IOUtils;Importorg.apache.hadoop.io.IntWritable;ImportOrg.apache.hadoop.io.MapFile;ImportOrg.apache.hadoop.io.Text; Public classMapfilewritedemo {Private Static FinalString [] Data={        "One,two,buckle My Shoe",        "Three,four,shut The Door",        "Five,six,pick Up Sticks",        "Seven,eight,lay them Straight",        "Nine,ten,a Big Fat Hen"    };  Public Static voidMain (string[] args)throwsException {//TODO auto-generated Method StubString uri=args[0]; Configuration conf=NewConfiguration (); FileSystem FS=Filesystem.get (Uri.create (URI), conf); Intwritable Key=Newintwritable (); Text value=NewText (); Mapfile.writer Writer=NULL; Try{writer=NewMapfile.writer (Conf,fs,uri,key.getclass (), Value.getclass ());  for(inti=0;i<1024;i++) {Key.set (i+1); Value.set (Data[i%Data.length]);            Writer.append (key, value); }        }finally{Ioutils.closestream (writer); }    }}

Run the program and use this program to build a mapfile:

When I enter the command Hadoop FS-LSR numbers.map, you can see:

Discover that Numbers.map is actually a folder containing data and index two files, and these two files are Sequencefile

The data file contains all the records, as follows:

The index file contains a portion of the key and the mapping of the key to its offset in the data file:

From the output can be seen: By default, only every 128 keys are included in the index file, of course, can also be adjusted, call Mapfile.writer instance of Setindexinterval () method to set the Io.map.index.interval property

Read operation of 2.MapFile

The process of iterating through all the entries in the file in Mapfile is similar to the procedure in Sequencefile: Create a new Mapfile.reader instance first, and then call the next () method until the return value is False

File-based data structures: about Mapfile