Use gdal to obtain images from HDF and other datasets

When using gdal to read data from HDF, netcdf, and other datasets, two steps are generally required: first, to obtain the sub-dataset in the dataset; second, to read the image data from the sub-dataset obtained in the first step. There are many subdatasets in a general HDF image, such as frequently-used modem_data. When you use ENVI to open the image, the following dialog box is displayed to allow users to select the subdataset to be opened (1 ).

Figure 1 ENVI enable the image processing function

As shown in figure 1, ENVI lists the bands of all sub-datasets, but this is a little different from gdal reading. (A bit confusing here is that there are a total of 46 sub-data obtained using gdalinfo, and many images are of different sizes, here, the size of all the ENVI band images is 1354x2030. If you want to know, please let us know, aren't those smaller than 406x271 ?).

After reading the ENVI, we use the gdalinfo tool to view the data. The output content is as follows:

Driver: hdf4/hierarchical data format Release 4 files: F: \ data \ HDF \ mod 021KM. a2010287.0250.005.2010287121743.hdfsize is 512,512 coordinate system is ''metadata: % dead detector ev data = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 The nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn multicharacter is omitted here ................................... ..... southboundingcoordinate = 27.4153536499348 versionid = 5 westboundingcoordinate = 106.240739628195 subdatasets: subdataset_1_name = hdf4_eos: eos_swath: "F: \ data \ HDF \ mod 021KM. authorization ": Authorization: ev_1km_refsb subdataset_includesc = [15x2030x1354] ev_1km_refsb partition (16-bit unsigned integer) subdataset_2_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_2_desc = [15x2030x1354] required modis_swath_type_l1b (8-bit unsigned integer) subdataset_3_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. authorization ": Authorization: ev_1km_emissive subdataset_3_desc = [16x2030x1354] ev_1km_emissive partition (16-bit unsigned integer) subdataset_4_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_4_desc = [16x2030x1354] required modis_swath_type_l1b (8-bit unsigned integer) subdataset_5_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_5_desc = [2x2030x1354] required modis_swath_type_l1b (16-bit unsigned integer) subdataset_6_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_6_desc = [2x2030x1354] required modis_swath_type_l1b (8-bit unsigned integer) subdataset_7_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_7_desc = [2x2030x1354] required modis_swath_type_l1b (8-bit integer) subdataset_8_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_8_desc = [5x2030x1354] required modis_swath_type_l1b (16-bit unsigned integer) subdataset_9_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_9_desc = [5x2030x1354] required modis_swath_type_l1b (8-bit unsigned integer) subdataset_10_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. required ": required: Required subdataset_10_desc = [5x2030x1354] required modis_swath_type_l1b (8-bit integer) subdataset_11_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. height: height subdataset_11_desc = [406x271] height modis_swath_type_l1b (16-bit integer) subdataset_12_name = hdf4_eos: eos_swath: "F: \ data \ HDF \ mod 021KM. usage ": Usage: sensorzenith subdataset_12_desc = [406x271] sensorzenith partition (16-bit integer) subdataset_13_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. usage ": Usage: sensorazimuth subdataset_13_desc = [406x271] sensorazimuth modis_swath_type_l1b (16-bit integer) subdataset_14_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. sequence ": sequence: range subdataset_14_desc = [406x271] range sequence (16-bit unsigned integer) subdataset_15_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. warning ": Warning: solarzenith subdataset_15_desc = [406x271] solarzenith Merge (16-bit integer) subdataset_16_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. export ": Export: solarazimuth subdataset_16_desc = [406x271] solarazimuth modis_swath_type_l1b (16-bit integer) subdataset_17_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. authorization ": Authorization: gflags subdataset_17_desc = [406x271] gflags partition (8-bit unsigned integer) subdataset_18_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. usage ": Usage: ev_band26 subdataset_18_desc = [2030x1354] ev_band26 percentile (16-bit unsigned integer) subdataset_19_name = hdf4_eos: eos_swath:" F: \ data \ HDF \ mod 021KM. usage ": Usage: ev_band26_uncert_indexes subdataset_19_desc = [2030x1354] using round (8-bit unsigned integer) subdataset_20_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. required ": 0 subdataset_20_desc = [406x271] Latitude (32-bit floating-point) subdataset_21_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. required ": 1 subdataset_21_desc = [406x271] longpolling (32-bit floating-point) subdataset_22_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. required ": 2 subdataset_22_desc = [15x2030x1354] ev_1km_refsb (16-bit unsigned integer) subdataset_23_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 3 subdataset_23_desc = [15x2030x1354] partition (8-bit unsigned integer) subdataset_24_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. authorization ": 4 subdataset_24_desc = [16x2030x1354] ev_1km_emissive (16-bit unsigned integer) subdataset_25_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 5 subdataset_25_desc = [16x2030x1354] partition (8-bit unsigned integer) subdataset_26_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 6 subdataset_26_desc = [2x2030x1354] ev_250_aggr1km_refsb (16-bit unsigned integer) subdataset_27_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 7 subdataset_27_desc = [2x2030x1354] partition (8-bit unsigned integer) subdataset_28_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 8 subdataset_28_desc = [2x2030x1354] partition (8-bit integer) subdataset_29_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. authorization ": 9 subdataset_29_desc = [5x2030x1354] ev_500_aggr1km_refsb (16-bit unsigned integer) subdataset_30_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 10 subdataset_30_desc = [5x2030x1354] partition (8-bit unsigned integer) subdataset_31_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 11 subdataset_31_desc = [5x2030x1354] partition (8-bit integer) subdataset_32_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. a2010287.0250.005.2010287121743.hdf ": 12 subdataset_32_desc = [406x271] height (16-bit integer) subdataset_33_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. required ": 13 subdataset_33_desc = [406x271] sensorzenith (16-bit integer) subdataset_34_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. detail ": 14 subdataset_34_desc = [406x271] sensorazimuth (16-bit integer) subdataset_35_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. duration ": 15 subdataset_35_desc = [406x271] range (16-bit unsigned integer) subdataset_36_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. required ": 16 subdataset_36_desc = [406x271] solarzenith (16-bit integer) subdataset_37_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. authorization ": 17 subdataset_37_desc = [406x271] solarazimuth (16-bit integer) subdataset_38_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. authorization ": 18 subdataset_38_desc = [406x271] gflags (8-bit unsigned integer) subdataset_39_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. authorization ": 19 subdataset_39_desc = [2030x1354] ev_band26 (16-bit unsigned integer) subdataset_40_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. authorization ": 20 subdataset_40_desc = [2030x1354] ev_band26_uncert_indexes (8-bit unsigned integer) subdataset_41_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. criteria ": 21 subdataset_41_desc = [16x10] noise in thermal detectors (8-bit unsigned integer) subdataset_42_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. partition ": 22 subdataset_42_desc = [16x10] change in relative responses of thermal detectors (8-bit unsigned integer) subdataset_43_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. unknown ": 23 subdataset_43_desc = [203x16x10] DC Restore change for thermal bands (8-bit integer) subdataset_44_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. response ": 24 subdataset_44_desc = [203x2x40] DC Restore change for reflective 250 m bands (8-bit integer) subdataset_45_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. shard ": 25 subdataset_45_desc = [203x5x20] DC Restore change for reflective 500 m bands (8-bit integer) subdataset_46_name = hdf4_sds: Unknown:" F: \ data \ HDF \ mod 021KM. a2010287.0250.005.2010287121743.hdf ": 26 subdataset_46_desc = [203x15x10] DC Restore change for reflective 1 km bands (8-bit integer) corner coordinates: upper left (0.0, 0.0) lower left (0.0, 512.0) upper right (512.0, 0.0) lower right (512.0, 512.0) Center (256.0, 256.0)

From the output information above, we can see that this HDF data actually has 46 sub-Data. From the above subdatasets, there are a total of 46 sub-datasets, and gdal lists the description of these data, including the data size band number and data type. The following uses the first subdataset as an example to illustrate the general meaning of the information output by gdal. The first subdataset is described as follows:

  SUBDATASET_1_NAME=HDF4_EOS:EOS_SWATH:"F:\Data\HDF\MOD021KM.A2010287.0250.005.2010287121743.hdf":MODIS_SWATH_Type_L1B:EV_1KM_RefSB  SUBDATASET_1_DESC=[15x2030x1354] EV_1KM_RefSB MODIS_SWATH_Type_L1B (16-bit unsigned integer)

It can be seen that a subdataset consists of two rows. The first row indicates the "path" of the subdataset. The path here can be considered as the path of the subdataset stored in the HDF file. This path is used to read data using gdal later. The second line indicates the description of the Child data. For example, 15 × 2030 × 1354 indicates that the child data has 15 bands, with the length and width 2030 and 1354 respectively. Ev_1km_refsb modis_swath_type_l1b on the weekend indicates the processing level and type of the data. The 16-bit unsigned integer in the following arc indicates that the child data set is a 16-bit unsigned integer.

Then, how can we use gdal to obtain the image information in the subdataset? We still use gdalinfo as a tool to describe it. To  about each sub-dataset in HDF, you must first obtain the path of the sub-dataset in HDF. This path needs to be determined by the subdataset_1_nam above. Take the first child dataset as an example. The command line is as follows, as shown in Figure 2:

gdalinfo.exe HDF4_EOS:EOS_SWATH:"F:\Data\HDF\MOD021KM.A2010287.0250.005.2010287121743.hdf":MODIS_SWATH_Type_L1B:EV_1KM_RefSB

Figure 2 Use gdalinfo to view the output information of the subdataset

Because too much output information cannot be displayed in all cases, I will not post it here. With the above experiment, we can write a program to read HDF data. First, we can use gdal to open HDF data, obtain the sub-dataset path in subdatasets, and then use gdalopen to open it. As for this subdatasets, it is actually in the gdal metadata, which can be obtained using the function gdaldataset: getmetadata. The sample code is as follows:

Gdalallregister (); const char * pszsrcfile = "F: \ data \ HDF \ mod 021KM. export "; gdaldataset * pdataset = (gdaldataset *) gdalopen (pszsrcfile, ga_readonly); If (pdataset = NULL) {printf (" cannot open this file. Please check whether the file exists! "); Return re_filenotexist;} Char ** papszsubdatasets = gdalgetmetadata (gdaldataseth) pdataset," subdatasets ");

In the last line of this Code, papszsubdatasets contains the path and description of the stored sub-data. Papszsubdatasets is actually a string array. Next, parse the string array and obtain the path and description of the HDF subdataset from it, if the sub-dataset obtained by an HDF file is null, that is, if the papszsubdatasets variable is null, The HDF data itself is a single data with no sub-dataset. The following code parses the string array of the Child Dataset:

vector<string> vSubDataSets, vector<string> vSubDataDesc;if ( papszSUBDATASETS == NULL ){string papszMetadata = GDALGetDriverShortName((GDALDriverH)pDataSet);vSubDataSets.push_back(papszMetadata);vSubDataDesc.push_back(papszMetadata);}

In the above Code, if the current HDF data does not have a subdataset, it is itself a dataset. Two vectors <string> are defined here to store the path and description information of the sub-dataset. The following code is used to parse HDF data containing child datasets. Here we need a little STD: String function, mainly used to intercept the substr function for the string.

Else {int icount = cslcount (papszsubdatasets); If (icount <= 0) {If (pprocess! = NULL) pprocess-> setmessage ("This HDF file has no sub-data! "); Gdalclose (gdaldriverh) pdataset); Return re_success;} For (INT I = 0; papszsubdatasets [I]! = NULL; I ++) {if (I % 2! = 0) continue; string tmpstr = string (papszsubdatasets [I]); tmpstr = tmpstr. substr (tmpstr. find_first_of ("=") + 1); const char * tmpc_str = tmpstr. c_str (); string tmpdsc = string (papszsubdatasets [I + 1]); tmpdsc = tmpdsc. substr (tmpdsc. find_first_of ("=") + 1); gdaldataseth htmpdt = gdalopen (tmpc_str, ga_readonly); If (htmpdt! = NULL) {vsubdatasets. push_back (tmpstr); vsubdatadesc. push_back (tmpdsc); gdalclose (htmpdt) ;}// end for} gdalclose (gdaldriverh) pdataset );

The code above explains a little: first, use the cslcount function to obtain the number of this string array. If it is not greater than 0, it means no. Directly disable HDF data return. If it is greater than 0, traverse this string array. Generally, the number of this string array must be an even number. You can also determine in advance. If the subscript of the array is an odd number, skip this step. That is to say, we can directly retrieve strings whose subscript is 0, 2, and 4. The following string is connected by a "=" (For details, refer to the above example ). The left side of the equal sign describes the sequence of the dataset, and the right side shows the path of the value, that is, the Child dataset. Use the substr function to extract the path and store it to the previously defined vector <string>. At the same time, the obtained description information is also stored. (This description information is optional, if user interaction is required, the path of the subdataset is correct.

Through the above instructions, we will understand how to use gdal to open HDF data (including hdf4, hdf5, and netcdf. The following is a simple demo I wrote. The core function is to use the above Code.
First, open HDF data, as shown in 3:

Figure 3 open HDF data

After you click open, a dialog box is displayed, as shown in Figure 4. The description of all sub-datasets in the open HDF data is displayed in the list. You can select to specify the sub-dataset to be opened, the list in this dialog box shows the description parsed in the code above. Click "OK" to locate the path of the sub-dataset by selecting the number of sub-dataset, and then use gdal to display the sub-dataset or perform other processing.

Figure 4 description of the Child dataset that is listed after HDF data is opened