How to Use libmaxminddb,

How to Use libmaxminddb,
Name

Libmaxminddb-database used to process MaxMind database files

Introduction

#include 
 
  int MMDB_open(    const char *const filename,    uint32_t flags,    MMDB_s *const mmdb);void MMDB_close(MMDB_s *const mmdb);MMDB_lookup_result_s MMDB_lookup_string(    MMDB_s *const mmdb,    const char *const ipstr,    int *const gai_error,    int *const mmdb_error);MMDB_lookup_result_s MMDB_lookup_sockaddr(    MMDB_s *const mmdb,    const struct sockaddr *const    sockaddr,    int *const mmdb_error);int MMDB_get_value(    MMDB_entry_s *const start,    MMDB_entry_data_s *const entry_data,    ...);int MMDB_vget_value(    MMDB_entry_s *const start,    MMDB_entry_data_s *const entry_data,    va_list va_path);int MMDB_aget_value(    MMDB_entry_s *const start,    MMDB_entry_data_s *const entry_data,    const char *const *const path);int MMDB_get_entry_data_list(    MMDB_entry_s *start,    MMDB_entry_data_list_s **const entry_data_list);void MMDB_free_entry_data_list(    MMDB_entry_data_list_s *const entry_data_list);int MMDB_get_metadata_as_entry_data_list(    MMDB_s *const mmdb,    MMDB_entry_data_list_s **const entry_data_list);int MMDB_dump_entry_data_list(    FILE *const stream,    MMDB_entry_data_list_s *const entry_data_list,    int indent);int MMDB_read_node(    MMDB_s *const mmdb,    uint32_t node_number,    MMDB_search_node_s *const node);const char *MMDB_lib_version(void);const char *MMDB_strerror(int error_code);typedef struct MMDB_lookup_result_s {    bool found_entry;    MMDB_entry_s entry;    uint16_t netmask;} MMDB_lookup_result_s;typedef struct MMDB_entry_data_s {    bool has_data;    union {        uint32_t pointer;        const char *utf8_string;        double double_value;        const uint8_t *bytes;        uint16_t uint16;        uint32_t uint32;        int32_t int32;        uint64_t uint64;        {mmdb_uint128_t or uint8_t[16]} uint128;        bool boolean;        float float_value;    };    ...    uint32_t data_size;    uint32_t type;} MMDB_entry_data_s;typedef struct MMDB_entry_data_list_s {    MMDB_entry_data_s entry_data;    struct MMDB_entry_data_list_s *next;} MMDB_entry_data_list_s;
 

Description

The libmaxminddb Library provides the ability to process MaxMind DB files. Databases and results are represented by different data structures. Call MMDB_open () to open the database. You can use MMDB_lookup_string () to find the IP address, or use MMDB_lookup_sockaddr () to point to the sockaddr structure pointer.

If you search for IP addresses in the database, the MMDB_lookup_result_s structure is returned. If this structure indicates that the database has data of this IP address, you can use many functions to obtain the data. These include MMDB_get_value () and MMDB_get_entry_data_list ().

Data Structure

All data structures exported from the maxminddb. h header file of this database are in the form of typedef struct foo_s {...} foo_s. Therefore, you can reference them without using the struct prefix.

This database provides the following data structures:

MMDB_s

This is the handle of the MaxMind DB file. We only record some fields in this structure for public use. All other fields may be changed for internal use only.

typedef struct MMDB_s {    uint32_t flags;    const char *filename;    ...    MMDB_metadata_s metadata;} MMDB_s;

Uint32_t flags-indicates that the database is opened. MMDB_metadata_s and MMDB_description_s

You can obtain this structure from the MMDB_s structure. It contains metadata read from database files. Note: You can easily access metadata by calling MMDB_get_metadata_as_entry_data_list.

typedef struct MMDB_metadata_s {    uint32_t node_count;    uint16_t record_size;    uint16_t ip_version;    const char *database_type;    struct {        size_t count;        const char **names;    } languages;    uint16_t binary_format_major_version;    uint16_t binary_format_minor_version;    uint64_t build_epoch;    struct {        size_t count;        MMDB_description_s **descriptions;    } description;} MMDB_metadata_s;typedef struct MMDB_description_s {    const char *language;    const char *description;} MMDB_description_s;

Most of these structures are self-explanatory.

Ip_version should always be 4 or 6. Binary_format_major_version should always be 2.

The database metadata is not required to include a language or description, so the count of these parts of the metadata can be zero. All other fields of MMDB_metadata_s should be filled.

MMDB_lookup_result_s

This structure is returned as the IP address search result.

typedef struct MMDB_lookup_result_s {    bool found_entry;    MMDB_entry_s entry;    uint16_t netmask;} MMDB_lookup_result_s;

If found_entry is false, other members of the structure do not contain meaningful values. Always check whether found_entry is true.

The entry is used to search for IP address-related data.

Netmask indicates the subnet to which the IP address belongs. For example, if you search for the address 1.1.1.1 In An IPv4 database and the returned netmask is 16, the address is part of the 1.1.0.0/16 subnet.

If the database is an IPv6 database, the returned network mask is always an IPv6 prefix length (from 0 to 128), even if the database also contains an IPv4 network. If you are looking for an IPv4 address and want to convert the network mask to an IPv4 network mask value, you can subtract 96 from the value.

MMDB_result_s

You do not need to mine this structure. You will get this result from the MMDB_lookup_result_s structure and pass it to various functions.

MMDB_entry_data_s

This structure is used to return a single IP segment entry. These entries can point to other entries in turn, just like maps and arrays. Some members of this structure are not recorded because they are only used internally.

typedef struct MMDB_entry_data_s {    bool has_data;    union {        uint32_t pointer;        const char *utf8_string;        double double_value;        const uint8_t *bytes;        uint16_t uint16;        uint32_t uint32;        int32_t int32;        uint64_t uint64;        {mmdb_uint128_t or uint8_t[16]} uint128;        bool boolean;        float float_value;    };    ...    uint32_t data_size;    uint32_t type;} MMDB_entry_data_s;

If data is found for a given query, the has_data member is true. For more information, see MMDB_get_value (). If this parameter is set to false, other values in the structure are meaningless.

The union at the beginning of the structure defines the actual data. To determine which union member is filled, check the type member. The union pointer member should not be filled in any data returned by the API. The pointer should always be resolved internally.

Data_size members are only related to utf8_string and bytes data. Utf8_string does not end with null. You must use data_size to determine its length.

The type member can be compared with one in the MMDB_DTYPE _ * macro.

128-bit integer

The processing of uint128 data depends on how your platform supports the 128-bit integer type. Using GCC 4.4 and 4.5, we can write it as unsigned int _ attribute _ (_ mode _ (TI ))). Using a newer version of GCC (4.6 +) and clang (3.2 +), we can simply write "unsigned _ int128 ".

To solve these differences, this library defines a mmdb_uint128_t type. This type is defined in the maxminddb. h header file, so you can use it in your own code.

For older compilers, we cannot use an integer. Therefore, we use a 16-byte array to replace the uint8_t value. This is the raw data in the database.

This library provides a public macro MMDB_UINT128_IS_BYTE_ARRAY. If the value is true, the uint128 value is returned as a byte array. If the value is false, the values are returned as mmdb_uint128_t integer.

Macro of Data Type

This database provides a macro for each data type defined by MaxMind DB.

MMDB_DATA_TYPE_UTF8_STRING MMDB_DATA_TYPE_DOUBLE MMDB_DATA_TYPE_BYTES MMDB_DATA_TYPE_UINT16 MMDB_DATA_TYPE_UINT32 MMDB_DATA_TYPE_MAP MMDB_DATA_TYPE_INT32 MMDB_DATA_TYPE_UINT128 MMDB_DATA_TYPE_UINT128 MMDB_DATA_TYPE_ARRAY MMDB_DATA_TYPE_BOOLEAN MMDB_DATA_TYPE_FLOAT

There are also some types for internal use only:

MMDB_DATA_TYPE_EXTENDED MMDB_DATA_TYPE_POINTER MMDB_DATA_TYPE_CONTAINER MMDB_DATA_TYPE_END_MARKER

If you see one of the returned data, it must be an error. It may be because the database is damaged or generated incorrectly, or the libmaxminddb code has an error.

The pointer value and MMDB_close ()

Uint128 members of the utf8_string, bytes, and (maybe) structure direct pointer to the database data section. This may be a memory block of malloc or mmap. In both cases, after MMDB_close () is called, these pointers become invalid.

If you need to reference this data later, use the appropriate function (strdup, memcpy, etc.) to copy the data.

MMDB_entry_data_list_s

This structure encapsulates the linked list of the MMDB_entry_data_s structure.

typedef struct MMDB_entry_data_list_s {    MMDB_entry_data_s entry_data;    struct MMDB_entry_data_list_s *next;} MMDB_entry_data_list_s;

This structure allows you to view the entire map or array data through a traversal table.

MMDB_search_node_s

This structure encapsulates two records in the search node. This is useful if you want to write code that traverses the entire search tree instead of looking for a specific IP address.

typedef struct MMDB_search_node_s {    uint64_t left_record;    uint64_t right_record;    uint8_t left_record_type;    uint8_t right_record_type;    MMDB_entry_s left_record_entry;    MMDB_entry_s right_record_entry;} MMDB_search_node_s;

Two record types Use one of the following values:

MMDB_RECORD_TYPE_SEARCH_NODE-the record points to the next search node. MMDB_RECORD_TYPE_EMPTY-the record is a placeholder, indicating data without IP addresses. The search should end here. MMDB_RECORD_TYPE_DATA-records are used for data in the database data section. Use record entries when searching for record data. MMDB_RECORD_TYPE_INVALID-the record is invalid. Find the invalid node or the database is damaged.

MMDB_entry_s is valid only when the type is MMDB_RECORD_TYPE_DATA. Attempts to use entries of other record types will result in incorrect or invalid data.

Status Code

This library returns (or fills in) the status code of many functions. These status codes are:

MMDB_SUCCESS-everything is normal MMDB_FILE_OPEN_ERROR-An error occurred while trying to open the MaxMind DB file. MMDB_IO_ERROR-IO operation failed. Check errno for more details. MMDB_CORRUPT_SEARCH_TREE_ERROR-find the IP address in the search tree and give us an impossible result. The database is damaged or generated incorrectly, or the libmaxminddb code has an error. MMDB_INVALID_METADATA_ERROR-some contents in the database are incorrect. Including lost metadata and impossible values (for example, ip_version is 7 ). MMDB_UNKNOWN_DATABASE_FORMAT_ERROR-database metadata indicates that its master version is not 2. This database can only process version 2. MMDB_OUT_OF_MEMORY_ERROR-memory allocation call (malloc, etc.) failed. The entries in the MMDB_INVALID_DATA_ERROR-data section contain invalid data. For example, a uint16 field claims to be longer than two bytes. The database may be damaged or generated incorrectly. MMDB_INVALID_LOOKUP_PATH_ERROR-the path passed to MMDB_get_value, MMDB_vget_value, or MMDB_aget_value contains an array offset of a negative integer or an integer greater than LONG_MAX. MMDB_LOOKUP_PATH_DOES_NOT_MATCH_DATA_ERROR-the query path passed to MMDB_get_value, MMDB_vget_value, or MMDB_aget_value does not match the data structure of the entry. There are many possible causes. The search path can include keys not in the map. The search path can include indexes greater than the array. It may also happen when the path expects to find a map or array that does not exist.

All status codes should be considered as int values.

MMDB_strerror ()

const char *MMDB_strerror(int error_code)

This function requires status code and returns a string that interprets the status.

Function

This Library provides the following exported functions:

MMDB_open ()

int MMDB_open(    const char *const filename,    uint32_t flags,    MMDB_s *const mmdb);

This function opens the handle of the MaxMind DB file. The returned value is the status code defined above. Always check the return value of this call.

MMDB_s mmdb;int status =    MMDB_open("/path/to/file.mmdb", MMDB_MODE_MMAP, &mmdb);if (MMDB_SUCCESS != status) { ... }...MMDB_close(&mmdb);

The incoming MMDB_s structure can be allocated in the stack or from the heap. However, if it is enabled successfully, it will contain heap allocated data, so you need to use MMDB_close () to close it. If the returned status is not MMDB_SUCCESS, ensure that all allocated memory is released before the returned status.

Currently, flags are provided as follows:

MMDB_MODE_MMAP-use mmap () to open the database.

Passing flags of other values may produce unpredictable results. In the future, we can add additional flags, which can be bitwise OR in combination with the mode, and other modes.

You can also pass 0 as the flags value. In this case, the database will be opened with the default flags. However, these default values may be changed in future versions. The default value is MMDB_MODE_MMAP.

MMDB_close ()

void MMDB_close(MMDB_s *const mmdb);

Release any allocated or mmap memory retained from the MMDB_s structure. It does not release the memory allocated to the structure itself! If you allocate a structure from the heap, you are responsible for releasing it.

MMDB_lookup_string ()

MMDB_lookup_result_s MMDB_lookup_string(    MMDB_s *const mmdb,    const char *const ipstr,    int *const gai_error,    int *const mmdb_error);

This function is used to find IP addresses ending with null. Internally, it calls getaddrinfo () to parse the address into a binary format. Then it calls MMDB_lookup_sockaddr () to find the address in the database. If you have already resolved an address, you can directly call MMDB_lookup_sockaddr () without having to parse the address twice.

int gai_error, mmdb_error;MMDB_lookup_result_s result =    MMDB_lookup_string(&mmdb, "1.2.3.4", &gai_error, &mmdb_error);if (0 != gai_error) { ... }if (MMDB_SUCCESS != mmdb_error) { ... }if (result.found_entry) { ... }

This function always returns an MMDB_lookup_result_s structure, but you should also check the gai_error and mmdb_error parameters. If any of these errors occurs, the returned structure is meaningless.

If no error occurs, make sure that the found_entry member in the returned result is true. If not, this means that the IP address has no entries in the database.

This function can be used with IPv4 addresses even if the database contains IPv4 and IPv6 addresses. The IPv4 address is ": xxx. xxx. xxx. xxx, instead of re- ing to the IPv6 address assigned for IPv4 ing: ffff: xxx. xxx. xxx. xxx block.

If you pass the IPv6 address to a database with only IPv4 data, the found_entry is false, but the mmdb_error status is still MMDB_SUCCESS.

MMDB_lookup_sockaddr ()

MMDB_lookup_result_s MMDB_lookup_sockaddr(    MMDB_s *const mmdb,    const struct sockaddr *const sockaddr,    int *const mmdb_error);

This function is used to find the IP address resolved by getaddrinfo.

Except that getaddrinfo () is not called, this function is the same as MMDB_lookup_string () function.

int mmdb_error;MMDB_lookup_result_s result =    MMDB_lookup_sockaddr(&mmdb, address->ai_addr, &mmdb_error);if (MMDB_SUCCESS != mmdb_error) { ... }if (result.found_entry) { ... }

Data lookup Function

There are three functions related to IP address search.

int MMDB_get_value(    MMDB_entry_s *const start,    MMDB_entry_data_s *const entry_data,    ...);int MMDB_vget_value(    MMDB_entry_s *const start,    MMDB_entry_data_s *const entry_data,    va_list va_path);int MMDB_aget_value(    MMDB_entry_s *const start,    MMDB_entry_data_s *const entry_data,    const char *const *const path);

These three functions allow three slightly different calling styles, but they all do the same thing.

The first parameter is the value of MMDB_entry_s. In most cases, this will be from the MMDB_lookup_string () or MMDB_lookup_sockaddr () returned by MMDB_lookup_result_s.

The second parameter is a reference to the MMDB_entry_data_s structure. This will be filled with the data being searched (if any ). If not found, the has_data member of this structure will be false. If has_data is true, you can view data_type members.

The last parameter is the search path. The path consists of map keys (for example, "city") or array indexes (for example, "0", "1") used in the search. This allows you to manipulate complex data structures. For example:

{    "names": {        "en": "Germany",        "de": "Deutschland"    },    "cities": [ "Berlin", "Frankfurt" ]}

You can use the following code to find an English name:

MMDB_lookup_result_s result =    MMDB_lookup_sockaddr(&mmdb, address->ai_addr, &mmdb_error);MMDB_entry_data_s entry_data;int status =    MMDB_get_value(&result.entry, &entry_data,                   "names", "en", NULL);if (MMDB_SUCCESS != status) { ... }if (entry_data.has_data) { ... }

If we want to find the first city, the search path will be "cities", "0 ". If no search path is provided, the entries corresponding to the top-level map are obtained. The search path must always end with NULL, no matter which function you call.

The MMDB_get_value function accepts variable parameters. All parameters after the MMDB_entry_data_s * structure pointer are the search path. The last parameter must be NULL.

The MMDB_vget_value function accepts a va_list as the search path. The last element retrieved by va_arg () must be NULL.

Finally, MMDB_aget_value accepts the string array as the search path. The last member of the array must be NULL.

To obtain data of all entries at a time, call MMDB_get_entry_data_list ().

For each of the three functions, the return value is the status code defined above.

MMDB_get_entry_data_list ()

int MMDB_get_entry_data_list(    MMDB_entry_s *start,    MMDB_entry_data_list_s **const entry_data_list);

This function allows you to retrieve all the data in a complex data structure at a time, instead of repeatedly calling MMDB_get_value () to find each data.

MMDB_lookup_result_s result =    MMDB_lookup_sockaddr(&mmdb, address->ai_addr, &mmdb_error);MMDB_entry_data_list_s *entry_data_list, *first;int status =    MMDB_get_entry_data_list(&result.entry, &entry_data_list);if (MMDB_SUCCESS != status) { ... }// save this so we can free this data laterfirst = entry_data_list;while (1) {    MMDB_entry_data_list_s *next = entry_data_list = entry_data_list->next;    if (NULL == next) {        break;    }    switch (next->entry_data.type) {        case MMDB_DATA_TYPE_MAP: { ... }        case MMDB_DATA_TYPE_UTF8_STRING: { ... }        ...    }}MMDB_free_entry_data_list(first);

You can explain the data structure of entry_data_list. List to traverse connections in depth first. Take this structure as an example:

{    "names": {        "en": "Germany",        "de": "Deutschland"    },    "cities": [ "Berlin", "Frankfurt" ]}

This list includes the following items:

MAP-top-level map UTF8_STRING-"name" Key MAP-"name" Key map UTF8_STRING-"en" Key UTF8_STRING-"en" key value UTF8_STRING-"de" Key UTF8_STRING- "de" key value UTF8_STRING-"cities" Key ARRAY-"cities" key value UTF8_STRING-ARRAY [0] UTF8_STRING-ARRAY [1]

The Return Value of the function is the status code defined above.

MMDB_free_entry_data_list ()

void MMDB_free_entry_data_list(    MMDB_entry_data_list_s *const entry_data_list);

The MMDB_get_entry_data_list () and MMDB_get_metadata_as_entry_data_list () functions allocate the linked list structure from the heap. Call this function to release the MMDB_entry_data_list_s structure.

MMDB_get_metadata_as_entry_data_list ()

int MMDB_get_metadata_as_entry_data_list(    MMDB_s *const mmdb,    MMDB_entry_data_list_s **const entry_data_list);

This function allows you to obtain the database metadata as a linked list of the MMDB_entry_data_list_s structure. This can be a more convenient way to directly use the metadata structure to process metadata.

MMDB_entry_data_list_s *entry_data_list, *first;int status =    MMDB_get_metadata_as_entry_data_list(&mmdb, &entry_data_list);if (MMDB_SUCCESS != status) { ... }first = entry_data_list;... // do something with the dataMMDB_free_entry_data_list(first);

The Return Value of the function is the status code defined above.

MMDB_dump_entry_data_list ()

int MMDB_dump_entry_data_list(    FILE *const stream,    MMDB_entry_data_list_s *const entry_data_list,    int indent);

This function receives the chain table of the MMDB_entry_data_list_s structure and returns its string to stream. The indent parameter is the initial indent level for generating output. Nested data structures (maps, arrays, etc.) increase progressively.

Stream must be a file handle (stdout ). If your platform provides functions similar to GNU open_memstream (), you can use it to capture output in string format.

The output is formatted in JSON-ish format, but the value is marked with its data type (except for maps and arrays displayed with "{}" and "[]" respectively ).

The specific output format may be changed in future versions, so you should not rely on the specific format generated by this function. It is intended to display data to users in a readable manner and for debugging purposes.

The Return Value of the function is the status code defined above.

MMDB_read_node ()

int MMDB_read_node(    MMDB_s *const mmdb,    uint32_t node_number,    MMDB_search_node_s *const node);

Read specific nodes in the search tree. The third parameter is a reference to the MMDB_search_node_s structure of the function.

The return value is the status code. If the passed node_number is greater than the number of nodes in the database, this function returns MMDB_INVALID_NODE_NUMBER_ERROR; otherwise, MMDB_SUCCESS is returned.

The first node in the search tree is always node 0. If you want to traverse the entire search tree, you will first read the node 0, and then track the records that constitute the node based on the type of each record. If the type is MMDB_RECORD_TYPE_SEARCH_NODE, the record will contain an integer for the next search node.

MMDB_lib_version ()

const char *MMDB_lib_version(void)

This function returns the library version of a string, similar to "2.0.0 ".

Example

#include 
 
  #include 
  
   #include 
   
    #include 
    
     int main(int argc, char **argv){    char *filename = argv[1];    char *ip_address = argv[2];    MMDB_s mmdb;    int status = MMDB_open(filename, MMDB_MODE_MMAP, &mmdb);    if (MMDB_SUCCESS != status) {        fprintf(stderr, "\n  Can't open %s - %s\n",                filename, MMDB_strerror(status));        if (MMDB_IO_ERROR == status) {            fprintf(stderr, "    IO error: %s\n", strerror(errno));        }        exit(1);    }    int gai_error, mmdb_error;    MMDB_lookup_result_s result =        MMDB_lookup_string(&mmdb, ip_address, &gai_error, &mmdb_error);    if (0 != gai_error) {        fprintf(stderr,                "\n  Error from getaddrinfo for %s - %s\n\n",                ip_address, gai_strerror(gai_error));        exit(2);    }    if (MMDB_SUCCESS != mmdb_error) {        fprintf(stderr,                "\n  Got an error from libmaxminddb: %s\n\n",                MMDB_strerror(mmdb_error));        exit(3);    }    MMDB_entry_data_list_s *entry_data_list = NULL;    int exit_code = 0;    if (result.found_entry) {        int status = MMDB_get_entry_data_list(&result.entry,                                              &entry_data_list);        if (MMDB_SUCCESS != status) {            fprintf(                stderr,                "Got an error looking up the entry data - %s\n",                MMDB_strerror(status));            exit_code = 4;            goto end;        }        if (NULL != entry_data_list) {            MMDB_dump_entry_data_list(stdout, entry_data_list, 2);        }    } else {        fprintf(            stderr,            "\n  No entry for this IP address (%s) was found\n\n",            ip_address);        exit_code = 5;    }    end:        MMDB_free_entry_data_list(entry_data_list);        MMDB_close(&mmdb);        exit(exit_code);}