Grid Data warehouse receiving

Spatial Database technology space database technology uses Relational Database Management System (RDBMS) to manage spatial data. It mainly solves the problem of data interfaces between spatial data stored in relational databases and applications, that is, spatial databaseengine ). To be more precise, the space database technology is used to solve the problem of accessing the geometric attributes of Spatial Data Objects in relational databases. Its main tasks are:

1) Use relational databases to store and manage space data;
2) read spatial data from the database and convert it to the format that can be received and used by GIS applications;
3) import the spatial data in the GIS application to the database for relational database management. Therefore, the spatial database technology is the channel for the entry and exit of spatial data into and out of relational databases.

I. purpose of creating a database:
The purpose of establishing an image database is to organize massive image data produced in different layers to comply with unified specifications and standards, and effectively organize and manage the data, it facilitates the query and distribution of spatial data and other applications. After the database is built, the data is standardized and standardized, and unified encoding and format are adopted. The data is effectively organized in the plane direction, the split data should be organized into a logically seamless whole. in the vertical direction, various data can be superimposed and combined through consistent spatial coordinate positioning; it has efficient functions such as space data query, scheduling, roaming, and data distribution, and can be seamlessly integrated with other systems to serve other applications. From the perspective of applications, the overall goal of the ghost library is to manage the normal image data with multiple scales, resolutions, and data sources, allows you to roam, browse, and use images quickly and seamlessly from the full picture to the details, from the whole to the local area, from the low resolution to the high resolution, supports the storage and management of Chinese and Distributed Image datasets (distributed storage within the LAN), providing an efficient and seamless platform for massive data applications.

Ii. database creation principles:

In short, there are two methods: layered and segmented ". Two methods refer to raster dataset and raster directory ). Its storage and management methods are "layered and segmented ".
In personal Geodatabase, raster data can be stored as a raster directory table or a raster dataset. A raster dataset is a continuous single data. The raster directory table is a collection of multiple rasters. Each storage method has its own advantages and limitations, but there are also rules that must be followed by both methods.
You can create an empty container for both the raster directory table and the raster dataset, and then load data to it, or dynamically create a container. You can use the data management tool in arctoolbox to create and load a grid directory table or grid dataset. These tools can be found on the arccatalog user interface.

1. Raster Data

To put it simply, "raster dataset" is ArcGIS's abstraction of the Raster Data Model. Its English format is rasterdataset. Any physical raster file (such as the erdasimagine file and ArcGIS asciigrid file, (TIFF files, etc.) The Raster Data Model abstracted by ArcGIS exists in the form of rasterdataset in the memory. rasterdatset is generally composed of at least one rasterband band, for example, a simple gray-scale image is composed of data in one band, and a general color synthetic image is composed of three or more bands, the multiclass spectrum consists of multiple channels (we call rasterband a channel ). Computer displays generally provide three display channels: R (red), g (green), B (blue ), therefore, even if we have a multi-band image data at hand, we can only display the three data bands at the same time, for example, red, green, and blue are used for the 5, 4, and 3 bands of TM images. We can also understand the Raster Data Model in this way. rasterdatset is composed of multiple bands. We understand the band as a layer, in this way, each rasterdatset is composed of multiple "layers", and each layer is composed of two-dimensional arrays with row and column attributes. In order to abstract the internal implementations, arcGIS uses the rasterband class to wrap this two-dimensional array, and provides various methods to operate its internal two-dimensional array ..
Raster datasets are suitable for aerial or Wei slices with standard coordinate systems. When you import image files in this way, SDE concatenates them into a complete large map, however, this method imposes very strict requirements on each image frame. To achieve splicing, the adjacent sides of the adjacent image frame must not have a small amount of point overlapping and misplacement, and the image frame must have a coordinate system. However, there is also a problem with the management method of combining images. What should I do if I want to update images in a small area in the future? For example, the ghost Building in New York is not hit. In fact, you only need to update the data at the point in Manhattan, if you want to use rastermap to manage all the images and maps in New York, it means you need to delete all the original data and re-import the whole city photos, dozens of G, it is not a small number. If it is, it is estimated that the image data volume in New York should be around 300 GB. How can this problem be completed in just a few days.

 

2. Raster directory

The "raster directory" can be simply understood as a common table data model. Its English format is rastercatalog, each record in the "raster directory" is composed of a "raster dataset" and multiple metadata information that describes the "raster dataset. Through this definition, we can see that it has a great advantage for building a basic database to use the "raster directory" to manage the split image data, because the split image data often has many metadata attributes. "Grid directory" management also provides a convenient portal for us to develop basic data systems.
The raster directory is a photo album. You can store pictures of everything, and SDE only stores and maintains a Directory for them. This can be seen from the oraclespatial table corresponding to rastercatalog, it feels like a consortium with dismember pictures. If you use an ESRI desktop tool (such as ArcMap and arccatalog) to preview this rastercatalog, you will see a spliced image! Using this method to manage airline films will not solve the problem mentioned above in the rastermap method. For example, if the ghost tower is gone, we will repeat it there, update the image corresponding to the original rastercatalog, and the rest is unnecessary.

3. "Grid dataset" and "Grid directory"
In short, for "raster dataset" and "raster directory", ArcGIS is actually a memory model for raster data abstraction, as the core data model of Geodatabase, "raster dataset" generally stores geographic background data in the basic database. The requirement is that the data does not change frequently (for example, it is not suitable for storing the background data that is updated frequently ); "raster directory" is generally used to manage image data with attribute information, such as split data or multiple data periods in the same region. Note that, the same "Grid directory" must have the same space reference.
When creating a grid directory table, you must set the XY field (a set of all input grid space ranges), but not the grid dataset. Like all other datasets in ArcGIS, we strongly recommend that you pre-define the spatial reference, geographical coordinate system, or projection coordinate system to import the grid data of a directory table or dataset. Such a coordinate system does not need to be consistent with the coordinate system of the Directory table or dataset. 1 p + N/A % Z + | "Y
A raster dataset has a spatial reference. During the Mosaic (MOSAIC) process, pixels in different coordinate systems are dynamically projected to the correct position. In the raster directory table, each raster has its own space reference, which is different from the Space reference of the ry space reference and the raster column, these grids are dynamically projected only during display or analysis.

 

4. Image pyramid

"Layered" refers to the index of the image pyramid (pyramid. The basic idea is to generate a pyramid from the bottom of the sample, and take a certain level of the pyramid as the operation object as needed to improve the overall efficiency. Of course, like other things in this world, efficiency improvement comes at a cost. This is the extra space overhead brought about by tower construction. The more levels it creates, the more convenient it is to query, of course, the larger the data redundancy is. + D4 C5 R9 Q9 ^: J1 {%'
If a pyramid is created for a large Raster image, these images can be quickly displayed. In addition to display on the screen, the pyramid also contains a lot of other information. If there is no pyramid, you need to access the raster dataset during display, and then perform a lot of calculations to select which raster pixels are displayed. Pyramid is a method for storing raster images in a copy mode that reduces resolution step by step. By selecting a resolution similar to the display area, you only need to perform a small amount of queries and calculations to reduce the display time.
Every time you use ArcGIS to open an image, creatingpyramids will be displayed in the status bar. At this time, the image's gold tower is being built. Currently, portal applications such as Google Map, visualearth, and mapbar use to pre-process the map into an image pyramid, and split the chunk into four-tree codes. The maptile is dynamically called during the zoom and Pan operations. This architecture omits out the components that consume a lot of server performance.
Each layer of the image pyramid has its resolution, so according to your current operation, such as amplification (whether it is a pull box or a fixed proportion), zoom out, roaming (this operation does not involve changes to the image resolution) to calculate the image resolution required for this operation and the geographic coordinate range that will be displayed within the current view range, then, the resolution is matched with the resolution of the created image pyramid. The resolution of the layer of the image pyramid is the closest to that of the image pyramid, then, based on the display range of the current view after the operation, determine the number of corresponding images on the image pyramid of the layer, and then draw them out.

 

5. Block/tile Storage

"Block" means that each frame is stored by block or tile and indexed by grid. When storing a graph in a database, SDE does not store a silly row and a row, but divides the graph into several same size blocks, each of which cannot exceed 16 kb, generally, 128*128 is used. The order of partitioning is from top to bottom, from left to right. The benefit of partitioning is that it can reduce disk I/O. But how can there be such a good image, and the length and width are all integer times of 128? In fact, there is almost no such good image. If the length and width of the image cannot be exceeded by 128, the SDE policy is to add zero. Add a number of zeros (RGB: 000000) to the right and lower sides of the image, that is, black spots, so that processed images can be divided.

Iii. database creation process

After all the software and hardware are ready (this includes the SDE database file created during the installation of the ArcSDE software), you can create an image database. In enterprise-level databases, the storage structure of raster data includes blocks, indexes, image pyramid creation, and is often compressed. It is precisely because the data has attributes such as partition, index, and pyramid structure that only data blocks that meet the query range and accuracy will be returned each time the raster data is queried, instead of returning the entire dataset every time. Compression can reduce the amount of data exchange between customers and servers, store large, seamless raster datasets and reach several TB of raster directories, and enable them to be quickly displayed on the client.
1) Data Preparation
Because the image database has strict requirements for storing raw data in the database, this is also a part of ensuring the integrity of the image database. Therefore, you should organize and place the original images before receiving the database, at the same time, make sure that each original image file has a unique and correct coordinate file stored in the same directory and count the data volume for future use.
2) compression mode
As the image data volume is huge, in order to reduce storage space and improve display efficiency, the size of the block is compressed before the raster data is stored in the geographic database. The compression method can be lossy (JPEG and MPEG-4) or lossless (lz77 ). Lossless Compression means that the pixel values in the grid data set will not be changed. The amount of compression depends on the type of the image metadata. The more consistent the image, the higher the compression ratio.
The main benefit of compressing data is that it can save storage space. Another advantage is that it provides overall performance because it reduces the data packets exchanged between the server and the customer application.

 

Lossy compression can be selected for the following reasons:
A. If the raster data is only used as a background image, you do not need to analyze it.
B. Fast data loading and retrieval!
C. The required storage space is small, because the compression ratio can reach or (more compression ratio can be obtained if you select 20001. for example, or.

Lossless compression can be selected for the following reasons;
A. Raster datasets are used to obtain new data or for visual analysis.

B. The compression ratio is between and.
C. do not intend to retain original data #

D. The input data has been lossy and compressed
Even if you do not compress the raster data, the storage capacity of the enterprise-level geographic database can meet the requirements. However, we recommend that you compress the raster data. If you cannot determine the compression method, use the default lz77 (lossless compression ).

3) data warehouse receiving

ArcSDE manages images in two ways: consecutive raster datasets and raster directories. Each grid directory is independent of each other, facilitating updates and database maintenance. You can query and access a single dataset and add user-defined fields to the table based on user-defined attributes. Therefore, the raster directory rastercatalog is used to store image data.
The procedure is as follows:
After successfully connecting to the database server (detailed introduction before the connection process), create a grid directory. Right-click "new" -- "raster catalog ".
Import data to the raster directory. Right-click "Grid directory"> "LOAD"> "load data" to import the grid data. (4) building a pyramid
After receiving a large amount of data for a long time, use the arccatalog tool to connect to the database, select the image database item, right-click and select "buildpyramids, the system began to create an image pyramid. After creating the pyramid, the image database is basically created. To work with the image, you can import vector data to the database to form a truly meaningful image database.
When a pyramid is created, a dataset (. RRD) file with reduced resolution will be created. For an uncompressed raster dataset, the size of the (. RRD) file generated by the dataset is approximately 8% of that of the source raster dataset. You cannot create a pyramid for a grid directory, but you can create a pyramid for each grid data set ..

 

Create a pyramid (INDEX) in arccatalog)

1. Right-click the grid dataset in the catalog tree.

2. Click build pyramids.
3. If not selected, it points to the raster dataset of the pyramid you want to create.

4. Click OK.

 

Change default settings for creating a pyramid

1. Click the Tools menu, and click options.

2. Click the raster tab.

3. Click the appropriate option to describe the creation of the pyramid.