Deep Learning Image Database Summary (for collection)

Deep Learning Database Summary

Thanks for the collection.

Source: https://blog.csdn.net/chaipp0607/article/details/71403797

The preparation of the data is necessary to train the model, which is obviously time-consuming, so we can use the existing open source image Library to quickly prepare for the initial work in the introductory phase: ImageNet

Imagenet is an image database organized according to the WordNet hierarchy (currently only nouns), where each node of a hierarchy is depicted by hundreds of and thousands of images. Currently, each node in the database has an average of more than 500 images. We hope imagenet will be a useful resource for researchers, educators, students, and everyone who shares our passion for images.  
Imagenet some features:  
Imagenet is the world's largest open source image library, and up to Now (2017.5) imagenet a total of more than 14 million images. Including more than 20,000 synset (s), Synset is a collection of synonyms, synsnet can be understood as imagenet collation of the label.  
Speaking of WordNet hierarchy, it is necessary to mention what is Wordnet,wordnet is the open-source lexicon of Princeton University, which can be understood as a dictionary. Each word may have several different semantics, corresponding to different sense. And each of the different semantics (sense) may correspond to multiple words, such as topic and subject in some cases is synonymous, WordNet by Princeton University psychologists, linguists and computer engineers, a cognitive linguistics based English dictionary. It is not the light that arranges the words in alphabetical order and makes up a "network of words" in terms of the meaning of the words. WordNet mainly has 3 main concepts Synset, Wordsense and Word. And Imagenet is the application of the concept of synset, but only imagenet currently only nouns.  
Because of the copyright problem of the picture, the picture in Imagenet provides the download in the form of the URLs, that is to say imagenet only provides the picture where, but does not provide the picture itself directly.  
 
When we search for a synset in imagenet, we can see his hierarchy wordnet on the left, and the download address for URLs in download. Cifar

Collected and collated by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton, Cifar selected 60,000 of the 800,000 images in visual dictionary and divided them into CIFAR-10 and CIFAR-100.
The CIFAR-10 dataset contains 60,000 32*32 color images, a total of 10 categories. There are 50,000 training images and 10,000 test images. The dataset is divided into 5 training blocks and a test block with 10,000 images per block. The test block contains 1000 images randomly selected from each class. The training blocks contain these images in a random order, but some training blocks may contain more images than other classes. Training blocks Each class contains 5,000 images.
The CIFAR-100 DataSet contains 100 small classes, each of which contains 600 images, with 500 training images and 100 test images. Class 100 is grouped into 20 large classes.
MNIST

MNIST the famous datasets in deep learning-mnist, almost all the getting started examples of deep learning tutorials are handwritten digit recognition, and the libraries they use are MNIST. It's like showing "Hello World" when we learn a language.
The Mnist dataset contains a total of 70,000 samples, namely the handwritten numeral 0~9, with a sample size of 28*28.

labeled Faces in the Wild

The labeled Faces in the Wild database collects more than 13,000 face images, including more than 5,000 people. Each person is tagged in addition to the person's name, but also contains other information, such as gender, age, etc.

Quick Draw

Quick Draw is a collection of 50 million pictures published by Google, which is divided into 345 categories, which are actually like this:

So it seems that the data set is still very boring, its release is actually originated in Google's Autodraw, this is a can put your graffiti into the painting of artificial intelligence technology tools, that is (Google is always doing something very fun):

Google has also published papers and blogs about the technology behind it. In fact, the technology used by Autodraw is based on Google's previous graffiti experiment "quick, draw!". Recently, Google released the data set behind the project, which is the quick draw data set mentioned earlier. This project is also posted on git, and the data set is described in detail in Git's address, which is briefly described below:

The original data is in the Ndjson file and is segmented by category, in the following format:

This data set is stored in the Google Cloud storage service, which is categorized in the Ndjson file. See the list of files in the Cloud Console, sorted by the following data sets:

Raw files (. Ndjson)
Simplified drawings files (. ndjson)
Binary files (. bin)
Numpy bitmap files (. npy)

Where the original file and the simple strokes are. Ndjson are stored in the form of a binary file (. bin) and a NumPy bitmap (. npy) file.

Binary files (. bin)
We also provide a customized binary format for simplified painting and metadata that can be used for efficient compression and loading. Examples/binary_file_parser.py gives an example of how to load the file with Python.

Numpy bitmap (. npy)
All the simplified paintings have also been converted into 28x28 grayscale bitmaps, saved as numpy. npy format. The file can be loaded via the Np.load () function. Ai-challneger

Ai-challneger is a competition initiated by the Innovation Workshop, which has 6 projects, each with a matching set of data sets, such as the scene classification project, so far has provided three data sets, respectively, is the training set (train), The validation set (valuation) and test set a (test_a) contain a picture 5w+,7k+,7k+, which contains a 80-class scene map that supports direct download of the original image.
Kaggle Cats vs. Dogs

Cat and dog War data set, the famous Kaggle contest data, a two categorical data set, training concentrated cat and dog pictures of 12500, test the central cat and dog pictures A total of 12500, support the original download.
notmnist

Notmnist Data set the name is to salute Mnist, which provides a picture from A to J is a letter, the size of the picture is 28*28, and the picture is not handwritten letters, but comes from a variety of strange images on the network, such as the letter A:

The data set provides two versions, of which the large version, each class about 5.3W of pictures, the total number in 53W, the data between categories is average. Pascal VOC

The PASCAL VOC challenge is a benchmark for the identification and detection of visual objects, providing standard image annotation datasets and standard evaluation systems for detection algorithms and learning performance. PASCAL VOC photo collection consists of 20 directories: human; Animals (birds, cats, cows, dogs, horses, sheep); vehicles (airplanes, bicycles, boats, buses, cars, motorcycles, trains); Indoors (bottles, chairs, tables, potted plants, sofas, televisions). The PASCAL VOC Challenge is no longer held after 2012 years, but its data set images are good quality, well-labeled and ideal for testing algorithmic performance.

COCO Common Objects Dataset

The

Coco DataSet, sponsored by Microsoft, has not only the category, location information, but also the semantic text description of the image, and the open source of the Coco DataSet has made great progress in the semantic understanding of image segmentation in recent two or three years, and has almost become a "standard" data set for the performance evaluation of the image semantic comprehension algorithm. Google's open source show and tell generation model is tested on this dataset.  
currently includes the following competitions:  
1. Target Detection (COCO Detection Challenge), consisting of two races:  
output target border (using bounding box Output), which is what we often call target detection (object detection)  
requires the object to be separated from the image (object segmentation output), which is what we call the semantic segmentation of the image (Semantic Image segmentation)  
2. Images labeled (COCO captioning Challenge)  
Specifically, it is a sentence that accurately describes the information on the image (producing image Captions that is informative and accurate). So how do you rate this? It is now based on manual scoring.  
3. Human key point detection (COCO KeyPoint Challenge)  
The race requirement is to find out where the person is and then locate some key points in the body (the KeyPoint Challenge involves Simultaneously detecting people and localizing their keypoints).  
cityscapes

cityscapes data scene includes 50 different cities (mainly in Germany), Spring and summer fall three seasons of the day scene, there are a lot of dynamic targets at different levels of scene and diverse backgrounds. The scene does not include heavy rain or snow, as this scenario needs to be handled with special techniques.
Image data is divided into 30 categories: In addition to 5000 frames of fine-grained (pixel-level), labeled a picture time control around 1.5h, fine-labeled data divided into the following figure training validation test set, not randomly divided, but to ensure that each partition of the dataset contains a variety of scenarios. Finally, 2975 are used for training, 500 for verification, and 1525 for testing. In addition, there are 20000 weakly labeled frames, which are used only for training, and a picture is controlled within 7min.

Continuous update ...