Menu

Sergey Kosov

About Sergey Kosov

http://www.project-10.de/Kosov/

Sergey Kosov received his Diploma in Applied Mathematics from the Kirgiz-Russian Slavic University, Kirgyzstan in 2004, and M.Sc. degree in Computer Science from the Saarland University, Germany in 2008. From 2008 to 2013, he worked as a researcher in the Max Plank Institute for Informatics and Leibniz University, Germany. Currently he is an external Ph.D. student at Pattern Recognition Group in University of Siegen, Germany. His research interests include classification with conditional random fields and deep neural networks, motion estimation with optical flow, 3-D reconstruction as well as movie industry.

Posts by Sergey Kosov:

human_faces_collection_various_draft_design_6826407

Rorschach Test as The First Human Classification Approach

, , , 0

Human personality classification. How, in principle, can one measure a complex and, in fact, unique human personality with a finite number of labels?  Ascribe a human to one of a few categories? If it seems impossible to solve such a task, the human brain copes with it with a hurrah. We, actually, from the first moments of familiarizing with a new person can already write him down as a family man or a bachelor, a spendthrift or a miser, a careerist or an eternal student. How does this happen? It turns out that our brain loves and is able to simplify everything it encounters. By criterion brain knowns only, in the first 8 seconds, we can determine very precisely the social status of a person, whether he or she is in a relationship, whether we can trust him or her, etc.[1] It is as if we are labeling a person. 

The first impression, of course, is not always the most correct and can change over time. But still, if the brain classifies people so effectively, can we find a scientific or pseudoscientific approach? Mankind made a lot of attempts in the past: signs of the zodiac in astrology, types of personalities in socionics, psychological archetypes of Carl Jung and many more. All these approaches are based on extracting a few features (descriptors) from a human and then giving its personality description based on these features.

Feature Extraction

In machine learning we also preprocess the original input data to transform it into some new space of features where we hope, the classification problem will be easier to solve. The main idea of this preprocessing is to reduce the variability of input data for each class. This makes it much easier for a subsequent classification algorithm to distinguish between the different classes. We call this preprocessing stage feature extraction. Note that new test data must be preprocessed using the same steps as the training data. [2]

human classification with DGM library
Human personality classification with machine learning: feature extraction is an essential stage of input data preprocessing.

Thus, astrology uses only one feature – the date of birth. The archetypes of Carl Jung – two binary features: whether a test person relies more on intuition or sensation, thinking or feeling. Of course, Jung’s features are much more advanced than the ‘date of birth’ feature for classification. However, the first truly scientific method of classifying a person was proposed by Hermann Rorschach in 1921.

Rorschach Test

Look at this image. So it could be? An evil monster? A couple of friendly bears? For almost a century, 10 such ink spots have been used as a kind of mystical personality test.

One of the ten cards in the Rorschach test
One of the ten cards in the Rorschach test. The images themselves are only one component of the test, whose focus is the analysis of the perception of the images.

Developed at the beginning of the 20th century by psychologist Hermann Rorschach, the test is not really about the concrete things we see, but about our common approach to perception. As an amateur artist, Herman was fascinated by how visual perception varies from person to person. He transferred his passion for medicine and learned that our perception process not only registers sensory information but also transforms it. When he started working in a psychiatric hospital in Switzerland, he began to develop a series of bizarre paintings to gain a new understanding of this mysterious process. Using his ink-stained drawings, Rorschach began asking hundreds of people, the same question: “What can it be?”

However, for Rorschach was not really important what the subjects saw, but how they approached the task. What details of the image they were focusing on or ignoring. Whether they can see the movement on the cards. Did the color on some of the ink spots help to give a clearer answer or distract and crowd out the rest? Some people are inhibiting, giving the same answer for several spots, others give unusual and rich descriptions. The answers were as varied as the ink spots, offering different kinds of perception problems – some easier to interpret, others – harder.

Using Inkblots to Describe Human Personality

Rorschach developed a system to encode people’s responses, reducing a wide range of interpretations to a few average numbers. These numbers could serve as ideal features for modern machine learning engines (such as DGM library). But that time Rorschach himself acted as such engine – he had the empirical data to quantify all the tested people.

The analysis of the general approach of the tested person gives a real understanding of his personality and sychology. And as Rorschach tested more and more people, the number of models increased. Healthy subjects with similar personalities often used surprisingly similar approaches. Patients with the same mental illness are also similar in their responses, making the test a reliable diagnostic tool. It can also diagnose some conditions that are difficult to determine by other available methods. [3] Thus, the Rorschach test was the first scientific method for human personality classification.

References

[1] J. Willis and A. Todorov Making Up Your Mind After a 100Ms Exposure to a Face. Psychological Science 17, 2006.
[2] S. Kosov Multi-Layer Conditional Random Fields for Revealing Unobserved Entities. PhD Thesis, Siegen University, 2018 [PDF]
[3] D. Searls How does The Rorschach Inkblot Test Work? TED-Ed talk [video]

Tenerife 4K

Tenerife 4K

, , 0

After 3 years of work, 5 journeys to Tenerife island, 10.5 thousands of frames, 235 Gigabytes of data I am proud to present you this timelapse, which carries a part of astrophotography and the unique nature of the Teide National Park.

There are only a few places in the world where you can go up by car to observe the clear and almost pristine starry sky. To find such a place, it is enough to find out where the major international observatories are. One such place is Tenerife Island. Like Hawaii, Tenerife was born by a volcano – Teide. It is here, in the Teide National Park, at an altitude of more than 2.2 km above sea level, where the air is thinner and cleaner, and the nights are darker, you can enjoy the views of the Milky Way, the constellations and the many flashing satellites of the Earth flying over your head.

I sincerely hope that I have managed to convey in this work a little bit of that unique feeling of freedom when you find yourself at the top of the world under the unthinkably huge Universe which looks at you through billions of stars.

Please watch it in fullscreen, 4K and with headphones! I hope you enjoy it! And you are welcome to like / share if you like it!

Camera:
    Canon 6D

Lenses:
    Samyang 24mm f/1.4 ED AS IF UMC
    ZEISS Planar T* 50mm f/1.4 ZE
    Canon EF 15mm f/2.8 Fisheye
    Canon EF 70-200mm f/2.8L USM

Track:
    ЦИФЕi – Cyber Dreams

tree_dgm

DGM library v.1.7.0 has been just released

, , , , 0

This is our grand winter release DGM 1.7 which incorporates Complete (dense) graphical models, supports recently released OpenCV 4.0 and now available also for Linux (Ubuntu) operating system. The library has also undergone massive refactoring with emphasis on simplifying the user interface. The library is now easier to use than ever before. 

Classical CRF models are composed of unary potentials on individual observations and pairwise potentials on neighboring observations. The resulting adjacency CRF structure is limited in its ability to model long-range connections within the observations and generally results in excessive smoothing of observed object boundaries. Dense CRF establishes pairwise potentials on all pairs of observations, enabling greatly refined labeling. The main challenge is the size of the model. DGM library now includes efficient inference algorithm for dense CRF models in which the pairwise edge potentials are defined by a linear combination of Gaussian kernels in an arbitrary feature space. The resulting approximate inference algorithm is sub-linear in the number of edges in the model [1].

In this release we introduce two powerful tools for simplifying work with graphical models:

  1. Graph extensions, which significantly simplify building and filling the 2D graphical models used for image classification. In addition, with help of the pairwise graph extension the training of the edges was also simplified in terms of the required user code.
  2. Factory methods, which allow for creating objects of a library in a way such that it doesn’t have tight coupling with the class hierarchy of the library. Such factories allow you to switch from one model to another (pairwise graph to dense, Bayes classifier to neural network classifier, etc.) with changing just one argument – the flag of the factory method function.

Our library was conceived primarily as an educational project – a useful tool for university students and researchers who experiment with various machine learning algorithms for semantic segmentation and labeling of images. That is why from the very first versions of the DGM library we paid a lot of attention to the efficient and qualitative evaluation of algorithms and results obtained with the help of our library. Since students and researchers often have situations when they have to change the library code for their specific tasks, we decided that it was time to review the quality of our code. Starting from this release we incorporate an automatic code review solution CodeFactor and try to keep our current grade score A (97,6%) at the same level. Together with the in-depth documentation this will allow our users easily go into the code and understand the implemented algorithms fast.

Finally, we have changed all the demo code snippets to illustrate the benefits of factory methods and graph extensions and added two more tutorials: one for dense CRFs and another – for the control model parameters training. Every tutorial includes a ready-to-use demo application within the library. These demo applications are ideal base for starting your own project and starting to use our DGM library.

[1] P. Krähenbühl and V. Koltun Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

chart 16

Weekly CoT News

, 0

Natural Gas (Chart 1) has reached a 3-year high (Chart 2) in terms of short positions of commercials. This fact is based on recent Commitment of Traders Report (CoT) and indicates that it is more likely that the price will fall. The observation of the net positions of the commercials also support this conclusion. The net positions of the commercials have reached a bearish area compared to the last 52 weeks (Chart 3). Considering these 2 facts together it is more likely that we will see falling prices in the next few weeks or months. Here are the relevant charts:


Rough Rice (Chart 4) has reached a 3-years low (Chart 5) in terms of short positions of commercials. This fact is based on recent Commitment of Traders Report (CoT) and indicates that it is more likely that the price will rise. The observation of the net positions of the commercials also support this conclusion. The net positions of the commercials have reached a bullish area compare to the last 52 weeks (Chart 6). Considering these 2 facts it is more likely that we will see rising prices in the next few weeks or months. Here are the relevant charts:


These news were generated with help of DGM library

brent oil

Time to Sell Brent Oil, Gasoline or Wheat

, , 1

Brent oil, gasoline and wheat (SR) have reached a 3-years high in terms of short positions of commercials. This is an extreme and it indicates that it is more likely that the price will fall. This interesting fact is supported by the net positions of the commercials, which have reached a bearish area compare to the last 52 weeks. Counting these 2 facts together it is more likely that we will see falling prices in the next few weeks or even months.

GASOLINE BLENDSTOCK (RBOB) - NEW YORK MERCANTILE EXCHANGE
Gasoline

WHEAT-SRW - CHICAGO BOARD OF TRADE
Wheat (SR)

These news were generated using the DGM library.
RG-High-Sierra-Blog

Conditional Random Fields say Hello to macOS

, , , 0

DGM library v.1.6.0 has been just released

We are glad to present our next big release of DGM, v.1.6.0, which summarizes the v.1.5.x line with further improvements and bug fixes. This is the first cross-platform release: since now on the DGM library is available also for macOS. The binaries built for macOS High Sierra (OS X 10.13) are now available for download. See the changelog for details.

DGM is a C++ library which extends popular OpenCV by implementing various tasks in probabilistic graphical models for Conditional Random Fields. In particular, the DGM learning units include:

  • Artificial Neural Networks     
    		•
  • Random Forests Model
    		•
  • Support Vector Machines
    		•
  • Sequential Gaussian Mixture Model
    		•
  • k-Nearest Neighbors
    		•
  • Bayesin Model
    		•

    The library is also supplied with advanced feature extraction and visulaization modules. The demo code could be run directly after installation and may serve as a base for user projects.

    119_img

    Vaihingen Double Layer Dataset (Vaihingen-DL) is released

    , , , 0

    Introduction

    The Vaihingen-DL dataset contains aerial images of Vaihingen village in Germany, associated with corresponding digital surface models (DSM) and two ground truth images – one for the base and the second – for the occlusion layer.

    Base layer Occlusion layer
    class 0 Road class 0 Void
    class 1 Traffic island (asphalt) class 1 Tree
    class 2 Sidewalk class 2 Car
    class 3 House class 3 Bridge
    class 4 Grass
    class 5 Agriculture
    class 6 Water
    class 7 Sealed
    class 8 Traffic island (vegetation)
    class 9 Beach
    class 10 Railway


    In ground truth images, the classes are marked with pixels with the above mentioned values (e.g. classes “Sidewalk” and “Car” have values 4).

    Usage

    The Vaihingen-DL dataset can be used to test image segmentation, feature extraction, classification approaches, etc especially for occluded areas. It has two layers of reference labels, thus the occluded areas of the scenes are also covered with ground truth labels. Two layers of labels could be used with the multi-layer CRF classification framework which is a part of the Direct Graphical Models library.

    Copyright of the images in the Vaihingen-DL fully belongs to their owners. In no event, shall owners be liable for any incidents, or damages caused by the direct or indirect usage of the images. The dataset should be only used for non-commercial research and/or educational purposes.

    Download

    The Vaihingen-DL dataset can be downloaded from this link: Vaihingen-DL.rar (61MB). If you use the dataset in your publications, please cite it, using this BibTex file.

    PLATINUM (US$ / OZ)

    Time to buy platinum!

    0

    Platinum has reached a 3-years low in terms of short positions of commercials. This is an extreme and it indicates that it is more likely that the price will rise. This interesting fact is supported by the net positions of the commercials, which have reached a bullish area compare to the last 52 weeks. Counting these 2 facts together it is more likely that we will see rising prices in the next few weeks or even months.

    EMDS

    Environmental Microorganism Dataset (EMDS) has been just released

    , , , 0

    Introduction

    The EMDS dataset contains environmental microorganism (EM) images downloaded from the Internet, associated with corresponding binary ground truth images. In total there are 21 classes of EMs. Each class is represented with 20 EM images with the corresponding binary ground truth bitmap. In ground truth images, EMs are marked with white (value: 255) and the background is marked with black (value: 0).

    List of EMs:

    class 1   Actinophrys       class 8 Paramecium class 15   Keratella Quadrala
    class 2 Arcella class 9 Rotifera class 16 Euglena
    class 3 Aspidisca class 10 Vorticella class 17 Gymnodinium
    class 4 Codosiga class 11 Noctiluca class 18 Gonyaulax
    class 5 Colpoda class 12 Ceratium class 19 Phacus
    class 6 Epistylis class 13 Stentor class 20 Stylongchia
    class 7 Euglypha class 14   Siprostomum       class 21 Synchaeta

    Usage

    The EMDS dataset can be used to test Image segmentation, feature extraction, classification approaches, etc. Copyright of the images in the dataset fully belongs to their owners. In no event, shall owners be liable for any incidents, or damages caused by the direct or indirect usage of the dataset. The dataset should be only used for non-commercial research and/or educational purposes.

    Download

    The EMDS dataset can be downloaded from this link: EMDS-4.rar (110MB). If you use the dataset in your publications, please cite it, using this BibTex file.

    nn_screening

    Efficient K-Nearest Neighbours

    , , 2

    The K-nearest neighbours classifier (KNN) is a type of instance-based learning, or lazy learning, where the function is only approximated locally and all computation is deferred until classification. Thus, the KNN approach is among the simplest of all discriminative approaches, but this classifier is still especially effective for low-dimensional feature spaces. However, the application of the KNN model in practical applications is problematic because of its low-speed performance for large datasets represented in high-dimensional feature spaces and for the large number of neighbors – K. In this article we address exactly this problem of the KNN model.

    The input for the KNN algorithm consists of the K closest training samples in the feature space and the output is a class label l. An observation (or testing sample) y is classified by a majority vote of its neighbours, with the observation being labelled by the class most common among its K nearest neighbours (see figure below, center). In case of K = 1 the class of that single nearest neighbour is simply assigned to the observation y.

    Efficient K-Nearest Neighbours
    The original distributions of 160'000 samples from the dataset
    hist2d_CvKNN_pred
    Resulting k-Nearest Neighbors decision map
    Efficient K-Nearest Neighbours
    k-Nearest Neighbors classifier

    In order to estimate the potentials we consider the class of every neighbour as a vote for the most likely class of the observation. If the number of neighbours, having class l is Kl we can define the probability of the association potentials as: (see figure above, right)

    p(x=l,|,y)=frac{K_l}{K}

    It can be useful to assign weight to the contributions of the neighbors, so that the nearer neighbors contribute more to the average than the more distant ones. For example, a common weighting scheme consists in giving each neighbor a weight of 1 / r, where r is the distance to the neighbor. For our weighting scheme we modify this idea as follows: let r will be the Euclidean distance from the test sample to the nearest training sample in feature space and ri – Euclidean distance to every found neighbor. Then we can rewrite the previous equation with weighting coefficient:

    p(x=l,|,y)=frac{1}{K}sum_i{frac{1_l}{(1+r_i-r)^2}},

    where 1l means 1 if the class of the training sample is l and 0 otherwise.

    Optimization

    The search algorithm aims usually to find exactly K nearest neighbors. However it may happen, that distant neighbors do not affect probability p(x = l|y) much. For example, the nearest neighbor with ri = r contributes value of 1 / K to the probability. And a neighbor, twice as distant from the testing sample (ri = 2r) will contribute only 1 / K(1 + r)2. For the optimization purpose we stop the search once the distance from the test sample to the next nearest neighbor exceeds 2r. Thus, only K’ ≤ K neighbors in area enclosed between two spheroids of radii r and 2r are considered (see figure below) and weighted according to the equation: p(x = l|y) = Kl / K’.

    Illustration of the nearest neighbors screening: if the distance to the nearest neighbor is r, we take into consideration only those neighbors that lie closer then 2r distance.
    Illustration of the nearest neighbors screening: if the distance to the nearest neighbor is r, we take into consideration only those neighbors that lie closer then 2r distance.

    The neighbors are taken from a set of objects for which the class is known. This can be thought of as the training set for the algorithm, though no explicit training step is required. A peculiarity of the KNN algorithm is that it is sensitive to the local structure of the data.

    Evaluation

    Our implementation of the KNN model in DGM C++ library is based on the KD-tree data structure, which is used to store points in k-dimensional space. Leafs of the KD-tree store feature vectors with corresponding groundtruth and every such feature vector is stored in one and only one leaf. Tree nodes correspond to axis-oriented splits of the space. Each split divides space and dataset into two distinct parts. Subsequent splits from the root node to one of the leafs remove parts of the dataset until only small part of the dataset (a single feature vector) is left.

    KD-trees allow to efficiently perform searches “K nearest neighbors of N”. Considering number of dimensions k fixed, and dataset size N training samples, the time complexity for building a KD-tree is O(N · logN) and for finding K nearest neighbors – close to O(K · logN). However, its efficiency decreases as dimensionality k grows, and in high-dimensional spaces KD-trees give no performance over naive O(N) linear search.

    In order to evaluate the performance of our KNN model, we perform a number of experiments: 2r-KNN, 4r-KNN, 8r-KNN, 16r-KNN and 32r-KNN – models, where the nearest neighbors enclosed between two spheroids of radii r and 2r (4r, 8r, 16r and 32r respectively) are only taken into account. In the ∞r-KNN experiment all the K neighbors were considered. And finally the KNN experiment is the OpenCV implementation of KNN (CvKNN) based on linear search. The overall accuracies and the timings for all 7 experiments are given in table below:

    2r-KNN 4r-KNN 8r-KNN 16r-KNN 32r-KNN ∞r-KNN CvKNN
    Training: 4659 sec 4659 sec 4659 sec 4659 sec 4659 sec 4659 sec 102 sec
    Classification: 8,3 sec 22,2 sec 52,8 sec 97,2 sec 134,9 sec 216,1 sec 45,3 sec
    Accuracy: 81,39 % 81,65 % 81,97 % 82,11 % 82,33 % 82,42 % 82,36 %

    Accuracies and timings for Intel® Core™ i7-4820K CPU with 3.70 GHz required for training on 1016 scenes and classification of 1 scene.

    Our 2r-KNN model gives almost the same overall accuracies as the reference KNN model, but needs almost 5.5 times less time. The training time of the xr-KNN models, which includes the building of the KD-tree, takes 78 minutes, what is much more slower then 1,7 minutes for KNN training. However, the training in practical applications is performed only once and could be done offline, when the classification time is more critical for the whole classification engine performance. In the table above we can also observe almost linear increase of the classification time with increasing the outer spheroid radius to 4r, 8r, etc. Figure below shows the classification results for the experiments 2r-KNN – ∞r-KNN.

    hist2d_KNN_R_2
    2r-KNN
    hist2d_KNN_R_4
    4r-KNN
    hist2d_KNN_R_8
    8r-KNN
    hist2d_KNN_R_16
    16r-KNN
    hist2d_KNN_R_32
    32r-KNN
    hist2d_KNN_R_inf
    ∞r-KNN

    Load More