Project 4 / Scene Recognition with Bag of Words

In this project I implemented scene recognition with 2 different feature selecting methods: tiny images and bag of SIFT features, and 2 different classifiers: K-nearest-neighbors and Support Vector Machine Classifier. I tried to make the code run as in parallel as possible, through using cellfun.

K-Nearest-Neighbor

Using tiny images as features I was able to tweak my KNN algorithm. I tested several values for K and found that 14 worked best. 13 and 15 resulted in less accuracy than 14.

Bags of SIFT features

Bags of SIFT features replaces tiny images as a feature selector. For this algorithm I implemented two methods. build_vocabulary.m and get_bags_of_sifts.m. I had 2 parameters to play with in each of these methods. These parameters tweak the results of the function vl_dsift. This function returns sift features form an inputted image. The parameters were step and size. I found that keeping size the same in both build_vocabulary.m and get_bags_of_sifts.m worked best. I settled on size=16 since that was the recommended feature size. For step I found that in build_vocabulary.m setting it to 16 and in get_bags_of_sifts.m setting it to 8 gave fast and moderately accurate results.

Support Vector Machine Classifier

This method makes use of the function vl_svmtrain. This function has one parameter lambda. I found that setting it to 2 provides consistent results.

Results in a table

Tiny images and K-nearest-neighbors Bags of SIFTs and K-nearest-neighbors
Accuracy (mean of diagonal of confusion matrix) is 0.205 Accuracy (mean of diagonal of confusion matrix) is 0.533

Best results scene classification results visualization

Bags of SIFTs and Support Vector Machine Classifier


Accuracy (mean of diagonal of confusion matrix) is 0.605

Category name Accuracy Sample training images Sample true positives False positives with true label False negatives with wrong predicted label
Kitchen 0.770
InsideCity

Store

Office

Bedroom
Store 0.380
InsideCity

InsideCity

InsideCity

Mountain
Bedroom 0.070
Industrial

Industrial

Suburb

Kitchen
LivingRoom 0.600
Store

Industrial

Highway

InsideCity
Office 0.790
Kitchen

Bedroom

Kitchen

Kitchen
Industrial 0.060
TallBuilding

Street

Highway

Store
Suburb 0.920
Kitchen

Street

Kitchen

LivingRoom
InsideCity 0.430
Store

Store

Store

Store
TallBuilding 0.550
Store

Store

LivingRoom

LivingRoom
Street 0.760
Industrial

Industrial

Suburb

InsideCity
Highway 0.820
OpenCountry

Industrial

Suburb

Mountain
OpenCountry 0.500
Mountain

Coast

Forest

Mountain
Coast 0.710
OpenCountry

Mountain

Mountain

OpenCountry
Mountain 0.760
Highway

Store

Suburb

Forest
Forest 0.950
Store

LivingRoom

Mountain

Mountain
Category name Accuracy Sample training images Sample true positives False positives with true label False negatives with wrong predicted label