Building a Wall Construction Detection Model with Keras.

I am building a project to detect wall construction types from images of Pompeii. I am using Waleed Abdulla’s Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. Also, I employ the technique described by Jason Brownlee in How to Train an Object Detection Model with Keras to detect kangaroos in images. Instead of kangaroos, I want to detect the type of construction used in building walls.

This is a brief post describing the preparation of images for training as well as the initial results. The images used are from the Pompeii Artistic Landscape Project and provided courtesy of Pompeii in Pictures. The original images were photographed by Buzz Ferebee and they have been altered by the program used for predictions. An example of an image showing the model’s detection of construction type opus incertum is below. Cinzia Presti created the data used to select the images for training.

To build this model, images were selected for training. Given the construction type is visible in only parts of each image, rectangles in each image show where the construction type is visible.

Each of the images has a corresponding xml file containing the coordinates of the rectangles that contain the objects used to train on. See file 00096.xml below:

<annotation>
	<folder>opus_incertum</folder>
	<filename>00096.jpg</filename>
	<path>/home/student/data_5000_project/data/images/construction_types/raw/opus_incertum/pompeiiinpictures Ferebee 20600 May 2016 DSCN8319.JPG</path>
	<source>
		<database>pompeiiinpictures.com</database>
	</source>
	<size>
		<width>1024</width>
		<height>768</height>
		<depth>3</depth>
	</size>
	<segmented>0</segmented>
	<object>
		<name>opus_incertum</name>
		<truncated>0</truncated>
		<difficult>0</difficult>
		<bndbox>
			<xmin>16</xmin>
			<ymin>579</ymin>
			<xmax>257</xmax>
			<ymax>758</ymax>
		</bndbox>
	</object>
	<object>
		<name>opus_incertum</name>
		<truncated>0</truncated>
		<difficult>0</difficult>
		<bndbox>
			<xmin>507</xmin>
			<ymin>563</ymin>
			<xmax>703</xmax>
			<ymax>749</ymax>
		</bndbox>
	</object>
	<object>
		<name>opus_incertum</name>
		<truncated>0</truncated>
		<difficult>0</difficult>
		<bndbox>
			<xmin>829</xmin>
			<ymin>570</ymin>
			<xmax>1007</xmax>
			<ymax>752</ymax>
		</bndbox>
	</object>
</annotation>

The program to create the xml annotation files also saves images using a standard numeric file name (ex.: 00001.jpg) and width of 1024 pixels.

Initial Results

The “Actual” column of images below shows images used in training the model. The white rectangles show the boundary boxes contained in the corresponding xml file for the image. Some images don’t have a white rectangle. These images were deemed by me not to have a good enough sample for training so I didn’t make an xml file for them.

The “Predicted” column shows what the model considers to be opus incertum construction. Frequently it’s correct. It does make errors too, considering the blue sky in row 5 is recognized as stone work. I want to see if further training can correct this.

A couple things to note: It’s bad practice to run a model on images used to train it, but I am doing this here to verify it’s functioning. Later, I also need to see how the model performs on images with no opus incertum.

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

References

Abdulla, Waleed. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. GitHub repository. Github, 2017. https://github.com/matterport/Mask_RCNN.

Brownlee, Jason. How to Train an Object Detection Model with Keras. Machine Learning Mastery. https://machinelearningmastery.com/how-to-train-an-object-detection-model-with-keras/.

Dunn, Jackie and Bob Dunn. Pompeii in Pictures.

Ferebee, Buzz. Pompeii Photographic archive. 2015-2017