By Andrea Siotto
This article is the third of a series on a project of mapping the ship lost during the First World War.
We explored the data collection with web scraping in Python here.
Once the data was collected and organized in a csv file with the name of the ship, the date of sinking, the country of the ship, the coordinates, the description, and a list of the links offered by Wikipedia, there was a big problem to solve: half of the ships did not have coordinates.
For the majority of these almost four thousand ships without coordinates the location of sinking was present in the description in a format such as “(125 km) north east of random-lighthouse.” There was still hope, but either I would have to spend months on doing the job manually or I could implement software that automated the task as much as possible.
A decent plan was necessary: I wanted the software to find the location of the “random lighthouse or place”, then to find the bearing, to translate it into degrees, to retrieve the distance in the text, and to find on the map the approximate coordinates of the sinking location.
Let’s see the different tasks one by one.
To find the location seems an easy task: we could query Google Maps. However, often it is difficult to find locations that are important for sailing such as lighthouses and moreover light vessels, especially if their name was changed after one hundred years. To solve this problem, I decided to find a database of such locations with their coordinates. It was not a simple task, but once I found the relevant webpages I could use a mix of Python scripting, Excel, and Notepad++ to create another csv file with such locations. My program in C# first interrogates the list of locations, if it does not find them then it asks Google Maps and if successful then I can manually add the location to the list with a button. In the rare case that it does not find the location I can search online, then go on the map in the program, and add the location. Adding the location to the list is fundamental, because the more I add, the less I need to search with long navigations in the browser.
To find the bearing and distance I had to analyze the description. At the end of this article you can find the code of the two functions that find the bearings and convert them into degrees. There is one key element that I strongly suggest you to learn well: regular expressions. They will solve so many troubles for you that you will start to love them (after the first frustrations). A great help in learning them is this website, where you can test your expressions and learn them in the best way: trying them.
Once I had the coordinates of our reference place (for example the lighthouse), bearing, and distance, I could calculate the approximate location where the ship sank. I tried to use complicated functions that considered the shape of the earth, but after some frustrating problems, I decided that I could find a practical solution with some easy math and considering the earth as a sphere. The locations are imprecise anyway, and an error of even ten kilometers was not a problem for our task.
The program then saves the ship on another file with the added coordinates.
A few considerations:
Why C#? – Because it is the easiest way (that I know) for making a program in windows with all the fancy stuff like buttons, textboxes, and menus. All this stuff is fundamental to implement a program that helps in managing data.
How did you put Google Maps in your program? – There is a browser integration into the program. It was necessary to have a visualization of the position of the ship and check that there was not an obvious problem (such as the ship sunk in the middle of Ireland). To incorporate the browser in your program I used CefSharp, a library that I would recommend strongly.
Are you an expert in User Interfaces? – I am an historian, so not at all. And I am obviously not an expert programmer. But I kept my mind open, I modified things doing test runs, and I looked for the least number of clicks and copy-and-paste possible; slowly I noticed that I was using mostly 3 buttons and I decided to group them and increase their dimension. I basically evolved the program for my personal needs and preferences.
The Code.
/*
This file contains the snippets of a program that automate as much as possible the finding of the position of a shipwreck based on a descriptive coordinate system.
Andrea Siotto
*/
// This function takes a string (a description of the events with a possible bearing and distance of the sinking location from a given place),
// searches for a combination of words (the bearing),
// and returns a list of the bearings found.
private List searchAndTransfDirections(string description) // searches for the definition of bearing and distances
{
List result = new List(); // the list where to put the results
// this block of coding is the real magic: with the help of the regular expressions (regex), it searches for strings such as “(128 km) east by north” or similar
string pattern = @”(\(\d+ km\)|\(\d+,\d+ km\)|\(\d+.\d+ km\)) (east|south|north|west) by (east|south|north|west)”;
string pattern1 = @”(\(\d+ km\)|\(\d+,\d+ km\)|\(\d+.\d+ km\)) (east|south|north|west) (east|south|north|west) (east|south|north|west)”;
string pattern2 = @”(\(\d+ km\)|\(\d+,\d+ km\)|\(\d+.\d+ km\)) (east|south|north|west) (east|south|north|west) by (east|south|north|west)”;
string pattern3 = @”(\(\d+ km\)|\(\d+,\d+ km\)|\(\d+.\d+ km\)) (east|south|north|west) (east|south|north|west) of”;
string pattern4 = @”(\(\d+ km\)|\(\d+,\d+ km\)|\(\d+.\d+ km\)) (east|south|north|west) of”;
MatchCollection m = Regex.Matches(description, pattern,RegexOptions.IgnoreCase);//searches the “direction” by “direction” case
if (m.Count > 0)
{
for (int ctr = 0; ctr 0)
{
for (int ctr = 0; ctr 0)
{
for (int ctr = 0; ctr 0)
{
for (int ctr = 0; ctr 0)
{
for (int ctr = 0; ctr < m.Count; ctr++)
{
result.Add(m[ctr].Value);
}
}
}
}
}
}
return result; // it returns the list of the bearings found.
}
// this function takes a string with a description of the bearings and transforms them in degrees.
private string convertBearingstoDegree(string bearings)
{
// declaration of the matrix containing all the descriptions and the respective degrees.
string[,] directions = new string[2,32]
{
//new string[32]
{
“North”,
“North by East”,
“North North East”,
“North East by North”,
“North East”,
“North East by East”,
“East North East”,
“East by North”,
“East”,
“East by South”,
“East South East”,
“South East by East”,
“South East”,
“South East by South”,
“South South East”,
“South by East”,
“South”,
“South by West”,
“South South West”,
“South West by South”,
“South West”,
“South West by West”,
“West South West”,
“West by South”,
“West”,
“West by North”,
“West North West”,
“North West by West”,
“North West”,
“North West by North”,
“North North West”,
“North by West”,
},
//new string[32]
{
“0”,
“11.25”,
“22.5”,
“33.75”,
“45”,
“56.25”,
“67.5”,
“78.75”,
“90”,
“101.25”,
“112.5”,
“123.75”,
“135”,
“146.25”,
“157.5”,
“168.75”,
“180”,
“191.25”,
“202.5”,
“213.75”,
“225”,
“236.25”,
“247.5”,
“258.75”,
“270”,
“281.25”,
“292.5”,
“303.75”,
“315”,
“326.25”,
“337.5”,
“348.75”
}
};
string pattern = @”(east|south|north|west) (east|south|north|west) by (east|south|north|west)”;
string pattern1 = @”(east|south|north|west) by (east|south|north|west)”;
string pattern2 = @”(east|south|north|west) (east|south|north|west) (east|south|north|west)”;
string pattern3 = @”(east|south|north|west) (east|south|north|west)”;
string pattern4 = @”(east|south|north|west)”;
// in this block of code the function searches for the matching bearings to see if the string is a bearing
string controlledBearings = “”;
string match=Regex.Match(bearings, pattern).Value;
if (match != “”) { controlledBearings = match; }
else
{
match=Regex.Match(bearings, pattern1).Value;
if (match != “”) { controlledBearings = match; }
else
{
match=Regex.Match(bearings, pattern2).Value;
if (match != “”) { controlledBearings = match; }
else
{
match = Regex.Match(bearings, pattern3).Value;
if (match != “”) { controlledBearings = match; }
else
{
match = Regex.Match(bearings, pattern4).Value;
if (match != “”) { controlledBearings = match; }
}
}
}
}
string result = “-1”;
for (int x = 0; x < 32; x++)
{
if (string.Equals(directions[0, x], controlledBearings, StringComparison.OrdinalIgnoreCase))// if the bearing is found in the list then it returns the value
{
return directions[1,x];
}
}
return result; // if the value it is not found then it returns an empty string.
}