{"id":5409,"date":"2018-10-23T13:33:08","date_gmt":"2018-10-23T17:33:08","guid":{"rendered":"https:\/\/sites.temple.edu\/tudsc\/?p=5409"},"modified":"2023-01-24T12:58:54","modified_gmt":"2023-01-24T16:58:54","slug":"create-a-network-from-xml-and-visualize-it-a-soccer-player-passing-network","status":"publish","type":"post","link":"https:\/\/sites.temple.edu\/tudsc\/2018\/10\/23\/create-a-network-from-xml-and-visualize-it-a-soccer-player-passing-network\/","title":{"rendered":"A Network Analysis of Soccer Passes: Parsing XML with Python to Visualize Sports in Gephi"},"content":{"rendered":"

By Luling Huang<\/p>\n

A large portion of a soccer match is just players passing without any goals. What are some passing patterns out of the randomness in the game? Which players are the hubs of passing in a team? With the right data-set, and a few easy-to-use Python<\/em> programming scripts, we can build a player passing network to find out.<\/p>\n

In this post, I will demonstrate how to use the Python ‘lxml<\/a>‘ package (with XPath<\/a>) to parse XML<\/a>. I will then show how to use the ‘networkx<\/a>‘ package to build and export network graphs in GEXF<\/a>\u00a0for Gephi<\/a>. The demonstration will use a great data-set with detailed information about a single football\/soccer match made publicly available by the Manchester City Analytics project with Opta Sports. A player passing network will be created.<\/p>\n

You will find this post helpful if your network analysis project’s raw data input is in XML<\/em> (a pretty common document format to store large amount of data), and if your preferred network visualization tool is Gephi<\/a> (a free and powerful software), or if you don’t mind some casual reading on the beautiful game. (Here is a serious reading on the relationship between player passing network and team performances:\u00a0\u00a0Grund’s (2012) study<\/a>.)<\/p>\n

\n

Beautiful Soup vs lxml?<\/strong><\/p>\n

I have used Beautiful Soup<\/a> multiple times for webscraping. However, lxml<\/em> with XPath<\/em> is way faster than Beautiful Soup<\/em> to parse an XML<\/em> document. Martin (2017)<\/a> showed some results from testing the speeds of the two. The takeaway here is that if your XML<\/em> file is small, it doesn’t matter to choose which one to use. But if you have a large file (e.g., 50 MB), you will notice Beautiful Soup<\/em> is significantly slower than lxml.<\/p>\n

\n

What is this thing called GEXF?<\/strong><\/p>\n

When I thought about a network data structure before, an adjacency matrix<\/a> or an edgelist always came up first. But let’s say I have some new information on whether a node (say a person) in my network is a bowler, a surfer, or a nihilist. Then I want to present a graph with bowlers only. How do I add some personal attribute information to my network data?<\/p>\n

Well, we can just prepare multiple data files (say two CSV<\/em> files: one for network and one for node attributes) and make some connection among them. But isn’t a single file able to store all the relevant information a better idea? GEXF<\/em> (Graph Exchange XML Format) can do just that. GEXF<\/em>‘s hierarchical data structure makes it possible to add multiple node and edge attributes on top of a network structure.<\/p>\n

\n

In Python<\/strong>:<\/p>\n

In this section, I will share my Python<\/em> code that parses a data-set in\u00a0XML<\/em>, builds a player passing network, and exports the network to GEXF<\/em>. I will use the one-match data available from\u00a0the Manchester City Analytics project with Opta Sports. A minimum level of programming knowledge is needed to understand this section, but nothing too fancy here.<\/p>\n

Some prerequisites:<\/p>\n

<\/p>\n

\n
import<\/span> networkx<\/span> as<\/span> nx<\/span>\nfrom<\/span> lxml<\/span> import<\/span> etree\nfrom<\/span> itertools<\/span> import<\/span> groupby\nfrom<\/span> operator<\/span> import<\/span> itemgetter\n<\/pre>\n<\/div>\n
The F24 data (see below) has 1,673 events for both teams in a single match. An event can be a pass, a tackle, a shot, or a substitution, etc. The F7 data contains player names,\u00a0 which will be matched to player IDs in F24.<\/p>\n
Import data and parse XML<\/em>:<\/p>\n
<\/p>\n
\n
data =<\/span> etree.<\/span>parse('Bolton_ManCityF24.xml'<\/span>)\nall_events =<\/span> data.<\/span>findall('\/\/Event'<\/span>)\nmeta =<\/span> etree.<\/span>parse('Bolton_ManCityF7.xml'<\/span>)\n<\/pre>\n<\/div>\n
Next, we are looking for all the events that represents Man City’s successful passes (427 in total). One complexity in the data: whereas successful passers are identified, there is no information for receivers of the passes. Here is how I dealt with this problem (a.<\/em>, b<\/em>., and c<\/em>. below):<\/p>\n
a<\/em>. Group all consecutive City’s good passes first;<\/p>\n
<\/p>\n
\n
posl =<\/span> [all_events.<\/span>index(e) for<\/span> e in<\/span> all_events if<\/span> e.<\/span>xpath('@team_id=\"43\" and @type_id=\"1\" and @outcome=\"1\"'<\/span>) is<\/span> True<\/span>]\ngroups =<\/span> []\nfor<\/span> k, g in<\/span> groupby(enumerate<\/span>(posl), lambda<\/span> ix: ix[0<\/span>]-<\/span>ix[1<\/span>]):\n    groups.<\/span>append([i[1<\/span>] for<\/span> i in<\/span> g])\n<\/pre>\n<\/div>\n
b<\/em>. Within each sequence of good passes, a current pass’s sender is treated as the previous pass’s receiver.<\/p>\n
c<\/em>. For the last good pass in a sequence (S<\/em>_n<\/em>), find the next City’s event. The actor (a City player) of this event is assumed to be the receiver of the last good pass in S<\/em>_n<\/em>.<\/p>\n
Now, we are ready to build a directed weighted network.<\/p>\n
Create two functions to make the script briefer first:<\/p>\n
<\/p>\n
\n
def<\/span> get_player_name<\/span>(player_id):\n    p_id =<\/span> 'p'<\/span> +<\/span> player_id\n    player_name =<\/span> meta.<\/span>xpath('\/\/Player[@uID={!r}]\/PersonName\/Last\/text()'<\/span>.<\/span>format(p_id))[0<\/span>]\n    return<\/span> player_name<\/pre>\n<\/div>\n
\n
def<\/span> add_new_actorpair<\/span>(graph,sd_id,rcv_id,sd_name,rcv_name):\n    graph.<\/span>add_edge(sd_id,rcv_id)\n    graph.<\/span>node[sd_id]['label'<\/span>] =<\/span> sd_name\n    graph.<\/span>node[rcv_id]['label'<\/span>] =<\/span> rcv_name\n    graph[sd_id][rcv_id]['weight'<\/span>] =<\/span> 1<\/span>\n    return<\/span> graph\n<\/pre>\n<\/div>\n
Grand finale (finishing b<\/em>. and c<\/em>. above):<\/p>\n
<\/p>\n
\n
G =<\/span> nx.<\/span>DiGraph()\nfor<\/span> g in<\/span> groups:\n    for<\/span> fst, snd in<\/span> zip<\/span>(g,g[1<\/span>:]):\n        sd_id =<\/span> all_events[fst].<\/span>xpath('@player_id'<\/span>)[0<\/span>]\n        rcv_id =<\/span> all_events[snd].<\/span>xpath('@player_id'<\/span>)[0<\/span>]\n        sd_name =<\/span> get_player_name(sd_id)\n        rcv_name =<\/span> get_player_name(rcv_id)\n        if<\/span> G.<\/span>has_edge(sd_id,rcv_id) is<\/span> True<\/span>:\n            G[sd_id][rcv_id]['weight'<\/span>] +=<\/span> 1<\/span>\n        else<\/span>:\n            add_new_actorpair(G,sd_id,rcv_id,sd_name,rcv_name)\n    last_pass_idx =<\/span> g[-<\/span>1<\/span>]\n    end_pass_sdid =<\/span> all_events[g[-<\/span>1<\/span>]].<\/span>xpath('@player_id'<\/span>)[0<\/span>]\n    end_pass_rcvid =<\/span> all_events[g[-<\/span>1<\/span>]].<\/span>xpath('following-sibling::Event[@team_id=\"43\"][1]'<\/span>)[0<\/span>].<\/span>xpath('@player_id'<\/span>)[0<\/span>]\n    end_sd_name =<\/span> get_player_name(end_pass_sdid)\n    end_rcv_name =<\/span> get_player_name(end_pass_rcvid)\n    if<\/span> G.<\/span>has_edge(end_pass_sdid,end_pass_rcvid) is<\/span> True<\/span>:\n        G[end_pass_sdid][end_pass_rcvid]['weight'<\/span>] +=<\/span> 1<\/span>\n    else<\/span>:\n        add_new_actorpair(G,end_pass_sdid,end_pass_rcvid,end_sd_name,end_rcv_name)\n<\/pre>\n<\/div>\n
Write to GEXF<\/em>:<\/p>\n
<\/p>\n
\n
nx.<\/span>write_gexf(G,'whatever_name.gexf'<\/span>,encoding=<\/span>'utf-8'<\/span>,version=<\/span>'1.2draft'<\/span>)\n<\/pre>\n<\/div>\n
The above one liner is one of the reasons to use the networkx<\/em> package to create a GEXF<\/em> file. Convenient! But we also lose some control on getting the GEXF<\/em> file we really want. Note that you can always write a GEXF<\/em> file from scratch (1) by using lxml<\/em> to construct XML<\/em> elements; (2) or by just writing a plain text file without using lxml<\/em>.<\/p>\n
\n
What is the pattern?<\/strong><\/p>\n
\"\"<\/p>\n
In Gephi<\/em>, the GEXF<\/em> file can be visualized as the above graph. Node size is ranked by degree<\/a>; Node color is differentiated by betweenness centrality<\/a>, where darker blue indicates a higher betweenness centrality; Edge width is ranked by weight (i.e., number of successful passes between two players). Tevez is the only substitute included. Player locations are presented based on the formation and position information in the F24 data.<\/p>\n
The graph shows that Man City’s attacking focus was predominantly on its left flank. The betweenness centrality<\/em> shows that the left back Kolarov was the most important link-up player who connected the otherwise disconnected defensive line and attacking line.<\/p>\n
Interestingly, Tevez, who substituted for the starting center forward, Dzeko, and only played 25 minutes, had the third highest betweenness in the squad. At the time Tevez started playing, City was leading 3-2, but Bolton just scored a goal a few minutes ago. One may interpret that Tevez’s task as a sub striker was to draw back a bit more and to connect the defenders\/midfielders and other attacking players more than Dzeko.<\/p>\n","protected":false},"excerpt":{"rendered":"
By Luling Huang<\/p>\n","protected":false},"author":3163,"featured_media":5416,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[303,286,2],"tags":[64,270,360,357,182,359,71,6,361,358],"class_list":["post-5409","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-computer-science","category-cultural-studies","category-grad-students","tag-coding","tag-gephi","tag-gexf","tag-lxml","tag-network-analysis","tag-networkx","tag-python","tag-top-news","tag-visualization","tag-xml"],"_links":{"self":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/5409","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/users\/3163"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/comments?post=5409"}],"version-history":[{"count":0,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/5409\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media\/5416"}],"wp:attachment":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media?parent=5409"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/categories?post=5409"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/tags?post=5409"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}