

{"id":506,"date":"2015-06-18T09:21:07","date_gmt":"2015-06-18T13:21:07","guid":{"rendered":"https:\/\/sites.temple.edu\/tudsc\/?p=506"},"modified":"2023-01-24T12:50:16","modified_gmt":"2023-01-24T16:50:16","slug":"digital-video-analysis-and-retrieval-part-ii-introduction-of-digital-video-analysis-tools","status":"publish","type":"post","link":"https:\/\/sites.temple.edu\/tudsc\/2015\/06\/18\/digital-video-analysis-and-retrieval-part-ii-introduction-of-digital-video-analysis-tools\/","title":{"rendered":"Digital Video Analysis and Retrieval (PART II) -Introduction of Digital Video Analysis Tools"},"content":{"rendered":"<p>By Ping Feng<\/p>\n<p><!--more--><\/p>\n<p>In Part I, I discussed the\u00a0<a href=\"http:\/\/www.cinemetrics.lv\" target=\"_blank\" rel=\"noopener noreferrer\">Cinemetrics<\/a>\u00a0project, a prominent database with film editing statistics. Though it is still largely based on manual collaboration, its decent amount and richness of data have been utilized by many scholars to either conduct further cinematic analysis or theorize their methodologies. For example, Mike Baxter has worked on papers such as\u00a0<a href=\"http:\/\/www.cinemetrics.lv\/dev\/Evolution_paper_for_Cinemetrics.pdf\"><em>Evolution in Hollywood editing patterns?<\/em><\/a> and\u00a0<a href=\"http:\/\/www.cinemetrics.lv\/dev\/baxter_q3.pdf\"><em>Cutting patterns in D.W.Griffith\u2019s silent feature films<\/em> <\/a>by using descriptive data and R code (Baxter, 2012). More of his theoretical principle and data analysis methodology can be found\u00a0<a href=\"http:\/\/cinemetrics.lv\/links.php\" target=\"_blank\" rel=\"noopener noreferrer\">here<\/a>\u00a0.<\/p>\n<p>Another example is Nick Rederfern&#8217;s work. He maintains a\u00a0blog &#8211; <a href=\"http:\/\/nickredfern.wordpress.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Research Into Film<\/a>\u00a0&#8211;\u00a0where some of his film studies are presented \u00a0by using quantitative approach theorized by Cinemetrics.<\/p>\n<p><a href=\"http:\/\/www.shotlogger.org\" target=\"_blank\" rel=\"noopener noreferrer\">Shot Logger<\/a> is another open source database assembled the Cinemetrics but at a\u00a0smaller scale. It provides visual style analysis and a database with editing statistics for 928 instances of 246 films and TV programs and a gallery of 282,157 frames captured (Jeremy Butler, data updated till April 8, 2015). The drawback for this site is that the data, such as Average Short Length, Median Short Length, is mainly presented numerically, not graphically.<\/p>\n<p>However, in addition to the sheer volume of the digital video available in competing users\u2019 attention, the unique nature of digital video with both temporal and spatial elements also requires a more automatic way to collect, retrieve, and analyze the data. For instance, in order to collect the number of cuts or the appearance of certain face in a one-hour long video for analysis\u00a0manually,\u00a0it will be both time-consuming and expensive to transit and stream the giant size of video for at least one hour or so. Therefore, the need for efficient analyzing and retrieving video images has called for an automatic way of metadata extraction and indexing.<\/p>\n<p>One\u00a0project, completed by\u00a0undergraduate Fredric Brodbeck, is also called &#8220;Cinemetrics&#8221;, but with a\u00a0different concept. He developed an precise\u00a0tool to &#8220;automatically&#8221; extract, process, and visualize movie data. A video demo of his project can\u00a0be found\u00a0<a href=\"http:\/\/cinemetrics.fredericbrodbeck.de\">here<\/a>, and his code is available <a href=\"https:\/\/github.com\/freder\/cinemetrics\/\">here<\/a>.<\/p>\n<p>Some organizations have already developed tools or algorithms to automatically extract, analyze, and visualize the metadata of digital video editing statistics or visual images. For example,<\/p>\n<ul>\n<li><a href=\"http:\/\/ldt.iri.centrepompidou.fr\" target=\"_blank\" rel=\"noopener noreferrer\">Lignes de Temps<\/a>\u00a0features video-flow annotation and cut-detection tool, but it is all in French, thus less popular<\/li>\n<li>Edit 2000\u00a0is an excellent tool and platform to\u00a0upload edited video files and obtain editing decision lists automatically with both numeric summary and visual display.<a href=\"https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM.png\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-517\" src=\"https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM.png\" alt=\"Screen Shot 2015-06-09 at 9.58.25 AM\" width=\"673\" height=\"335\" srcset=\"https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM.png 810w, https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM-300x149.png 300w, https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM-700x348.png 700w, https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM-232x115.png 232w, https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM-464x231.png 464w, https:\/\/sites.temple.edu\/tudsc\/files\/2015\/06\/Screen-Shot-2015-06-09-at-9.58.25-AM-624x310.png 624w\" sizes=\"auto, (max-width: 673px) 100vw, 673px\" \/><\/a><\/li>\n<li><a href=\"http:\/\/lab.softwarestudies.com\" target=\"_blank\" rel=\"noopener noreferrer\">Software Studies Initiative<\/a>\u00a0is a software-based platform focusing on cultural analytics and discovering\u00a0cultural patterns by analyzing the big data of digital video and visual images. Though it has been more focused on visual image analysis, some of the prominent projects include: <a href=\"http:\/\/lab.softwarestudies.com\/2015\/05\/visualizing-instagram-selfies-cities.html?utm_source=feedburner&amp;utm_medium=feed&amp;utm_campaign=Feed%3A+SoftwareStudies+%28Software+Studies+Initiative%29\" target=\"_blank\" rel=\"noopener noreferrer\">Visualize Instagram: Selfies, Cities and Protests<\/a>, \u00a0<a href=\"http:\/\/lab.softwarestudies.com\/2015\/03\/on-broadway-new-interactive-urban-data.html?utm_source=feedburner&amp;utm_medium=feed&amp;utm_campaign=Feed%3A+SoftwareStudies+%28Software+Studies+Initiative%29\" target=\"_blank\" rel=\"noopener noreferrer\">On Broadway-a New Interactive Urban Data Visualization from Selfiecity<\/a>, \u00a0<a href=\"http:\/\/lab.softwarestudies.com\/2010\/12\/2008-us-presidential-campaign-ads.html\" target=\"_blank\" rel=\"noopener noreferrer\">Political Video Ads<\/a>, <a href=\"http:\/\/lab.softwarestudies.com\/2008\/09\/filmhistoryviz-1500-feature-films.html\" target=\"_blank\" rel=\"noopener noreferrer\">Cinema Histories-Patterns across 1100 Feature Film, 1900-2008<\/a>.<\/li>\n<\/ul>\n<p>Further, there are other exciting\u00a0projects that work to extract content related metadata automatically:<\/p>\n<ul>\n<li><a href=\"http:\/\/www.dlib.org\/dlib\/december02\/marchionini\/12marchionini.html\" target=\"_blank\" rel=\"noopener noreferrer\">Open Video Digital Library<\/a> by Marchionini and Geisler (2002), University of North Carolina at Chapel Hill. In this system, key frames were first\u00a0extracted using MERIT software (University of Maryland).<\/li>\n<\/ul>\n<ul>\n<li>Informedia Digital Video Library by Carnegie Mellon University has integrated speech, natural language process technologies to automatically recognize speech in video soundtrack and transcribe it into text information in alignment with linear video segments and index to create &#8220;video paragraphs&#8221; or &#8220;video skimming&#8221; for efficient retrieval.<\/li>\n<\/ul>\n<ul>\n<li><a href=\"http:\/\/research.ibm.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">IBM\u2019s Cue Video<\/a> summarizes a video and extracts key frames. It acquires spoken documents from video via speech recognition.<\/li>\n<\/ul>\n<ul>\n<li>IBM Research TRECVID-2004 Video Retrieval System is content-based automatic retrieval project focusing on four tasks of shot boundary detection, high-level feature detection, story segmentation, and search by using IBM Cue Video the team has previously developed.<\/li>\n<\/ul>\n<ul>\n<li><a href=\"http:\/\/www.ee.columbia.edu\/ln\/dvmm\/\">Digital Video Multimedia Group<\/a> at Columbia has been engaged in multi-media content analysis, data extraction from images, videos with the efforts to build large-scale search engines, machine leaning and recognition system for automatic index and retrieval of the data.<\/li>\n<\/ul>\n<ul>\n<li>The Fischlar System\u00a0at Dublin City University(1999) is an digital library with several hundred hours of video content, operating via TREC Video Retrieval track.\u00a0It can\u00a0detect and removes the advertisement from the video shots and\u00a0analyzes the remaining content by spoken dialogue indexing, speech\/music, discrimination, face detection, anchorperson detection, shot clustering, and shot length cue, all of which are implemented based on Support Vector Machine algorithm. Finally, it applies the story-segment program to combine several shots into story segment and save the result in the database.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>By Ping Feng<\/p>\n","protected":false},"author":7448,"featured_media":513,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[297,2],"tags":[27,33,24,25,29],"class_list":["post-506","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-film-studies","category-grad-students","tag-automatic-video-data-extraction","tag-film-editing","tag-video-analysis","tag-video-archive","tag-visual-analysis"],"_links":{"self":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/506","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/users\/7448"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/comments?post=506"}],"version-history":[{"count":0,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/posts\/506\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media\/513"}],"wp:attachment":[{"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/media?parent=506"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/categories?post=506"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.temple.edu\/tudsc\/wp-json\/wp\/v2\/tags?post=506"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}