Undergraduate Project Proposal - Centre for Digital Video Processing, DCU

The Wayback Machine for TV NEWS

Imagine a scenario in which a person has been out of the country for a number of weeks and comes back to find a particular story grabbing the headlines. This project will aim to put each new story in its context by locating previous stories from this news item and linking them together in a timeline. In the CDVP we have the teletext text (all spoken words by the newsreader) of two years of news stories as well as a representative image from each story in each program. By using the teletext text it is possible to construct a chronographic progression through the news story from the early days right up until last nights news by using conventional text IR methods. An element of this project would be to display this timeline (navigable) to a user on both a desktop (lab) machine and on a mobile device such as an iPAQ. Suitable for 2 CA students. Supervisor: Dr.Cathal Gurrin

Image Search Engine with shared recollection

Recently in the CDVP, we have been downloading images from a collection of almost 100,000,000 web pages that we have stored locally. Therefore we have available a large number of images and their source web pages. The goal of this project is to build an image search engine for a small subset of these images (e.g. 10/20,000). However this image engine will not be simply a reworking of Google's image search system. Rather it will take the text search facility of Google, extend that by including low-level feature matching for the images (e.g. colour). This in itself is not innovative, however by taking into account what previous users of the system have selected as relevant for a query, we can leverage this information to improve future searches (shared recollection). The goal of which is to provide better quality retrieval. Suitable for 2 CA students. Supervisor: Dr.Cathal Gurrin

Updatable Search Engine

The aim of this project to twofold. Firstly to replicate some of the functionality of the Google search engine, i.e. text search for web pages incorporating a pagerank measure and image search facilities. The underlying index structure of this search engine must support the addition, updating and removal of documents from the index and support tens of thousands of web pages/images and provide speedy query turnaround time. We will provide the web pages and the images from these pages to support this project. Suitable for 2 CA students. Supervisor: Dr.Cathal Gurrin

Distributed Web Retrieval

Information retrieval has traditionally been concerned with location of relevant documents within a single collection of documents. The proliferation of documents collections particularly on the World Wide Web has led to growing interest in the topic of distributed information retrieval (DIR). In DIR a single query is sent to a (potentially very large) number of separate search engines. The documents returned by these separate engines must then be merged into a single list for presentation to the user. The key research challenge is how to combine these lists optimally to deliver maximum retrieval accuracy to the user. One approach is to attempt to predict the presence of relevant documents in each collection and combine this information with the query-document matching scores from each collection to form a overall score for each document. This project will use the SPIRIT web document collection to explore the application of existing merging techniques to web data. The project will then move on to explore the extension of these methods to web data, for example existing methods do not attempt to use between document links in document scoring for the merging process. In considering how this might be done it may be useful to consider that documents may contain links to documents not contained in this database, but which are contained in other databases. This project would be suitable for 2xCA4 students with interests in learning more about web retrieval methods and contributing to the development of new web retrieval algorithms. Supervisor: Dr. Gareth Jones

Text Mining - Tell Me About...

When using search engines users sometimes only have a vague idea of what they want to know. In order to address their information need they can end up browsing between a large number of documents developing their knowledge, in effect finding out what they need to know before ultimately finding the answer to their question. Existing search engines fail to address the needs of this type of user effectively. After entering their vague query a ranked list of individual documents is returned, the user must then browse among these separate documents in an attempt to find out what they need to know. The aim of this project is to consider the needs of this class of web user and develop search technologies and an interface to enable the user to explore an integrate space of information relating to a vague topic. This would involve features such as automatically linking documents with related content and indicating these links to the user. For example, if the user wants to find out more about a specific topic within a document, they should be directed straight to potentially useful additional documents. This project would involve the use of information retrieval techniques, document link analysis and the exploration of novel web browsing interfaces and metaphors. It is expected that the project would make use of the SPIRIT web collection. This project would be suitable for 2xCA4 students with interests in learning more about web retrieval methods and contributing to the development of new browsing and navigation tools for web data. Supervisor: Dr. Gareth Jones

Video Summarisation

Browsing multimedia data to locate particular information can be an inefficient and time-consuming process. In the simplest solution the user can view the material from the beginning. This can be particularly slow, for example if the answer to a question is 20 minutes into a video it takes 20 minutes to find the answer. In some prototype systems a graphical application is used to indicate likely relevant points in the video from which playback can be commenced. An alternative approach is to show the user a transcription of the video with a sequence of still keyframes from the video, as in the Físchlár system. This allows the user to view many possible relevant sections simultaneously, these can then be selected for playback. The problem with this approach is that locating relevant material to answer your question among the complete transcription and keyframes can also be time consuming, The aim of this project is to develop a video summarisation system for transcription and keyframes. This will involve use of text summarisation techniques, but also potentially information image analysis of the keyframes to decide which parts of the text and keyframes to shown to the user. This could be extended to showing excerpts from the complete video or to develop the tool to expand the summary in relevant sections or link sections of related stories. This project would be suitable for 2xCA4 students with interests in learning more summarization and retrieval methods and multi-model presentation of information. Supervisor: Dr. Gareth Jones

Automated Alignment and Segmentation of Audio Transcriptions for the Físchlár System

Each day the Físchlár-News system records 30 minutes of news from RTE. Físchlár records the audio and video signals along with the teletext subtitles. The subtitles are used to index the content of the news broadcasts for retrieval. However, the text is not well aligned with the audio signal since it is often broadcast a variable number of seconds after the corresponding audio. This means that the text cannot be used for intelligent processing of the audio-video signal such as automatically segmenting it into individual stories. The first objective of this project would be to use speech recognition techniques to align the text with the words spoken into the audio soundtrack. The project may then move on implement story segmentation algorithms based on the text transcriptions and integrate these with existing work on Físchlár-News using information from the video data stream for segmentation. Story segmentation requires identification of the points in the audio-visual stream where one story ends and the next begins. This is generally easy for humans to determine, but automatic segmentation is difficult and research will be required within the project to choose and extend existing algorithms to maximise segmentation accuracy. This project would be suitable for 2xCA4 students interested in multimedia information management, it would be helpful if at least one student is taking the Speech Processing module. The project would provide opportunities to apply methods from Speech Processing and Multimedia Information Retrieval to real world data and their integration within a multimedia digital library. Supervisor: Dr. Gareth Jones

Automatically Linking Teaching and Learning Resources

It is now common practice for lecturers to make electronic versions of their lecture slides available on webpages to support module teaching. This only represents a starting point for the potential of electronic information management tools to support teaching and learning. For example, an existing project developed a novel application to automatically link lecture slides to audio recordings of the relevant section of the associated lecture. Using this application students can play back the original lecture material while viewing the slide. This can be very useful since lectures often contain descriptions and examples not available in textbooks. This approach is potentially very powerful, for example allowing automated search for material within lectures, rather than having to listen to a complete audio recording to find the relevant part.

The aim of this project would be to explore methods by which this approach could be extended to further linking of study resources and information. Some examples of possible areas of investigation could be the automated generation of search queries from lectures slides and audio recordings, these might be used for example to find relevant sections of online textbooks or locate web pages containing related material, another could be the automated linking of materials between lectures, so that a student could simply follows links to related material. The system could be personalised for individual students by linking slides and the presentation to notes made by the student either during a lecture or in private student. Such notes would provide further information for automated information seeking.

The overall aim is to develop a learning environment which integrates the available information resources to improve the efficiency with which students navigate their way through related information searching and linking, and to bring to their attention resources that they might otherwise not be able to locate. This project would be suitable for 2xCA4 students with interests in learning more about existing information retrieval methods and their use in a novel application. Supervisor: Dr. Gareth Jones

Context-Aware Information Management

Conventional search engines are designed for use on desktop computers and offer little by way of personalisation for individual users. Thus any user entering a search request will receive the same ranked list of documents. Context-aware information management aims to improve the reliability and usefulness of information delivered to a user by taking into account the context in which the search is taking place. Context here refers both to the user operating the system and environmental metadata, such as location or time. We are particularly interested in retrieval in mobile computing environments where users require reliable data and are unlikely to wish to engage in extensive document browsing. For example, a user may be in a city centre and want to know the location of the nearest restaurant that serves their favourite dish, or the location of the nearest bus stop to get them home, or perhaps they want to know the latest sports results. Context-aware retrieval combines database technologies, information retrieval techniques and issues from human-computer interaction and user interface design.

Two projects are suggested for this year as follows:

Agents for Context-Aware Information Delivery

Information can be delivered to a user using different media, e.g. as text or graphics or as the output of a speech synthesiser. The most appropriate form for the information delivery for mobile applications will often depend on the activities of the user. For example, a user who is walking or driving is best served by audio delivery, while a user who is stationary and able to view a screen can often obtain information more efficiently if it is delivered visually. The bandwidth of visual information delivery is much higher than audio delivery, thus it will often be necessary to automatically summarise text documents before they can be delviered via audio. The objective of this project would be to develop a prototype information delivery management agent. The agent would monitor a user's activity based on simulated context input and then decide on how and when to deliver information. The project would require formal analysis of user context and how this can be formally described for an agent, automated transformation of information into the appropriate form, e.g. showing directions on a graphical map or delivering them via speech synthesis, summarisation of text information for audio delivery. The system could be extended to include automated learning of user responses to agent behaviour and modification of the agent. Techniques cover would include agent technology, context-aware technology, automated summarisation, speech synthesis, information visualiation. This project would be suitable for 2xCA4 students with interests in learning about machine learning and agent technology, and their application in data management for mobile context-aware envrionments. Supervisor: Dr. Gareth Jones

Simulating Context-Aware Environments

Context-aware computing systems are designed to behave differently depending user context, for example different weather conditions, user location, who the user is with. When testing these systems it is often difficult to examine all anticipated since many context are not available on demand; we can't make it rain or change temperature for example to test how system responds. One approach to this problem is to build virtual worlds where the user is in control of the context. The output of environment sensors can be varied exactly as they would in the real-world and the behaviour of the context-aware system monitored. In general terms the context-aware application does not need to know that it is dealinng with simulated as opposed to real data. One approach to simulating environments for context-aware testing is the use of compuer games development engines which enable programmers to develop new worlds and control the environment. The aim of this project would be to use an existing games engine to develop a simulated context-aware environment and integrate this with a basic context-aware information management tool. This project would be suitable for 2xCA4 students with interests in learning about context-aware information management and virtual world simulation. Supervisor: Dr. Gareth Jones

Video Shot Cut Detection (2 people)

A video shot is a single piece of continuous video shot with a single camera. A video shot cut is an abrupt or gradual change from one video shot to another. Detecting video shot cuts is a key starting point for the semantic analysis of video that is researched within the Centre for Digital Video Processing (CDVP). We originally detected cuts by looking for abrupt changes in the colour histogram of video frames, but more recently have achieved much faster results by analysing the content of the encoded MPEG-1 video stream itself. We now have access to a large database of video material with the shot cuts determined by a number of research groups worldwide. We want to do three things at this stage: (i) compare the results of our approach with the best in the rest of the world; (ii) make our approach more robust to encoder parameters and encoder types, which can alter the MPEG-1 video stream significantly and (iii) extend our approach to deal with gradual transitions. The work will be based around an MPEG-1 video decoder written in C that has been developed within the Centre for Digital Video Processing (CDVP). Supervisor: Prof. Alan Smeaton and Dr. Noel Murphy

Robust Football Tracking in Video

In televised soccer matches, the football is usually the centre of attention. Various types of personalisation, summarisation and packaging of televised soccer require that the football be located, and where possible, tracked through the video frames. We have already done preliminary work on football location and tracking and in this project we want to make this work more generally applicable and more robust. The existing work is based on an MPEG-1 video decoder written in C that has been developed within the Centre for Digital Video Processing (CDVP). The existing approach is based on colour and region analysis of the content of video frames in a context-specific manner and there is scope for extending this approach to frequency information available in the encoded MPEG-1 video stream and to motion analysis over time. At the end of this project you will have learned loads about MPEG-1 and digital video analysis and will have sharpened your software development skills. Supervisor: Prof. Alan Smeaton and Dr. Noel Murphy

MPEG-4 Object-based Encoder (2 people)

MPEG-4 is the last in a series of Multimedia content encoders that were standardised during the 1990s. It is important for research in the Centre for Digital Video Processing (CDVP) because one version of it supports the coding of objects within a video frame and we do research on visual object segmentation and tracking. The versions of MPEG-4 encoders and players commercially available do not support this Video Object mode, so we are in the process of developing our own version. A freely available MPEG-4 encoder FAME encoder can do Video Objects, but not for all types of encoded frame. In this project we will focus on extending the FAME encoder to satisfy our requirements. This is a project in software development requiring good skills at C and/or C++. Supervisor: Prof. Alan Smeaton and Dr. Noel Murphy

Building a Linked Graph of Físchlár-News-Stories

The F�schl�r-News-Stories system records RTE main evening news daily and automatically segments the broadcast into individual news stories. FNS then allows users to browse news stories and as each story is viewed, the text closed captions associated with each story are used to find the "closest" or most similar stories, which are probably related in some way. Thus a story about "bank robber caught in Templemore sent for trial" would be close to a story about "bank robber caught in Templemore gets 10 years". However, one of the drawbacks with this approach is that from each news story we can jump to its closest or most similar story, but there is no global, archive-wide, computation of story-story similarities. This project will develop a system to analyse the archive each day (when new stories have been added) and find groups of stories related to last night's new stories. If the final work is effective we will consider incorporating it into the Fischlar-News-Stories system. Project suitable for 2x CA4 students. Supervisor: Prof. Alan Smeaton

Video Mosaicing

The technique of image mosaicing is now well-known and easily evailabe. This involves taking a sample image, dividing it into a grid and for each element of the grid, substituting pixels from the original image by a complete, separate image taken from an image database. See http://mazaika.tripod.com/mazaika.html for an example. The aim of this project is to build a mosaic in the same way, but instead of combining a set of still images to replace a candidate source image in a mosaic pattern, the task is to replace a short clip of video with a series of other video clips arranged and played as a kind of "video wall" so as to create the illusion of the original video clip, but made up of sub-image clips. This will require taking a source clip of say 10 seconds, dividing it into a grid of 4x5 and replacing each of the 20 elements of the grid by another clip of video. To make the project managable, a source database of video clips - all of the same duration as the original, will each be indexed by their global and regional colour arranged over time, sampled say every 0.5 seconds. Global and regional colour from each element of the 4x5 grid will be used as a "query" to the database of video clips to find the most similar, and a new video will be composited in this way. The project requires computer science insights in order to develop, implement, and test a database similarity measure which works, it requires good software engineering to be able to build the system, and requires extensive disk space, and memory and processing requirements, in order to be able to compute. Most of the basic tools to do this are available in the Centre for Digital Video Processing so this project is about putting them together in a novel, creative and visually attractive way. Suitable for 2x CA4 students with very good computing skills and no fear of the unknown ! Supervisor: Prof. Alan Smeaton

Video Copy Identification

This project requires the creation of video "fingerprints" for a set of TV commercials. Fingerprints will be based on automatically detected audio and video features such as colour, texture, motion, pitch, etc. For each commercial in a "database" we will generate a fingerprint and then match these fingerprints against a live TV station transmssion, possibly 24x7. This will allow us to determine which commercials are broadcast, and when, and how often. The challenges in this project will be to determine which features should be included in the fingerprint, and in what combinations, and how fingerprints should be matched. The project should allow the inclusion of new adverts in the "to be monitored" database, automatic feature generation and matching for those new adverts. The project will use much of the feature extraction work from the Centre for Digital Video Processing, as well as the broadcast TV capture and storage from the F�schl�r system. The project is suitable for two students with an interest in image/video processing, and good software engineering skills. There are several published papers in this area to which we can provide reference. Supervisor: Prof. Alan Smeaton

ER1 Robot Navigation Around the Postgrad Lab in Computing

The ER1 is a robot http://www.evolution.com/er1/ which is effectively a laptop with wheels which can move and turn, it has a video camera, microphone, speaker and connection to the wireless network. it is programmer through an interface which allows you to write a series of IF X THEN DO Y rules. So, if it receives email, hears a noise, sees a particular colour or recognises a particular object, then it can move, turn, take a picture or video, send an email, utter a sound, or a combination of these. It is potentially very powerful, but limited by the programming interface. This project is to take my ERI and develop a programming environment to allow development of small and useful applications. The robot's world will be limited to the postgrad area of the School of Computing (outside my office) and the useful applications could be to see if Steve Blott is in his office (go down corridor, stop outside his door, take picture and email it to me), collect my printouts from the printer at the other end of the corridor, etc., and many others. Supervisor: Prof. Alan Smeaton

Build and deploy a CCTV Capture and Indexing Tool for CA Labs

There are several AXIS cameras http://www.axis.com/products/cam_210/index.htm deployed in the CA labs. These are IP-based meaning you canwrite an application on a PC to capture, store, manipulate the images from these. The aim of his project is to build and deploy a system for capturing and storing images from 1 or more cameras - that's the easy part ! The hard part is to build and deploy a system to query and browse the database of CCTV images. Querying can be by date/time, by camera (all easy so far), or by "event" (the hard part). As images are captured, they are analysed by comparing to previous images to detect things that happen. for example, most of the time nothing happens, then somebody walks into a lab sits down at a machine and stays relatively still for 10 minutes at a PC before packing up his/her bag, then leaving the lab. the "events" are things which are visually very different from what has preceeded them. So the action of walking across in front of the camera is visually very different from haviong an empty room. Sitting down at a PC reading email will generate images which are very similar to each other, but then standing up, packing bag and leaving, will be "events". So, capture CCTV images, store them, analyse them for "events" and then allow a user to query based on time and on event occurrence. Supervisor: Prof. Alan Smeaton

My Average Face

Using video footage of a single person, this project will use face recognition software to track the person's face throughout a video clip and will extract facial poses on a periodic basis - say every second. This will build up a "database" of faces for a given person. Using a fairly established technique, each face can be analysed into an Eigenface - a virtual representation of the face used in many face matching algorithms. Normally face-to-Eigenface transformation is used so that Eigenfaces can be matched against each other, but the process is reversible so it is possible to generate an (approximate) face from an Eigenface. This project is to take the set of faces extracted from a video clip, generate an Eigenface for each, combine these into an averaged Eigenface and then reverse the process to generate a virtual face, from an averaged Eigenface. We don't know what this generated Eigenface will look like but it will be the average face for an individual, taken from a video clip. This might have applications in security for face matching. Supervisor: Prof. Alan Smeaton

DCU's Who is who based on Faces

Using the same technology as in "My Average Face", generate an Eigenface for each image in the DCU student picture database, cluster these to see "who is like who" among the student population. Supervisor: Prof. Alan Smeaton

Handwritten Word Matching in the ISOS Project

The ISOS project has built up an archive of several thousand ultra high resolution images of Old Irish manuscripts, some of which are over 800 years old (see www.isos.dias.ie/). This project involves image analysis to segment handwritten words and letters from one given manuscript, and to use some hand-annotation of the segmented words to characterise the writing styles of individual scribes (people who did the hand writing). The idea is to analyse how different letters are written in order to determine if the same scribe is responsible for writing entire manuscripts or if several different people with slightly different writing styles, are responsible. The project will use a patented algorithm for shape matching which we have developed but have not used it for this application before. The project involves software engineering, image analysis and user interface aspects and is suitable for two students.
Related project: www.cl.cam.ac.uk/users/jgd1000/scribes.html
Supervisor: Prof. Alan Smeaton

Visual Detection of Chemical Sensing Events

The objective of this project is to develop a suite of image/video analysis tools for video captured from a camera trained on a test chamber of chemical sensors. Test data will be provided of a colourless gas being introduced to the test chamber. When the sensors come into contact with the reactant they change colour. The approaches to be developed should be able to detect when and where this change has occurred. This output should be represented in a user fiendly GUI that illustrates visually what is taking place in the chamber. Time permitting, the project will be extended so that this demonstration system works in real-time.
Skills: C/C++/Java Supervisor: Dr. Noel O'Connor

3-D Depth Analysis using Colour Segmentation

Finding pixel correspondences in natural stereo image pairs plays an important role in a large number of applications, e.g. robot navigation, augmented reality, and telecommunications. The aim of a stereo correspondence search is to match points in two images such that the two corresponding pixels are projections of the same point in 3-D space. However, due to problems such as: projective distortions, unstructured texture regions, and occluded areas across images, individual pixel intensities/colours do not provide sufficient and distinct information for a unique correspondence. This project will investigate how the use of colour-based object segmentation within the images could improve the quality of the point correspondence by restricting the search area to within the object boundaries. The tasks to be completed are: (i) implementation of a defined stereo correspondence algorithm, (ii) develop and implement enhancement of the correspondence algorithm based on object segmentation information (segmentation tool will be provided), (iii) detailed comparison of the two approaches.
Skills: C/C++ Supervisor: Dr. Noel O'Connor

Face Detection and Matching in Movies

The detection of faces in images or video clips is an essential tool for many applications in fields such as security or advanced computer/human interaction. Furthermore, it is a useful pre-processing step for analysis tasks such as event detection and major cast identification in broadcast content. Although it is a complex problem there are a number of existing solutions available. This project will involve evaluating some of these solutions in the context of a larger scale event detection system currently under development, in order to assist in the task of dialogue detection in movies.
Skills: C/C++ Supervisor: Dr. Noel O'Connor

Visual Analysis of Motorway Scenes for Traffic Monitoring [suitable for M.Eng]

There is an ongoing research project within the Centre for Digital Video Processing (CDVP) - see: www.cdvp.dcu.ie - that attempts to analyse traffic flow based on audio data captured using low-cost microphones deployed in strategic locations. When test audio data was generated, accompanying video footage was also captured. The ultimate goal of the project is to use a combination of audio and visual information in order to provide robust traffic data. The objective of this project is to investigate the different forms of visual analysis possible with the available test data. This will involve investigation of the feasibility of tasks such as:

robust background change detection given a known background (ideally this should be adaptable to varying lighting conditions, or rain)
subsequent (region-based and/or snake) processing to separate individual cars
tracking of detected cars over subsequent video frames

Technologies: The student will need to be proficient in C and/or C++. Novice C programmers are likely to find this project too demanding. The EE554 module is strongly preferred. Supervisor: Dr. Noel O'Connor

Linking Stories using News Icons in TV News Broadcasts [suitable for M.Eng]

An approach for automatically segmenting individual stories in TV news broadcasts has been developed within the Centre for Digital Video Processing (CDVP) - see: www.cdvp.dcu.ie. The approach is based on combining the results of audio-visual analysis using a Support Vector Machine (SVM). The output of this system is a set of time stamps that indicate when an individual news story starts and ends and a set of extracted representative images for the news story. One of these images is chosen to be the news anchor person and typically it contains a background graphic/logo, that we term a news icon, that gives some indication of the content of the news story. The aim of this project is to develop an approach to extracting and characterising this graphic/logo in order that it can be matched against other extracted news icons. If two icons are matched with a high degree of confidence, then this indicates that these news stories are related and should be linked in the news retrieval system.

Design and Implementation of a Content-based Information Retrieval System [suitable for M.Eng]

There is an ongoing research effort within the Centre for Digital Video Processing (CDVP) - see www.cdvp.dcu.ie - to build a content-based information retrieval (CBIR) system that allows efficient indexing of images, audio and video clips as well as efficient flexible queries on the underlying database. This system will be used in an experiment designed to bench-mark different information retrieval systems during Aug 2004. The aim of this project to design and build essential indexing componenst of such a system. This will require the student to assist in building and populating the media database as well as integrating existing state of the art image/video indexing tools. The image indexing tools targeted will focus on well known techniques for face detection and/or camera motion analysis.

Technologies: The student will need to be proficient in C, C++, Java, web server technologies and SQL (or similar). The EE554 module is strongly preferred. Supervisor: Dr. Noel O'Connor

EM-based image region segmentation for retrieval [suitable for M.Eng]

Automatically dividing an image into a number of arbitrarily-shaped sub-regions is a key tool in image and video analysis. This is normally referred to as the process of region-based image segmentation and many different techniques exist. One popular technique models each image region as a multivariate Gaussian probability density function (pdf) and the entire image feature space as a Gaussian mixture model. By estimating the parameters of this mixture a segmentation of the image can be obtained. Typically the Expectation Maximisation (EM) algorithm is used to estimate the parameters of the mixture. In addition to being useful as the basis for a segmentation process, the resultant estimated pdfs are also a description of the image regions themselves and their relationships. As such, it is proposed that these pdfs could be used in order to index images for subsequent retrieval from a database. This project will investigate using an existing approach to EM-based region segmentation in an image retrieval framework using a small database of test images.

This is an extremely challenging project requiring strong mathematical ability and very good programming skills (C, C++). Novice programmers and those with limited mathematical aptitude are likely to find this project too demanding. The EE554 module is strongly preferred. Supervisor: Dr. Noel O'Connor