Tinghuai Wang

I am a research manager in Huawei Helsinki Research Center, where I have been leading the research in machine learning and computer vision. My recent research interests focus on multimodal foundation models and reinforcement learning.

I received my PhD from the University of Surrey and MSc from KTH and RWTH Aachen in machine learning and computer vision. I have more than 10 years experience in research and innovation roles at various industry research labs (Huawei Helsinki Research Center, Nokia, HP Labs, Sony Research Labs). I have published 40 peer-reviewed international journals and conference publications, including top venues ICML, AAAI, IJCAI, ICCV, BMVC, TVCG, TGRS, TMM, CVIU. My publications have won two IEEE best paper awards (MESH 2009, ICME 2015). I also (co-)invented 84 granted/pending patents filed in US and EU, with more than two dozens of technology transfers to influential products. I have been serving as programme committee member for top AI venues such as AAAI, IJCAI, and area chair of ICIP 2018, CVMP 2013-2016 and SIGRAPH/NPAR 2012-2016. I was awarded as distinguished PC member of IJCAI 2018.

Email  /  Google Scholar  /  Google Patents /  Github

I'm generally interested in computer vision/graphics, machine learning, and reinforcement learning. Representative papers and patents are listed as follows. Full list of publications can be found in my Google Scholar page.

We propose a new Gaussian processes (GPs) based method for learning probabilistic subgoal representations in Hierarchical Reinforcement Learning (HRL).

We propose a novel adversarially guided subgoal generation framework for goal-conditioned HRL to mitigate the issue of non-stationarity in off-policy training.

We propose two novel factor graph optimization (FGO)-based models to improve smartphone positioning accuracy by integrating pedestrian dead reckoning (PDR) with GNSS data. The first model combines position data from both PDR and GNSS, while the second tackles heading angle errors and smartphone misalignment by integrating PDR step length with GNSS.

We investigate the tradeoffs between signal-to-noise ratio (SNR), spatial resolution, and spectral resolution in hyperspectral imaging for crop identification, addressing gaps in existing research. Results show that overall accuracy decreases with lower SNR and coarser spectral resolution but improves with lower spatial resolution. This study bridges the gap between hyperspectral sensor design and practical crop identification.

We design an architecture to encode the multiple spectral contextual information in the form of spectral pyramid of multiple embedding spaces. In each spectral embedding space, we propose graph attention mechanism to explicitly perform interpretable reasoning in the spatial domain based on the connection in spectral feature space.

We propose a novel pruning loss to explicitly enforces the optimizer to focus on promising candidate filters while suppressing contributions of less relevant ones. In the meanwhile, we further propose to enforce the diversities between filters and this diversity-based regularization term improves the trade-off between model sizes and accuracies. Nokia proposal to ISO/IEC DIS 15938-17 Information technology — Multimedia content description interface — Part 17: Compression of neural networks for multimedia content description and analysis

We propose a categorical attention mechanism to propagate consistent category-oriented information across multi-granularity contextual interpretations to close the semantic gap residing in CNN feature hierarchy; a cross-granularity contour enhancement mechanism is also proposed to propagate rich boundary cues from early layers to deep layers. These novel mechanisms boost the essentials in segmentation, i.e., region-wise semantic coherence and accurate object contour localization.

We propose a novel and efficient non-parametric affinity model to achieve efficient instance segmentation on mobile devices. We also present a portrait image dataset with instance level annotations dedicated to evaluating portrait instance segmentation algorithms.

Method for Nokia 8 Portrait Segmentation.

We address semantic video object segmentation via a novel cross-granularity hierarchical graphical model to integrate tracklet and object proposal reasoning with superpixel labeling.

We present a set of images for helping NPR practitioners evaluate their image-based portrait stylisation algorithms.

Video object detection method for Nokia OZO camera.

We present a novel algorithm that detects recurring primary object and learns cohort object proposals over space-time in video. Our core contribution is a graph transduction process that exploits both appearance cues learned from rudimentary detections of object-like regions, and the intrinsic structures within video data.

This paper presents a method to detect and segment recurring object from multi-view images. By harnessing a top-down explicit notion of object, our method overcomes the limitations of previous bottom-up methods that often mis-segment an object and de- livers high quality segmentation.

Method for Nokia OZO camera positioning.

Method for Nokia OZO VR content rendering.

Method for Nokia OZO VR content view synthesis.

We combine geodesic distance information with the flexibility of level set methods in energy minimization, leveraging the complementary strengths of each to promote accurate boundary placement and strong region connectivity while requiring less user interaction.

We present a a robust and efficient approach for segmenting images with less and intuitive user interaction, particularly targeted for mobile touch screen devices.

We present TouchCut; a robust and efficient algorithm for segmenting image and video sequences with minimal user interaction i.e., only a single finger touch to identify the object of interest in the image or first frame of video. This approach to visual object cut-out provides a practical solution for image and video segmentation on compact touch screen devices, facilitating spatially localized media manipulation.

We present a novel multi-view video matting method suitable for incorporation into a 4DPC pipeline. The key contributions of this method are 1) the propagation of appearance and spatial information across views using superpixel matching and a novel MRF that solves the mattes simultaneously across all views, 2) the temporal propagation of appearance and spatial information forward in time.

Interactive segmentation method for Nokia Lumia Phone.

We survey the field of non-photorealistic rendering (NPR), focusing on techniques for transforming 2D input (images and video) into artistically stylized renderings.

We present the first machine learning model which is capable of learning artistic style for portraits by analyzing training data from a human artist. Given a training pair — a source image and painting of that image — a non-parametric model of style is learned by observing the geometry and tone of brush strokes local to image features.

We describe a new system for searching video databases using free-hand sketched queries. Our query sketches depict both object appearance and motion, and are annotated with keywords that indicate the semantic category of each object. We parse space-time volumes from video to form graph representation, which we match to sketches under a Markov Random Field (MRF) optimization.

We present a novel algorithm for video segmentation and our core contribution is a multi-frame probabilistic motion diffusion model to incorporate labelling priors from previous frames to influence the segmentation in new frame.

We present a system to breathe life into home digital media collections, drawing upon artistic stylization to create a ‘‘Digital Ambient Display’’ that automatically selects, stylizes and transitions between digital contents in a semantically meaningful sequence.

We present a sketch based image retrieval system built on a bag of regions which encodes the complete information of salient shapes at various level of details in the form of enclosed contours of regions presenting a coherent visual appearance.

We interpret the sequence of editting operations applied to footage as a ‘program’ comprising cutting, panning and zooming constructs. We develop a Genetic Programming (GP) framework for representing and evolving such programs. Under this framework, the search for an aesthetically pleasing video edit becomes a search for the optimal genetic program. Our aesthetic criterion promotes the inclusion of people in shots, whilst penalising rapid shot changes or shot changes in the presence of camera motion.