Consistent Filtering of Videos and Dense Light-Fields Without Optic-Flow, VMV 2019

Abstract

A convenient post-production video processing approach is to apply image filters on a per-frame basis. This allows the flexibility of extending image filters—originally designed for still images—to videos. However, per-image filtering may lead to temporal inconsistencies perceived as unpleasant flickering artifacts, which is also the case for dense light-fields due to angular inconsistencies. In this work, we present a method for consistent filtering of videos and dense light-fields that addresses these problems. Our assumption is that inconsistencies—due to per-image filtering—are represented as noise across the image sequence. We thus perform denoising across the filtered image sequence and combine per-image filtered results with their denoised versions. At this, we use saliency based optimization weights to produce a consistent output while preserving the details simultaneously. To control the degree-of-consistency in the final output, we implemented our approach in an interactive real-time processing framework. Unlike state-of-the-art inconsistency removal techniques, our approach does not rely on optic-flow for enforcing coherence. Comparisons and a qualitative evaluation indicate that our method provides better results over state-of-the-art approaches for certain types of filters and applications.

Citation

Sumit Shekhar, Amir Semmo, Matthias Trapp, Okan Tarhan Tursun, Sebastian Pasewaldt, Karol Myszkowski, Jürgen Döllner
Consistent Filtering of Videos and Dense Light-Fields Without Optic-Flow
International Symposium on Vision, Modeling, and Visualization (VMV), 2019

@inproceedings {SSTTPMD2019,
booktitle = {Vision, Modeling and Visualization},
editor = {Schulz, Hans-Jörg and Teschner, Matthias and Wimmer, Michael},
title = {{Consistent Filtering of Videos and Dense Light-Fields Without Optic-Flow}},
author = {Shekhar, Sumit and Semmo, Amir and Trapp, Matthias and Tursun, Okan Tarhan and Pasewaldt, Sebastian and Myszkowski, Karol and Döllner, Jürgen},
year = {2019},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-098-7},
DOI = {10.2312/vmv.20191326}
}

© 2019 The Authors. This is the author's version of the work. It is posted here for your personal use. Not for redistribution.

Downloads

Acknowledgements

We thank Florian Wagner for his help with generating results. We thank Max Reimann for valuable discussion. We thank the reviewers for their insightful comments. The project was supported by the Fraunhofer and Max Planck cooperation program within the German pact for research and innovation (PFI), the Research School on “Service-Oriented Systems Engineering” of the Hasso Plattner Institute, and the Federal Ministry of Education and Research (BMBF), Germany (mdViPro, 01IS18092).