Dr. Connelly Barnes have led the research on patch-based algorithms over the years. Some of his most famous patch-based research are:
- PatchMatch (SIGGRAPH 2009, EECV 2010, CAF)
- Image Melding (SIGGRAPH 2012)
- Patch-based HDR video (SIGGRAPH Asia 2013)
- Synthesis of Complex (SIGGRAPH 2015)
- PatchTable (SIGGRAPH 2015)
Artistic Media
- Digital Bas-Relief (SIGGRAPH 2007)
- RealBrush (SIGGRAPH 2013)
- DecoBrush (SIGGRAPH 2015)
Compilers for visual programs
- Halide (PLDI 2013, CACM 2017)
- Image Perforation (SIGGRAPH 2016)
- VizGen (SIGGRAPH Asia 2016)
- Style Transfer (SIGGRAPH 2014)
CNN Porjects (Ongoing)
- Risser et al. 2017. Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses (arXiv)
- Collage / Composition / Replacement: Who and Where? Automatic Semantic Aware Person Composition, Tan et al. arXiv 2017
The paper PatchTable was presented at ACM SIGGRAPH 2015. Dr. Barnes created an algorithm called “PatchMatch,” which finds correspondences between small square regions (or patches) from one image to another image. That algorithm accelerated this correspondence-finding task by an order of magnitude or more, and is widely used, but it does not scale well to retrieval tasks where there is a large database of images. PatchTable therefore presents a data structure that reduces patch search times when one is querying a large database of images. This is done by offloading as much of the computation as possible to a pre-computation stage that takes modest time, so patch queries can be as efficient as possible. PatchTable is based on a locality sensitive hashing scheme.
The authors show experimentally that PatchTable accelerates the patch query operation by up to 9x over k-coherence, up to 12x over TreeCANN, and up to 200x over PatchMatch.
Their fast algorithm allows us to explore efficient and practical imaging and computational photography applications. They show results for artistic video stylization, light field super-resolution, and multi-image inpainting.
