While free-hand sketching has long served as an efficient representation to convey characteristics of an object, they are often subjective, deviating significantly from realistic representations. Moreover, sketches are not consistent for arbitrary viewpoints, making it hard to catch 3D shapes. We propose 3Doodle, generating descriptive and view-consistent sketch images given multi-view images of the target object. Our method is based on the idea that a set of 3D strokes can efficiently represent 3D structural information and render view-consistent 2D sketches. We express 2D sketches as a union of view-independent and view-dependent components. 3D cubic Bézier curves indicate view-independent 3D feature lines, while contours of superquadrics express a smooth outline of the volume of varying viewpoints. Our pipeline directly optimizes the parameters of 3D stroke primitives to minimize perceptual losses in a fully differentiable manner. The resulting sparse set of 3D strokes can be rendered as abstract sketches containing essential 3D characteristic shapes of various objects. We demonstrate that 3Doodle can faithfully express concepts of the original images compared with recent sketch generation approaches
From multiple-view images, we aim to reconstruct view-consistent sketch representation. We represent 3D strokes as a union of view-independent and view-dependent components. For view-independent components, we use 3D cubic Bézier curves to represent 3D feature lines. For view-dependent components, we use contours of superquadrics to express a smooth outline. We propose a differentiable rendering pipeline for our 3D stroke representation. Then we optimize the compact parameters of 3D stroke primitives to minimize the perceptual loss.
Results on NeRF synthetic dataset. We use only Bézier curves for this examples.
Comparison with baselines methods.
@article{10.1145/3658156,
author = {Choi, Changwoon and Lee, Jaeah and Park, Jaesik and Kim, Young Min},
title = {3Doodle: Compact Abstraction of Objects with 3D Strokes},
year = {2024},
issue_date = {July 2024},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {43},
number = {4},
issn = {0730-0301},
url = {https://doi.org/10.1145/3658156},
doi = {10.1145/3658156},
journal = {ACM Trans. Graph.},
month = {jul},
articleno = {107},
numpages = {13},
keywords = {3D sketch lines, 3D strokes, differentiable rendering}
}