LOGen: Toward Lidar Object Generation by Point Diffusion

A step toward augmenting lidar point clouds with object diversity

 

Ellington Kirby, Mickael Chen, Renaud Marlet, Nermin Samet
Paper Code

barrier

bike

bus

car

construction vehicle

motorcycle

pedestrian

traffic cone

trailer

truck

Real (white) and generated (green) objects by LOGen for all ten instance classes of nuScenes.

Abstract

A common strategy to improve lidar segmentation results on rare semantic classes consists of pasting objects from one lidar scene into another. While this augments the quantity of instances seen at training time and varies their context, the instances fundamentally remain the same. In this work, we explore how to enhance instance diversity using a lidar object generator. We introduce a novel diffusion-based method to produce lidar point clouds of dataset objects, including reflectance, and with an extensive control of the generation via conditioning information. Our experiments on nuScenes show the quality of our object generations measured with new 3D metrics developed to suit lidar objects.

The objects generated by LOGen are indistinguishable from real objects in certain classes

MY ALT TEXT

SPVCNN is trained with 3 (left) or 4 (right) channels (coordinates + intensity) on nuScenes (train set) with real objects. It is tested on nuScenes (val set) where each object is replaced with a generated object of the same class, box size and sensor viewing angle, and with the same number of points.

LOGen can generate intensities for lidar point cloud objects

LOGen is the first diffusion method that leverages transformer architecture for lidar point cloud generation

MY ALT TEXT

LOGen conditions the generation on the following box information: the box center (x, y, z), the box length, width and height (l, w, h) and ϕ the angle between the object heading ψ and the ray from the object bounding box center to the sensor.

LOGen can generate new objects from novel views by interpolating the viewing angle of the condition

reference ground-truth

recreation, rotation 0

novel view, rotation 1/5

novel view, rotation 2/5

novel view, rotation 3/5

novel view, rotation 4/5

Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4
Image 1
Image 2
Image 3
Image 4
Image 4
Image 4

Novel objects produced by LOGen. Recreations are generated using the conditioning information of a real object, the rest of the objects are created from novel views by interpolating the viewing angle ϕ of the condition.

BibTeX

@inproceedings{logen,
        author = {Ellington Kirby and Mickael Chen and Renaud Marlet and Nermin Samet},
        title = {LOGen: Toward Lidar Object Generation by Point Diffusion},
        booktitle = {arXiv},
        year = {2024},
      }