NUGETs architecture overview

The NUGETs package is meant to allow people to easily run their own models to investigate approximating solutions to computational geometric problems with neural networks. See running for more information on the mechanics of how to execute a run/train a model. Refer to our paper for more technical details on each element of our Workflow.

Workflow

The NUGETs package is separated out into three major parts:

Datasets: this is the raw point cloud data.
Task: maps each dataset to a label, depending on the problem you are interested in.
Model: model which is used to approximate solutions. We follow the Encode-Process-Decode paradigm of (Hamrick).

For example, if I am interested in approximating the minimum enclosing ball for the ModelNet dataset, we start with the raw point cloud data representing each object in ModelNet. Then, it gets mapped into the following training point (S, c, r) where c and r are the center and radius of the minimum enclosing ball, respectively. The model the learns to approximate c and r.

Datasets

This part of our workflow controls loading in the raw point cloud data. Note that this does not control the labelling process for the problem that you are interested in. We include several point cloud datasets for use, including synthetic point clouds generated from Gaussian distributions, ModelNet, 3D LiDAR scans from terrains, etc. .. note:

If you have your own raw point cloud dataset, it is easy to add with the CustomPointCloudDataset object. In the yaml file where you have set your experimental configuration (see `running`), do the following under where you :

.. code-block:: yaml

   dataset: CustomPointCloudDataset
   dataset_config:
      filename: <your-custom-dataset-filename>
      length: <number-of-point-clouds-to-include>
      dim: <dimension>

Tasks

The Task object is where the labels for each problem are generated. For instance, in the case of the minimum enclosing ball, a Dataset` object is passed to the Task and labels representing the center and radius of the minimum enclosing ball are generated. The Task object will automatically create and cache the train/validation/test split.

We additionally include configuration parameters which allow the user the specify an out-of-distribution dataset that they would like the model to be evaluated on during training. All metrics will automatically be logged to WandB.

Model

Each model consists of an encoder, a processor, and a decoder. Because the encoder and decoder are often task specific (e.g. a model predicting the minimum enclosing ball on the plane should decode to a three-dimensional output with one dimension of the predicted radius and two for the predicted center), they are constructed within each task object.

Each processor model in our benchmark is called a BackBone. The abstract data class is defined in nugets/models/backbone.py. See our reference for all available defined BackBone models. Currently, we assume that each Dataset is a point cloud so the currently implemented and tested BackBone models assume that the incoming data is in the form of a point cloud. However, there are some initial (untested) implementations of graph learning BackBone models as well.