Linkedin

Specifix

About the project

This is an ongoing university project, and due to an NDA, I am unable to disclose all details. For this project, I am working on two key components: a 3D Variational Autoencoder (VAE) and a Reinforcement Learning (RL) environment.

3D Variational Autoencoder

The goal of the 3D Variational Autoencoder is to take two separate parts of an object, which fit together, and output the complete object. To achieve this, we convert our .obj files, which consist of faces and vertices, into voxelized objects for processing by the autoencoder.

Data:

Showcase Cube
Two halves of a cube
Two halves of cube after voxelizing
Two halves of a sphere
Two halves of a sphere after voxelizing
Two halves of a cylinder
Two halves of a cylinder after voxelizing

Reinforcement Learning

The objective of the Reinforcement Learning model is to move two parts of an object back together so that they align correctly.

The environment is built using PyBullet, with additional calculations performed in Trimesh to calculate the reward. The PyBullet and Trimesh environments are synchronized at each step to ensure they work in parallel. The RL model manipulates the object in PyBullet, extracting its coordinates and orientation. We then convert the orientation into a rotation matrix and apply both the coordinates and rotation matrix to the object in the Trimesh environment.

Results

Since the project is still in progress, there are no results available to share at this time.