Abstract
Robotic systems are increasingly used in dental implant surgery, but fully autonomous robotic surgery is held back by data scarcity: manual annotation is expensive and there are no large datasets of missing teeth. Previous deep learning approaches detect missing teeth on 2D X-ray images, which lack the depth and rotation information needed to place an implant in a real-world coordinate system.
We introduce MT3D, a dataset for 3D missing tooth detection. We built it by removing individual teeth from 175 fully-dentate intraoral scans selected from the public Teeth3DS dataset, creating 12 distinct tooth loss cases per scan. We also propose a 3D detection framework that predicts the 3D bounding box of the missing tooth region directly from a point cloud. The box gives the position and orientation for subsequent implant placement. Experiments show high detection accuracy, a step toward autonomous dental implant surgery.
Interactive Results
Test-set predictions on an upper-jaw scan. Drag to rotate, scroll to zoom, right-drag to pan.
MT3D Dataset
Teeth3DS contains 1,800 intraoral scans from 900 patients with per-tooth instance labels. Because real clinical scans often contain damaged or imperfect teeth that could be confused with real tooth loss, we keep only scans with 14 or more teeth (FDI notation), which leaves a curated subset of 175 scans. From each scan we remove a single tooth at one of 12 target positions, and label each sample with a point cloud and an oriented 3D bounding box.
Generation pipeline
1 · Dental arch line
Fit a spline curve through the center of each tooth, then apply PCA to the curve to get the arch normal vector.
2 · Healed-ridge surface
Take the adjacent gingiva and proximal-surface points of the removed tooth, generate medium points along the arch normal, grid-interpolate between them, then smooth with a moving average over the surrounding points.
3 · 3D box label
The vertical axis is a weighted sum of the surface normal, arch normal and tooth heading. The horizontal axis comes from the arch line near the missing tooth. We fit the box to the removed tooth's points.
3D Missing Tooth Detection
The encoder is based on Point Transformer. It applies self-attention only inside kNN neighborhoods, which captures local geometric detail efficiently. After global average pooling, an MLP head predicts 10 parameters: the center (x, y, z), the rotation as a quaternion (w, rx, ry, rz), and the size (h, w, l).
The model is trained with a multi-task objective. An auxiliary axis loss maximizes the cosine similarity between predicted and ground-truth rotation axes, which stabilizes the orientation regression:
L = L1(center) + L1(size) + MSE(quaternion) + Lcos(axes)Results
We trained for 100 epochs on a single RTX 3090 (batch size 8, SGD with a cosine schedule) and evaluated on the test split. 3D IoU measures box overlap, center distance measures position error, and per-axis cosine similarity measures orientation accuracy.
| Loss | IoU (%) ↑ | Center Dist. (cm) ↓ | Cosine Similarity ↑ | ||
|---|---|---|---|---|---|
| X | Y | Z | |||
| w/o size, axis loss | – | 1.2900 | 0.9820 | 0.9797 | 0.9884 |
| w/o axis loss | 62.91 | 3.0140 | 0.9739 | 0.9703 | 0.9607 |
| Ours (full) | 68.85 | 0.9196 | 0.9923 | 0.9897 | 0.9934 |
Ablation of loss configurations. Without the size and axis losses the model fails to regress valid boxes. Adding the auxiliary axis loss improves every metric.
Qualitative results
