instant-ngp fails to predict rgb when the scene box is too tight?

[hoanhle]

Describe the bug
I’m trying to limit the region of interest for instant-ngp as much as possible to avoid unnecessary computation. However, when I make it very tight around the object, it fails to predict any color. When I enlarge the region of interest, it starts predicting colors again, which is puzzling to me, since I can clearly see the "wasted space".

To Reproduce
Steps to reproduce the behavior:
Check out this branch: #3364 and data: https://drive.google.com/drive/folders/1LlWcbTCr2EO9g-VhNowhW1IRiMHbLzgZ?usp=sharing

Hardcode scene box values in nerfstudio/data/dataparsers/nerfstudio_dataparser.py and visualize it here #3363

The tight box

        scene_box = SceneBox(
            aabb=torch.tensor(
               [[-0.048, -0.447, -0.414], [0.064, 0.483, 0.324]], dtype=torch.float32
            )
        )

The big box

        scene_box = SceneBox(
            aabb=torch.tensor(
                [[-0.096, -0.894, -0.828], [0.128, 0.966, 0.648]], dtype=torch.float32
            )
        )

Train instant-ngp with the following param

ns-train instant-ngp  --output-dir /home/leh19/workspace/experiment/outputs/base_line --vis viewer+tensorboard  --experiment-name painting_1 nerfstudio-data  --data /home/leh19/test_run_1/JPG/painting_2/processed 

main difference: disable the scene contraction and use orientation method = align

method_configs["instant-ngp"] = TrainerConfig(
    method_name="instant-ngp",
    steps_per_eval_batch=500,
    steps_per_save=2000,
    max_num_iterations=30000,
    mixed_precision=True,
    pipeline=DynamicBatchPipelineConfig(
        datamanager=VanillaDataManagerConfig(
            dataparser=NerfstudioDataParserConfig(load_3D_points=True, orientation_method="align"),
            train_num_rays_per_batch=4096,
            eval_num_rays_per_batch=4096,
        ),
        model=InstantNGPModelConfig(eval_num_rays_per_chunk=8192, disable_scene_contraction=True),
    ),
    optimizers={
        "fields": {
            "optimizer": AdamOptimizerConfig(lr=1e-2, eps=1e-15),
            "scheduler": ExponentialDecaySchedulerConfig(lr_final=0.0001, max_steps=200000),
        }
    },
    viewer=ViewerConfig(num_rays_per_chunk=1 << 12),
    vis="viewer",
)

Expected behavior
Whenever the tight region of interest is used, the model fails to predict any rgb at all, while with the not so tight (simply 2x) the model quickly converges.

Not able to predict any rgb

Predicting fine

I can clearly see the wasted space in viser when use the not tight box.
See example render with "white floaters" around
https://github.com/user-attachments/assets/6f432305-2640-4fe6-a8d1-ed0da53dfd31

I would love some pointers on why this behaviour could happen? What am I missing here?

[hoanhle]

lesson learned: be careful with rgba images.