hhhhh789
add ckpt mpts
ed4aa37
[2024-09-27 10:26:44,283][hydra.utils][INFO] - Instantiating <cdvae.pl_data.datamodule.CrystDataModule>
[2024-09-27 10:26:45,071][hydra.utils][INFO] - Loading preprocessed cached data from /sharefs/material/cdvae/data/mpts_52/preprocess_data_train_27380.pt
[2024-09-27 10:26:48,375][hydra.utils][INFO] - Loading preprocessed cached data from /sharefs/material/cdvae/data/mpts_52/preprocess_data_train_27380.pt
[2024-09-27 10:26:50,264][hydra.utils][INFO] - Loading preprocessed cached data from /sharefs/material/cdvae/data/mpts_52/preprocess_data_val_5000.pt
[2024-09-27 10:26:50,897][hydra.utils][INFO] - Loading preprocessed cached data from /sharefs/material/cdvae/data/mpts_52/preprocess_data_test_8096.pt
[2024-09-27 10:26:51,451][hydra.utils][INFO] - Instantiating <cdvae.pl_modules.vm_bfn_csp_plmodel.VM_BFN_CSP_PL_Model>
[2024-09-27 10:26:53,637][hydra.utils][INFO] - Passing scaler from datamodule to model <StandardScalerTorch(means: -1.1277656555175781, stds: 1.0188579559326172)>
[2024-09-27 10:26:53,640][hydra.utils][INFO] - Adding callback <LearningRateMonitor>
[2024-09-27 10:26:53,640][hydra.utils][INFO] - Adding callback <EarlyStopping>
[2024-09-27 10:26:53,641][hydra.utils][INFO] - Adding callback <ModelCheckpoint>
[2024-09-27 10:27:07,632][hydra.utils][INFO] - Adding callback <GradientClip>
[2024-09-27 10:27:07,635][hydra.utils][INFO] - Adding callback <EMACallback>
[2024-09-27 10:27:07,635][hydra.utils][INFO] - Instantiating <WandbLogger>
[2024-09-27 10:27:07,637][hydra.utils][INFO] - W&B is now watching <{cfg.logging.wandb_watch.log}>!
[2024-09-27 10:27:18,108][hydra.utils][INFO] - Instantiating the Trainer
[2024-09-27 10:27:18,132][hydra.utils][INFO] - Starting training!