-
Notifications
You must be signed in to change notification settings - Fork 59
Description
Hi there!
I was trying to finetune the distillbert model following the command from https://wandb.ai/hhousen/transformerextsum/runs/296s2066/overview
and removing all unavailable tags.
I used my custom json dataset following the guideline at https://transformersum.readthedocs.io/en/latest/extractive/convert-to-extractive.html
and txt files were properly generated by your code.
However, I run into this problem when I try to finetune the model.
Any idea why this could happening? Thank you in advance!
Training command:
python main.py --model_name_or_path distilbert-base-uncased --no_use_token_type_ids --pooling_mode sent_rep_tokens --data_path ../datasets/new_data/ --max_epochs 3 --accumulate_grad_batches 2 --warmup_steps 1800 --gradient_clip_val 1.0 --optimizer_type adamw --use_scheduler linear --do_train --do_test --data_type txt --dataloader_type map
Full logger:
Sanity Checking DataLoader 0: 0%| | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/ubuntu/TransformerSum/src/main.py", line 509, in
main(main_args)
File "/home/ubuntu/TransformerSum/src/main.py", line 137, in main
trainer.fit(model)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 770, in fit
self._call_and_handle_interrupt(
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 723, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 811, in _fit_impl
results = self._run(model, ckpt_path=self.ckpt_path)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1236, in _run
results = self._run_stage()
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1323, in _run_stage
return self._run_train()
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1345, in _run_train
self._run_sanity_check()
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1413, in _run_sanity_check
val_loop.run()
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 204, in run
self.advance(*args, **kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 154, in advance
dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/loops/base.py", line 204, in run
self.advance(*args, **kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 128, in advance
output = self._evaluation_step(**kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 226, in _evaluation_step
output = self.trainer._call_strategy_hook("validation_step", *kwargs.values())
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1765, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/pytorch_lightning/strategies/strategy.py", line 344, in validation_step
return self.model.validation_step(*args, **kwargs)
File "/home/ubuntu/TransformerSum/src/extractive.py", line 778, in validation_step
outputs, mask = self.forward(**batch)
File "/home/ubuntu/TransformerSum/src/extractive.py", line 275, in forward
outputs = self.word_embedding_model(**inputs, **kwargs)
File "/home/ubuntu/miniconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'source'