Open
Description
Hi, I'm having problems running the bash tests/minimal_test.sh command. I am not sure why this is happening but I believe it starts when I try to set the environment.
I already added execution permissions for the set_environment.sh by running chmod a+x set_environment.sh but when I try to execute it, nothing seems to happen:
(base) luis@luis-BOHK-WAX9X:~/transition-amr-parser$ touch set_environment.sh
(base) luis@luis-BOHK-WAX9X:~/transition-amr-parser$ . set_environment.sh
(base) luis@luis-BOHK-WAX9X:~/transition-amr-parser$
Futhermore, running bash scripts/download_and_patch_fairseq.sh seems that it is also doing nothing:
(base) luis@luis-BOHK-WAX9X:~/transition-amr-parser$ bash scripts/download_and_patch_fairseq.sh
(base) luis@luis-BOHK-WAX9X:~/transition-amr-parser$
Everything is installed properly:
base) luis@luis-BOHK-WAX9X:~/transition-amr-parser$ python tests/correctly_installed.py
pytorch 1.8.1+cu102
fairseq 0.7.2
spacy 2.2.3
[OK] correctly installed
But when I run bash tests/minimal_test.sh it does not test (I will leave the full message for convenience).
[normalize rules] months
[normalize rules] units
[normalize rules] cardinals
[normalize rules] ordinals
Read DATA/wiki25.jkaln
25 sentences
216/293 node types/tokens
35/285 edge types/tokens
241/383 word types/tokens
AMR contains 4 duplicate edges
{'ARG1': 4}
2021-05-13 11:33:55 [amr] Processing oracle
2021-05-13 11:33:55 [oracle] Parsing data
computing oracle: 100%|█████████████████████████| 25/25 [00:00<00:00, 92.61it/s]
2021-05-13 11:33:56 [oracle] Done
Not whitelisted actions used e.g. arcs for unconfirmed words
Counter({'LA': 2})
Blacklisted actions used e.g. duplicated edges
Counter({'RA': 1})
There were 18 disconnected nodes (:rel)
2021-05-13 11:33:56 [Totals:] 0.61
2021-05-13 11:33:56 [Totals:] Failed Entity Predictions:
Namespace(alignfile=None, batch_normalize_reward=False, bert_layers=None, bpe=None, cpu=False, criterion='cross_entropy', dataset_impl='mmap', destdir='DATA.tests/features/wiki25.jkaln/', entity_rules='DATA.tests/oracles/wiki25.jkaln//entity_rules.json', fp16=False, fp16_init_scale=128, fp16_scale_tolerance=0.0, fp16_scale_window=None, gold_annotations=None, gold_episode_ratio=None, joined_dictionary=False, log_format=None, log_interval=1000, lr_scheduler='fixed', machine_rules='DATA.tests/oracles/wiki25.jkaln//train.rules.json', machine_type='AMR', memory_efficient_fp16=False, min_loss_scale=0.0001, no_progress_bar=False, nwordssrc=-1, nwordstgt=-1, only_source=False, optimizer='nag', padding_factor=8, pretrained_embed='roberta.base', seed=1, source_lang='en', srcdict=None, target_lang='actions', task='translation', tbmf_wrapper=False, tensorboard_logdir='', testpref='DATA.tests/oracles/wiki25.jkaln//test', tgtdict=None, threshold_loss_scale=None, thresholdsrc=0, thresholdtgt=0, tokenize_by_whitespace=False, tokenizer=None, trainpref='DATA.tests/oracles/wiki25.jkaln//train', user_dir=None, validpref='DATA.tests/oracles/wiki25.jkaln//dev', workers=1)
| [en] Dictionary: 247 types
| [en] DATA.tests/oracles/wiki25.jkaln//train.en: 25 sents, 408 tokens, 0.0% replaced by <unk>
| [en] Dictionary: 247 types
| [en] DATA.tests/oracles/wiki25.jkaln//dev.en: 25 sents, 408 tokens, 0.0% replaced by <unk>
| [en] Dictionary: 247 types
| [en] DATA.tests/oracles/wiki25.jkaln//test.en: 25 sents, 408 tokens, 0.0% replaced by <unk>
| [actions] Dictionary: 127 types
| [actions] DATA.tests/oracles/wiki25.jkaln//train.actions: 25 sents, 1327 tokens, 0.0% replaced by <unk>
| [actions] Dictionary: 127 types
| [actions] DATA.tests/oracles/wiki25.jkaln//dev.actions: 25 sents, 1327 tokens, 0.0% replaced by <unk>
| [actions] Dictionary: 127 types
| [actions] DATA.tests/oracles/wiki25.jkaln//test.actions: 25 sents, 1327 tokens, 0.0% replaced by <unk>
Using cache found in /home/luis/.cache/torch/hub/pytorch_fairseq_master
fatal: not a git repository (or any of the parent directories): .git
running build_ext
/home/luis/anaconda3/lib/python3.8/site-packages/torch/utils/cpp_extension.py:369: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
skipping 'fairseq/data/data_utils_fast.cpp' Cython extension (up-to-date)
skipping 'fairseq/data/token_block_utils_fast.cpp' Cython extension (up-to-date)
copying build/lib.linux-x86_64-3.8/fairseq/libbleu.cpython-38-x86_64-linux-gnu.so -> fairseq
copying build/lib.linux-x86_64-3.8/fairseq/data/data_utils_fast.cpython-38-x86_64-linux-gnu.so -> fairseq/data
copying build/lib.linux-x86_64-3.8/fairseq/data/token_block_utils_fast.cpython-38-x86_64-linux-gnu.so -> fairseq/data
copying build/lib.linux-x86_64-3.8/fairseq/libbase.cpython-38-x86_64-linux-gnu.so -> fairseq
copying build/lib.linux-x86_64-3.8/fairseq/libnat.cpython-38-x86_64-linux-gnu.so -> fairseq
loading archive file http://dl.fbaipublicfiles.com/fairseq/models/roberta.base.tar.gz from cache at /home/luis/.cache/torch/pytorch_fairseq/37d2bc14cf6332d61ed5abeb579948e6054e46cc724c7d23426382d11a31b2d6.ae5852b4abc6bf762e0b6b30f19e741aa05562471e9eb8f4a6ae261f04f9b350
| dictionary: 50264 types
Using roberta.base extraction in cpu (slow, wont OOM)
25it [00:00, 84.66it/s]
There were missing actions
Counter({'LA(op1)': 1, 'LA(domain)': 1, 'RA(ARG1)': 1})
Using cache found in /home/luis/.cache/torch/hub/pytorch_fairseq_master
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
File "/home/luis/.cache/torch/hub/pytorch_fairseq_master/hubconf.py", line 49, in <module>
import fairseq.data.token_block_utils_fast # noqa
ModuleNotFoundError: No module named 'fairseq.data.token_block_utils_fast'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 152, in save_modules
yield saved
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 193, in setup_context
yield
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 254, in run_setup
_execfile(setup_script, ns)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 43, in _execfile
exec(code, globals, locals)
File "/home/luis/.cache/torch/hub/pytorch_fairseq_master/setup.py", line 268, in <module>
do_setup(package_data)
File "/home/luis/.cache/torch/hub/pytorch_fairseq_master/setup.py", line 179, in do_setup
setup(
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/luis/anaconda3/lib/python3.8/distutils/core.py", line 134, in setup
ok = dist.parse_command_line()
File "/home/luis/anaconda3/lib/python3.8/distutils/dist.py", line 483, in parse_command_line
args = self._parse_command_opts(parser, args)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/dist.py", line 903, in _parse_command_opts
nargs = _Distribution._parse_command_opts(self, parser, args)
File "/home/luis/anaconda3/lib/python3.8/distutils/dist.py", line 546, in _parse_command_opts
raise DistutilsClassError(
distutils.errors.DistutilsClassError: command class <class 'torch.utils.cpp_extension.BuildExtension'> must subclass Command
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/luis/anaconda3/bin/fairseq-preprocess", line 33, in <module>
sys.exit(load_entry_point('fairseq', 'console_scripts', 'fairseq-preprocess')())
File "/home/luis/transition-amr-parser/fairseq-stack-transformer/fairseq_cli/preprocess.py", line 295, in cli_main
main(args)
File "/home/luis/transition-amr-parser/fairseq-stack-transformer/fairseq_cli/preprocess.py", line 212, in main
make_state_machine(args, src_dict, tgt_dict, tokenize=tokenize)
File "/home/luis/transition-amr-parser/transition_amr_parser/stack_transformer/preprocess.py", line 300, in make_state_machine
make_binary_bert_features(args, validpref, outprefix, src_dict.eos_index, src_dict.pad_index, tokenize)
File "/home/luis/transition-amr-parser/transition_amr_parser/stack_transformer/preprocess.py", line 210, in make_binary_bert_features
pretrained_embeddings = PretrainedEmbeddings(
File "/home/luis/transition-amr-parser/transition_amr_parser/stack_transformer/pretrained_embeddings.py", line 180, in __init__
self.roberta = torch.hub.load('pytorch/fairseq', name)
File "/home/luis/anaconda3/lib/python3.8/site-packages/torch/hub.py", line 339, in load
model = _load_local(repo_or_dir, model, *args, **kwargs)
File "/home/luis/anaconda3/lib/python3.8/site-packages/torch/hub.py", line 365, in _load_local
hub_module = import_module(MODULE_HUBCONF, hubconf_path)
File "/home/luis/anaconda3/lib/python3.8/site-packages/torch/hub.py", line 74, in import_module
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/luis/.cache/torch/hub/pytorch_fairseq_master/hubconf.py", line 56, in <module>
sandbox.run_setup(
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 257, in run_setup
raise
File "/home/luis/anaconda3/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 193, in setup_context
yield
File "/home/luis/anaconda3/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 164, in save_modules
saved_exc.resume()
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 139, in resume
raise exc.with_traceback(self._tb)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 152, in save_modules
yield saved
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 193, in setup_context
yield
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 254, in run_setup
_execfile(setup_script, ns)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/sandbox.py", line 43, in _execfile
exec(code, globals, locals)
File "/home/luis/.cache/torch/hub/pytorch_fairseq_master/setup.py", line 268, in <module>
do_setup(package_data)
File "/home/luis/.cache/torch/hub/pytorch_fairseq_master/setup.py", line 179, in do_setup
setup(
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/home/luis/anaconda3/lib/python3.8/distutils/core.py", line 134, in setup
ok = dist.parse_command_line()
File "/home/luis/anaconda3/lib/python3.8/distutils/dist.py", line 483, in parse_command_line
args = self._parse_command_opts(parser, args)
File "/home/luis/anaconda3/lib/python3.8/site-packages/setuptools/dist.py", line 903, in _parse_command_opts
nargs = _Distribution._parse_command_opts(self, parser, args)
File "/home/luis/anaconda3/lib/python3.8/distutils/dist.py", line 546, in _parse_command_opts
raise DistutilsClassError(
distutils.errors.DistutilsClassError: command class <class 'torch.utils.cpp_extension.BuildExtension'> must subclass Command
Any idea why this might be happening?
Metadata
Metadata
Assignees
Labels
No labels