Skip to content

Add tls to ftp #1581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Aug 12, 2024
Merged

Add tls to ftp #1581

merged 10 commits into from
Aug 12, 2024

Conversation

bartvaneswhiffle
Copy link
Contributor

@bartvaneswhiffle bartvaneswhiffle commented Apr 17, 2024

See #1580.

It is basically a copy of the FTP class in fsspec, with a modification to use FTP_TLS and one addtional change.

Please have a look at this.

self.ftp = FTP_TLS(timeout=self.timeout)
self.ftp.connect(self.host, self.port)
self.ftp.login(*self.cred)
self.ftp.prot_p()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look at this.

@bartvaneswhiffle
Copy link
Contributor Author

I've added tests (mostly copied from test_ftp.py, not all are yet working. Could use some help here :)

@martindurant
Copy link
Member

Thanks for contributing!

Before going further, do you think it's possible to provide the extra features as options on the original FTP filesystem rather than create a new variant that is largely the same?

@bartvaneswhiffle
Copy link
Contributor Author

No worries, in a sense we could give two additional arguments on the init of FTPFileSystem:

  • ssl=False
  • prot_p=False

Then we could modify the _connect to use either FTP or TLS_FTP and modify the init to activate prot_p if it's equal to True after setting the connection.

How does this sound to you?

@martindurant
Copy link
Member

Yes, exactly.

For testing, you would need a secured server, of course, but you could essentially run all FTP tests with standard and secure servers.

@bartvaneswhiffle
Copy link
Contributor Author

I've removed the the ftp_tls class and integrated it with the ftp class. Two tests are currently still failing: the test with caching (test_complex) and the test with transaction (test_transaction), not sure how to fix those?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this an exact copy of he insecure tests? You can use pytest.mark.parametrize to run each of the existing tests, once with secure=False and once with True.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not possible since it will need a different fixture.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is totally possible to have a parametrised fixture, or two fixtures that you choose between. https://stackoverflow.com/a/29407931/3821154 shows an example.

@martindurant
Copy link
Member

Ping: is this PR still progressing?

@bartvaneswhiffle
Copy link
Contributor Author

Thanks for the reminder, so far I've not had the time to further implement the tests.

@bartvaneswhiffle
Copy link
Contributor Author

I've found some time to update the tests. The pyftpdlib server has been replaced with script that starts a pyftpdlib server that accepts TLS. Furtermore, docstrings and codestyle improvements have been made.

@martindurant: Do you understand why the CI / downstream fails?

@martindurant
Copy link
Member

Do you understand why the CI / downstream fails

Please merge from master and try again

@bartvaneswhiffle bartvaneswhiffle changed the title Add ftptls Add tls to ftp Aug 12, 2024
@bartvaneswhiffle
Copy link
Contributor Author

Do you understand why the CI / downstream fails

Please merge from master and try again

All checks have passed :)

@martindurant martindurant merged commit 32ce723 into fsspec:master Aug 12, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants