Skip to content

Removed empty bytes at channel subchunk ends for issue #31 #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Sep 14, 2020
Merged

Removed empty bytes at channel subchunk ends for issue #31 #40

merged 16 commits into from
Sep 14, 2020

Conversation

ggirelli
Copy link
Contributor

@ggirelli ggirelli commented Aug 15, 2020

I tried to address issue #31.

This edit checks if the chunk data size does not match the expected, based on image axes size. It then verifies that all unwanted bytes are zero-bytes and removes them if that is the case. It also supports (unseen) cases in which more than one unwanted zero-byte is present per step.

Scenarios:

  • If the unwanted byte distribution is not consistent (i.e., not a multiple of column length -i.e., height-), an assert stops execution.
  • If unwanted non-zero bytes are present, then a warning is triggered and the parsing crashes as usual down the line.
  • If all unwanted bytes are zero-bytes and properly distributed, they are removed and a warning is issued.

Checks if the chunk data size does not match the expected, based on image axes size. Verifies that all unwanted bytes are zero-bytes and removes them. Two triggered warnings: one for unwanted bytes and one for bytes removed (more of an INFO though).
Copy link
Member

@rbnvrw rbnvrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ggirelli for taking a look at this, I really appreciate it! 👍
Did you try this also for the example files in issue #31? And does it work?
I've left some small comments to make the warnings hopefully more user friendly and to avoid some code duplication.
On top of that, because it is possibly a dangerous change to the parser, we really need to add some unit tests to check the proper functioning of this part of the code. After that, I'd be really happy to merge it and I hope it will solve these long-standing problems!

@ggirelli
Copy link
Contributor Author

Hej @rbnvrw, thanks for the code revision! Yes, I tested it on the files provided on issue #31 and it works fine. I'll have a better look at your revs and prepare the unit tests next week 😄

@rbnvrw
Copy link
Member

rbnvrw commented Aug 21, 2020

That sounds fantastic! Thank you.

@ggirelli
Copy link
Contributor Author

Hi @rbnvrw, I tried to address all your code review points, thanks again for the comments. I added a simple unit test for this. As I did not have a smaller sample image with this issue and creating an artificial one would take a bit longer than I am willing to put on this, the code download the sample stitched image from #31 (2019 MB!) the first time one runs the tests, and then keeps using the same file from then onward. So the first time the test is run it takes some time (1-2 min) depending on your connection speed (and requires internet connection!). If you have any ideas on how to improve this, please let me know! 😄

@ggirelli ggirelli requested a review from rbnvrw August 31, 2020 11:04
@rbnvrw
Copy link
Member

rbnvrw commented Sep 14, 2020

Thank you very much @ggirelli for your efforts! 🥇

I think this looks good. Might decide to migrate the file to something like git LFS but I'll have to look into that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants