Skip to content
This repository was archived by the owner on Apr 26, 2023. It is now read-only.

Release Notes

Jump to bottom
Gregory Sharov edited this page Dec 27, 2017 · 30 revisions
Scipion Logo

v1.2.beta (2017-12-27) Caligula (work in progress)

Release note

We are very pleased to announce the release of a new minor version of Scipion. We have put our efforts in improving the Streaming functionality to work better in facilities. We have also updated some EM packages versions and done some bug-fixing and enhancements.

New features

Picking and particle extraction in streaming

We have push our streaming capabilities until particle extraction, this means that all pickings (not manual one of course) can run in streaming mode and the paticle extraction can be done also on the fly. Yep…​we are getting closer to have 2D classification and and rough initial model on the fly.

HTML Summary report redesigned

We have revisited our HTML report and have polished it. We’ve added a table with details and images per micrograph/ctf, a defocus coverage chart and a resolution histogram.

Improved CTF model

We have added phase shift, max. resolution and CTF fit quality parameters to our Scipion CTF model. Old parameters are still kept for compatibility with previous versions.

EM Packages

  • Added Relion v2.1.0 support: several new options in refinement & classification protocols, new local resolution and initial model protocols, refactored particle polishing protocol

  • Added dark/gain reference correction to Unblur protocol

  • Motioncor2 package updated to version 1.0.4. Also, now input tif files are read directly by the program without any conversion. Moreover, we have fixed the bug causing wrong drift plots.

Others

  • New protocol (xmipp3 - ctf selection) to make a selection of meaningful CTFs based on the defocus values, the astigmatism, and the resolution

  • New protocol (scipion - average frames) for computing raw frame averages (for simple diagnosis, nothing else)

  • New protocol (scipion - picking difference) to compute the difference between a reference SetOfPartices and another set (usually a negative reference).

  • Further improvements of streaming protocols:

    • Do not import files that are already imported (when continuing or restarting a stopped/failed streaming protocol)

    • Possibility to schedule jobs that link to previous unfinished ones (still in development)

  • Improved the performance during input set selection, especially when a lot of SetOfClasses2D/3D where generated by several runs of Relion

  • New scripts:

    • create_project.py script to create a project from a Scipion workflow file (json),

    • schedule_project.py to schedule all protocols given a project name (already existing project)

  • File browser now has shortcuts as well as Search function and keyboard navigation

  • Shorting protocol names: now when you copy a protocol, the new name will be oldName (copy N), where N is a number

v1.1 (2017-06-14) Balbino

Release note

We are very pleased to announce the release of a new version of Scipion. It’s been over a year since the previous and first version and we have been working on 3 main goals for this release:

  • Consolidation: We put and will always put our best effort into making Scipion a robust and reliable software. We have improved performance, usability and fixed multiple bugs.

  • EM packages integration: We have updated several EM packages to their latest versions (relion 2.0.4) and added new ones (motioncor2, gctf, gautomatch, …). Single movie alignment protocol (as in Scipion 1.0) has been split into several ones for each program.

  • Streaming capabilities: To speed up first preprocessing steps we have enabled Scipion to work in “streaming mode”, allowing users to compute aligned movies and estimate CTF as soon as a movie or micrograph comes out of the microscope PC.

New features

EM Packages

Xmipp

Xmipp has also been greatly improved and many new methods have been added. Please see xmipp release notes for details.

Ethan picker

Automated detection of spherical particles from electron micrographs.

gAutomatch

GPU-accelerated particle picking program developed by K. Zhang allows template-based and “gaussian-blob” (no references) picking. All advanced parameters (exclusive picking, filtering etc.) are available.

gCTF

GPU-accelerated program for CTF determination, refinement and evaluation. At this moment movie options, CTF refinement for particles and tilt refinement options are not supported yet.

Imagic

We have added Imagic MSA classification method. Further information is available from our wiki.

Localized reconstruction

A general method for the localized three-dimensional reconstruction of substructures bound to a larger particle. After determination of the particle orientations via conventional methods, local areas corresponding to the subunits ('subparticles') can be extracted and treated as single particles.

magDistortion

This program from Grigorieff’s lab allows to estimate and correct magnification distortions in electron micrographs. Correction is also available for particle coordinates. Hint: results of this protocol can be used for motion correction with motioncor2!

Motioncor2

Completely re-written (after motioncorr/dosefgpu) software from D. Agard lab allows anisotropic beam-induced motion correction at single pixel level across the whole frame using GPUs. Options for dose filtering, correction of magnification distortion and saving movie stacks are available.

Relion 2.0

We have updated Relion to its latest available version (2.0.4). You can benefit from the substantial reduction of computing time using Relion with your GPUs. But do not worry if you don’t have GPU, you can also use Relion on CPUs. We have also added new relion extract particle protocol and refactored the picking protocol. Moreover, new protocols for particle sorting, 3D mask creation, projection subtraction and CTF export to STAR file were added.

Streaming

Streaming processing allows to overlap data acquisition with the first steps of digital image processing. Protocols adapted for streaming can be executed even if the algorithm that produces the data that is going to be used as input has only produced part of the output dataset.

Import movies

Movies can now be imported in “streaming” mode. Also, individual frames can be used as input and stacked on-the-fly, making the movies available for later protocols immediately.

Movie alignment

All movie alignment protocols are able to use movies in streaming (motioncorr, motioncor2, xmipp-opticalflow, xmipp-crosscorrelation, unblur).

CTF estimation

ctffind3, ctffind4, gCTF are now able to work in streaming mode.

Monitors

We have designed monitors to follow the status of several running protocols within a Scipion project. They can track and plot values for “system status” (cpu, memory, swap), ctf values (defocusU, defocusV). Alerts can be setup to email the microscopist/user when certain values rise above/below a custom threshold.

HTML Report

The summary monitor generates an HTML report that summarizes the status of the system and the data being processed, plotting the defoci and system data.

Scipion box wizard

This wizard creates a folder structure for your project and can be customized to reflect different microscope or camera setups. You can choose what preprocessing steps you would like to do during movie streaming.

Consolidation

We have also done a lot of work to consolidate Scipion, improving the usability and adding small features to make Scipion a better and more robust software:

  • Extract coordinates protocol can now apply shifts to the particle coordinates.

  • Scipion install script now provides a possibility to choose package version for installation.

  • Export bibliographic references into bibtext file.

  • Notebook: write project notes in your favourite text editor.

  • Labels: any protocol can now be labelled with a name and/or color. This helps to orient within a protocol tree in large projects. (Use Ctrl+T to loop through modes)

  • Age mode: Coloring the boxes by “age”, the younger the bluest. (Use Ctrl+T to loop through modes)

  • Zooming and panning a project’s protocol tree (Use “Shift + mouse wheel”)

  • Linear picking mode, eraser size modification are now available in xmipp particle picker.

  • Project import: besides import/export of workflows, now it is also possible to import whole projects, e.g. from another computer.

  • Collect statistics: we collect and analyze usage statistics information to better understand the usage of the different protocols and prioritize maintenance and support. You can choose to enable or disable the collection of information at any time.

  • New scripts: create movie stacks, mirror directory, scipion box wizard.

  • Sort objects in browse windows: now all objects can be sorted by name, info or creation date. Useful when you have a lot of different object sets.

  • Highlight direct connections of selected protocols.

  • Several performance improvements to decrease project loading time.

  • Improved movie model: added initial dose (pre-exposure) and dose per frame.

v1.0.1 (2016-06-30)

  • Several protocol fixes:

    • Fixed bug when creating the output for Frealign (in some cases some information from input particles was not properly propagated)

    • Fixed some bugs in movie alignment protocols (summovie and unblur) and tests added

    • Some minor bugs fixed in Relion protocols

    • Bugs fixed in Resmap protocol when using two half volumes

  • Fixed several bugs in Spider protocols:

    • converting input particles with alignment

    • wrong regular expression for replacing some variables in script template

    • parsing of the resulting dendrogram

    • some additional validations and removed unused code

  • Bugfixes and inprovements in Xmipp protocols:

    • Protocols screen-classes merged into one: compare-reprojections

    • Complete refactoring of operate-particles and operate-volumes protocols (previously called 2D and 3D calculator). Tests added

  • Picking and Viewer:

    • Warning if particles are picked in a temporary folder and the SetOfParticles was not created

    • Improved implementation of assign-tiltpairs protocol in Xmipp and some refactoring of picking methods

    • Fixed bug that caused GUI to freeze sometimes

    • Some bugs fixed when displaying and exporting particles

    • Sorting arrows displayed after sorting by a column. Hourglass displayed while sorting.

    • Some bug fixed when creating subset from classes

  • Other fixes or improvements:

    • ImageHandler’s methods convert and writeStack now accepts alignment parameters

    • Fixed bug when displaying Movies summary (sqlite files were not closed)

    • Fixed bug when spawning Eman process to write particles

    • Added REMOTE_MESA_LIB environment var for using OpenGL in remote desktops

    • Created a LegacyProtocol class to read deprecated protocols

    • Cleanup in some tests and added new ones for core classes or functions

v1.0.0 (2016-02-20) Augusto

  • Allows to combine several EM software packages (~ 100 protocols):

    • All protocols from Xmipp

    • Most of protocols from Relion

    • MDA protocols from Spider

    • Some protocols from Eman2/Sparx

    • From Grigorieff lab: CTFFIND, FREALIGN, unblur and summovie.

    • A few tools from Bsoft

    • ResMap, gEMpicker, dogpicker, motioncorr

  • Full tracking and reproducibility:

    • Display runs as a list or a tree.

    • Inspect the parameters of a previous run

    • Repeat one or several runs

    • Export/Import a workflow template

  • Data analysis:

    • Visualization and operation with Sets. (Particles, Micrographs, CTFs, etc)

    • Visualization of Volumes

    • Resolution and angular distribution plots

Clone this wiki locally