Scipion demo for ICS course (configuration)

Scipion demo for the course of International School of Crystallography

Setting up the machines

Here you can find the instructions to set up your machine to be able to follow the Scipion tutorial session.

There will be two options to follow up the tutorial session:

1. To use a virtual machine on Amazon cloud (AWS): A number of AWS instances will be ready during the session to be used by the students. Connections details will be given at that time, and the only required software will be a web browser and a reasonable internet connection.

   Get access (remember to accept the https risk):

      https://IP.xxx.yyy.zzz/vnc.html  --> psswd: scipionICS
   

   Resize the screen with the gear at left edge menu --> remote size.

2. The alternative plan: To set up your own Scipion machine using VirtualBox: A VirtualBox virtual appliance, that you could easily launch from your own computer using Oracle VirtualBox application, is ready to be downloaded from here. Then follow the next steps:

   1.1. Create a new Virtual Machine (Type Linux and Version Ubuntu).

   1.2. Assign at least 6 GB memory and 4 CPUs (Settings > System > Motherboard/Processor).

   1.3. Choose as existing hard disk the virtual appliance downloaded (vdi).

   1.4. Create.

   1.5. Start it and login with user scipion and password scipionICS. By default the virtual machine screen is quite small, to resize it open a terminal and type xrandr -q and choose the resolution that fits better your laptop screen and then set it up with xrandr -s wxh (e.g. 1280x1024) or by Start > Preferences > Monitor Settings.

You can open Scipion by clicking on the desktop shortcut. There are two demo projects that you can visualize prior to the session (you could try to run some steps yourself but processing might be slow since number of cores and RAM are virtualized and not very high and GPU is not present). Raw movies are not present either to save some disk space. Project ScipionCombined shows a representative workflow where many of the Scipion possibilities are shown while project Acquisition_Simulation presents a finished workflow of the Acquisition Demo <https://github.com/I2PC/scipion/wiki/Acquisition-simulation-project-for-the-ISC-course>_.

Datasets

The dataset is the beta-galactosidase EMPIAR 10061 (4k with 38 frames).

The full set to simulate the large acquisition is made of ~150 movies (~250Gb) and can it be got from /home/dmaluenda/dataLocal/rawData/10061_dwnsp2/*.

The sample of that dataset to be used by the students can be reached from /home/dmaluenda/dataLocal/rawData/10061_dwnsp2/*1_*.mrc.

This datasets should be placed where RAWDATA_SIM at ~/em-facilities/usingAPI_demo/scipionbox.conf is pointing out (~/rawData/EMPIAR_10062_2dwsamp).

Projects

We will have two stored projects ScipionCombined and AcquisitionDemo. They can be reach from carver at /home/dmaluenda/ScipionUserData/ScipionCombined and /home/dmaluenda/ScipionUserData/AcquisitionDemo.

Streaming workflows

We will run two streaming workflows one from a JSON template and the other from a script using the Scipion's API.

From JSON template

The first do not need any new configuration. Just open a web browser, download the All2Monitor_GPU_.... workflow from the http://workflows.scipion.i2pc.es. Then import the downloaded JSON to an empty Scipion project.

Check that the movies path is pointing to the RAWDATA_SIM folder, ~/rawData/EMPIAR_10062_2dwsamp by default. After editing some from parameter SAVE the protocol (do NOT execute the protocol!).

When you are happy with all the parameters of all the protocols , you can launch the whole workflow by right-click in the Import Movies + Restart workflow.

From Scipion's API script

• The script is in a git repository, thus

   git clone https://github.com/I2PC/em-facilities

• All needed code is in the usingAPI_demo directory.

• The launcher is assuming that Scipion is in ~/scipion and this repository at ~/em-facilities, if not, please edit in the first lines of the launcher (usingAPI_demo/script.sh).

• Also, usingAPI_demo/scipionbox.conf contains config variables. The most machine dependent variables are:

  • DEPOSITION_PATH: any directory where the script will simulate the acquisition creating symbolic links.

  • SCIPION_PROJECT: ~/ScipionUserData by default.

  • RAWDATA_SIM: Where the real data is located (20 or 200 movies, see datasets sections above).

  • NUM_CPU: Number of CPUs of the machine.

• Once the configuration is clear, we can launch the Acquisition Simulation by

   sh ~/em-facilities/usingAPI_demo/script.sh

NOTE: That several threads are call by this script, so to Stop it you need either to look for all the process and kill them or close the terminal where it was launch (also a killall python works, but be careful in AWS because vnc use a python script...). To avoid this, we can make a desktop launcher/shortcut calling the same command above. Then when we close the opened terminal, all tasks will be killed.