Advanced data collection workflows

A new workflow tool has been developed to allow complex experiments to be performed in a user friendly manner through the standard beamline GUI MXCuBE. A new "Advanced" tab can be found in the collection methods tab of MXCuBE 2.0.  These workflows provide various experimental tools, which can be selected and added to the queue like ordinary data collections.

Users then interact with the experiment through dynamic content in a new tab (next to or "behind" the centring tab).  In this way novel experiment workflows can be seamlessly integrated into MXCuBE. 


The individual workflows are described below.

Jump directly to a specific workflow:

  1. Visual reorientation

  2. Kappa reorientation

  3. Dehydration

  4. Line and mesh scans

  5. X-ray centring

  6. Automated data collection

  7. Enhanced EDNA characterisation

  8. Burn strategy

  9. Troubleshooting workflow


Workflows are developed in DAWN (Data Analysis WorkbeNch), an eclipse-based workbench for scientific data analysis, which is a collaborative effort between the European Synchrotron Radiation Facility, the European Molecular Biology Laboratory and the Diamond Light Source.

The use of workflows in the implementation of complex MX experiments has been described in Brockhauser et. al. (2012) Acta Cryst. D68, 975-984, please cite this article if you use workflows for your experiments.



Visual reorientation

The visual reorientation workflow is a fast way of aligning a crystal paralell to the spindle, by reorienting a sample along two selected positions.

After launching the workflow, select and save two centred positions in the centring tab. The minikappa goniometer will then align these two points onto the rotation (phi) axis.




Kappa reorientation

The use of Kappa goniometers for crystal reorientations can be useful in a number of circumstances in MX (Brockhauser et al., 2011). These include the case where Bijvoet pairs of reflections (a reflection and the Friedel pair of its symmetry equivalent, e.g. hkl and hk¯l) can be measured on the same diffraction image by properly aligning an even-fold symmetry axis along the spindle. Hence, anomalous differences can be measured at the same time and radiation damage induced non-isomorphism within these Bijvoet pairs can be minimized, resulting in more accurate measurements of the anomalous differences. Aligning a specific symmetry axis can result in collecting a complete dataset within a reduced rotation range (Dauter, 1999) so that the total dose can be lowered leading to less severe radiation damage.  Another example of an advantageous crystal reorientation is the alignment of the densest axis in reciprocal space, usually corresponding to the longest unit cell axis. By aligning this axis parallel to the spindle, the overlap of spots can be minimized.


A workflow has been implemented in MXCuBE using a DAWN workflow to guide the user through the steps required to reorient a crystal. This is a three step, iterative protocol which contains the initial characterisation of the sample; the calculation of a set of preferred orientations; and testing the diffraction quality and predicting data collection statistics at different orientations until a satisfactory result is achieved.

The workflow provides a menu allowing to chose from the following re-orientation strategies:


  • Cell:   aligns a reciprocal unit cell axis along the spindle.
    (Useful for verifying crystal symmetries as well as investigating spot overlaps.)
  • Anomalous:   aligns an even-fold symmetry axis along the spindle to measure the anomalous signal between the Bijvoet pairs on the same images.
  • Smart Spot Separation:   maximises spot separation while maintaining the highest possible completeness by avoiding the blind zones during data collection. This is achieved by a slight mis-alignment of the longest unit cell edge of the crystal when approaching the optimal orientation. 
  • Smallest Overall Oscillation:   reorients the crystal such that collection of a single sweep complete dataset requires the minimum overall oscillation.


The workflow is started by selecting "Kappa Reorientation" in the "Advanced" tab and adding it to the queue.

  • A menu is presented where the user selects the reorientation option (as explained above).
  • The user then selects the characterisation required by EDNA, i.e. number of images, angle between those images and the exposure time. 
  • An EDNA characterisation is launched and a proposed reorientation of the crystal is presented. 
  • After accepting the reorientation the crystal is moved and the user must validate the centred position or re-centre the crystal.
  • Another EDNA characterisation is launched.
  • If the orientation is optimal for the requirements a data collection strategy is proposed, and if a better orientation was found a new reorientation is proposed.
  • The final strategy can be edited, on clicking "continue" the strategy is sent to the queue and collected.




The dehydration workflow allows the design of dehydration protocols that are then coupled to data collection and on-line data analysis through MXCuBE.

The dehydration of crystals of macromolecules has long been known to have the potential to increase their diffraction quality.  A number of methods exist to change the relative humidity that surrounds crystals, but for reproducible results, with complete characterisation of the changes induced, a precise humidity control device coupled with an X-ray source is required. The EMBL Grenoble outstation has developed a humidity control device, the HC1, that is available on the ESRF MX beamlines (Sanchez-Weatherby et al. (2009) Acta Cryst. D65, 1237-1246, Russi et al. (2011) J. Struct. Biol. 175, 236-243), and many systems have shown major improvements in diffraction quality using the device.

For details on how to perform these experiments please see here. Contact Matthew Bowler for help.


The workflow is started by selecting the "Dehydration" workflow in the "Advanced" tab and adding it to the queue. The user then selects the starting relative humidity (this can be determined from these equations) and a dehydration protocol is defined. The key factors are the step decrease in RH and the equilibration time between each step.  We generally recommend for an initial experiment a step size of 0.5-1% RH and an equilibration time of 5 minutes.  Once changes have been characterised these parameters must be adapted to the system under study.




Once the gradient has been defined, the process is launched.

  • First an image is collected. This is important as the initial diffraction quality of each crystal must be assessed at the beginning of an experiment.
  • The gradient is launched and the user is given the options to continue automatically (recommended to prevent clicks for each dehydration step) and whether to collect data or not.
    (This option is useful when a protocol is known but radiation damage needs to be avoided, see below.)
  • At the end of the gradient these options will again be presented, allowing users to change to a new RH and to collect data or not. 
  • This allows multiple rounds of dehydration with or without data collection, and can include gradients that increase the RH as well. 
  • Graphs can be viewed by double-clicking on a queue entry:




Images and the results of processing are stored in a sub-directory called ".../dehydration_###". 

Graphs are produced at each step and are updated as new results are obtained:


HC1_workflow_example1_Cell2.png   HC1_workflow_example2_Labelit6.png


A video of the process:




Line Scan

This workflow scans along one dimension through the beam and moves to the position with the highest diffraction intensity.

Crystals are often difficult to visualise in one particular direction. The line scan workflow can be used to scan through this orientation to locate the crystal.

The line scan workflow is combined with a mesh scan in the X-ray centring workflow in order to centre the optimal diffraction volume (i.e., in three dimensions) to the beam.


Mesh Scan

The mesh scan workflow moves the sample through a user-defined mesh and moves to the position with the highest diffraction intensity.

Locating (a) small crystals on a large support or (b) finding the optimal position within a large crystal can be done with the mesh scan workflow. The support is moved through the beam according to a user defined mesh, and then moved to the position with the highest intensity.

The mesh scan workflow is combined with a line scan in the X-ray centring workflow in order to centre the optimal diffraction volume (i.e., in three dimensions) to the beam.



X-ray Centring: Finding the best diffraction volume

The large multi-component complexes and membrane proteins that are now routinely studied in structural biology tend to produce either very small crystals or crystals that can be extremely heterogeneous in their diffraction properties. In order to locate very small crystals or the optimum region of a crystal larger than the X-ray beam, mesh scans have been developed to collect images at numerous points specified within a grid. 

This workflow scans the largest face (xxx really?? xxx) of a loop using a mesh scan, and then performs a line scan 90 degrees away on the best area of diffraction determined from the first scan.  The best volume is then centred onto the X-ray beam, ready for final characterisation and data collection.


As the routine is based on only 2 orientations is is essential that the loop/pin is centred well before starting the workflow. If the centre of rotation is not known, the centring will not work. Practically this means that the end of the pin should be centred and not the centre of the loop.  --- Huh??? xxxx


Contact Matthew Bowler for help.


The workflow is started by selecting the "X-ray Centring" workflow in the "Advanced" tab and adding it to the queue.


Then a mesh over the area of interest must be defined: use the "Grab" tool (above the sample video) and specify the number of vertical and horizontal steps:




  • The mesh can be drawn before or after starting the workflow.
  • All images have a "mesh-" prefix and are stored in a subdirectory of the current working directory called "mesh###".
  • Images are collected at each of the intersecting points of the mesh with run number 1 for the first mesh and 2 for the second dimension 90 degrees away.
  • The next step is to define exposure time and image prefix:




After the data collection has finished, a map showing the results of analysing the images is produced:




  • The workflow will then rotate the crystal 90 degrees away from the current position and move the crystal to the best position determined from the first orientation. 
  • A line scan is then proposed - here the number of steps and distance can be altered from the default (the default is the distance determined from the shadow of the loop divided by the beamsize). 
  • By clicking on "continue" the collection is loaded to the queue.
  • Analysis is performed and a second map is generated.
  • The crystal is then automatically centred to the optimum diffraction volume. 
  • You can now continue collecting data.




Video How-To






Example 1: Locating and centring a 20 um needle to a 5 um beam (ID23-2). 
After the workflow centring procedure, a full data set was collected and the structure was solved.



Example 2: The workflow can also be used to locate and centre small crystals.
In this case the best 10 um crystal was centred to a 10 um beam:







Troubleshooting workflow

The troubleshooting workflow checks various functions for any problems and suggest potential solutions.

If any of the workflows are failing, the troubleshooting workflow should be launched to find a solution.

If all tests are OK,  an information message will be displayed (see below). If the workflow detects errors, remedies will be suggested.





Automated data collection


... to be done ...



Burn strategy

The burn strategy workflow allows the measurement of a crystal's susceptibility to radiation damage by directly measuring it, sacrificing one crystal (or a portion of it). The method has been described in Leal et al. (2011) J. Synch. Rad. 18, 381-386, please cite this article if you use this workflow.

When the burn strategy is selected, the crystal is first characterised. A series of burn runs are then added to the queue and collected. The analysis of images collected after the burn runs will be used to calculate the crystal's sensitivity to radiation damage.


Video How-To




Enhanced EDNA characterisation

EDNA (Incardona et al., 2009) had the initial goal of automating the sequence of taking reference images, characterising these images and calculating an optimised data collection strategy taking into account user requirements and radiation damage. The short-coming of the current implementation is that if reference images are not optimally collected (for example, using an incorrect exposure time, oscillation width and/or detector resolution) it is difficult, if not impossible, to calculate the optimal data collection strategy. EDNA/BEST can give advice on the optimal detector resolution (Popov & Bourenkov, 2003). However, the user has to manually follow suggested values and restart the collect and characterise pipeline.

In this workflow the characterisation pipeline has been enhanced by simply adding the optional step of automatically re-collecting reference images with a different exposure time or oscillation value and detector resolution. If the workflow is run in interactive mode the user can easily intervene and interrupt the workflow at certain stages as well as change the proposed new data collection parameters through prompts displayed in MXCuBE, if needed.


The workflow is started by selecting the "Enhanced EDNA" workflow in the "Advanced" tab and adding it to the queue.

  • Chose the number of images and the complexity of strategy, and the image prefix and directory.
  • By clicking on "Continue", two reference images are collected.
  • If the current parameters are correct for the calculation of a strategy, the strategy will be copied to the queue as with a normal EDNA run.
  • However, if the resolution obtainable is higher or other parameters of the strategy can be enhanced a new set of reference images will be collected with the beamline parameters automatically updated (i.e the detector will be moved to a higher resolution etc.)
  • A new strategy will then be calculated and copied to the queue.