Remote Access
Contains info on remote access to MX beamlines at the ESRF
2 Prepare your experiment
3 How to get started
4 Troubleshooting
5 Known Issues
Here you will find instructions on what you need on your PCs, how to access the beamlines and what software you need to run. If you require more information please contact Gordon Leonard or Deborah Davison.
When the beam-line is left unattended during a Remote Access experiment please display the yellow remote access signs (these can be found on the beam-line) on the Experimental Hutch and Control Cabin doors.
IMPORTANT: Following recent changes to the A-form that users are required to complete when visiting the ESRF for experiments, please note that ALL users for a given experimental session MUST be declared on the A-form for the experimental session in question. This includes those who will use the beam-line remotely when other members of a BAG Group are at the ESRF. Users who will use the beamlines remotely should check the "remote user" box on the A-Form (see bottom panel below) and give arrival/departure dates as the start/end dates of your experimental session. That remote users are declared on the A-form is a specfic requirement of both the ESRF User Office & the ESRF Saftey Group.
Prerequisites
Software requirements
To perform the remote access, you will need at least one computer with a relatively large screen (at least 17'', optimal 21'', the bigger the better! ) and access to high speed network (at least 512Kb/s, the faster the better !).
To connect to ESRF firewall, you will need the latest version of NXclient from Nomachine on your PCs at home. The client is available in windows, Mac and linux variants. A page has already been written on the configuration you need to access the ESRF computer systems. Log in with your normal user/password. Once connected this provides you with a linux desktop ported to your computer from inside the ESRF firewall. Some ESRF web pages are accessible only from intranet. Using NXclient you can read these pages (eg. safety training)
Please use bet (port 5222) for a better performance !
Known problems:
Timeout on connection to firewall - check that your laboratory firewall is not blocking your access;
If your computer crashes and loose the connection with ESRF, it might be difficult for you to reconnect easily since you will have already a session opened. You can kill your already running session before restarting a fresh one through this link.
Some More Hints on Configuring Your NXClient
As well as configuring NXClient with the correct hostname and port number, it is also important to configure the correct connection type. This should be set as "LAN" - using any other connection type may result in difficulties with image display when using adxv remotely.
Prepare your experiment
Here you will find a useful web page for preparing your experiment. Below are some extra suggestions to make your experiment run more smoothly.
Check List ( under construction)
- Sample Sheets
- Safety Training
- Dewar Sending
- Sample information in ISPYB
- Contact your Local Contact
Please be aware that all administration, especially sample sheets should be completed at least 3 days before the start time of your remote experiment. Failure to do so may result in you being denied access to the experiment.
A special safety training for remote access is now available here. Follow the links to the external web site and log onto the ESRF User portal using your SMIS username and password. On the top menu select safety and then on the left 'Safety Training for remote MX experiments'.
Please make sure that all material is marked with your experiment number - transport cases AND dewars themselves. You will not be there to keep track of your equipment please make it easy for the Local Contact to do so !! Please include a list of puck barcodes in the transport box so that there is a quick way of checking that the right pucks and in the right dewar.
REMEMBER, if you would like your dewar returned to you please provide an address label and Courier information (Name and account number ).
Please use dewar tracking though IspyB. If you are unable to do so, a dewar return form can be downloaded here.
Please fill in the shipment form in ISPYB - this helps sample tracking once your samples are on site.
Contact your LC to make sure that your dewar has arrived and to confirm the start time of the experiment.
You will need to exchange contact telephone numbers so that you can contact each other easily during the experiment.
Here is a page for Local Contacts about beamline preparation for a remote experiment.
How to get started
You will need the following:
- Your experiment number and password
- The name of the beamline and its control computer or the computer dedicated to remote access (For a list of MX computers see this page.)as well as it's password
- Contact details for Local Contact ( including phone number)
- Start time for your experiment
Connecting to the ESRF firewall
Start the NX Client on you computer in your home lab and login with your experiment number and password
Starting automatically the software necessary to perform your experiment
Please use this ONLY during your scheduled time or after agreement with your local contact since it will affect the people working on the beamline at that moment.
Double click on your desktop's icon for the appropriate beamline. This will allow you to open a series of windows necessary to perform your experiment.
(Be patient : don't double click twice on these icons otherwise you might end up having a hard time closing all the windows opened twice...)
Your desktop on TET or BET with the icons pointing to the various beamlines to generate a remote access session looks like that:
After double clinking on one icon, you will be asked several times via pop-up windows, to provide the beamline computer password and your experiment number password :
Please read carefully the message on the window to enter the appropriate password.
In a first phase, a set of windows will open:
- green background : control PC terminal window
- salmon background : processing terminal window
- pink background : Rsync terminal window on coral
- grey background : ADXV (load) terminal window on coral
- firefox instance pointing to the beamline videos
Once all the windows have appeared, type startup on the terminal PC window (green background) and idappli, Mxcube, ADXV (autoload) and a vncviewer to the SC GUIwill open automatically.
Thanks to the Williams BAG for providing the script that we have adapted.
launching the softwares manually:
This section will guide you through each window opened and will show you how to re-open it manually in case of problem.
Connecting to the beam-line computer & starting MxCUBE
At the moment you should log in to the PC used for the remote access (see this page to find the control PC name). To do this type:
ssh -X opidname@controlpcname, eg ssh -X opid14@hpcc12
then at the command prompt type mxCuBE. Once mxCuBE has started log in to mxCuBE as if you were on the beamline ie. Use your experiment number and password.
Once logged in to mxCuBE, the top bar will become yellow and you can then ask for control of the beamline by clicking the "Ask for control" button on the bottom right hand side of the mxCuBE screen. Before you do this, however, it is best to inform your local contact - either by phone, or using the 'chat' - that you are ready to take control of the beam-line. Upon being given control of the beam-line you will be asked to give details including your name, home laboratory name, contact telephone number and e-mail address. These are required by the ESRF Safety Group and will allow the ESRF's Exerimental Hall Operators to contact you in case of problems outside normal local contacting hours. You will also be asked whether users from your group are at the ESRF to help in case of minor problems. Once you've taken control of the beam-line the top bar of mxCuBE will turn green. You should also see in the bottom right-hand corner of mxCuBE who is accessing the beamline. You can give control to other members of the same BAG group. There "chat" tab at the bottom of the mxCuBE window can be used to communicate with your Local Contact.
If for any reason intervention by the Local Contact requires that he/she take control of MxCuBE he/she will inform you of this and a message to this effect will appear. Note that upon losing control of MxCuBE, the top band becomes yellow.
Experimental hutch visualisation
In the NXClient session use a new window to a beam-line data processing cluster (i.e. for ID14-1 ssh -X mx???@proc141; proc142 for ID14-2 etc) and start mozilla/firefox. From here view you can view the beamline web cameras which will allow to see what is happening in the experimental hutch of the beam-line you are accessing remotely. The web addresses are only accessible from inside the ESRF firewall.
Beam-line Web Camera Address
ID14-1 http://id141video1/view/index.shtml
ID14-2 http://id142video1/view/index.shtml
ID14-4 http://id144video1/view/index.shtml
ID23-1 http://id231video1/view/index.shtml
ID23-2 http://id232video1/view/index.shtml
ID29 http://id29video1/view/index.shtml
You can connect these web cams using the ESRF ssl gateway with your experiment account and usual password.
Starting Image visualisation
Connect to the control PC for your experiment: ssh -X opidname@controlpcname
At the command line type: start_adxv_autoload_ra
VERY IMPORTANT: Before running the start_adxv_autoload_ra script remotely MAKE SURE THAT THERE ARE NO OTHER INSTANCES OF ADXV IN AUTOLOAD MODE RUNNING ON THE BEAM-LINE CONTROL PC. Failure to do this will almost certainly cause a log out of the local (i.e. ESRF) users on the BL control PC and the loss of control of the beam-line!
Note that adxv in load mode (just type adxv at the prompt) can be run on any of the beam-line data processing clusters (see above for details of how to access these) without affecting the performance of the BL control PC.
Starting Data Collection
Here is a link to some of things you should know before collecting data on the ESRF MX-beamlines.
By default, the Local Contact will load the user baskets from top to bottom (from transport dewar) in SC position 1 to position 5.
Starting Data processing
You can log in to our clusters (proc141, proc142, proc143, ... , proc29) using "ssh-X" command and process data.
Starting your Data transfer
sFTP : a network protocol that provides file transfer and manipulation functionality over any reliable data stream.
rsync : to keep two directories "synchronised" (i.e identical contents). For example from a beamline to upload data to a remote place :
rsync -a -r -e ssh --progress --stats --compress /data/id23eh1/inhouse/opid231/20070829 user@remote.com:/esrf_data
The command above will connect to remote.com with user "user", and will compare the remote directory /esrf_data/20070829 (if it exists, otherwise it will be created) with the local /data/id23eh1/inhouse/opid231/20070829, then it will copy the differences from ESRF to the remote computer.
Connection is secured through ssh. It shows progress (--progress) and uses compression (--compress), and it considers the files as "archive" (-a). It does the same for all sub-directories (-r).
Users have found the following commands useful:
/usr/bin/rsync -av -r --rsh='ssh -p5022' --progress --stats --compress ix999@firewall.esrf.fr:/data/proposal/ix999/id23eh1/20071109/id14eh1/data/ .
scp -r -P 5022 fx999@firewall.esrf.fr:/data/proposal/fx999/ .
Troubleshooting
Spec Problems
If you are worried about something to do with the spec session, or if the session dies (red motors in mxCuBE) type the relevant stop command for the beamline you are using:
14-1 - stop_eh1
14-2 - stop_eh2
14-4 - stop_eh4
23-1 - stop_eh1
23-2 - stop_eh2
29 - stop_exp
To start spec and visualise it on your PC type the following beamline specific commands:
14-1 - start_eh1_ra
14-2 - start_eh2_ra
14-4 - start_eh4_ra
23-1 - start_eh1_ra
23-2 - start_eh2_ra
29 - start_exp_ra
If you wish to just restart spec on the beamline control pc then type the beginning of the relevant command without the "_ra" part.
Detector Problems
Most detector problems can be solved by opening a terminal and typing start_detector_ra. Then force the dark current image at the start of the data collection. If this fails, it might be worth looking at the detector troubleshooting webpage.
On-axis camera problem
If MxCube has lost the connection with the on-axis camera (the sample image freezes) you should restart the camera device server falconDS. If you are using ID29 there is a specific page here.
Known Issues
- a. Time out when connecting to firewall - problem with your local firewall.
- b. NX sessions keep crashing: run remote beamline control via BET, port 5222.
- c. Authentification failure when trying to connect to firewall. You do not have the correct experiment number or password
- d. adxv_autoload_ra starts for the remote users but remains white. The NXclient configuration needs to be changed to LAN
- e. Data collection on Pilatus takes longer than expected: This is due to video streaming filling the bandwidth. the situation can be dramatically improved by (a) keeping mxCuBE in the Collect Tab during data collection and (b) minimising Firefox applications displaying the feed from the cameras in the experimental hutch.
- f. Connection is very slow: Make sure that you connect to bet (port 5222) and NOT tet. There is no protocol for killing processes on tet so if there are many session performance levels drop.
ID29 Remote access web page
here