Sample Changer Troubleshooting

 

If the following instructions don't help you to solve your problem, call your local contact.

After 10pm, call the floor coordinator (25 25) who may be able to help you.

 

Quick links

 

 

 

Samples are loaded but not exactly centered on the magnet

If the samples are loaded but on the side of the magnet rather than on the center of it, we recommend to :

- perform a RESET of the sample changer (on the Sample changer touch screen or from MXCube as expert user)

- run the macro "minidiff_init" in spec(exp) (or "microdiff_init" in spec(exp) on ID29) which will re-initialize the motors involved by the loading/unloading  procedure

- do a reconfig in spec (exp)

 

Samples are dropped by the SC when unloading

 If the phi axis and the sample charger arm seem to be misaligned, it is possible that some motors have lost steps (specially the centring table motors). In such a case, you can try to run the macro "minidiff_init" in spec(exp) (or "microdiff_init" in spec(exp) on ID29) which will re-initialize the motors involved by the loading/unloading  procedure.

If only one motor seems to be "guilty" (exemple: sampy), you can try "minidiff_init sampy" in the spec(exp) session (or "microdiff_init sampy" in spec(exp) on ID29). Then only sampy will be reinitialize.

Caution: the macro is valid only for sampx, sampy, phi, phix, phiy and phiz

 

Non-fatal Alarms

During the loading/unloading of a sample, a multitude of checks are made by the SC device server. These are carried out to avoid losing samples during the loading/unloading procedure. While generally reliable, these checks can occasionally lead to the reporting of false error messages concerning the operation of the sample changer. Some of these messages should, at the current time, therefore be treated with caution. Below is a list of non-fatal alarms reported by the SC device server, the meaning of the alarm reported (in some cases blindingly obvious!) and the appropriate action to take to remedy the problem.

 

Alarm

Problem

Solution

Note

A sample has been detected on axis Load operation fails because there is already a sample on axis Remove the sample on axis before loading a new one  
Bad device state The electronic rack is in a faulty state Retry If still no success after ~3 tries call your Local Contact
Basket index not detected The camera didn't detected the basket to be in the correct position after the move Retry. Move the basket to a different position. If still no success after ~3 tries call your Local Contact This may be a false alarm.
Basket well open The blue basket port cover has been removed/is not in place Put the blue basket port cover back on!  
- Camera communication timeout
- Camera error
- Camera answer format
Device communication error Retry. If problem persists restart the rack. If persists call the local contact.  
Cap is lost A cap was lost between phi axis and SC during transfer Call Local Contact  
Cap not closed After the operation the cap allowing samples to enter/exit remained open Try then to close it using the sample changer software or the buttons on the “setup” touch screen. Check if there is something blocking the cap. This often happens when the vial is cracked. Remove the vial if this is the case and try to load a new one. If it does not work reset the rack If the cap remains open your samples may be at risk. If no solution found you should  call your Local Contact.
Cap not open The cap allowing samples to enter/exit the SC can't open Call Local Contact  
Cap detected on vial There's a sample on the vial into which you would like to unload a sample Move to a position where there is an empty vial This is sometimes reported falsely. If you are sure there is not a sample in the vial in the position try the operation again. If still no success after ~3 tries call your Local Contact
Cryo didn't move back The cryostream did not move back during transfer. Transfer is aborted and sample or empty vial returned to SC Call Local Contact  
No air pressure   Call Local Contact  
No cap detected on vial There's no cap on the vial in the position from where you are trying to load a sample Move to a position that contains both vial and sample This is sometimes reported falsely. If you are sure there is a sample in position try the operation again. If still no success after ~3 tries call your Local Contact
No sample detected on axis Unload operation fails because there no sample mounted on axis    
No vial was detected No vial in the loading/ unloading position from where you are trying to load/ unload a sample Move to a position where there's a vial This is sometimes reported falsely. If you are sure there is a vial in the position try the operation again. If still no success after ~3 tries call your Local Contact
Phase clock error Internal hardware error Try to restart the control rack. If the problem persists Call Local Contact  
Pmac communication error/Pmac is Offline Device communication error Retry. If problem persists restart the rack. If persists call the local contact.

 
Safety cover open The sample changer safety cover is open Close the safety cover  
Sample lift is not down The sample lift did not move into the correct position after loading/loading There may be ice building up in the SC preventing the sample lift from moving properly. Reset the SC. If this doesn't help call your local contact.  
Sample not loaded - detected in sample changer The vial didn't stay on axis after loading This error often occurs when non-SPINE standard vials are put in the SC or if the sample changer is not well aligned.  
Sample not unloaded - detected on the axis The sample remains on axis after unloading This error often occurs when non-SPINE standard vials are put in the SC or if the sample changer is not well aligned.  
Smart Magnet detection ambiguous – check if the sample is loaded There has been ambiguous information about the sample detection on axis. Check if sample is mounted. If not the sample may be lost. Make sure it is not lying under the SC arm.  
Transfer refused The sample transfer was refused   There are a number of reasons for this error. The solutions for which are described below
Vial lost A vial was lost between phi axis and SC during transfer Check that this really is the case. If it is true remove your cap and vial before any further action.
If it is not and sample has been successfully loaded/unloaded carry on as normal.
If false error message persists call your Local Contact
 

 

"Transfer refused" - what to try

The most common causes of a Transfer refused alarm are that the mini- or mico-diffractometer cannot reach its preset default loading position.

For ID29: points 1 to 3 do not apply.

1) The Sample changer may not have been declared in the SPEC control program. To check whether this is the problem type the following in SPEC and try loading/unloading the sample again.

EXP> SCInUse 1

2) Check the message just above the prompt line in the experimental hutch version of SPEC - this runs on the beam-line control computers in a blue window. After the successful loading/unloading of a sample using the SC this should read "SAMPLE CHANGER LOCKED/ MINIDIFF CAN MOVE"

sclocked-minidiffmove.jpg (SCLocked-MinidiffMove)

If after a Transfer refused alarm is received the message reads "SAMPLE CHANGER CAN LOAD/UNLOAD SAMPLE / MINIDIFF LOCKED" this means that the mini-diffractometer cannot move to its preset loading position and the sample transfer will time-out. To cure this problem simply type the following in SPEC:

 EXP> SCMinidiffGetControl

The mini-diffractometer is now free to move to its loading position and the next transfer, when requested, should proceed normally.

3) If carrying out steps 1 and/or 2) still results in a Transfer refused alarm then type the following in the experimental hutch version of SPEC:

EXP> SCMoveToLoadingPosition

This command moves the mini- or micro-diffractometer to its default loading position and prepares the Sample Changer for loading. If the loading position cannot be reached an error message will appear. In this case please call your Local Contact

4) Ultimately reboot the windows SC control PC. To access it try the following in a new window on the beam-line control computer:

p1-id29~ vncviewer pcsc3-id29 (i.e. name of SC control computer for beam-line)

This will lead to the displaying of the screen of the SC control computer. 

Being a Windows machine, over time memory use may build u with clogged software effectively leading to a crash due to lack of memory.

Close the software and reboot. Login as the normal beamline user (e.g. opid23) and password (it may take a few seconds to login); restart the software using the icon in the centre of the screen. If the sample changer software offers you the possibility to install a more recent version, click "No".

 

5) If the transfer was refused without any evident reason and the SC is not even bringing up the sample lift, you can try to restart the PMAC controller.

 crate-on-off.jpeg (PMAC)

 Press the Red Button, switch off the Control Electronics. Wait few seconds and restart the controller switching on the Control Electronics again and press the Green Button.

 

1)Depress red motors power supply momentary switch

2)Control Electronics selector to off

3)Main Supply and heating rocker switch to off

4)Wait 5 reconds

5)Main Supply and heating rocker switch to on

6)Control Electronics selector to on

7)Green motors power supply momentary switch

 

Fatal Errors

If one of these occurs you MUST call you local contact as an intervention on the SC will almost certainly be required. The most common of this type of errors are 'basket jams' (Basket selector jammed) and 'vial jams' (Vial jammed) the number of which can be minimised, or even abolished by ensuring that only SPINE standard sample holders and vials, are loaded into the SC and that all vials are in good condition. The following describes how to cure vial & basket jams. These procedures should be only carried out either by Local Contacts or (with the appropriate agreement) by experienced users of the SCs.

 

 

Alarm Problem Solution Note
Arm clip error The clip that holds the vial during sample transfer collided with something - Transfer is aborted Check by seeing that no vial prevent the sample changer arm to work correctly. If any remove it and reset SC. Call Local Contact if no vial was found The SC may need realigning
Arm lift error While loading or unloading a sample the transfer arm collided with something Call Local Contact  
Basket selector jammed The carousel that moves the baskets into the correct loading position does not turn properly If you feel comfortable follow these instructions, otherwise call your Local Contact  
LN2 level too low The liquid nitrogen level in the SC is too low Call Local Contact if persistent  
No 24V The electronic rack is switched off

Switch it on

 
Start error Internal hardware error Try to restart the control rack. If the problem persists Call Local Contact  
Vial jammed The vial did not move completely down the chimney during loading/unloading Check that this really is the case. If it is not try the operation again. If still no success after ~3 tries, follow the instructions below or call your Local Contact  

 

Vial jammed

A vial jam means that either an empty vial or a vial containing a pin is stuck in the chimney of the SC, and normally occurs just after a loading or unloading event. When one of these occurs you have to manually push the vial back down into the SC baskets using the following procedure :

1) Go into the experimental hutch and look at the touch screen. As a result of the error, it will now be showing the "Setup General" screen.

Startup-Reset-Screen

2) Press "Cap Closed" and use the menu which appears to open the cap - when you press on this the brass cap covering the hole where the samples are pushed in and out of the SC will slide open.

cap_open_vial

3) Looking down this hole, you should see either the base of a pin with its 2D matrix, or an empty vial (empty vials are more difficult to see!). Take a screwdriver or a pencil and insert it in the hole until you meet a resistance and just wiggle until you feel the vial drop.

4) Close the brass cap by pressing again on the "Cap Closed" button on the touch screen and using the menu which appears to close the cap.

5) Press on Exit on the touch screen and all should now be ready to use again (the status in MxCuBe should automatically change back from red to green once the vial is unjammed and you should not have to Reset in the software).

 

Basket Selector jammed

1) Go into the experimental hutch and look at the touch screen. As a result of the error, it will now be showing the "Setup General" screen.

2) Take the top white cover off by unscrewing the allen screw in the middle of this cover, gently remove it & its associated perspex protective guard, then remove the blue basket port cover.

Cover-Screw

3) Place a screwdriver through the holes in the chimney as shown in the figure below so that you can insert it into the screw end of the basket selector motor (see below).

Screwdriver1

Screwdriver2

4) Turn the screwdriver a couple of turns - try both directions and turn in whichever direction is easiest to move - the baskets should move round as you turn. Warning: while you have to use some force to get the baskets to turn do not overdo things!

5) Once the baskets are moving freely replace the white cover & its associated perspex protective guard and tighten the Allen screw, then replace the blue basket port cover (NO FURTHER OPERATION OF THE SC SHOULD BE CARRIED OUT WITHOUT THESE IN PLACE) .

6) Press the "Exit" button on the touch screen. This should bring you back to the Sample Transfer screen. The status bar in MxCube should automatically change from red to green and you can now try selecting a different basket. Everything should move normally. If this is the case then you will be able to continue working normally. If you still have problems please alert your Local Contact.

 

Communication troubles between SC software and SC

When you get as error on the SC software:

Failed to flush PMAC command buffer.

not connected to PMAC.

please look if they are a pmacserver.exe into the C:\WINDOWS\SYSTEM32 folder. If not, you can copy it from another SC PC.

 

Non Standard Sample Holders and Vials

The SC must ONLY be used with SPINE Standard Sample Holders and Vials (see below) whch are comercially available from Hampton Research (vials, bases & pins in the CrystalCap HT™ range of products) & Marresearch (bases only).

Hampton Research

Molecular Dimensions (MDL)

 

SpineStandards

Non SPINE Standard Sample Holders and Vials

Under no circumstances should non-standard sample holders and vials be introduced into the SCs. Using non SPNE standard vials and/or pins is very likely to block or damage the Sample Changer and could result in the loss of crystals and/or beam-time. Examples of the types of vials and sample holder that should NOT be used in conjunction with the SCs are shown below. Please also note that the SCs are not compatible with the 'oxford top-hat system'

HamptonVialHamptonVialSpinePin

Vials in bad condition

Using vials in bad condition is also very likely to block or damage the Sample Changer and could result in the loss of crystals and/or beam-time. It is thus important, before using the Sample Changer, to ensure that all vials are in good condition, i.e. that they are not broken or cracked. Examples of vials in bad condition that should NOT be used in conjunction with the SCs are shown below.

badvial1badvial2