Processing 2d images from the CCD camera
bm2img program allows some preprocessing of the image. Simultaneously, the counters information can be extracted by the perl script wspec2xl.pl. In case of the direct illumination CCD refer to wv2ph or edf2ph programs. The following links document on some processing problems or procedures.
These programs can be downloaded using ftp://ftp.grenoble.cnrs.fr/xnd/BM2 . In this directory, you will find an archive with program sources and binary executables LINUX.
- The following lines are an example of bm2img use but its parameters may slightly differ from the current release in use on BM2 beamline.
- Converting a winview CCD image (.spe) to a 32 bit EsrfDataFormat (.edf)
- Dark current correction
- Flat field correction
- Outputs
- bm2img input parameters
- Processing series of data
- Radial distribution for SAXS image
- Normalisation using spec files
Procedure
All data are stored internally as 32 bits signed integer, this choice has been done in order to reach a good dynamic without losing the counting accuracy. When the image are read they are corrected from dark current (if a dark image has been set) and from flatfield (if set). Then various operation can be done on the 32 bits image. When writing an output file, the dynamic is only modified on the resulting file not on the inernel image. If you rescale the image, remember that integer operations are order dependent.
Converting a winview CCD image (.spe) to a 32 bit EsrfDataFormat (.edf).
The following command transforms the winview file dani10.spe into the 32 bit file dani10.edf
bm2img dani10 %X
Dark current correction
The following command transforms the winview file dani10.spe into a 32bit file corrected for dark current... dani10d.edffile.
bm2img D=dark dani10 %X
The following similar command applies to a preprocessed dark edf file that is the sum of 15 frames.
bm2img D=dark.edf,U15,X dani10 %X
A reduced view of a SAXS image can be seen below. In the central part it shows a small angle scattering image using 2*2 binning. To show the influence of the correction on 8 bit .gif images, intensities higher than 3 times the image average have been truncated; this occurs near the beamstop and/or on cosmic rays. When an images has been stored in edf format, it is alaways assumed that this image is made by only one frame, then it is mandatory to use the U parameter to correct this if the dark file contains more than one frame.
Flat field correction
The following images show the transformation of the data. In the central part they show a small angle scattering image using the same binning as for the above flatfield. To display the influence of the flatfield correction on 8 bit .gif images, intensities higher than 3 times the image mean have been truncated : this occurs near the beamstop and/or on cosmic ray spots, that have not be fully cleaned.
bm2img D=dark F=flat.edf,X dani10 %X
| dark corrected | dark and flatfield corected |
| bm2img D=dark dani10 %X | bm2img D=dark F=flat.edf,X dani10 %X |
 |
 |
Outputs
The output name is by default
name(d)(f)(_x).$EXTENSION
( ) is added to the name,
"d", if the dark correction has been made
"f", if the flat correction has been made
"_x", if the program is runned a second time, for instance with a modified bm2img.ini.$EXTENSION indicates the format of the file.
| command | %X | %Y | %Z |
| dynamic | 32 bits | 16 bits | 8 bits |
| default | name.edf | ||
| .edf | name.edf | ||
| .gel | name.gel | ||
| .tif | name.tif | ||
| .txt | name.txt | ||
Most of the viewers can handle 8-bits tiff, 16-bits gel (or tiff) is also a standard format. Edf file can be read by fit2d or viewFile.
The txt output can be used to extract some binded rows or columns of the images, it does not depend on the selected dynamic. Obviously trying to output a full image using this mode may produces some disk full error!!!
bm2img default parameters
The online help
The command bm2img -h will display its principal key words. The bm2img processing reads images and acts when a % command is encountered.usage : bm2img [D=|F=]img_file[,Bb,Ee,X,Aa|Mm,Uu] img_in... [%M=val] [%[X] [img_out...]]
By default img_file are of ".spe" type, but to force
to read an edf format, type img_file.edf
D=dark_file to substract (must be defined before image),
F=flatfield_file (must be defined before image)
,B option to begin at frame b
and
,E to end at frame e,
,X don't correct
,A add a files with increasing
number (or Merge them, DMI specific)
,U force used frames to u, i.e.
takes for instance a single edf image to a *u multiframe (mandatory when
dark and data have not the same frame numbers)
example: D=dark3,B2,A5 --> adds frame starting at 2 in dark_files dark3 to dark8
Detail recall of the commands
%M=val (%A=val) multiply current image by val (or add to) before reading next arguments
%M=val/SPEC, divide by the monitoring regitered in the dataspec file defined in the bm2img.inifile.
%C=vah[,val] cut counts higher (lower) than vah*mean before reading next arguments
%S=val rescalling dynamics to val
%X[[name].edf|.gel|.tif|.txt] produce outfile (edf default with 32bits)
%Y[[name].edf|.gel|.tif|.txt] produce 16bits outfile (gel default) rescalling the output if mandatory
%Z[[name].edf|.gel|.tif|.txt] produce 8bits outfile (tif default) rescalling the output if mandatory
%R produce radial distribution file (.rad), one angular sector is the default
%P[=m[,o],v]]] produce peak list file (.pks), m=5 peaks is the default, omega osc if not 0, sum>v*max,
v=1.5
%H produce an ascii map (h, k, l) of the file using the UB matrix
%F clean current image (but keeps dark and flat)Inside the program, the images are kept as integer with 32 bits
These parameters and others may be defined in bm2img.ini file
Valid options in bm2img.ini files are
They are recalled when running bm2img: MIND! a typing error is equivalent to skipping the command, with unattended results!
DIR = full path to search for data file
OUT = full path for result (.res) file
DEF = default file extension (if not default for the prg_name : spe2img, edf2img, gel2img)
CR = 0, set to 1 if you wish cr in rad file
DFORMAT = size of ineger field in output name, default same as entry
STDIN = full path for redirecting stdin
FLAT_NAME = full path of the flatfield file to use
DARK_NAME = full path of the dark file to use
SCA_DARK = scale factor to apply on dark
ADD_DARK = offset to add to dark curent
REALTIME = 0 or waiting time in secs to ensure data completion
PRINT = ... +64 cosmics, +128 cor, +256 sum, +512 raw
T_DATA = output mode (float...) for edf file Cosmic corrections:
O_COSMIC = file_name, STDOUT or default to image_name.cosmic
T_COSMIC = used for time removal (usually set to 3)
S_COSMIC = used for space removal (3)
R_COSMIC = ratio on the standard deviation used for time and
space removal (1.3)
H_COSMIC = lower cosmics on one frame for time removal (40)
P_COSMIC = lower cosmics on the frame sum for spacial removal
(40)
frame settings:
X_MIN =
Y_MIN =
X_MAX =
Y_MAX =
X_BIND =
Y_BIND =
BEG_FRAME =
END_FRAME =
MULT =
P_EXCLUDED = x1 y1 [v]
C_EXCLUDED = x1 y1 r [v]
T_EXCLUDED = x1 y1 x2 y2 x3 y3 [v]
R_EXCLUDED = x1 y1 x2 y2 [v]
T_EXCLUDED = x1 y1 x2 y2 x3 y3 [v]
C_ROI = x1 y1 r [v]
R_ROI = x1 y1 x2 y2 [v]
by default the [v] value is set to zero
ROI, Region Of Interest, excludes pixel outside the defined
Circle/Rectangle.
ORIG = pixel_x(=0) pixel_y(=0) center of detector
PLANE = a x+ b y+ c z= d e ecart
monitor normalisation
SPEC=/mat_dsk1/visit/name/dataspec/fourc.26Oct01 full path to
the corresponding dataspec file
WSPEC2XL=wspec2xl.pl -M
XL_COL=n, where n is the column number on output of the
wspec2xl.pl
script.
following group only concerns radial distribution
RADIUS = radius binning
RAD_LIM = low limit value in radial distribution (not set
is no limit)
N_SECTOR = number of angular sectors (num 0 to N_SECTOR-1)
A_SECTOR = origin (degree) of the first sector
P_SECTOR = if not empty, binning specifications without
space radial distribution
example, N=8,A=-22.5,P=0+4,2+6,1+3+5+7 print 3 curves
Processing series of data
The first way to process a series of data is to use a script shell but it is more efficient to use bm2img internal redirection if all the images use the same dark and flat.
Use STDIN=inputfile in the bm2img.ini file and fill this file with all the order you need. Do not forget to clean the memory (%F) between each image to process or they will be added together. The following lines show an example of inputfile.D=v2zr51.edf,U5,X img39 %X %R %F img40 %X %R %F img41 ...In case of numerous frames for the data, the dark and the flatfield files, it is time saving to perform cosmic and hotpixel removal : bm2img adds all frames and records a single name.edf image.
The program edf2img can be triggered with the associated edf2img.ini command file, with the same options as defined previously
edf2img D=dark,X F=flat,X dani10 %R
will correct dani10.edf from dark.edf and flat.edf
without
cosmic research, perform the radial distribution and record it in the file
dani10df.rad.
Radial intensity distribution for SAXS images
In SAXS experiments, it is not always necessary to write a corrected image in a file. You can obtain the radial distribution directly using the following command. In case of anisotropy, more than one angular sector can be specified.Then the following line can be used in many cases, this line can be included in a shell script for processing a lot of images when processing conditions are defined.
bm2img D=dark F=flat.edf,X dani10 %R
An ascii output file is created with the extension ".rad"
specific parameter in the bm2img.ini file
| key | expected | default | |
| ORIG= | x y | CCD center | float values to define the center of the image |
|---|---|---|---|
| N_SECTOR= | n | 1 | number of sectors : angular resolution for anisotropic images
(one sector is obviously isotropic) |
| A_SECTOR= | n | 0 | if anisotropic, angular origin of the first sector
(the origin is the lower side of the sector related to the X axis) |
| P_SECTOR= | string | empty | print statement for binning sectors, see examples below |
| RADIUS= | n | 1 | binding radius parameter |
| RAD_LIM= | f | -Inf | allow to avoid Zero or Negative values in radial distribution
(values below rad_lim are set to rad_lim |
Default output file
If P_SECTOR is not found, the radial distrubution file looks like:# ./00jan147df.rad (400-900)/1*(400-800)/1 (00jan147df.gel) # center 545.90 626.80 -> 145.90 226.80, r_max 420.51 # 16 sectors with origine at 0.00 deg # r s/n e(s/n) n ... 0 0.00 -1.00 0.0 19.75 -1.00 0.1 0.00 -1.00 0.0 0.00 -1.00 0.0 0.00 -1.00 0.0 86.57 -1.00 0.3 0.00 -1.00 0.0 0.00 -1.00 0.0 0.00 -1.00 0.0 0.00 -1.00 0.0 120.00 -1.00 0.5 0.00 -1.00 0.0 0.00 -1.00 0.0 0.00 -1.00 0.0 49.61 -1.00 0.2 0.00 -1.00 0.0 1 47.73 -1.00 0.2 233.25 -1.00 0.9 62.13 -1.00 0.3 0.00 -1.00 0.0 94.52 -1.00 0.4 176.43 -1.00 0.7 142.05 -1.00 0.6 0.00 -1.00 0.0 164.44 -1.00 0.6 0.00 -1.00 0.0 141.03 -1.00 0.6 141.49 -1.00 0.6 0.00 -1.00 0.0 109.12 -1.00 0.4 206.39 -1.00 0.8 64.66 -1.00 0.3 2 243.13 -1.00 1.0 182.34 -1.00 0.7 182.87 -1.00 0.7 85.77 -1.00 0.3 262.33 -1.00 1.0 246.62 -1.00 1.0 157.70 -1.00 0.6 169.46 -1.00 0.6 254.40 -1.00 1.0 80.24 -1.00 0.3 241.18 -1.00 1.2 255.89 -1.00 1.0 140.50 -1.00 0.5 156.90 -1.00 0.6 203.42 -1.00 0.8 261.12 -1.00 1.0 3 253.68 -1.00 1.0 268.60 -1.00 1.4 231.71 -1.00 0.9 277.05 -1.00 1.0 271.35 -1.00 1.2 255.20 -1.00 1.5 246.90 -1.00 1.1 254.32 -1.00 1.0 260.36 -1.00 1.0 259.13 -1.00 1.1 258.15 -1.00 1.4 273.01 -1.00 1.4 282.03 -1.00 1.0 260.58 -1.00 1.1 267.47 -1.00 1.5 277.58 -1.00 1.1 4 279.85 -1.00 1.0 280.01 4.29 2.3 293.07 -1.00 1.6 281.30 -1.00 1.1 265.20 4.30 2.1 249.44 -1.00 1.3 257.70 -1.00 1.5 244.29 -1.00 1.4 255.63 -1.00 1.5 254.01 -1.00 1.6 270.05 -1.00 1.5 276.85 9.92 2.1 281.49 -1.00 1.3 294.67 -1.00 1.5 286.19 -1.00 1.2 271.42 5.35 2.1 5 289.64 5.37 2.1 304.81 16.00 2.1 300.79 -1.00 1.3 302.34 13.46 2.1 287.04 27.20 2.0 267.82 17.94 2.4 251.79 -1.00 1.7 268.03 18.86 2.1 242.62 20.99 2.0 259.82 -1.00 1.7 279.27 11.21 2.4 288.04 20.69 2.0 301.69 22.29 2.1 318.00 -1.00 1.4 288.49 6.81 2.3 274.35 9.72 2.0 6 297.29 13.17 2.0 315.34 9.24 2.9 333.89 10.67 2.3 336.10 24.43 2.0 329.26 13.30 3.0 294.24 20.22 2.0 249.00 1.66 2.1 260.46 15.27 2.2 252.05 14.14 2.3 263.75 14.57 2.1 284.13 11.89 3.0 314.98 5.94 2.2 355.74 36.16 2.0 338.35 20.64 2.4 320.42 13.93 2.8 295.34 36.50 2.0 7 333.05 26.76 2.8 331.37 31.35 2.7 372.27 27.82 2.1 378.13 14.81 2.9 352.68 8.67 3.1 325.46 23.81 2.8 257.67 8.26 2.3 258.64 10.90 3.1 257.92 7.65 3.1 294.31 15.25 2.3 316.24 15.00 2.9 363.37 34.69 3.1 401.71 28.77 2.2 390.97 31.50 3.0 356.85 36.73 2.7 362.83 24.98 2.9in which
| r | stand for the radius of the ring |
|---|---|
| s | is the sector number |
| s/n | is the counting per pixel |
| e(s/n) | is the standard deviation on the counting |
| n | is the pixel number taken in account in the ring |
Output file with binned sectors
The P_SECTOR string is interpreted to join the sectors in the output file. The string format consists in the number of the sectors and '+' or ',' signs, space are not allowed. The following example is consistent with sectors of 45deg, the first group is made by two sectors centered on the horizontal axis, the second on sectors from vertical axis and a third consists of the other sectors. Only the specified sectors are printed.# ./00jan147df.rad (400-900)/1*(400-800)/1 (00jan147df.gel) # center 545.90 626.80 -> 145.90 226.80, r_max 420.51 # 16 sectors with origine at 0.00 deg # print sectors binned according : '0+8,4+12' # 4 used sectors (of 16) are binned in 2 group : 0 ( 0 8) , 1 ( 4 12) # r s/n e(s/n) n ... 0 0.00 -1.00 0.0 0.00 -1.00 0.0 1 212.17 -1.00 0.8 94.52 -1.00 0.4 2 248.74 5.80 2.1 268.51 -1.00 1.5 3 257.01 9.49 2.0 276.33 6.17 2.2 4 265.64 13.20 2.5 271.52 7.71 3.3 5 266.17 16.05 4.1 294.42 15.03 4.1 6 273.35 14.83 4.3 340.04 15.23 5.0
Normalisation using spec files
The %M command calls the script wspec2xl.pl to do it, but this argument may be changed. Using the following lines in bm2img.iniSPEC=/mat_dsk1/visit/xx...xx/fourc.26Oct01 WSPEC2XL=wspec2xl.pl -M XL_COL=5the command bm2img alpdre33.edf %M=1e7/SPEC does...
# command : %M=1e7/SPEC spec2xl 1e7/SPEC running wspec2xl.pl -Malpdre33 /mat_dsk2/lcontact/berar/Spec/fourc.26Oct01 >bm2img.SPEC using 1e+07/1.404e+07=0.712 as coefficient # alpdre33.edf : 1 frame(s), mean 16742.1 sigma 60758.2 [0@0*0 - 466609@13*58] 5220!=0It is obvious that if you wish to keep some statistical meaning, the efficient coeffficient must be of some order of magnitude (100 by example) as data are integers inside bm2img. If you wish to avoid scanning the specfile at each image, you can process like the following:
- first run
wspec2xl.pl -malpdre /mat_dsk2/lcontact/berar/Spec/fourc.26Oct01 >summary.file
SPEC=summary.file WSPEC2XL=grep " * " XL_COL=5
# command : %M=1e7/SPEC
spec2xl 1e7/SPEC
running grep " alpdre33 " summary.file >bm2img.SPEC
using 1e+07/1.404e+07=0.712 as coefficient
# alpdre33.edf : 1 frame(s), mean 16742.1 sigma 60758.2 [0@0*0 - 466609@13*58] 5220!=0
However, after dark correction, data of this frame can be compared with data of other frames. Then numerous experiments (small angle scattering, crystal oscillation) can be performed with only one single frame.
Updated by jpsimon on November 29, 2001
A hydrogen reservoir made of … mostly hydrogen
Inauguration of Refurbished ROBL beamline
Debian for Scientific Facilities Days