SExtractor methods

(updated: 4/6/06)

Sextractor is a program used to identify and catalog objects in an image.
There are certain input parameters experimented with and the results shown

INDEX:
    default.sex
    applied
    masking bad regions
        flag      
        mihos's
    messing w/ parameters
       BACK_SIZE
       BACK_FILTERSIZE
       DETECT_THRESH / ANALYSIS_THRESH
       PHOT_AUTOPARAMS: Kron_fact & min_radius
    the Just Get It Done approach
       SExtractor masks + bright star masks = something useful

    attempts to reproduce negative-noise-spike detection methods


Example of default file:
default.sex [parameters which will be frequently adjusted are in green]

Catalog	------------	------------
CATALOG_NAME	test.cat	name of the output catalog
CATALOG_TYPE	ASCII_HEAD	"NONE","ASCII_HEAD","ASCII","FITS_1.0" or "FITS_LDAC"
PARAMETERS_NAME	default.param	name of the file containing catalog contents
Extraction	------------	------------
DETECT_TYPE	CCD	"CCD" or "PHOTO"
FLAG_IMAGE	flag.fits	filename for an input FLAG-image NOTE: this is similar to the Bad-Pixel-Mask of OBJMASK
DETECT_MINAREA	5	minimum number of pixels above threshold
DETECT_THRESH	1.5	<sigmas> or <threshold>,<ZP> in mag.arcsec-2
ANALYSIS_THRESH	1.5	<sigmas> or <threshold>,<ZP> in mag.arcsec-2
FILTER	Y	apply filter for detection ("Y" or "N")?
FILTER_NAME	default.conv	name of the file containing the filter
DEBLEND_NTHRESH	32	Number of deblending sub-thresholds
DEBLEND_MINCONT	.005	Minimum contrast parameter for deblending
CLEAN	Y	Clean spurious detections? (Y or N)?
CLEAN_PARAM	1.0	Cleaning efficiency
MASK_TYPE	CORRECT	type of detection MASKing: can be one of "NONE", "BLANK" or "CORRECT"
Photometry	------------	------------
PHOT_APERTURES	5	MAG_APER aperture diameter(s) in pixels
PHOT_AUTOPARAMS	2.5, 3.5	MAG_AUTO parameters: <Kron_fact>,<min_radius>
PHOT_PETROPARAMS	2.0, 3.5	MAG_PETRO parameters: <Petrosian_fact>, <min_radius>
SATUR_LEVEL	50000.0	level (in ADUs) at which arises saturation
MAG_ZEROPOINT	0.0	magnitude zero-point
MAG_GAMMA	4.0	gamma of emulsion (for photographic scans)
GAIN	0.0	detector gain in e-/ADU
PIXEL_SCALE	1.0	size of pixel in arcsec (0=use FITS WCS info)
Star/Galaxy Separation	------------	------------
SEEING_FWHM	1.2	stellar FWHM in arcsec
STARNNW_NAME	default.nnw	Neural-Network_Weight table filename
Background	------------	------------
BACK_SIZE	65	Background mesh: <size> or <width>,<height>
BACK_FILTERSIZE	3	Background filter: <size> or <width>,<height>
BACKPHOTO_TYPE	GLOBAL	can be "GLOBAL" or "LOCAL"
Check Image	------------	------------
CHECKIMAGE_TYPE	APERTURES, BACKGROUND	can be one of "NONE", "BACKGROUND", "MINIBACKGROUND", "-BACKGROUND", "OBJECTS", "-OBJECTS", "SEGMENTATION", "APERTURES", or "FILTERED"
CHECKIMAGE_NAME	aper.fits, back.fits	Filename for the check-image
Memory (change with caution!)	------------	------------
MEMORY_OBJSTACK	3000	number of objects in stack
MEMORY_PIXSTACK	300000	number of pixels in stack
MEMORY_BUFSIZE	1024	number of lines in buffer
Miscellaneous	------------	------------
VERBOSE_TYPE	NORMAL	can be "QUIET", "NORMAL" or "FULL"

Using this list of default parameters, the following CHECKIMAGEs are generated:

Original Image: CoreRS2.fits	SExtracted Image:
Close up on upper left corner: Original	SExtracted:

[clicking on either of the Close Up images will link to a page where you can toggle between them to better see the object selection]

These results aren't too bad (for doing nothing but using the defaults), but many improvements can be made:


Masking out bad regions

First, the default.sex settings are modified only to include a FLAG_IMAGE of the manually masked out bad regions {from previous page: Bad Regions}.

something w/ the 'FLAG_IMAGE' parameter here...

[to do]



alternatively...


Mihos's suggested method:
IRAF: imexpr
    expr    a = 0 ? 1 : 0
    out      weight.fits
    a         RMCore.pl   [manual bad regions as a .pl mask]
SExtractor
    in default.sex:
       WEIGHT_TYPE      MAP_WEIGHT
       WEIGHT_IMAGE   weight.fits
       INTERP_TYPE        VAR_ONLY
This generates:

SExtracted with default parameters: Full image	SExtracted using above method: Full image
Defaults: close up on upper left corner	Mihos: close up on upper left corner

[again, click on Close Up images for larger/alternating view]

These results look initially good, as it appears SExtractor is ignoring the requested regions, but there is more going on.
It is not just the selected regions that get affected, but other regions too.
This might be a problem, as removing the bright stars with this methods results in an increase in other objects detected.


Experimentation with various parameters:
All parameters except the one in question were held constant, while the specifed one was varied.

BACK_SIZE

BACK_SIZE contributes to the background map by using "a bi-cubic-spline interpolation over all the areas of size BACK_SIZE."
SExtrator for Dummies recommends setting BACK_SIZE larger than the average size of the objects in your image.

[clicking on any image will link to a page that allows cycling through them, from 4 to 900,000]

BACK_SIZE = 4	BACK_SIZE = 8	BACK_SIZE = 32
BACK_SIZE = 64 (default)	BACK_SIZE = 128	BACK_SIZE = 1024
BACK_SIZE = 900,000
back to main Virgo page

BACK_FILTERSIZE

BACK_FILTERSIZE is used before BACK_SIZE is. It is used in the smoothing that goes on "to get rid of the deviations resulting from bright or extended objects."

NOTE: since the input image to all of these is already smoothed, the effects of changing the value of BACK_FILTERSIZE are limited.

[clicking on any image will link to a page that allows cycling through them, from 1 to 10]

BACK_FILTERSIZE = 1	BACK_FILTERSIZE = 2	BACK_FILTERSIZE = 3 [default]
BACK_FILTERSIZE = 4	BACK_FILTERSIZE = 10

DETECT_THRESH / ANALYSIS_THRESH

DETECT_THRESH determines, in units of sigmas, what it takes for a group of pixels (larger than a separately specific minimum in number) to be detected as an object.
ANALYSIS_THRESH is the same idea, but marks the threshold where "CLASS STAR and FWHM [begin to] operate."

NOTE: in this exploration, DETECT_THRESH and ANALYSIS_THRESH were both set to the same quantity, specified by *_THRESH

[clicking on any image will link to a page that allows cycling through them, from 1 to 10]

*_THRESH = 0.5	*_THRESH = 1	*_THRESH = 1.5 [default]
*_THRESH = 2	*_THRESH = 2.5	*_THRESH = 3

PHOT_AUTOPARAMS: Kron_fact & min_radius

PHOT_AUTOPARAMS contains some of the key parameters for SExtractor's photometry. The Kron radius is "the typical size of the aperture," which is the ring/ellipse around objects. The Kron factor (Kron_fact) is "the number of Kron radii the aperture is set at", and the minimum radius (min_radius) is "the minimum radius for which this is done." If this radius is not specified (or is an unusable value) the minimum aperture in PHOT_AUTOAPERS is used here.

To best understand the connection between the Kron_fact and min_radius parameters, a grid-like array was formed with the Kron_fact varying on the x-axis, and min_radius varying on the y-axis.
Here is the full 10 x 12 array of images (will open in new window)
A small example:

K = 2, rad = 2	K = 2, rad = 2.5	K = 2, rad = 3
K = 2.5, rad = 2	K = 2.5, rad = 2.5	K = 2.5, rad = 3
K = 3, rad = 2	K = 3, rad = 2.5	K = 3, rad = 3

the Just Get It Done method:

In order to test the statistics program and see how our results are computing, the 'Quick and Dirty" approach was used to combine SExtractor and the original bright-star masks. An example mask is shown for Core and clicking on it links to a page with each region's Q&D mask on it.
All parameters (with only an exception or two) were defaults.

example: Core.fits

click to access original .fits files and final product .pl masks


Detecting Negative Noise Spikes and Understanding Overall Noise in the Data

SExtractor was used (with otherwise ridiculous settings) to mask objects like crazy using the Ring Median Subtracted Smoothed Images and their inverses
Kron radius = 2
min_radius = 3
sigma = 0.6 (yikes!)
DEBLEND_NTHRESH = 32 (this makes sure we don't combine multiple objects)

This (over-)masking gets basically every all of the information/objects in the image, from both noise and actual data. To find the amount of noise in the data (and thus the lowest magnitude of objects we could possibly have) we need to create histograms of the negative noise spikes and of the positive spikes (from data and noise) and compare.

First those parameters were used with SExtractor generating a catalog of objects found (with their fluxxes/magnitudes/etc) and a segmentation image which has non-zero values where ever actual objects were detected. To get this into a familiar .pl mask we use the iraf command imexpr as follows:

ecl> imexpr "(a>0)? 1 : 0" "mask.pl" "seg.fits"

this takes the segmentation image and makes a mask from it that we can combine with our other image masks for the overall mask

This same procedure is done on the inverse of the original image, with the inverse being generated with the imarith command:
ecl> imarith CoreRS.fits * -1 CoreRSinv.fits


Next the catalog exported by SExtractor was used to sort out objects with stellarity index less than 0.5 (as per the example paper - Feldmeier et al)

The magnitudes of these objects were binned up and plotted in a histogram in SM.

for Core:

The triangles reflect the positive noise spikes and the real data, the lines reflect the negative (inverse) noise spikes only.


These histograms are divided by each other to get the ratio of "goodness" as a function of magnitude:


After the outliers and funny points are removed, this cleans up to:



Using the fit line plotted here, we can do a Monte Carlo simulation to tell how likely it is that objects of a certain magnitude are real and not noise. After normalising this function, we select a random number between 0 and 1 and compare it to the value of the function for an object's magnitude. If the normalised fit curve is greater than the random number, then the data is considered a real source and masked out. If not it is left in the image.
There are maximum and minimum magnitude thresholds, too. For Core, any object brighter than -8 will be kept and any object fainter than -2 will be discarded. These are logical limits, judging from the graph above.

To see the histograms for the other fields, click:
Core Sub LPC FCJ 1 3 4 7


After a list of "real" objects has been generated, Prof. Mihos's 'mkspj4mask' program goes through and makes a new mask from the segmentation image and from the good object list. This mask is the cleaned segmentation image from the smoothed image, so is called CoreRSsegclean.fits
Here it is shown overlaid on top of Core. Click on it to go to a separate page and cycle between the Core image, the Core segmentation image, and the cleaned segmentation image


Core.fits with overlay (in green) of CoreRSsegclean.fits

Next we do a basic threshold brightness masking as we did before (way back here) for u = 25, 26, and 27. These masks are done on the original field and shown overlaid on it:

Core.fits with overlay = Coreu25.fits	Core.fits with overlay = Coreu26.fits	Core.fits with overlay = Coreu27.fits

To see the masks for the other fields, click:
Core Sub LPC FCJ 1 3 4 7


To make the final mask from these methods, the cleaned segmentation mask, the maximum surface brightness mask and the hand-made bright star mask (from back here) were all combined with imarith to generate 3 masks per field, one for each surface brightness limit. CoreMCu25.pl, CoreMCu26.pl, CoreMCu27.pl are shown overlaid on Core.fits

Core.fits with overlay = CoreMCu25.pl	Core.fits with overlay = CoreMCu26.pl	Core.fits with overlay = CoreMCu27.pl

Clearly u = 27 is a bit much, but when u = 26, this mask looks very promising.

For other fields, too, these levels worked well:
Core Sub LPC FCJ 1 3 4 7

Now, to do some statistics (on a separate page)


back to main Virgo page