v1.56.1: Support for SpectrophotometricColorCalibration, GUI updates.
v1.55: Support for BlurXTerminator.
v1.54: Code reorganization, banding reduction.
v1.53: Save processed channel images.
v1.52: Use already processed files, histogram stretch, star reduction.
v1.51: Star alignment settings.
v1.50: Manually select reference images.
AutoIntegrate is a PixInsight script to process FITS and other image files and run a basic image processing workflow to create a final image. Script has a GUI interface where some processing options can be selected. AutoIntegrate can do the whole image processing workflow including image calibration.
After running the script there will be integrated light images and an automatically processed final image. LRGB, color/OSC/DSLR and narrowband files are accepted.
AutoIntegrate is best suited for:
- When you are starting out with PixInsight and want to do a quick image processing before learning all the details.
- For quick calibration, alignment and integration (but not a replacement for WBPP), especially useful if you use remote observatories or other existing, pre-calibrated images or prepared masters.
- Quick generation of a reasonably good image, especially with filter combinations that may be difficult to manage when starting out.
- The possibility to have a quick look at a result and manually process the problematic steps if needed (so you do not spend a day processing an image whose color were badly balanced to start with). This is especially useful to see if your calibration/integration is good enough or leave artifacts/gradients/... that should be removed early.
I recommend that you use a hashtag #autointegratescript if posting images on social media. You can find my images processed with AutoIntegrate script on Instagram with that tag.
Images created with AutoIntegrate script
You can find my images created with AutoIntegrate script from the following links.
- APOD 2022 January 17, Chamaeleon Dark Nebulas
- APOD 2022 September 14, Waves of the Great Lacerta Nebula
- My Astrobin page
- My Instagram page
Below is a sample output from AutoIntegrate script with otherwise default options but Crop to common area was enabled.
I have created short YouTube videos that show the workflow with AutoIntegrate script. Some of these are created with an older version of AutoIntegrate script so dialog looks a bit different. Basic steps are still the same.
AutoIntegrate dialog screens
Below are the script dialog screens. Often it is best to start with default options, add files and hit the Run button.
Script remembers the last open or closed states of collapsible sections in the dialog. It is useful for example with smaller screens where it is possible to start with only minimal sections open.
Downloading the script
Using PixInsight update system
It is possible to automatically install and update AutoIntegrate script by adding it to the PixInsight update repository. Whenever PixInsight is started it will then check for updates to AutoIntegate. To enable automatic updates you need to add the following Url to the PixInsight Resources/Updates/Manage Repositories.https://ruuth.xyz/autointegrate
AutoIntegrate is added to the Script/Batch Processing menu.
In a page GitHub releases you can find the latest release and download it from there. Releases page has some detailed information of changes in each release. To download the source code click Source code (zip) and save the zip file. After download is complete unzip the file.
You can also go to GitHub AutoIntegrate main page to find the latest development changes and download it from there. GitHub master branch may contain test changes that are not yet available as release.
Running the script
Starting the script when using PixInsight update system
Click AutoIntegrate in the Script/Batch Processing menu to start the script.
Starting the script the script from the script editor
Starting from AutoIntegrate version 1.56 there is a dependency to ImageSolver script in PixInsight distribution. This means that ImageSolver files must be in the ../AdP directory relative to AutoIntegrate directory. You can do this by creating a ../AdP directory and copying the contents of Pixinsight-install-directory/src/scripts/AdP there.
Steps to start the script from the script editor
- Download the script
- Open Script Editor in PixInsight
- Open file AutoIntegrate.js
- Press F9 to start the script from the editor
Running the script
- Add files with add buttons
- Click Run and wait until the script completes
Using SpectrophotometricColorCalibration (SPCC) will clear the dialog window. Everything still runs fine. This is a problem in the SPCC process.
PixInsight strange errors.
StarXTerminator File I/O error.
When using StarXTerminator you may get a File I/O Error and stars are not removed from the image. This is an error that Russell Croman has acknowledged and a fix should be available soon. Right now the fix is to manually start and stop the StarXTerminator process. After that StarXTerminator works also from AutoIntegrate script.
AutoIntegrate does not show up in Script/Batch Processing menu.
If AutoIntegrate does not show up in the Script/Batch Processing menu you can try to reinstall scripts by deleting the updates.xri file in the PixInsight application folder. Or you can try to repaired the PixInsight installation, check for updates, close PixInsight to install updates, and restart PixInsight.
AutoIntegrate dialog does not fit on the screen.
AutoIntegrate dialog is quite large so in some cases it is possible that it is not fully visible on the screen and you cannot access the bottom row. In this case the problem can be the preview window size. It is possible to change the preview window size or disable preview in the Interface section. Note the some buttons like Exit and Run are duplicated at the top part of the dialog.
Files accepted by the script
Script accepts all files that are supported by PixInsight. It can be used with FITS files from telescopes, or RAW files from DSLR. It can be used with both calibrated and non-calibrated files. For non-calibrated files the script can do image calibration.
Calibrated files from remote telescope systems like Telescope Live, iTelescope, Slooh and Deep Sky West have been tested and they work fine.
When using RAW or OSC files, the script by default does debayering of the files. It is recommended to use the Pure RAW setting in PixInsight. If the files are already debayered you should choose None in the Debayer list.
How the script automatically recognizes different files
- First script checks if the file has a FILTER keyword that specifies the image as Luminance, Red, Green, Blue, Ha, SII, OIII or Color data. The following FILTER values are recognized: Luminance, Clear, L, Red, R, Green, G, Blue, B, SII, S, Halpha, Ha, H, OIII, O, Color. If there is a FILTER keyword but it is not one of the recognized values, the file is treated as a color file. Only the first letter of the FILTER keyword is used to detect the filter type.
- If the FILTER keyword is not found then files that end with _L, _R, _G, _B, _H, _S, _O or _C are treated as filter files. For example ngc6514_R.fit is treated as a Red filter file.
- Otherwise files are assumed to be color files.
If IMAGETYP keyword is set on files script can automatically detect bias, dark and flat calibration files when adding light files. Also flat dark files are recognized with a special value 'flat dark'.
Adding filter files manually
If automatic detection of the FILTER keyword does not correctly recognize the filter type, it is possible to manually add files for each filter. This can be enabled by checking 'Add manually' or 'Do not use FILTER keyword’ boxes.
There are separate buttons for each filter type recognized by the script. Also color/OSC/DSLR files can be added manually. Manual adding is useful if the FILTER keyword is missing or just for overriding filter keyword.
AutoIntegrate will by default put output files into subdirectories.
- AutoOutput contains intermediate files generated during processing
- AutoMaster contains generated master calibration files
- AutoCalibrated contains calibrated light files
- AutoProcessed contains processed final images. Also integrated images and log output is here.
By default output files are written into the same directory as the first light file in the file list. It is possible to give an absolute or relative path to the output directory.
There is an option to select the target type for simplified processing. Target type changes some default options so that they are more likely to match to the target.
Following target types are available
- Galaxy - Works well when the target is a lot brighter than the background.
- Nebula - Works well when the target fills the whole image or is not much brighter than the background.
Processing RGB files
Script automatically detects RGB files. If only LRGB, RGB or OSC/DSLR/color files are present it runs a basic RGB workflow.
Processing narrowband files
Narrowband images are supported by the script. They are processed mostly the same way as other images. Main difference is that you can choose the color palette that is used when mapping narrowband images to RGB channels.
Note that AutoIntegrate always uses narrowband mapping if any of the H, S, or O filters are present. So for example HaLRGB workflow always uses narrowband mapping.
Below is a sample output from AutoIntegrate script when using narrowband data and SHO color palette. Otherwise it was run with default settings but Remove green cast and Fix star colors options were checked.
Script can do image calibration if needed. Two basic workflows are supported:
- Calibrating with bias, darks and flats
- Calibrating with darks, flat darks (or dark flats) and flats
If any of the calibration files are not available the script tries to run calibration with the remaining files.
If there is only one file for bias, dark or flat dark it is assumed to be a master file. If you have multiple master files for example for different binning you can give multiple master files but in that case option Master files must be checked. When multiple master files are available, the script picks the one with the same resolution.
YouTube video of image calibration
Crop to common area
It is possible to automatically crop bad pixels on the sides of the images. This works by integrating all images, finding an area where all images are contributing and cropping channel images to that area.
Automatic cropping makes it easier to get an image with only good data in it. It also helps automatic processes such as stretching because only real data is included in the image.
To check how much cropping was an image file LowRejectionMap_ALL is left minimized on the screen.
Automatic cropping can be used also during AutoContinue. Cropping requires that the LowRejectionMap_ALL file is open in the PixInsight desktop. Cropping can be done only if starting from Integration_[LRGBHSO] or Integration_RGB_color files.
In the case of AutoContinue it is possible to edit the crop preview area in the LowRejectionMap_ALL image. AutoContinue will use the crop preview from LowRejectionMap_ALL. In this way it is possible to rerun the processing using a different crop if needed.
Below is an example that shows the difference between non-cropped image (left) and cropped image (middle). On the right is a LowRejectionMap_ALL file showing the cropped area.
Sometimes the cropping also helps with automatic processing. Below is an example that shows the processing difference between non-cropped image (left) and cropped image (right). Since bad pixels on the side are cropped, automatic processing can better stretch the images since there is only real data in the image.
AutoIntegrate automatically selects the reference images for star align, image integration and local normalization. By default the images with the best SSWEIGHT value are chosen.
It is possible to manually choose reference images. A reference image for star alignment is used to align all light images. For each filter reference images are used for image integration and local normalization.
There is a button that can be used to calculate SSWEIGHT values for each file and automatically mark reference images to the file list. It is useful for checking which files would be chosen and fine tune the selection if needed. SSWEIGHT values are calculated for raw files that are not debayered or cosmetically corrected so they may differ from those calculated during processing. If no reference images are selected manually then automatically selected images are marked to the file list after processing. They can be found also from the AutoIntegrate log file.
Note that manually selecting reference images for filters do not work with OSC images if the Extract channels option is used. In that case extracted channel images should be loaded separately as LRGB images to choose correct reference images.
AutoIntegrate offers a few options to stretch an image from linear to non-linear.
- Auto STF - Uses auto Screen Transfer Function.
- Masked Stretch - Uses MaskedStretch to stretch an image.
- Arcsinh Stretch - Uses ArcsinhStretch to stretch an image.
- Histogram stretch - Using simple histogram transformation with some clipping to get histogram median or peak to the target value.
- Hyperbolic - Generalized Hyperbolic Stretching using PixInsight process.
Auto STF is the default stretch. It is good in most cases. It can be fine tuned with STF targetBackground parameter. If you get too bright an image you can try lowering targetBackground value.
Masked Stretch is a good choice when Auto STF does not give a good image. For example with bright galaxy images Masked Stretch is a good choice as it keeps brightness in control and gives better saturation.
Arcsinh Stretch is another good choice when Auto STF does not give a good image. Typically Arcsinh Stretch gives a very well saturated image. There are a few parameters to fine tune stretching.
Histogram stretch is a very simple stretch that moves the histogram median or peak to the target value. Stretch is done iteratively in small steps. Also a selected amount of shadow clipping can be done.
Histogram stretch works best with images that are processed with the Crop to common area option.
Generalized Hyperbolic Stretching (GHS) is most useful on bright targets where AutoSTF may not work well. It often preserves background and stars well and also saturation is good. For very dim or small targets the implementation in AutoIntegrate does not work that well. Current implementation in AutoIntegrate is very experimental but may work on some images.
It is recommended that dark background is as clean as possible from any gradients with GHS. Consider using ABE on combined images and maybe also BackgroundNeutralization to clean the image background. Local Normalization can be useful too.
It is also recommended that the Crop to common area option is used. It cleans the image from bad data and makes finding the symmetry point more robust.
Generalized Hyperbolic Stretching is using PixelMath formulas from PixInsight forum member dapayne (David Payne). For more details see ghsastro site.
By default noise reduction is done in a linear phase (Channel image noise reduction). In case of channel images (LRGBSHO) noise reduction is done separately for each channel image. In the case of OSC image it is done for RGB image.
With option Combined image noise reduction noise reduction is done for combined RGB image and L image while they are still in linear phase. For OSC images this is the same as Channel image noise reduction.
With option Non-linear noise reduction noise reduction is done for combined RGB image and L image after they have been stretched to non-linear phase. For OSC images noise reduction is done for stretched RGB image.When using BlurXTerminator it is recommended to do noise reduction after BlurXTerminator is used. Either Combined image noise reduction or Non-linear noise reduction should be selected. But it is always good to experiment what is best for your own data.
By default color calibration is done using the ColorCalibration process. It is run on the linear RGB image before the image is stretched to non-linear. Optionally it can be run before ABE if the ABE option is selected.
Optionally SpectrophotometricColorCalibration (SPCC) can be used for color calibration. Option Color calibration using SPCC is used to enable SPCC. SPCC is run on linear RGB image. Some configuration options are available for SPCC.
SPCC requires that the image is plate solved. Plate solving is run automatically on Integration_RGB image which is also saved to disk.
If the image does not have center coordinates for plate solving then coordinates can be given in a new Image solving section. Coordinates can be given manually or searched using an online database (Sesame). Image solving needs correct focal length to work properly. If focal length is not embedded into image metadata it can be given in the Focal length field.
Automatic processing of all Narrowband palettes
Narrowband palette option All automatically processes all HSO palettes it knows. Every image is named with the palette option and automatically saved to disk as a .xisf file. Images are also left open on screen. With Save final image files selection it is possible to save them all for example as 16 bit TIFF files. You can also use extra options in images, then images with name Auto_+palette-name+_extra are created.
The fastest way to generate all palettes is run one processing with option Save processed channel images enabled. It generates Integration_channel_processed files. Open only Integration_channel_processed files on the desktop, select All narrowband palette and use AutoContinue to run all palettes from already processed channel files.
Extracting color channels from OSC files
When processing Color/OSC/DSLR files it is possible to extract separate color channels from color data. Channels can be extracted as LRGB, HSO or HOS data. After channels are extracted, processing continues as in mono filter processing.
Channels are extracted right after cosmetic correction and debayering. Channel files are saved to disk so they can be used for processing later.
Using separate channels enables different narrowband mappings when collecting narrowband data with OSC cameras. Also LRGB data can benefit from separate processing.
Thanks to René Bondo Hoffmann for providing OSC camera data using Optolong L-eXtreme filter. Image is processed to emulate the Hubble (SHO) color palette using the L-eXtreme SHO palette option. Additional color processing was done after the AutoIntegrate script run.
Fixing linear defects
It is possible to automatically fix linear column and row defects by using linear defect detection algorithm from PixInsight LinearDefectDetection.js script. Defect information is used by CosmeticCorrection to fix the defects.
Optionally it is possible to run Canon Banding Reduction script code inside AutoIntegrate. In section Banding, binning and cosmetic correction settings there are options to enable Banding reduction, select Protect highlights and Amount.
Banding reduction can be useful for Canon DSLR data and sometimes also for other OSC data too. Banding reduction is run just after debayering.
AutoContinue runs automatic processing from previously created LRGB/RGB, Narrowband or OSC/Color images. The idea with AutoContinue is that it is possible to run manually background extraction, histogram transformations or other steps on the integrated images and then continue automatic processing from there.
AutoContinue looks for images with fixed names and if it finds them it starts processing from those images. Image names are based on intermediate images created by the script but they have a special extension to separate them from automatically generated images. An exception is integrated channel images which can be used as a starting point to run different processing on automatically created images. They are useful for example for testing with different stretching options.
Starting points and the search order for AutoContinue are listed below
- AutoLRGB, AutoRGB, AutoRRGB or AutoMono - Final image for extra processing
- L_HT + RGB_HT - Manually stretched L and RGB images
- RGB_HT - Manually stretched RGB image
- Integration_L_DBE + Integration_RGB_DBE - Background extracted L and RGB images
- Integration_RGB_DBE - Background extracted RGB image
- Integration_L_DBE + Integration_R_DBE + Integration_G_DBE + Integration_B_DBE - Background extracted channel images
- Integration_H_DBE + Integration_S_DBE + Integration_O_DBE - Background extracted channel images
- Integration_RGB_color - Integrated Color RGB image
- Integration_RGB_narrowband - Integrated narrowband RGB image
- Integration_H + Integration_S + Integration_O - Integrated channel images
- Integration_L + Integration_R + Integration_G + Integration_B - Integrated channel images
- Integration_L_start + Integration_RGB_start - Integrated L and RGB image
- Integration_RGB_start - Integrated RGB image
Not all images must be present, for example the L image can be missing.
Explanation of terms used in image names:
- RGB = Combined image, can be RGB, Narrowband or Color image.
- HT = Histogram Transformation, image is manually stretched to non-linear state.
- DBE = Background Extracted, for example manual DBE is run on image.
- Integration = Individual light images are integrated into one image.
The following extra processing can be applied to the image
- StarNet, StarNet2 or StarXTerminator to separate stars and background
- Combine starless and star images using selected option
- Clip shadows
- Automatic Background Extraction
- Darker background with a luminance mask
- ExponentialTransformation with a luminance mask
- HDRMultiscaleTransform with a luminance mask
- LocalHistogramEqualization with a luminance mask
- Add contrast
- Different noise reductions
- Smaller stars with a star mask.
If multiple options are selected see tooltips on the script for execution order.
In case of Run or AutoContinue, extra processing steps are applied to a copy of the final image. A new image is created with _extra added to the name. For example if the final image is AutoLRGB then a new image AutoLRGB_extra is created. AutoContinue can be used to apply extra processing after the final image is created.
Narrowband specific extra processing
There are also narrowband specific extra processing options in the Narrowband processing section.
- Hue shift for more orange color or other hue shift, useful in SHO images
- Remove green cast, useful in SHO images
- Remove part of the green cast but leave some green color, useful in SHO images
- Fix magenta star colors often visible in SHO images
From a drop down list it is possible to select an image into which extra processing is applied. With the Apply button extra processing is by default run on a copy of the image. There is also a simple undo and redo mechanism to iterate with different extra processing options.
AutoIntegrate can automatically remove stars from an image. There are three options to remove stars and combine them back into the starless image. Starnet, Starnet2 or StarXTerminator can be used to remove stars. Some options work with linear data which require StarXTerminator.
Remove stars from channel images
With LRGB and narrowband images this option removes stars from L, R, G, B, H, S and O channel images separately before channels are combined and while images are still in the linear stage. Star images from channels are then combined to create a star image.
With color images (DSLR/OSC) the option removes stars after color calibration while the image is still in the linear stage.
Star image stretching and combining settings can be selected in the Image stretching settings section.
Remove stars before stretch
Removes stars from combined RGB or narrowband images just before stretching while it still is in the linear stage. Stars are used only from the RGB/narrowband image, stars from the L image are not used.
For OSC data this may not work well. Separating channels might help.
Star image stretching and combining settings can be selected in the Image stretching settings section.
This option needs the StarXTerminator process.
Remove stars after stretch
Removes stars from combined RGB or narrowband images just after stretching when image is not any more in linear stage. Stars are used only from the RGB/narrowband image, stars from the L image are not used.
Star image stretching and combining settings can be selected in the Image stretching settings section.
Extra remove stars
Removes stars from the final image to generate a starless image and a separate stars image.
For both normal and extra processing all there is an Unscreen stars checkbox. This option uses a bit of a different method to generate the stars image as described by Russell Croman. For details see this post: "Unscreening" and re-screening: recombining stars with starless images
The Unscreen method usually keeps star colors more correct than simple star removal. It is recommended to use the Screen method when combining star and starless images back together.
Star image stretching
When stars are removed before stretching then a different stretching can be used for the stars and potentially get better star colors. Star image stretching and combining settings can be selected in the Image stretching settings section.
Stars can be reduced either separately or when combining starless and star images.
Extra option Smaller stars reduce star sizes. Number of iterations can be selected when reducing star sizes. Value zero uses Erosion instead of Morphological Selection
When combining starless and star images it is possible to reduce stars using PixelMath expressions created by Bill Blanshan. There are three different methods: Transfer, Halo and Star. There is also a control parameter for each method. For details see this YouTube video
Combining starless and star images
For combining starless and star images there are three different options: add, screen and lighten. Add is a simple PixelMath operation to add stars back. Screen and lighten are equivalent to similar Photoshop blending modes.
Combine option can be set separately in the Image stretching settings section and Extra processing section.
Using Photometric Mosaic script
PixInsigth has great tools for creating mosaics. One such tool is the PhotometricMosaic script by John Murphy. You prepare linear images with scripts ImageSolver, MosaicByCoordinates and TrimMosaicTile and then create the mosaic using script PhotometricMosaic.
Using these tools with AutoIntegrate you first need to run a basic workflow which creates integrated linear images Integration_[LRGBHSO] for separate color channels data or Integration_RGB for OSC data. You can then create a mosaic for each channel or for OSC data.
Once you have the mosaic tiles you can load them into PixInsight with names Integration_[LRGBHSO] or Integration_RGB_color. You can then create the final image using the AutoContinue button.
Batch mode for mosaic images
Batch mode is intended to be used with mosaic images. In mosaics there are several sets of images and typically the same basic processing options are used for all images. In Batch mode AutoIntegrate script automatically asks for files for the next mosaic panel. You can freely choose the directories for images. Script creates a final processed image which has the same name as the directory where images were read. So if images are from directory P1 then the image name will be P1. At the end of the script all batch mode images are left open and all intermediate images are closed.
When batch files are ready for further processing is it possible to save all files by clicking a button in the section “Save final image files”. Batch mode adds a keyword to each image and it is used to find files for saving. It is possible to save images in PixInsight .xisf format, 16-bit TIFF format or 8-bit TIFF format. If you want to assemble the final image in Photoshop, save images in 16-bit TIFF format.
Batch mode keyword is saved on disk with .xisf format but not with TIFF format. If you want to process images later and use batch save then you should save files also in .xisf format.
Narrowband to RGB mapping
A special processing is used for narrowband to (L)RGB image mapping. It is used to enhance (L)RGB channels with narrowband data. It cannot be used without RGB filters. This mapping is similar to the NBRGBCombination script in PixInsight or as described in Light Vortex Astronomy tutorial Combining LRGB with Narrowband. You can find more details on parameters from those sources.
If narrowband RGB mapping is used then narrowband Color palette is not used.
With narrowband RGB mapping you can choose:
- Mapping of narrowband channels to (L)RGB channels
- Boost for (L)RGB channels
- Bandwidth for each filter
- Test the mapping with a test button
Software binning option is available for light files. It is done as the first operation for all light files or only for color channel files. It uses the IntegerResample routine for binning. For very noisy images binning should help to reduce noise at the costs of decreased resolution. If luminance data is good then it can be useful to do binning only for color data.
Window name prefix
It is possible to give the user specified prefix to image window names. This makes all generated window names unique and allows results from multiple processing runs to be open at the same time. These images are also iconized at different columns on the screen.
By default the icon column for a new window prefix is automatically managed by the script. Script tries to track used icon column positions and assign free ones for new runs.
Sometimes, like when using multiple workspaces, this automatic management does not work optimally. In that case it is possible to manually set the icon column for a window prefix. This can be enabled by setting option Manual icon column control. The change will be effective when restarting the script.
When manual icon control is enabled there is a new control Icon column. With that control it is possible to manually set the column where icons are placed. The default for the Icon column is Auto where the icon column is chosen automatically. Last icon column is remembered and automatically restored when the script starts.
AutoIntegrate includes image preview functionality. It is possible to see a stretched image by clicking through the file list files. Image preview is shown in a preview window that is part of the script dialog. The image in the preview window can be zoomed and moved around.
Files in the file list have a checked flag. Using image preview functionality bad files can be unchecked. Only checked files are used for processing.
Image preview window is also used to show image files during different steps of processing. During processing images are not shown in stretched mode.
By default image preview is shown in a separate tab. When clicking a file or starting to process the preview tab is set as default. Optionally using a Toggle preview button image preview can be shown as a separate window on the side of the dialog.
Preview window can be disabled in the interface settings section. Also preview window size can be changed there.
Blink in AutoIntegrate
If image preview is not used then AutoIntegrate includes a simple Blink functionality. It is possible to see a stretched image by clicking through the file list files. Blink window can be zoomed to 1:1. Currently the focus is locked on the script so the blink window is in a fixed position and size.
Files in the file list have a checked flag. Using blink functionality bad files can be unchecked. Only checked files are used for processing.
Saving settings to persistent module settings
It is possible to save all settings into PixInsight persistent module settings. These settings are automatically restored on startup.
All saved parameters, persistent module settings, process icon and setup file, will include only those values that are different from default values.
There is also a button to reset all parameters to default values.
Saving settings to script icon
It is possible to save your own default options to a script icon. It can then be used to recall default settings. Script icons can be saved to a file.
Parameter values are first restored from persistent module settings and then from the process icon. So process icon settings overwrite persistent module settings.
Settings are saved by dragging the lower left corner triangle icon to the desktop. By double clicking the icon a script window opens. Clicking the Apply global button starts the script with those settings that are saved to the icon. PixInsight also saves MD5 checksum to the script icon. When using the script icon with a modified script you need to clear the checksum field first.
Save and restore script setup
Current parameter settings and image file lists loaded into light and calibrate image pages can be saved to a setup file. That file can later be restored back to AutoIntegrate. Saved file keeps checked/unchecked status for files. The file must have a .json extension. Loading the setup from a Json file works in all file dialogs but there is also a separate button for it in the GUI.
Optionally it is possible to save just file lists and not current parameter settings. There is a separate button to save just file lists.
File paths that are in a subdirectory of the setup file are saved as relative paths. This makes it possible to move or share setup and images files and it still works in a new location. Relative paths are automatically restored as full paths.
Setup file saves only those parameters that have a non-default value. It is good to note that other parameters settings are not changed when restoring the setup. Also window prefix and output directory are saved to a setup file.
When available the setup Json file includes ssweight values for files.
Option Autosave setup saves current setup automatically after successful processing into AutosaveSetup.json file. Autosave is done only after the Run command, it is not done after the AutoContinue command.
File is saved to the lights file directory, or to the user given output directory. Saving to lights directory makes it possible to use relative paths to files so it is easy to move the setup file around or even share with lights files.
Setup can be later loaded into AutoIntegrate to see the settings or run the setup again possibly with different options. If the option Add window prefix to log files is selected the current prefix is added to the autosave setup file.
Use already processed files
Option Use processed files tries to use already processed files when possible. This option can be useful when adding files to an already processed set of files. Only files generated before image integration are reused.
Option works best with setup file that is saved after processing or with Autosave setup generated AutosaveSetup.json file because then star alignment reference image and possible defect info is saved.
With image calibration it is possible to use previously generated master files by loading already processed master files info calibration file lists. If only one calibration file is present then the script automatically uses it as a master file.
Sorting and filtering data
Sort and filter file lists
Filtering option is used to filter images using current weighting and filtering settings. If no filtering is set then files are just sorted based on weight calculation. For large data sets it may be useful to use multiple filterings. For example, sort first by PSF Signal and then by Eccentricity and after each sort uncheck bad files that are at the end of the list.
Automatic filtering of bad files
AutoIntegrate has a few ways to automatically filter out files based on image statistics calculated by the SubframeSelector process.
The simplest filtering uses a limit value for SSWEIGHT. Only files with calculated SSWEIGHT above the limit value are included. This is a good way to filter files if the expected SSWEIGHT values are known. Calculated SSWEIGHT values can be checked from the AutoIntegrate.log file after processing.
There are also a few ways to filter files by detecting outliers in the data set. This method does not require previous knowledge of the data. It can be useful in cases with huge data sets where manual filtering gets too complicated.
The following variables can be used for filtering outliers: SSWEIGHT, FWHM, Eccentricity, SNR, Star count, PSF Signal and PSF Power. Each variable is used independently to detect and filter outliers.
There are also three ways to calculate outlier filtering threshold values:
- Two sigma that filters out values that are two sigmas away from mean value.
- One sigma that filters out values that are one sigmas away from mean value.
- Interquartile range (IQR) measurement that is based on median calculations.
By default FWHM and Eccentricity are filtered for too high values, and all others are filtered for too low values. It is also possible to select filtering for both too low and high values.
Embedding AutoIntegrate engine
AutoIntegrate processing engine is separated from AutoIntegrate GUI so the engine can be called without the GUI. This makes it possible to create own interfaces to AutoIntegrate engine.
A sample to show how to call AutoIntegrate engine from a PixInsight script is in the following link: TestCalibrate.js
Processing Hubble data
A YouTube video shows how to process Hubble data using AutoIntegrate script. It includes basic steps to download Hubble data and shows how to do basic processing using AutoIntegrate script.
Some tips for using AutoIntegrate script
- Very often you get good results by running the script with default settings and then continue processing in PixInsight.
- Always before running the script you should check the quality of the data using the AutoIntegrate preview functionality or Blink tool. Both preview and blink tools are built into AutoIntegrate script. You can also use a separate Blonk process. You should remove all files that have movement like wind effect or bad clouds.
- If your data quality is not the best you may want to increase noise reduction.
- When processing bright galaxy data the image may come out too bright. You can try using MaskedStretch or reduce STF targetBackground value.
- When processing nebulosity data can try using MaskedStretch but then you need to increase targetBackground value.
- With extra processing it is possible to do basic enhancements like remove stars using StarNet, StarNet2 or StarXTerminator, increase contrast, run HDRMultiscaleTransform, LocalHistogramEqualization and make smaller stars automatically.
- When removing stars you often get better star color with unscreen option. When combining stars and starless in either PixInsight or Photoshop use screen option.
- Automatic cropping is very handy. But check LowRejectionMap_ALL and verify that the crop is ok.
- If you want to integrate only a single channel, you can do it by clicking Monochrome and Integrate only options, and then selecting the correct files.
- When processing comets it can be useful to keep the green color. This can be done using the No SCNR option. Also with comets the core may get overexposed. Choosing 'None' in image integration rejection can help with this.
If you want to test with local normalization or with different image integration rejection settings
you can do the following steps:
- Run normal processing to generate registered image files. You may also want to check the option Integrate only.
- Check option Start from ImageIntegration and select registered *_r.xisf files as Light files.
- Choose local normalization or image integration settings you want to test.
- Click the 'Run' button. Result files will be saved into the subdirectory AutoProcessed under the lights files directory.
- If you are getting satellite trails in your integrated image you can try to use Winsorised sigma rejection and lower High sigma value. Values like 2.2 or even 1.8 could be a good starting point. In general a lower High sigma value can be helpful if rejection is not good and you are using one of the Sigma rejection algorithms.
Tips for processing One Shot Color (OSC) or RAW files
- Default options are typically a pretty good start for color images. If there is a strong color cast and/or vignetting it is worth trying with Use ABE on combined images and Use BackgroundNeutralization options. Sometimes also choosing a different setting in the Link RGB channels option helps.
- For OSC/DSLR files PixInsight should be put into Pure RAW mode. It can be set in Format Explorer by double clicking RAW format. AutoIntegrate does debayering of images so no automatic debayering should be done.
Erik Westermann has a web site Remote Astrophotography. He talks about AutoIntegrate in his blog post PixInsight AutoIntegrate.js Processing Script. It has a lot of useful information so please check it out.
Glenn Newell discusses his latest (not so) secret PixInsight weapon, the AutoIntegrate script in the San Jose Astronomical Association (SJAA) meeting. Link to the YouTube presentation is: SJAA Imaging Special Interest Group 05 18 2021
Videos on PixInsight
Reading on PixInsight
Written information on PixInsight may not be as up to date as videos but they are still worth a check. At least the book Inside PixInsight by Warren A. Keller is good. Also the website Light Vortex Astronomy has a lot of useful information.
List of processes used by AutoIntegrate
Below are listed PixInsight processes that may be used by AutoIntegrate script with default LRGB processing. AutoIntegrate writes processing output to the Process Console window and to AutoIntegrate.log file. Details of processing steps can be found from the header block of the source code.
History of AutoIntegrate script
I started my astrophotography hobby with remote telescopes in 2017. I almost immediately got PixInsight and started working with FITS files. From the start I was happy with the results but also found that there are a lot of repetitive and complex tasks to be done. After a few months I even stopped my new hobby for a while. When I started astrophotography again after the summer 2018 I realized that there is this wonderful scripting capability in PixInsight. So I started developing my script. FITS files from remote telescopes were already calibrated so I did some first steps around ImageIntegration. This is where the script name comes from. Since then I have expanded the script into both directions, from image calibration to post processing.
Many thanks to Jean-Marc Lugrin for contributing brilliant Crop to common area code.
Many thanks to rob pfile for contributing the very useful Window name prefix and icon column position code. They are a great help when working with multiple images or multiple image versions.
Jean-Marc Lugrin has also created a great TestAutoIntegrate tool that automates testing. You can find it at GitHub. I have a slightly modified version of that code that works with the latest AutoIntegrate version.
Special thanks to Garth Hunt, Erik Westermann, Glenn Newell and Jari Backman for testing different versions of the script.
Thanks to Erik Westermann for doing a great work on documenting the script in his web site https://remoteastrophotography.com/.
Star Reduction equations using PixelMath are created by Bill Blanshan. For details see this YouTube video
Linear Defect Detection code is using the code from Vicent Peris. The PreviewControl module is from Andres del Pozo.
PixInsight scripts that come with the product were a great help when developing this script. Website Light Vortex Astronomy was a great place to find details and best practises when using PixInsight.
Routines ApplyAutoSTF and applySTF used by AutoIntegrate are from scripts that are distributed with PixInsight. Routines for Linear Defect Detection are from PixInsight scripts LinearDefectDetection.js and CommonFunctions.jsh that are distributed with PixInsight.
This product is based on software from the PixInsight project, developed by Pleiades Astrophoto and its contributors (https://pixinsight.com/).