Home Astro Mosaic AutoIntegrate Download .fit FITS utility

PixInsight AutoIntegrate.js Script

PixInsight repository link

AutoIntegrate is available as an automatic update in PixInsight. Repository link is below. For more details see Downloading the script section.

https://ruuth.xyz/autointegrate/

Contents

INFO

Latest updates

v1.73 - MultiscaleGradientCorrection
v1.72 - Ha to RGB mapping, true background
v1.71 - Fix to memory usage
v1.70 - DeepSNR, Signature, FastIntegration, Flowchart
v1.69 - GraXpert denoise
v1.68 - Small updates
v1.67 - Support for GradientCorrection
v1.66 - Live Flowchart, Ha to RGB mapping
v1.65 - Flowchart, GraXpert
v1.64 - Interface changes, narrowband colorization
v1.63 - Narrowband mapping, processing history
v1.62 - Foraxx, StarNet2, Annotate image

For more details on versions see GitHub page or forums.ruuth.xyz or Discord

Summary

The PixInsight AutoIntegrate script is a tool designed to streamline and simplify the process of integrating astronomical images in PixInsight. Here's a summary of its key features and functions:

  1. Automation of Integration Process: The script automates the complex and time-consuming task of combining multiple exposures into a single, high-quality image. This includes several steps like calibration, alignment, and stacking of images.
  2. User-Friendly: Aimed at making the integration process more accessible, especially for those who are either new to PixInsight or find its extensive options overwhelming.
  3. Customizable Parameters: Despite its automation, the script allows users to adjust certain parameters, providing flexibility to cater to different needs and preferences.
  4. Comprehensive Processing Steps: It includes the entire workflow required for image integration, such as applying calibration frames (flats, darks, bias), aligning images, and integrating them. Additional steps like noise reduction and histogram stretching are an integral part of the script.
  5. Enhanced Efficiency: By automating routine tasks, it saves time and reduces the likelihood of user error, especially in repetitive tasks.

The AutoIntegrate script is a powerful tool for astrophotographers looking to efficiently process their images while maintaining a high degree of control over the final output. It exemplifies the balance between automation and customization, making advanced image processing more accessible to a broader range of users.

(Generated by ChatGPT, if you have OpenAI account you can check AutoIntegrate Guide GPT)

Overview

AutoIntegrate is a PixInsight script to process FITS and other image files and run a basic image processing workflow to create a final image. Script has a GUI interface where some processing options can be selected. AutoIntegrate can do the whole image processing workflow including image calibration.

After running the script there will be integrated light images and an automatically processed final image. LRGB, color/OSC/DSLR and narrowband files are accepted.

AutoIntegrate is best suited for:

AutoIntegrate works with PixInsight version 1.8.9-1 or later.

Images created with AutoIntegrate script

You can find my images created with AutoIntegrate script from the following links.

Hashtag #autointegratescript

I recommend that you use a hashtag #autointegratescript if posting images on social media. You can find my images processed with AutoIntegrate script on Instagram with that tag.

Below is a sample output from AutoIntegrate script with otherwise default options but Crop to common area was enabled.


Data for this image is from Telescope Live remote telescope.

YouTube videos

I have created a set of short YouTube videos that show the workflow with AutoIntegrate script.

Video
Introduction to AutoIntegrate
Video
LRGB Processing using AutoIntegrate
Video
Narrowband Processing using AutoIntegrate
Video
Narrowband Processing using AutoIntegrate

Below are older videos that are still valid altough the dialog looks a bit different. Basic steps are still the same.

Video
Easy PixInsight workflow for astrophotos
Video
PixInsight Narrowband Processing
Video
Image Calibration and Basic Workflow
Video
Processing Hubble Data
Video
Hubble color palette using OSC data

AutoIntegrate dialog screens

Below are the examples of the script dialog screens. Often it is best to start with default options, add files and hit the Run button.

Script remembers the last open or closed states of collapsible sections in the dialog. It is useful for example with smaller screens where it is possible to start with only minimal sections open.

Downloading the script

Using PixInsight update system

It is possible to automatically install and update AutoIntegrate script by adding it to the PixInsight update repository. Whenever PixInsight is started it will then check for updates to AutoIntegrate. To enable automatic updates you need to add the following Url to the PixInsight Resources/Updates/Manage Repositories.

https://ruuth.xyz/autointegrate/

After you have added the link to AutoIntegrate repository you should check for updates by clicking Resources/Updates/Check for Updates. To install updates you need to restart PixInsight.

AutoIntegrate is added to the Script/Batch Processing menu.

Manual download and source code

In a page GitHub releases you can find the latest release and download it from there. Releases page has some detailed information of changes in each release. To download the source code click Source code (zip) and save the zip file. After download is complete, unzip the file.

You can also go to GitHub AutoIntegrate main page to find the latest development changes and download it from there. GitHub master branch may contain test changes that are not yet available as a release.

Test repository

There is also a test repository at https://ruuth.xyz/test/autointegrate/ that usually includes the latest changes from the GitHub master branch.

Running the script

Starting the script when using PixInsight update system

Click AutoIntegrate in the Script/Batch Processing menu to start the script.

Starting the script the script from the script editor

Starting from AutoIntegrate version 1.56 there is a dependency to ImageSolver script in the PixInsight distribution. This means that ImageSolver files must be in the ../AdP directory relative to AutoIntegrate directory. You can do this by creating a ../AdP directory and copying the contents of Pixinsight-install-directory/src/scripts/AdP there.

Steps to start the script from the script editor

  1. Download the script
  2. Open Script Editor in PixInsight
  3. Open file AutoIntegrate.js
  4. Press F9 to start the script from the editor

Running the script

  1. Add files with add buttons
  2. Click Run and wait until the script completes

Known problems

Problems already fixed

Files accepted by the script

Script accepts all files that are supported by PixInsight. It can be used with FITS files from telescopes, or RAW files from DSLR. It can be used with both calibrated and non-calibrated files. For non-calibrated files the script can do image calibration.

Calibrated files from remote telescope systems like Telescope Live, iTelescope, Slooh and Deep Sky West have been tested and they work fine.

When using RAW or OSC files, the script by default does debayering of the files. It is recommended to use the Pure RAW setting in PixInsight. If the files are already debayered you should choose None in the Debayer list.

How the script automatically recognizes different files

  1. First script checks if the file has a FILTER keyword that specifies the image as Luminance, Red, Green, Blue, Ha, SII, OIII or Color data. The following FILTER values are recognized: Luminance, Clear, L, Red, R, Green, G, Blue, B, SII, S, Halpha, Ha, H, OIII, O, Color. If there is a FILTER keyword but it is not one of the recognized values, the file is treated as a color file. Only the first letter of the FILTER keyword is used to detect the filter type.
  2. If the FILTER keyword is not found then files that end with _L, _R, _G, _B, _H, _S, _O or _C are treated as filter files. For example ngc6514_R.fit is treated as a Red filter file.
  3. If files do not end with a filter letter then the file name is checked for text '_Luminance_', '_Red_', '_Green_', '_Blue_', '_SII_', '_Halpha_', '_OIII_'.
  4. Otherwise files are assumed to be color files.

If IMAGETYP keyword is set on files script can automatically detect bias, dark and flat calibration files when adding light files. Also flat dark files are recognized with a special value 'flat dark'.

Adding filter files manually

If automatic detection of the FILTER keyword does not correctly recognize the filter type, it is possible to manually add files for each filter. This can be enabled by checking 'Add manually' or 'Do not use FILTER keyword’ boxes.

There are separate buttons for each filter type recognized by the script. Also color/OSC/DSLR files can be added manually. Manual adding is useful if the FILTER keyword is missing or just for overriding the filter keyword.

Output directories

AutoIntegrate will by default put output files into subdirectories.

If image calibration is used then calibrate files are put into own subdirectories.

By default output files are written into the same directory as the first light file in the file list. It is possible to give an absolute or relative path to the output directory.

Output images

Output images when using mono camera

Output images when using OSC/DSLR camera

Target type

There is an option to select the target type for simplified processing. Target type changes some default options so that they are more likely to match to the target.

Following target types are available

Processing RGB files

Script automatically detects RGB files. If only LRGB, RGB or OSC/DSLR/color files are present it runs a basic RGB workflow.

Processing narrowband files

Narrowband images are supported by the script. They are processed mostly the same way as other images. Main difference is that you can choose the color palette that is used when mapping narrowband images to RGB channels.

Note that AutoIntegrate always uses narrowband mapping if any of the H, S, or O filters are present. So for example HaLRGB workflow always uses narrowband mapping.

Below is a sample output from AutoIntegrate script when using narrowband data and SHO color palette. Otherwise it was run with default settings but Remove green cast and Fix star colors options were checked.


Thanks to Erik Westermann for providing narrowband data.

Image calibration

Script can do image calibration if needed. Two basic workflows are supported:

  1. Calibrating with bias, darks and flats
  2. Calibrating with darks, flat darks (or dark flats) and flats

If any of the calibration files are not available the script tries to run calibration with the remaining files.

If there is only one file for bias, dark or flat dark it is assumed to be a master file. If you have multiple master files for example for different binning you can give multiple master files but in that case option Master files must be checked. When multiple master files are available, the script picks the one with the same resolution.

YouTube video of image calibration

Video

Processing Flowchart

Generate Flowchart

Using the New Flowchart button the script will generate a flowchart of the processing workflow. Flowchart uses the current settings and images. A partially simulated minimal workflow is run to generate the flowchart information. To run the simulated workflow all relevant files must be loaded to the Files tab. A graphical version of the flowchart is printed to the preview window and a text version is printed to the process console.

Full Flowchart is available after processing. It is saved to the AutosaveSetup file and also to the setup file when available so it can be loaded later. A text version of flowchart is also printed to the AutoIntegrate log file.

Note that with a preview save button is it possible to save the flowchart image to a file.


LRGB Workflow using
default settings and calibrated images.

Narrowband workflow using
RC Astro tools and calibrated images.

LRGB workflow using image calibration.

Live Flowchart

Flowchart information is always generated during processing. It can viewed using Show Flowchart checkbox. During processing Flowchart is updated after each step. By checking and unchecking the Show Flowchart checkbox it is possible to switch between the current preview image and flowchart. Flowchart also shows processing time for each step.

Flowchart settings are in the Interface tab. Note that Flowchart settings are saved to persistent module settings but values are not reset with the Set default values button.

By default Flowchart settings option Flowchart show processing time is selected. This options shows the processing time for each step in the flowchart. If this option is not selected then only the flowchart data is shown.

By default Flowchart settings option Flowchart show processed image is selected. This options shows the processed image in the preview window and the flowchart data is shown on top of the image. If this option is not selected then only the flowchart data is shown.

If Flowchart settings option Get flowchart data before processing is selected then flowchart data is collected before processing. This is useful if you want to see the full flowchart during processing. Note that this option is not selected by default. Full flowchart is not available with AutoContinue or batch processing.


Live flowchart with default options

Live flowchart using option
Get flowchart data before processing.

Startup image

A startup image is shown in the preview window when the script starts. This startup image can be changed to a user selected image in the Interface section. The startup image can also be disabled there.

Embedding processing history

AutoIntegrate automatically embeds processing history into image metadata as FITS keywords. This helps to see how the image was processed. Processing history is saved using the HISTORY keywords.

Non-default processing options are saved after the base processing is run. With extra processing the processing option name with possible parameters are saved to the image metadata.

Processing history can be viewer using PixInsight File/FITS header selection.

It is also possible to view the processing history by loading the image as a target image in the AutoIntegrate script Extra processing tab. Then the processing history is printed to the process console using the AutoIntegrate processing history button.

Processing settings

Crop to common area

It is possible to automatically crop bad pixels on the sides of the images. This works by integrating all images, finding an area where all images are contributing and cropping channel images to that area.

Automatic cropping makes it easier to get an image with only good data in it. It also helps automatic processes such as stretching because only real data is included in the image.

To check how much cropping was an image file LowRejectionMap_ALL is left minimized on the screen.

Automatic cropping can be used also during AutoContinue. Cropping requires that the LowRejectionMap_ALL file is open in the PixInsight desktop. Cropping can be done only if starting from Integration_[LRGBHSO] or Integration_RGB_color files.

In the case of AutoContinue it is possible to edit the crop preview area in the LowRejectionMap_ALL image. AutoContinue will use the crop preview from LowRejectionMap_ALL. In this way it is possible to rerun the processing using a different crop if needed.

Below is an example that shows the difference between non-cropped image (left) and cropped image (middle). On the right is a LowRejectionMap_ALL file showing the cropped area.

Sometimes the cropping also helps with automatic processing. Below is an example that shows the processing difference between non-cropped image (left) and cropped image (right). Since bad pixels on the side are cropped, automatic processing can better stretch the images since there is only real data in the image.

Reference images

AutoIntegrate automatically selects the reference images for star align, image integration and local normalization. By default the images with the best SSWEIGHT value are chosen.

It is possible to manually choose reference images. A reference image for star alignment is used to align all light images. For each filter reference images are used for image integration and local normalization.

There is a button that can be used to calculate SSWEIGHT values for each file and automatically mark reference images to the file list. It is useful for checking which files would be chosen and fine tune the selection if needed. SSWEIGHT values are calculated for raw files that are not debayered or cosmetically corrected so they may differ from those calculated during processing. If no reference images are selected manually then automatically selected images are marked to the file list after processing. They can be found also from the AutoIntegrate log file.

Note that manually selecting reference images for filters do not work with OSC images if the Extract channels option is used. In that case extracted channel images should be loaded separately as LRGB images to choose correct reference images.

FastIntegration

AutoIntegrate supports the FastIntegration process. FastIntegration is a new process that is available in PixInsight version 1.8.9-2 or later. It is a faster version of the old ImageIntegration process. It is recommended to use FastIntegration instead of ImageIntegration if you want quick results or if you have a huge number of images.

FastIntegration has some setup options in Proccessing 2/Image Integration/FastIntegration section. By default FastIntegration skips cosmetic correction and also uses a subset of frames to find the reference image. It is possible to change these settings in the FastIntegration section.

FastIntegration supports also drizzle integration. Drizzle integration is useful when you have undersampled data and you want to increase the resolution of the image. Note that drizzle integration uses the normal DrizzleIntegration process so some performance benefits of FastIntegration are lost.

Find true background

A new method is added to find a true background area in an image. True background area is used by the BackgroundNeutralization process. It is enabled by default but can be switched off with the No auto background option. A new image named AutoBackgroundModel is created that shows the background area. The AutoBackgroundModel can be edited and it is used with AutoContinue if it already exists.

Image stretching

AutoIntegrate offers a few options to stretch an image from linear to non-linear.

Auto STF

Auto STF is the default stretch. It is good in most cases. It can be fine tuned with STF targetBackground parameter. If you get too bright an image you can try lowering targetBackground value.

Masked Stretch

Masked Stretch is a good choice when Auto STF does not give a good image. For example with bright galaxy images Masked Stretch is a good choice as it keeps brightness in control and gives better saturation.

Masked+Histogram Stretch

A minimal Histogram based stretch is done before Masked Stretch. This can help with stars that can be too pointlike with Masked Stretch. There is a separate parameter for the histogram prestretch target.

Histogram stretch

Histogram stretch is a very simple stretch that moves the histogram median or peak to the target value. Stretch is done iteratively in small steps.

Histogram stretch works best with images that are processed with the Crop to common area option.

Other stretch methods

There are also a few other stretch methods available. These could be described as experimental. They may not be as useful as other stretch methods but they can be helpful in some special cases.

In the Processing 1/Image Stretching section there are some settings for the stretch methods.

Noise reduction

By default noise reduction is done in a linear phase (Channel image noise reduction). In case of channel images (LRGBSHO) noise reduction is done separately for each channel image. In the case of OSC image it is done for RGB image.

With option Combined image noise reduction noise reduction is done for combined RGB image and L image while they are still in the linear phase. For OSC images this is the same as Channel image noise reduction.

With option Non-linear noise reduction noise reduction is done for combined RGB image and L image after they have been stretched to non-linear phase. For OSC images noise reduction is done for stretched RGB image.

When using BlurXTerminator it is recommended to do noise reduction after BlurXTerminator is used. Either Combined image noise reduction or Non-linear noise reduction should be selected. But it is always good to experiment what is best for your own data.

By default noise reduction uses MultiscaleLinerTransform. There are also several AI tools available for noise redution: NoiseXTerminator, GraXpert denoise or DeepSNR.

Color calibration

By default color calibration is done using the ColorCalibration process. It is run on the linear RGB image before the image is stretched to non-linear.

SpectrophotometricColorCalibration

Optionally SpectrophotometricColorCalibration (SPCC) can be used for color calibration. Option Color calibration using SPCC is used to enable SPCC. SPCC is run on linear RGB image. Some configuration options are available for SPCC.

SPCC requires that the image is plate solved. Plate solving is run automatically on Integration_RGB image which is also saved to disk. Some sources say that you should use Drizzle with scale 1 or 2 when using SPCC.

Sample narrowband color calibration output

Below is an image that shows results from different narrowband processing using different color calibration and AutoSTF auto/linked/unlinked channel options.

First row is using defaults with no color calibration and auto (unlinked) channels for AutoSTF. Second row is using SPCC with auto channel link mode. With narrowband images and SPCC the auto channel link mode AutoIntegrate uses linked channels for AutoSTF when there is H assigned into the red channel, otherwise AutoIntegrate uses unlinked channels.

Image solving

If the image does not have center coordinates for plate solving then coordinates can be given in a new Image solving section. Coordinates can be given manually or searched using an online database (Sesame). Image solving needs correct focal length to work properly. If focal length is not embedded into image metadata it can be given in the Focal length field.

Extracting color channels from OSC files

When processing Color/OSC/DSLR files it is possible to extract separate color channels from color data. Channels can be extracted as LRGB, HSO or HOS data. After channels are extracted, processing continues as in mono filter processing.

Channels are extracted right after cosmetic correction and debayering. Channel files are saved to disk so they can be used for processing later.

Using separate channels enables different narrowband mappings when collecting narrowband data with OSC cameras. Also LRGB data can benefit from separate processing.


Thanks to René Bondo Hoffmann for providing OSC camera data using Optolong L-eXtreme filter. Image is processed to emulate the Hubble (SHO) color palette using the L-eXtreme SHO palette option. Additional color processing was done after the AutoIntegrate script run.

YouTube video for creating Hubble color palette using OSC data

Video
YouTube video

Fixing linear defects

It is possible to automatically fix linear column and row defects by using linear defect detection algorithm from PixInsight LinearDefectDetection.js script. Defect information is used by CosmeticCorrection to fix the defects.

RC Astro tools support

In the Tools and Batching section there are options to use RC Astro tools. There are also some options to configure RC Astro tools which are described below.

BlurXTerminator

In the Processing 1 tab there are Sharpening settings for BlurXTerminator.

NoiseXTerminator

In the Processing 1 tab there are Noise reduction settings. With NoiseXTerminator the strength value changes denoise and detail values. Strength value has the following mapping to denoise and detail:

StarXTerminator

In the Processing 1 tab there are StarXTerminator settings.

There it is possible to select other than default AI model. You can see the default AI model in the tooltip. AI models are stored in PixInsight installation directory and have .pb extension. At least in Windows they are in PixInsight/library directory.

If you are getting tiling artifacts you can check Large overlap option. It runs slower but should give better results.

GraXpert support

It is possible to use GraXpert for gradient correction or for denoising the image.

For gradient correction GraXpert is used instead of GradientCorrection or AutomaticBackgroundExtractor. GraXpert can be used by checking the Use GraXpert option in the Tools and Batching section. Smoothing and correction settings can be configured. When GraXpert is used from the script it always uses the AI model when removing gradients.

By default no gradient correction is done. To use GraXpert for gradient correction you need to also check one of the gradient correction options in the Image processing parameters section.

For denoising GraXpert is used instead of NoiseXTerminator. GraXpert can be used by checking the Use GraXpert denoise option in the Tools and Batching section. Smoothing and batch sizer settings can be configured.

GraXpert settings

GraXpert is run as an external process so it must be installed on the computer before it can be used.

To use GraXpert first the path to the GraXpert binary must be set in the GraXpert section.

GraXpert AI model must be loaded manually before it can be used from AutoIntegrate. To load the AI model, run GraXpert manually once and close it. AutoIntegrate uses the default model.

Note that AutoIntegrate supports GraXpert version 2.2.0 or later.

StarNet and DeepSNR support

StarNet2 can be used to remove stars from the image. Before StarNet2 can be used it must installed to the PixInsight.

DeepSNR can be used to reduce noise the image. Before DeepSNR can be used it must installed to the PixInsight.

In the Processing 1 tab there are Noise reduction settings. With DeepSNR the strength setting has a reverse effect. The higher the value the less noise reduction is done. The strength value has the following mapping:

Banding reduction

Banding reduction for each light image

Optionally it is possible to run Canon Banding Reduction script code inside AutoIntegrate. In section Banding, binning and cosmetic correction settings there are options to enable Banding reduction, select Protect highlights and Amount.

Banding reduction can be useful for Canon DSLR data and sometimes also for other OSC data too. Banding reduction is run just after debayering so it is run separately for each light image.

Banding reduction extra processing option

There is also an extra processing option to run banding reduction on the final image.

AutoContinue

AutoContinue runs automatic processing from previously created LRGB/RGB, Narrowband or OSC/Color images. The idea with AutoContinue is that it is possible to run manually gradient correction or histogram transformation on the automatically processed images and then again continue automatic processing from there.

AutoContinue looks for images with fixed names and if it finds them it starts processing from those images. It is also possible to load integrated images for example from WBPP to the light files. If fixed image names are not found, option Integrated lights in Files tab is checked and light files have only one image for a filter then it is considered as an integrated light image.

Image names are based on intermediate images created by the script but they have a special extension to separate them from automatically generated images. An exception is integrated channel or RGB images which can be used as a starting point to run different processing on automatically created images. They are useful for example for testing with different stretching options or narrowband palettes.

Starting points and the search order for AutoContinue are listed below

  1. AutoLRGB or AutoRGB - Final image for extra processing
  2. L_HT + RGB_HT - Manually stretched L and RGB images.
  3. RGB_HT - Manually stretched RGB image
  4. Integration_filter_GCext - Gradient corrected integrated channel images
  5. Integration_RGB_GCext - Gradient corrected integrated color RGB image
  6. Integration_filter - Integrated channel images
  7. Integration_RGB - Integrated RGB image

Not all images must be present, for example the L image can be missing.

Explanation of terms used in image names:

Simple example of using AutoContinue

In the simplest case you can use AutoContinue after basic processing is done and all icons are still on the desktop. In that case you can for example change the stretching setting or some other options, set a new Window Prefix to avoid overwriting previous processing and then run AutoContinue.

Example to run manual gradient correction

Below is an example how to run manual gradient correction on images and then continue automatic processing from there. The example assumes that a base processing has been done with AutoIntegrate.

  1. First close all images on desktop. For mono camera, load channel images Integration_filter to the desktop. For example Integration_L, Integration_R, Integration_G and Integration_B. For color camera, load Integration_RGB image.
  2. If you want to avoid overwriting previously processed images set the Window prefix.
  3. Crop images to remove all black borders. If you want to use automatic cropping you should use option Save cropped images during base processing. Then it is possible to use cropped images with _crop extension.
  4. Run gradient correction on images. New images must have an extension like _DBE or _GraXpert. So gradient corrected Integration_R could be Integration_R_DBE, or Integration_RGB could be Integration_RGB_GraXpert.
  5. Select processing options and click AutoContinue.

Example to run manual stretching

Below is an example how to run manual stretching on images and then continue automatic processing from there. The example assumes that a base processing has been done with AutoIntegrate.

  1. First close all images on desktop. For mono camera, load images Integration_L_processed and Integration_RGB_processed to the desktop. For color camera, load Integration_RGB_processed image.
  2. If you want to avoid overwriting previously processed images set the Window prefix.
  3. Run stretching on images and rename images to L_HT and RGB_HT.
  4. Select processing options and click AutoContinue.

Extra processing

Extra processing allows running several PixInsight image tuning processes on image. Extra processing can be run as part of the basic image workflow or it can be run separately on selected images. When extra processing is run separately on a selected image it is easy to experiment the effect of different processing in a preview window with undo and redo buttons.

Some of the extra processing that can be applied to the image include:

If multiple options are selected see tooltips on the script for execution order.

In case of Run or AutoContinue, extra processing steps are applied to a copy of the final image. A new image is created with _extra added to the name. For example if the final image is AutoLRGB then a new image AutoLRGB_extra is created. AutoContinue can be used to apply extra processing after the final image is created.

Target image and apply button

From a drop down list it is possible to select an image into which extra processing is applied. With the Apply button extra processing is by default run on a copy of the image. There is also a simple undo and redo mechanism to iterate with different extra processing options.

Narrowband specific extra processing

There are also narrowband specific extra processing options in the Narrowband processing section. Below are listed some of the options.

Removing stars

AutoIntegrate can automatically remove stars from an image. There are three options to remove stars and combine them back into the starless image. Starnet2 or StarXTerminator can be used to remove stars. Some options work with linear data which require StarXTerminator.

Remove stars from light images

This option removes stars from individual light images. Star removal is done after star alignment and before optional comet alignment.

Remove stars from channel images

With LRGB and narrowband images this option removes stars from L, R, G, B, H, S and O channel images separately before channels are combined and while images are still in the linear stage. Star images from channels are then combined to create a star image.

With color images (DSLR/OSC) the option removes stars after color calibration while the image is still in the linear stage.

Star image stretching and combining settings can be selected in the Image stretching settings section.

Remove stars before stretch

Removes stars from combined RGB or narrowband images just before stretching while it still is in the linear stage. Stars are used only from the RGB/narrowband image, stars from the L image are not used.

For OSC data this may not work well. Separating channels might help.

Star image stretching and combining settings can be selected in the Image stretching settings section.

This option needs the StarXTerminator process.

Remove stars after stretch

Removes stars from combined RGB or narrowband images just after stretching when the image is not any more in linear stage. Stars are used only from the RGB/narrowband image, stars from the L image are not used.

Star image stretching and combining settings can be selected in the Image stretching settings section.

Extra remove stars

Removes stars from the final image to generate a starless image and a separate stars image.

Unscreen stars

For both normal and extra processing all there is an Unscreen stars checkbox. This option uses a bit of a different method to generate the stars image as described by Russell Croman. For details see this post: "Unscreening" and re-screening: recombining stars with starless images

The Unscreen method usually keeps star colors more correct than simple star removal. It is recommended to use the Screen method when combining star and starless images back together.

Star image stretching

When stars are removed before stretching then a different stretching can be used for the stars and potentially get better star colors. Star image stretching and combining settings can be selected in the Image stretching settings section.

Reducing stars

Stars can be reduced either separately or when combining starless and star images.

Extra option Smaller stars reduce star sizes. Number of iterations can be selected when reducing star sizes. Value zero uses Erosion instead of Morphological Selection

When combining starless and star images it is possible to reduce stars using PixelMath expressions created by Bill Blanshan. There are three different methods: Transfer, Halo and Star. There is also a control parameter for each method. For details see this YouTube video

Combining starless and star images

For combining starless and star images there are three different options: add, screen and lighten. Add is a simple PixelMath operation to add stars back. Screen and lighten are equivalent to similar Photoshop blending modes.

Combine option can be set separately in the Image stretching settings section and Extra processing section.

Extra processing to starless and stars images

Extra processing has some additional options when combining starless and stars images.

To use the Extra processing option combine you need to have starless and stars images open on the desktop. Starless image must be selected as the target image.

With a default Auto mode AutoIntegrate tries to automatically find the stars image based on the starless image name. It assumes that your starless image name has a text starless and stars image name has a text stars. So a starless image sameprefix_starless_whatever is matched with a stars image sameprefix_stars_doesnotmatterwhatishere. You should get an error if a matching stars image is not found.

Using the select button it is possible to manually select the stars image from images open on the desktop.

Reduce stars options can be used to reduce star sizes during combine. These options use PixelMath equations created by Bill Blanshan.

Creating mosaics

Using Photometric Mosaic script

PixInsigth has great tools for creating mosaics. One such tool is the PhotometricMosaic script by John Murphy. You prepare linear images with scripts ImageSolver, MosaicByCoordinates and TrimMosaicTile and then create the mosaic using script PhotometricMosaic.

Using these tools with AutoIntegrate you first need to run a basic workflow which creates integrated linear images Integration_[LRGBHSO] for separate color channels data or Integration_RGB for OSC data. You can then create a mosaic for each channel or for OSC data.

Once you have the mosaic tiles you can load them into PixInsight with names Integration_[LRGBHSO] or Integration_RGB_color. You can then create the final image using the AutoContinue button.

Batch mode for mosaic images

Batch mode is intended to be used with mosaic images. In mosaics there are several sets of images and typically the same basic processing options are used for all images. In Batch mode AutoIntegrate script automatically asks for files for each mosaic panel. You can freely choose the directories for images. Script creates a final processed image which has the same name as the directory where images were read. So if images are from directory P1 then the image name will be P1. At the end of the script only final batch mode images are left open and all intermediate images are closed.

When batch completed is it possible to save all files in a different formats by clicking a button in the section “Save final image files”. Batch mode adds a keyword to each image and it is used to find files for saving. It is possible to save images in PixInsight .xisf format, 16-bit TIFF format or 8-bit TIFF format. If you want to assemble the final image in Photoshop, save images in 16-bit TIFF format.

Batch mode keyword is saved on disk with .xisf format but not with TIFF format. If you want to process images later and use batch save then you should save files also in .xisf format.

Binning

Software binning option is available for light files. It is done as the first operation for all light files or only for color channel files. It uses the IntegerResample routine for binning. For very noisy images binning should help to reduce noise at the costs of decreased resolution. If luminance data is good then it can be useful to do binning only for color data.

Comet processing

Below is the suggested workflow with comet processing in AutoIntegrate:

Comet alignment will automatically skip star alignment and SCNR. Since already star aligned images (*_r.xisf) are used then Star alignment could invalidate coordinates given here and thus it is not used.

Note that using starless images may cause problems for example with ImageIntegration or BlurXTerminator. In some cases this shows as an error "Zero or insignificant PSF Signal Weight estimate" in the AutoIntegrate log files. With missing PSF error in ImageIntegration you can use an option ImageIntegration use ssweight. Sometimes on starless images PSF value can not be calculated. Then a manual value should be given or BlurXTerminator should not be used.

It is possible to manually run the CometAlignment process. Below are the steps to use AutoIntegrate with manual comet alignment:

Annotate images

Annotate images extra option can be used to add annotations to the final image. When used with the Run or AutoContinue button a new image with _Annotated postfix is created. Annotate image uses PixInsight AnnotateImage script.

Note that the image must have a correct astrometric solution embedded for annotation to work. When using SPCC color calibration the astrometric solution is automatically embedded.


Annotated image.

Add signature

Signature extra option is used to add signature to the final image. Signature image file must be given by the user. The signature file can be for example a .png image with a transparent background. Note that no borders are added to the image.

Optionally the signature image can be scaled relative to the main image. The scaling is given as percentage of the main image size. Signature position can also be chosen: 'Top left', 'Top middle', 'Top right', 'Bottom left', 'Bottom middle' or 'Bottom right'.

Narrowband specific settings

Narrowband palettes

By default a single narrowband palette is run. It can be selected in the Narrowband processing section. If there is H, S or O files then narrowband palette is always used. Note that even if there is just H files with LRGB files then narrowband palette is used.

Automatic processing of all palettes

Optionally it is possible to process multiple narrowband palettes in one run.

Narrowband palette option All automatically processes all HSO palettes it knows. Every image is named with the palette option and automatically saved to disk as a .xisf file. Images are also left open on screen. With Save final image files selection it is possible to save them all for example as 16 bit TIFF files. You can also use extra options in images, then images with name Auto_+palette-name+_extra are created.


All narrowband palettes known by AutoIntegrate, and two SHO palettes after extra processing.

Automatic processing of multiple palettes

With option Use multiple mappings it is possible to select needed narrowband palette mappings instead of all. Multiple mappings run the same way as all palettes option but only selected mappings are used.

The fastest way to generate all palettes is to first run processing with one palette and then use already integrated images for other palettes. For processing multiple palettes open Integration_channel files on the desktop, select All narrowband palette or multiple palettes and use AutoContinue to run palettes from already integrated channel files.

Foraxx palette

To run Foraxx palette during the normal processing you need to select Dynamic narrowband palatte like Dynamic SHO and check the option Narrowband mapping using non-linear data.

Foraxx extra option

In the extra processing section there is also an option to use the Foraxx palette for SHO and HOO images. Foraxx needs nonlinear images so it is best run after other processing. Foraxx palette is the same as Dynamic palettes with Narrowband mapping but for SHO images also Remove green cast and Orange/blue colors options are run for the image.

Narrowband mapping extra options

Another way to check different narrowband palettes is with the Narrowband mapping option in the Extra processing section. If the processed image is a SHO or HOO image then it is possible to try different narrowband mappings. When the Narrowband mapping option is used, channels are extracted from the SHO or HOO image and mapped again to create a new palette image.

Narrowband colorization

Extra processing option Colorize narrowband is used to map narrowband (or any other) data to RGB channels using user selected hue, saturation and weight.


Image colorized using the Eagle preset.

Ha to RGB mapping

Using narrowband color palette

Narrowband Color palette list has a few predefined options to map Ha to RGB. It is also possible to create a custom mapping by writing PixelMath expressions to the channel boxes.

Using Ha to RGB mapping section

Ha to RGB mapping section is used to map Ha to the RGB red channel. Different mapping options can be selected and the mapping can be tested with a test button.

There is also extra processing option Ha to RGB mapping to add Ha to RGB. In that case None combine mapping option can be used to skip mapping during processing. Then the mapping can be done using the extra option. Extra option mapping settings are taken from Ha to RGB mapping section.

With extra option it is possible to use a different stretching for the Ha image. So if the galaxy image is processed with masked stretch then Ha image can be stretched with AutoSTF and combined for example with 'Max 0.7' predefined setting.

Narrowband to RGB mapping

A special processing is used for narrowband to (L)RGB image mapping. It is used to enhance (L)RGB channels with narrowband data. It cannot be used without RGB filters. This mapping is similar to the NBRGBCombination script in PixInsight or as described in Light Vortex Astronomy tutorial Combining LRGB with Narrowband (Light Vortex Astronomy site is no longer available). You can find more details on parameters from those sources.

If narrowband RGB mapping is used then narrowband Color palette is not used.

With narrowband RGB mapping you can choose:
- Mapping of narrowband channels to (L)RGB channels
- Boost for (L)RGB channels
- Bandwidth for each filter
- Test the mapping with a test button

Setup options

Window name prefix

It is possible to give the user specified prefix to image window names. This makes all generated window names unique and allows results from multiple processing runs to be open at the same time. These images are also iconized at different columns on the screen.

By default the icon column for a new window prefix is automatically managed by the script. Script tries to track used icon column positions and assign free ones for new runs.

Sometimes, like when using multiple workspaces, this automatic management does not work optimally. In that case it is possible to manually set the icon column for a window prefix. This can be enabled by setting option Manual icon column control. The change will be effective when restarting the script.

When manual icon control is enabled there is a new control Icon column. With that control it is possible to manually set the column where icons are placed. The default for the Icon column is Auto where the icon column is chosen automatically. Last icon column is remembered and automatically restored when the script starts.

Image preview and interface configuration

AutoIntegrate includes image preview functionality. It is possible to see a stretched image by clicking through the file list files. Image preview is shown in a preview window that is part of the script dialog. The image in the preview window can be zoomed and moved around.

Files in the file list have a checked flag. Using image preview functionality bad files can be unchecked. Only checked files are used for processing.

Image preview window is also used to show image files during different steps of processing. During processing images are not shown in stretched mode.

By default image preview is shown in a separate tab. When clicking a file or starting to process the preview tab is set as default. Using the Toggle side preview button image preview can be shown as a separate window on the side of the dialog.

Preview window can be disabled in the interface settings section. Also preview window size can be changed there.


Image preview settings in Interface settings section

Image histogram

By default the image histogram is displayed below the image. Histogram is updated when the image is changed. Clicking on the histogram shows some statistics. Default size for histogram is calculated automatically. Size can be adjusted in the interface section and the histogram display can be disabled in the interface section.

Below are some configuration options for preview and generic interface


Image preview in a side window with Large preview and More tabs options (default view)

No Files Tab, no More tabs (old default view)

Full screen preview

Both dialog and preview window can be maximized to (almost) full screen. This is useful when using a small screen or otherwise when a bigger view of the image is needed.


Full screen dialog button.


Full screen image preview button.


Full screen image preview.

Single column interface option.

Saving settings to persistent module settings

It is possible to save all settings into PixInsight persistent module settings. These settings are automatically restored on startup.

All saved parameters, persistent module settings, process icon and setup file, will include only those values that are different from default values.

There is also a button to reset all parameters to default values.

Saving settings to script icon

It is possible to save your own default options to a script icon. It can then be used to recall default settings. Script icons can be saved to a file.

Parameter values are first restored from persistent module settings and then from the process icon. So process icon settings overwrite persistent module settings.

Settings are saved by dragging the lower left corner triangle icon to the desktop. By double clicking the icon a script window opens. Clicking the Apply global button starts the script with those settings that are saved to the icon. PixInsight also saves MD5 checksum to the script icon. When using the script icon with a modified script you need to clear the checksum field first.

Save and restore script setup

Current parameter settings and image file lists loaded into light and calibrate image pages can be saved to a setup file. That file can later be restored back to AutoIntegrate. Saved file keeps checked/unchecked status for files. The file must have a .json extension.

File paths that are in a subdirectory of the setup file are saved as relative paths. This makes it possible to move or share setup and images files and it still works in a new location. Relative paths are automatically restored as full paths.

Setup file saves only those parameters that have a non-default value. It is good to note that other parameters settings are not changed when restoring the setup. There is an option Reset on setup load to use only those settings that are loaded from the setup file. Other parameters are reset to default values.

Also window prefix and output directory are saved to a setup file.

When available the setup Json file includes ssweight values for files.

Autosave setup

Option Autosave setup saves current setup automatically after successful processing into AutosaveSetup.json file. Autosave is done only after the Run command, it is not done after the AutoContinue command. Starting from version 1.65, Autosave setup is enabled by default.

File is saved to the lights file directory, or to the user given output directory. Saving to lights directory makes it possible to use relative paths to files so it is easy to move the setup file around or even share with lights files.

Setup can be later loaded into AutoIntegrate to see the settings or run the setup again possibly with different options. If the option Add window prefix to log files is selected the current prefix is added to the autosave setup file.

Use already processed files

Option Use processed files tries to use already processed files when possible. This option can be useful when adding files to an already processed set of files. Only files generated before image integration are reused.

Option works best with setup file that is saved after processing or with Autosave setup generated AutosaveSetup.json file because then star alignment reference image and possible defect info is saved.

With image calibration it is possible to use previously generated master files by loading already processed master files info calibration file lists. If only one calibration file is present then the script automatically uses it as a master file.

Sorting and filtering data

Sort and filter file lists

Filtering option is used to filter images using current weighting and filtering settings. If no filtering is set then files are just sorted based on weight calculation. For large data sets it may be useful to use multiple filterings. For example, sort first by PSF Signal and then by Eccentricity and after each sort uncheck bad files that are at the end of the list.

Automatic filtering of bad files

AutoIntegrate has a few ways to automatically filter out files based on image statistics calculated by the SubframeSelector process.

The simplest filtering uses a limit value for SSWEIGHT. Only files with calculated SSWEIGHT above the limit value are included. This is a good way to filter files if the expected SSWEIGHT values are known. Calculated SSWEIGHT values can be checked from the AutoIntegrate.log file after processing.

Additional image filtering is based on two measures. It is possible to filter images for example with FWHM and Eccentricity values. Images that do not meet filtering criteria are not processed.

There are also a few ways to filter files by detecting outliers in the data set. This method does not require previous knowledge of the data. It can be useful in cases with huge data sets where manual filtering gets too complicated.

The following variables can be used for filtering outliers: SSWEIGHT, FWHM, Eccentricity, SNR, Star count, PSF Signal and PSF Power. Each variable is used independently to detect and filter outliers.

There are also three ways to calculate outlier filtering threshold values:

By default FWHM and Eccentricity are filtered for too high values, and all others are filtered for too low values. It is also possible to select filtering for both too low and high values.

Other information

Embedding AutoIntegrate engine

AutoIntegrate processing engine is separated from AutoIntegrate GUI so the engine can be called without the GUI. This makes it possible to create own interfaces to AutoIntegrate engine.

A sample to show how to call AutoIntegrate engine from a PixInsight script is in the following link: TestCalibrate.js

Processing Hubble data

A YouTube video shows how to process Hubble data using AutoIntegrate script. It includes basic steps to download Hubble data and shows how to do basic processing using AutoIntegrate script.

Video

Some tips for using AutoIntegrate script

Tips for processing One Shot Color (OSC) or RAW files

Other resources

AutoIntegrate resources

Erik Westermann has a web site Remote Astrophotography. He talks about AutoIntegrate in his blog post PixInsight AutoIntegrate.js Processing Script. It has a lot of useful information so please check it out.

Glenn Newell discusses his latest (not so) secret PixInsight weapon, the AutoIntegrate script in the San Jose Astronomical Association (SJAA) meeting. Link to the YouTube presentation is: SJAA Imaging Special Interest Group 05 18 2021

Videos on PixInsight

There are a huge number of videos available on PixInsight. A few that I have seen often recommended are Visible Dark on YouTube or Adam Block on YouTube or on his site.

Reading on PixInsight

Written information on PixInsight may not be as up to date as videos but they are still worth a check. At least the book Inside PixInsight by Warren A. Keller is good.

List of processes used by AutoIntegrate

Below are listed PixInsight processes that may be used by AutoIntegrate script with default LRGB processing. AutoIntegrate writes processing output to the Process Console window and to AutoIntegrate.log file. Details of processing steps can be found from the header block of the source code.

History of AutoIntegrate script

I started my astrophotography hobby with remote telescopes in 2017. I almost immediately got PixInsight and started working with FITS files. From the start I was happy with the results but also found that there are a lot of repetitive and complex tasks to be done. After a few months I even stopped my new hobby for a while. When I started astrophotography again after the summer 2018 I realized that there is this wonderful scripting capability in PixInsight. So I started developing my script. FITS files from remote telescopes were already calibrated so I did some first steps around ImageIntegration. This is where the script name comes from. Since then I have expanded the script into both image calibration and post processing.

Credits

Many thanks to Jean-Marc Lugrin for contributing brilliant Crop to common area code.

Many thanks to rob pfile for contributing the very useful Window name prefix and icon column position code. They are a great help when working with multiple images or multiple image versions.

Jean-Marc Lugrin has also created a great TestAutoIntegrate tool that automates testing. You can find it at GitHub. I have a slightly modified version of that code that works with the latest AutoIntegrate version.

Special thanks to Garth Hunt, Erik Westermann, Glenn Newell and Jari Backman for testing different versions of the script.

Thanks to Erik Westermann for doing a great work on documenting the script in his web site https://remoteastrophotography.com/.

AutoIntegrate Normalization option uses similar PixelMath expressions as Bill Blanshan in his Narrowband Normalization using PixInsight Pixelmath script. See more information in his YouTube channel AnotherAstroChannel.

Narrowband colorizing is inspired by Steven Miller's YouTube channel Entering Into Space, NBColourMapper script from Mike Cranfield and Adam Block, and CombineImages script by Dean Carr.

Star Reduction equations using PixelMath are created by Bill Blanshan. For details see this YouTube video

Linear Defect Detection code is using the code from Vicent Peris. The PreviewControl module is from Andres del Pozo.

Foraxx and Dynamic palettes, credit The Coldest Nights.

PixInsight scripts that come with the product were a great help when developing this script. Website Light Vortex Astronomy was a great place to find details and best practices when using PixInsight. Unfortunately that site is no longer available.

Routines ApplyAutoSTF and applySTF used by AutoIntegrate are from scripts that are distributed with PixInsight. Routines for Linear Defect Detection are from PixInsight scripts LinearDefectDetection.js and CommonFunctions.jsh that are distributed with PixInsight.

This product is based on software from the PixInsight project, developed by Pleiades Astrophoto and its contributors (https://pixinsight.com/).

Feedback: forums.ruuth.xyz or info@astroimagetools.net