smooth

1 Menu Information
2 Brief Information
3 Additional Information
4 Command Line Usage
5 Variables
6 Examples
7 Algorithm
8 More Information
9 Related X-Functions

Menu Information

Smoothing

Brief Information

Perform smoothing to irregular and noisy data.

Additional Information

This feature is updated in 8.0 SR3. For more details, please refer to Release Notes.

Command Line Usage

1. smooth iy:=Col(2);

2. smooth iy:=Col(2) method:=aav npts:=51;

3. smooth iy:=Col(2) boundary:=periodic;

4. smooth iy:=Col(2) polyorder:=3;

5. smooth iy:=Col(2) method:=pf percent:=70;

Variables

Display Name	Variable Name	I/O and Type	Default Value	Description
Input	iy	Input XYRange	<active>	Specifies the input XY data range to be smoothed
Method	method	Input int	sg	Specifies the smoothing method to be used. The default is Savitzky-Golay. Option list aav: Adjacent-Averaging This method essentially takes the average of a certain number (which you specify) of data points around each point in your data and replaces that point with the new average value. sg: Savitzky-Golay This method is actually performing a local polynomial regression. It is best suited to remove noise without affecting the overall signal shape. Original data values are replaced with fitted values. pf: Percentile Filter This method is also known as rank-order filter. It essentially finds the specified percentile of data points in the data window around each point in your data and replaces that point with the percentile. It is best suited for removing shot noise. fft:FFT Filter This method is based on FFT low-pass filter. It should be used when the frequency of noise is higher than the true signal. In essense, it removes the high-frequency components with a parabolic window. Frequencies not less than the cutoff frequency will be discarded.
Weighted Average	weight	Input int	0	This parameter is available only when Method is Adjacent-Averaging. When this parameter is set to 1 (checked), weighted averaging will be used.
Points of Window	npts	Input int	5	If FFT Filter is not selected for Method, this parameter specifies the number of points in the moving window. Otherwise, you can use this parameter to control the cutoff frequency for the FFT filter. The greater this value, the greater the degree of smoothing. A value of zero for this parameter will leave the data unsmoothed. See the Algorithm part for the computation of cutoff frequency.
Boundary Condition	boundary	Input int	none	Specifies the boundary condition. It is not available when FFT Filter is selected for Method. Option list none:None Does no padding reflect:Reflect Pads data by reflecting signal at both ends repeat:Repeat Pads data by repeating the ending point at both ends periodic:Periodic Treats data as periodic signal extrap:Extrapolate Uses linear extrapolation to pad signal at both ends
Polynomial Order	polyorder	Input int	2	This parameter applies only to the Savitzky-Golay smoothing option. It specifies the polynomial order.
Percentile	percent	Input double	50	This parameter determines the percentile to be used in percentile filtering. The default is 50%, which corresponds to replacing the signal point with median value of the data points in the data window.
Cutoff Frequency	cutoff	Input double	0	This parameter is visible in GUI when FFT Filter is selected for Method. It is a read-only variable showing the cutoff frequency of the low-pass filter calculated from Points of Window. Please do not use it in script.
Pad to Power of 2 Points	pad	Input int	0	This parameter is available only when FFT Filter is selected for Method. It is available only from script. It specifies whether to pad the number of data points to power of 2. When this is set to 1, the result will match the result calculated with Origin prior to the Version 8. See more details in the Algorithm part.
Filter Type	filter	Input int	new	This parameter is available only when FFT Filter is selected for Method. It is available only from script. It specifies the filter type. When old is selected, the result will match the result calculated with Origin prior to the Version 8. Option list: new:Low-pass parabolic low-pass parabolic filter old:Low-pass parabolic(7.5) low-pass parabolic filter which is used in Origin 7.5
Baseline to Subtract	baseline	Input int	two_ends_1	This parameter is available only when FFT Filter is selected for Method. It is available only from script. It specifies whether to subtract a baseline before fft filtering is used. Option list: two_ends:Straight Line Obtained from End Points Subtracts the input data with a straight line constructed from the two end points before fft filtering is applied. two_ends_1:Straight Line Obtained from the mean of 1% End Points Subtracts the input data with a straight line constructed from the mean of 1% end points before fft filtering is applied. none:None No subtraction is performed on input data before fft filtering is applied.
Output	oy	Output XYRange	<new>	Specifies the destination of smoothed signal. See the syntax here.

Examples

1. To perform Savitzky-Golay filtering using default settings, to XY data in columns 1, 2 of the active worksheet, use the script command:

smooth (1,2)

2. To perform default smoothing to active data plot in graph, use the script command:

smooth %c

3. To change the method of smoothing, either open the smoothing dialog and select a method from the drop down menu by typing:

smooth -d

Or specify which method of smoothing to use from the script line by typing, for example,

smooth m:=2

m:=1 corresponds to the (default) Savitzky-Golay method, m:=0 executes the Adjacent-Averaging method, and m:=2 corresponds to the percentile filter method.

4. To perform smoothing to column 1, 2 data using a pre-saved smoothing theme file, save your preferences in the smooth dialog, and then execute it by typing (for example, but using your own saved-theme title) the script command:

smooth -t "my shot noise removal theme.oth"

For more examples, please refer to XF Script Dialog (press F11).

Algorithm

Moving window in adjacent-averaging, Savitzky-Golay or percentile filter method

When the smoothing method is adjacent-averaging, Savitzky-Golay or percentile filter, each smoothed data point is computed from data points within a moving window. Let { $f i | i = 1,2,..., N$ } be the input data points and let { $g i | i = 1,2,..., N$ } denote the output data points. Each $g i$ is computed from { $f m | i - f l o o r (n p t s / 2) < m < i + f l o o r (n p t s / 2)$ }, where npts is the value of the Points of Window variable. However, when the smoothing method is FFT filter, moving window is not used. Instead, the whole signal is processed.

The adjacent-averaging method

The adjacent-averaging method performs the simplest possible averaging procedure: each $g i$ is the average of the data points within the moving window. If the Weighted average option is used, the average will be computed using weighted averaging. In this case, a parabolic weight is used, with the weight area normalized to 1. For a window whose center is in i, the weight which corresponds to the jth (j=0, 1, ... npts-1) point is:

$w_j=\frac{1-(j-i)^2}{NormalizedFactor}$

The Savitzky-Golay method

The Savitzky-Golay method performs a polynomial regression to the data points in the moving window. Then $g i$ will be computed as the value of the polynomial at position i.

The percentile filter method

For the percentile filter, the pth quantile of the points in the moving window is assigned as $g i$ , where p is specified by the parameter, Percentile. The pth quantile (or 100 pth percentile) is computed from the empirical distribution function as follows:

Let $npts \cdot p/100=j+g$

where j is the integer part of $npts \cdot p/100$ , and g is the fractional part of it.

Then we can compute the pth quantile, which is denoted by y, with the following equations:

$y = \begin{cases} x_j, & \mbox{if } g = 0 \\ x_{j+1}, & \mbox{if } g > 0 \end{cases}$

where $x j$ is the jth (j=0, 1, ... npts-1) point in the moving window.

The FFT Filter method

When the FFT Filter method is selected, Origin performs the following:

If the baseline variable is set to 0 (two_ends) or 1 (two_ends_1) , this X-Function subtracts a baseline (see below for its construction) from the input data. Otherwise, skip this step.
If the Pad variable is set to 1, pad the dataset so that its size is the power of 2. Otherwise, skip this step.
Perform FFT on the dataset acquired in last step.
Apply filtering with the low-pass parabolic filter. The filter type depends on the filter variable. See below for details.
Perform IFFT on the filtered spetrum.
If baseline subtraction is performed in the first step, add the baseline back to the dataset acquired in last step.

The method used to construct the baseline depends on the baseline variable. If baseline is 0, this X-Function connects the first and last data points in the input data to form the baseline; If baseline is 1, it calculates the mean of the first 1% data points and the mean of the last 1% data points and then connects these mean values with a straight line.

The type of filter used in the filtering process depends on the filter variable. If filter is 0, the low-pass parabolic filter described in the Algorithm part of the X-Function document for FFT Filter will be used. Otherwise, the following window (which is used in Origin 7.5) will be used: $W(t) = \begin{cases} 1- \frac{4 p^2}{n^2}t^2, & 0 \le t \le \frac{n}{2p} \\ 0, & \frac{n}{2p} < t < n-\frac{n}{2p} \\ 1- \frac{4 p^2}{n^2}{(1-t)}^2, & n-\frac{n}{2p} \le t \le n \end{cases}$

Note: In Origin 7.5, padding is always used and baseline subtraction always uses the two ends method. To achieve same results as in Origin 7.5, set pad to 1, filter to 1 and baseline to 0.

More Information

For more information, please refer to our User Guide.

Related X-Functions

fft1, fft_filters, wtsmooth