Pre-Processing

Pre-Processing

Temperature and frequency corrections for Raman spectra

Raman spectra can be corrected from temperature and excitation line effects using this function.

Spectra.tlcorrectionFunction.
tlcorrection(data::Array{Float64},temp::Float64,wave::Float64;correction="long",normalisation="area",density=2210.0)

Inputs

data: Array{Float64}
	input spectrum with x and y in first and second columns respectively
temp: Float64
	temperature in °C
wave: Float64
	wavenumber at which the spectrum was acquirred in nm

Options

correction: String
	equation used for the correction. Choose between "long", "galeener", or "hehlen". Default = "long".
normalisation: String
	indicate if you want to normalise your signal or not. Choose between "intensity", "area", or "no". Default = "area".
density: Float64
	density of the studied material in kg m-3, to be used with the "hehlen" equation. Default = 2210.0 (density of silica).

Outputs

x: Array{Float64}
	x values
long: Array{Float64}
	corrected y values
eselong: Array{Float64}
	errors calculated as sqrt(y) on raw data and propagated after the correction.

Notes

This correction uses the formula reported in Galeener and Sen (1978), Mysen et al. (1982), Brooker et al. (1988) and Hehlen et al. (2010).

The "galeener" equation is the exact one reported in Galeener and Sen (1978), which is a modification from Shuker and Gammon (1970) for accounting of (vo - v)^4 dependence of the Raman intensity. See also Brooker et al. (1988) for further discussion.

The "long" equation is that of Galeener and Sen (1978) corrected by a vo^3 coefficient for removing the cubic meter dimension of the equation of "galeener". This equation has been used in Mysen et al. (1982), Neuville and Mysen (1996) and Le Losq et al. (2012).

The "hehlen" equation is that reported in Hehlen et al. (2010). It actually originates before this publication (Brooker et al. 1988). It uses a different correction that avoid crushing the signal below 500 cm-1. THerefore, it has the advantage of keeping intact the Boson peak signal in glasses.

source

Smoothing signals

Smoothing the signal is achieved with the smooth function.

Spectra.smoothFunction.
smooth(x,y;method="whittaker", window_length=5, polyorder = 2, Lambda = 10.0.^5, d=2, ese_y=1.0)

smooth the provided y signal (sampled on x)

Inputs

x: vector
	Nx1 array of x values (equally spaced).
y: vector
	Nx1 array of y values (equally spaced).
method: str
	Method for smoothing the signal;
	choose between savgol (Savitzky-Golay), GCVSmoothedNSpline, MSESmoothedNSpline, DOFSmoothedNSpline, whittaker, 'flat', 'hanning', 'hamming', 'bartlett', 'blackman'.

Options

window_length: int
	The length of the filter window (i.e. the number of coefficients). window_length must be a positive odd integer.
polyorder: int
	The order of the polynomial used to fit the samples. polyorder must be less than window_length.
Lambda: float
	smoothing parameter of the Whittaker filter described in Eilers (2003). The higher the smoother the fit.
d: int
	d parameter in Whittaker filter, see Eilers (2003).
ese_y: ndarray
	errors associated with y (for the gcvspline algorithms)

Outputs

y_smo: ndarray
	smoothed signal sampled on x.

Note

Use of GCVSmoothedNSpline, MSESmoothedNSpline, DOFSmoothedNSpline requires installation of gcvspline. See gcvspline documentation. See also documentation for details on GCVSmoothedNSpline, MSESmoothedNSpline, DOFSmoothedNSpline.

savgol uses the scipy.signal.savgol_filter() function.

References

Eilers, P.H.C., 2003. A Perfect Smoother. Anal. Chem. 75, 3631–3636. https://doi.org/10.1021/ac034173t

Scipy Cookbook: https://scipy-cookbook.readthedocs.io/items/SignalSmooth.html?highlight=smooth

source

Baseline subtraction

Baseline subtraction can be made with using the baseline function:

Spectra.baselineFunction.
baseline(x::Array{Float64},y::Array{Float64},roi::Array{Float64},basetype::AbstractString;polynomial_order=1, s = 1.0, lam = 10^5, p = 0.01, ratio = 0.01, niter = 10, p0_exp = [1.,1.,1.],p0_log =[1.,1.,1.])

Allows subtracting a baseline under a x y spectrum; uses the baseline function from the rampy Python package.

Parameters

x_input : ndarray
	x values.
y_input : ndarray
	y values.
bir : ndarray
	Contain the regions of interest, organised per line.
	For instance, roi = np.array([[100., 200.],[500.,600.]]) will
	define roi between 100 and 200 as well as between 500 and 600.
	Note: This is NOT used by the "als" and "arPLS" algorithms, but still is a requirement when calling the function.
	bir and method probably will become args in a futur iteration of rampy to solve this.
methods : str
	"poly": polynomial fitting, with splinesmooth the degree of the polynomial.
	"unispline": spline with the UnivariateSpline function of Scipy, splinesmooth is
				 the spline smoothing factor (assume equal weight in the present case);
	"gcvspline": spline with the gcvspl.f algorythm, really robust.
				 Spectra must have x, y, ese in it, and splinesmooth is the smoothing factor;
				 For gcvspline, if ese are not provided we assume ese = sqrt(y).
				 WARNING: Requires the installation of the gcvspline Python package prior to use in the Python ENV used by Julia.
				 See website for install instructions
	"exp": exponential background;
	"log": logarythmic background;
	"rubberband": rubberband baseline fitting;
	"als": automatic least square fitting following Eilers and Boelens 2005;
	"arPLS": automatic baseline fit using the algorithm from Baek et al. 2015
			 Baseline correction using asymmetrically reweighted penalized least squares smoothing, Analyst 140: 250-257.

Options

polynomial_order : Int
	The degree of the polynomial (0 for a constant), default = 1.
s : Float
	spline smoothing coefficient for the unispline and gcvspline algorithms.
lam : Float
	float, the lambda smoothness parameter for the ALS and ArPLS algorithms. Typical values are between 10**2 to 10**9, default = 10**5.
p : Float
	float, for the ALS algorithm, advised value between 0.001 to 0.1, default = 0.01.
ratio : float
	ratio parameter of the arPLS algorithm. default = 0.01.
niter : Int
	number of iteration of the ALS algorithm, default = 10.
p0_exp : List
	containg the starting parameter for the exp baseline fit with curve_fit. Default = [1.,1.,1.].
p0_log : List
	containg the starting parameter for the log baseline fit with curve_fit. Default = [1.,1.,1.,1.].

Returns

out1 : ndarray
	Contain the corrected signal.
out2 : ndarray
	Contain the baseline.

Example

Consider a Y signal sampled at X. We want to fit a polynomial baseline (2nd order) in the regions of interest comprised between x1 and x2, as well as x3 and x4.

```julia-repl
julia> Y_corr, Y_baseline = baseline(X,Y,[x1 x2; x3 x4],"poly",polynomial_order=2)
```
source

Frequency shifts correction

In case your spectra are shifted from a reference value, Spectra offers several functions that allows you to correct it from this shift.

To correct a spectrum from a shift of P wavenumbers, you can simply call:

Spectra.xshift_directFunction.
xshift_direct(original_x::Array{Float64}, original_y::Array{Float64}, p::Float64)

To correct a spectrum for a p shift in X.

Used in xshift_correction.

Inputs

original_x: Array{Float64}
	x values
original_y: Array{Float64}
	y values associated with x
p: Array{Float64}
	value of how much x should be shifted

Outputs

original_x: Array{Float64}
	same as input
corrected_y: Array{Float64}
	the y values corrected from the p shift in original_x
p: Array{Float64}
	same as input.
source

Sometime, two signals from the same mineral show a shift in the X axis, while they share a common X axis. To correct from such thing, you can use the function:

xshift_correction(full_x::Array{Float64}, full_shifted_y::Array{Float64}, ref_x::Array{Float64}, ref_y::Array{Float64},shifted_y::Array{Float64})

To correct a shift between two spectra using a reference peak.

Inputs

full_x: Array{Float64}
	x values that are not good
full_shifted_y: Array{Float64}
	y values associated with full_x
ref_x: Array{Float64}
	x values that are good
ref_y: Array{Float64}
	y values associated with ref_x
shifted_y: Array{Float64}
	y values associated with a selected range of full_x that corresponds to ref_x (for instance, a specific peak that you want to use to correct the shift).

Outputs

full_x: Array{Float64}
	same as input
corrected_y: Array{Float64}
	the full_shifted_y values corrected from the shift
p: Array{Float64}
	same as input.

refx is the common X axis of two particular refy and shiftedy signals, that should be for instance an intense and well defined peak in your spectra. If refy and shifted_y do not share the same X axis, you can use first the Dierckx spline to re-sample one of them and have both sharing a common X axis. See the examples for further details.

source

array manipulation

For spectra recorded with decreasing frequencies, use the flipsp() function to put them back with increasing frequencies (necessary for some algo)

Spectra.flipspFunction.
flipsp(spectra::Array{Float64})

Flip an array along the row dimension (dim = 1) if the first column values are in decreasing order.

source

You can also resample a signal at wanted x_new values with resample()

Spectra.resampleFunction.
resample(x::Array{Float64},y::Array{Float64},x_new::Array{Float64})

Resample a y signal associated with x, along the x_new values.

Uses the Dierckx spline library.

source