Data Import tutorial (Horiba AquaLog)
This is a tutorial that will demonstrate how to
- import data from a Horiba AquaLog fluorometer into MATLAB and
- assemble a dataset.
This tutorial shows focusses on data exported from a Horiba AquaLog. In the AquaLog software, the data export was performed using the
HYI export option under the file menu. The advantage of this export is that sample, blank, and absorbance scans are exported per sample and confusion is minimized in this way.
Note: AquaLog instruments come spectrally calibrated from the manufacturer. No emission or excitation correction factors will be applied here. It is your responsibility to ensure proper calibration from time to time.
Also, different instruments may not be correcting for spectral biases themselves and these corrections have to be applied manually!
You can copy & paste the code sections in this tutorial directly into your own MATLAB script. If this document is open in the MATLAB help browser (called with
doc), you can right-click on code sections and evaluate them line-by-line (also by pressing
F9 on windows)
If you want the code without any comments given in this tutorial, type:
The drEEM toolbox must be installed for this command to work.
The sections of this tutorial
- Data Import tutorial (Horiba AquaLog)
- Set up the toolbox
- Read in fluorescence files
- Read in absorbance files
- Check file alignment
- Resize EEMs depending on absorbance data
- Correct EEMs
- Consolidate the data
- Save the data
Set up the toolbox
Before we start this tutorial, we need to make sure that the drEEM toolbox is properly installed and the functions are ready to import a example dataset. This set of data is called
demofiles.zip and is stored in the drEEM toolbox folders.
If you downloaded the drEEM toolbox to your matlab user folder (the folder returned when you call
userpath), simply call
cd 'C:\Users\expertuser\somefolder\drEEM\' % Change this!
then, we ensure proper installation of drEEM and availability of the demo files:
Next, we should provide some necessary details and make choices on where the data should be saved. This section can later be modified when different settings are used or the next data import occurs.
datalocation = [fileparts(which('dreem_demofiles_2.zip')),filesep,'demofiles_AL_HYI']; savelocation = 'C:\Work\folder1\imported\Xin.mat'; % Where should the dataset be saved to? flusize = 'A2..DR126'; % EEMs limits, open in Excel and note where the EEM begins and ends. abssize = 'A4..J124'; % Absorbance limits, open in Excel and locate the absorbance column.
Read in fluorescence files
Now that we have the toolbox installed and the information collected, we can begin the data import. First, the samples are imported:
cd(datalocation); [X,Emmat,Exmat,fl,outdata]=readineems(3,'*Waterfall Plot Sample.dat',flusize,[0 1],0,0); DS = assembledataset(X,Exmat(1,:),Emmat(:,1),'RU','filelist',fl,); clearvars -except DS datalocation savelocation flusize abssize
Next, the blanks:
[X_b,Emmat_b,Exmat_b,fl_b,outdata_b]=readineems(3,'*Waterfall Plot Blank.dat',flusize,[0 1],0,0); DSb = assembledataset(X_b,Exmat_b(1,:),Emmat_b(:,1),'RU','filelist',fl_b,);
Let’s clean up a bit before we continue
clearvars -except DS DSb datalocation savelocation flusize abssize
Read in absorbance files
In this next step, we read in absorbance scans. They are automatically marked with the filename
'* - Abs Spectra Graphs' for this specific instrument.
[S_abs,W_abs,wave_abs,filelist_abs]=readinscans('Abs','dat_1_10',abssize,0,0,'* - Abs Spectra Graphs');
Next, we generate an absorbance dataset
A=[wave_abs;S_abs]; %add wavelengths to Absorbance scans Abs.A=A; %wavelengths are in the first row. They will be removed later. Abs.filelist=cellstr(filelist_abs); Abs.wave=wave_abs;
Let’s clean up a bit before we continue
clearvars -except DS DSb Abs datalocation savelocation flusize abssize
Check file alignment
Even though the AquaLog automates file export, it can happen that some files go missing. Since they were separately imported, we must check that all samples match up correctly: eems, blanks and Abs. Otherwise, we will see an error message later on, when all data are combined into one dataset. This section of code intends to give more specific information on which type of file may be missing.
Also: Always confirm that files are matched correctly. Otherwise, a sample may be matched with the wrong blank or an inappropriate absorbance scan!
This is the reason a pause was written into this code.
try disp([cellstr(num2str((1:DS.nSample)')) DS.filelist DSb.filelist]) catch error('Missmatch: DS.nSample, DS.filelist, DSb.filelist') end try disp([DS.filelist DSb.filelist Abs.filelist]) catch error('Missmatch: DS.filelist, DSb.filelist, Abs.filelist') end try disp([cellstr(num2str((1:DS.nSample)')) DS.filelist DSb.filelist Abs.filelist]) catch error('Missmatch: DS.nSample, DS.filelist, DSb.filelist, Abs.filelist') end warning('Check that the samples are aligned, press enter when ready') pause;
Resize EEMs depending on absorbance data
If absorbance was measured 240-600 then you can not calculate inner filter effect correction factors for any other wavelengths. These data would have to be deleted.
disp('Abs wavelengths: min and max') AbsRange=[min(Abs.wave) max(Abs.wave)]; disp(AbsRange) disp('EEM wavelengths: min and max') EEMRange=[min([DS.Em' DS.Ex']) max([DS.Em' DS.Ex'])]; disp(EEMRange)
Now, we remove wavelengths below 240 and above 600 from DS and Blanks
Xin=subdataset(DS,,... logical(double(DS.Em<AbsRange(1))+ double(DS.Em>AbsRange(2))),... logical(double(DS.Ex<AbsRange(1))+ double(DS.Ex>AbsRange(2)))); B=subdataset(DSb,,... logical(double(DS.Em<AbsRange(1))+ double(DS.Em>AbsRange(2))),... logical(double(DS.Ex<AbsRange(1))+ double(DS.Ex>AbsRange(2))));
Next, we correct the EEMs for the presence of inner filter effects and normalize EEMs by the Raman peak intensity. In drEEM, this is all done by
The data in this tutorial do not contain an excitation scan at Ex = 350 nm, so we will extract the Raman scan in Blanks at 351 nm instead. After extraction, we will add the Raman scans to the blank dataset
RamanWavel=351; Sr=squeeze(B.X(:,:,DSb.Ex==351)); W=[B.Em';Sr]; %2D Matrix of matched (350 nm) Raman scans B.W=W; [XcRU Arp IFCmat BcRU XcQS QS_RU]=fdomcorrect(Xin.X,Xin.Ex,Xin.Em,... [Xin.Em ones(size(Xin.Em))],[Xin.Ex ones(size(Xin.Ex))],... % These are dummy correction factors (ones) B.W,[351 381 426],Abs.A,B.X,B.W,,);
Consolidate the data
Now, we are ready to assemble the final dataset.
Note: AquaLog instruments are spectrally calibrated. No emission or excitation correction factors were applied here. It is your responsibility to ensure proper calibration from time to time. Also, different instruments may not be correcting for spectral biases!
Xin.X=XcRU; %X is in Raman Units Xin.RamanArea=Arp; %Raman Peak Area Xin.IFE=IFCmat; %IFE correction factors Xin.Abs_wave=Abs.A(1,:); %Abs wavelengths Xin.Abs=Abs.A(2:end,:); %Abs data Xin.Ram_wave=B.Em'; %Water Raman wavelengths Xin.Emcor='internally corrected by AquaLog'; %corrected by AquaLog not drEEM Xin.Excor='internally corrected by AquaLog'; %corrected by AquaLog not drEEM Xin.RamOpt=[350 381 426]; %Raman integration range on Em spectrum Xin.RamSource='Water Raman scans extracted from blanks'; clearvars -except Xin
Finally, we want to make sure that all data are present and the dataset is fully compatible with drEEM.
This should return a message like this:
>> checkdataset(Xin) Checking the validity of your dataset... Success: All required dataset variables are present Success: The size of your EEM cube is consistent with Ex, Em, and i (incl. nEx, nEm, nSample) Success: The fields i, Ex, and Em are correctly stored as column vectors. Success: Emission and excitation wavelengths steadily increase. Success: EEMs content suggests that Ex and Em information is correct. Success: NaN-percentage < 20% Success. The analysis indicates that your dataset is correctly formatted.
Save the data
Last step: Save the data!