Performance considerations for saving and loading matlab(*.mat) files

matlabsave
matlab
matlabsavelegacy
performance

#1

Variables in Lumerical Sript Workspace can be saved, among others, into a *.mat file by matlabsave(); and matlabsavelegacy(); commands. This allows the user to load the data into Matlab or in another Lumerical product.

matlabsave command saves the data in the latest v7.3 format, which is using HDF5 format and uses data compression.

matlabsavelegacy saves the files in an older format such as v6, which does not use HDF5.

In most cases, using the latest format is the best option and requires little or no considerations. Nevertheless, if we save large volumes into the matlab file and we will load them repeatedly, there are some performance considerations to be taken into account.
Most notably, loading the compressed data can require more time if the compression ratio is not significant and it could be better to use the older, uncompressed format.

EXAMPLE
In the following example, we will generate a large matrix and save them into different versions of matlab file. Afterwards we load them in Matlab and measure the time required to complete the loading of each file.

A script to generate a large three dimensional matrix populated by random data in Lumerical and saving them with matlabsave and matlabsavelegacy (Note: This can take quite long time ~1.5GB of data):

clear;

#Define matrix size
i=1000;
j=1000;
k=200;

#min and max values
min_value=1e-6;
max_value=1e6;

#Seed for randreset
seed=869;


#Generate the matrix and fill it with random values
randreset(seed);

m=matrix(i,j,k);

    for (n=1:length(m)) {
    m(n)=rand(min_value,max_value,1);
    }


matlabsave("v73_file.mat");
matlabsavelegacy("v6_file.mat");

?'done';

A script to load the files in Matlab and measure the time required to complete the loading of each file:

tic
load('v6_file.mat');
t1=toc;
t1
clear;

tic
load('v73_file.mat');
t2=toc;
t2

The test results show that on my workstation (12 cores, 30GB RAM and conventional HDD) it takes ~9 seconds to load the legacy file and ~15 seconds to load the v7.3 file saved with matlabsave command:

The difference can be even more pronounced with different hardware and dataset configuration.