File won't load after simulation


#1

I’m batch running simulations by generating lots of files with slightly different parameters and then loading each file to extract the simulation data. I’m doing this on a linux cluster.

In my latest simulation run my data extraction code gets stuck on the 4th file at the load file command. I’ve also tried opening the 4th file manually but nothing happens, the command line stays busy but doesn’t open the file. The simulation log file shows nothing out the ordinary, the simulation completed successfully, it’s just the file won’t load.

Anyone have any experience with this problem?


#2

@md14196,

What do you mean the file won’t load?

I’ve also tried opening the 4th file manually but nothing happens, the command line stays busy but doesn’t open the file. The simulation log file shows nothing out the ordinary, the simulation completed successfully, it’s just the file won’t load.

Is this the first time this error occurs when generating your simulations using your script or batch command? Do you use a scheduler to run your simulations? Can yo share the [simulation_file]_p0.log file? Or the commands you use to run the simulations on the cluster?


#3

Hi lyap,

When I used the command ‘load(‘filname.fsp’)’ the command line goes busy but nothing happens. This is also true if I try and load to the GUI.

I’ve never had this issue before generating and simulating files on the cluster using this same piece of code. All I changed in the generation script was the number of parameters to step over. The log file is normal, and the same as all the other log files for files that open normally. I have tried to rerun the simulation and still the same file does not load.

Here is the log file for the file that won’t load:

FDTD Solutions Version 8.16.1022 (Linux 64bit)
 expires (month/day/year): 5/12/2017
using config file: /home/md14196/.config/Lumerical/FDTD Solutions.ini
license host: <a href=http://ze-ls0.fen.bris.ac.uk:8095>ze-ls0.fen.bris.ac.uk:8095</a>
current time: Thu Mar 23 20:58:30 2017
Running on host: iqt.phy.bris.ac.uk
All processes are communicating and ready to proceed to simulation...
Running fsp file: /opt/lumerical/fdtd/bin/fdtd-engine-mpich2nem pillarsweep4.fsp
number of processors is 12
processors are distributed as: 1x2x6
Running 3D Simulation
Starting meshing
Simulation size in gridpoints: 24 x 19 x 17
Refined 0 mesh cells (0%)
Completed meshing!
Meshing complete in 0 seconds of CPU time
0 0
Starting meshing
Simulation size in gridpoints: 24 x 19 x 17
Refined 0 mesh cells (0%)
Completed meshing!
Meshing complete in 0 seconds of CPU time
0 0
Starting meshing
Simulation size in gridpoints: 24 x 19 x 17
Refined 0 mesh cells (0%)
Completed meshing!
Meshing complete in 0.01 seconds of CPU time
0 0
finished material properties
Selecting 3d kernel for splitFieldDirection=0 and optLevel=2
-----------------------------------------
Beginning initialization of 3D Simulation
Starting to add sources to simulation
Reverted to narrowband source injection
Adding 3D Gaussian Source propagating in Z
Starting to add boundary conditions to simulation
Adding 3D Periodic BC
Adding 3D Periodic BC
Adding 3D Parallel BC
Adding 3D Parallel BC
Adding 3D PML BC
Adding 3D Parallel BC
Starting to add monitors to Simulation
Adding analysis monitor
Adding 3D DFT monitor
Adding 3D DFT monitor
Finished initialization of 3D Simulation
-----------------------------------------
All processes are communicating and ready to proceed to simulation...
Starting 38769 total iterations
0% complete. Max time remaining: 21 secs. Auto Shutoff: 1
1% complete. Max time remaining: 20 secs. Auto Shutoff: 1
3% complete. Max time remaining: 19 secs. Auto Shutoff: 0.00081566
5% complete. Max time remaining: 18 secs. Auto Shutoff: 0.000135983
11% complete. Max time remaining: 16 secs. Auto Shutoff: 4.93271e-05
21% complete. Max time remaining: 14 secs. Auto Shutoff: 2.09504e-05
25% complete. Max time remaining: 13 secs. Auto Shutoff: 9.46058e-06
Early termination of simulation, the autoshutoff criteria are satisfied.
Completed 9769 iterations, or 2.519772e-13s of Simulation Time
Process 0 starting to collect data
    collecting monitor data
    collecting source data
Reverted to narrowband source injection
Finished collecting data
Simulation completed successfully at: Thu Mar 23 20:58:34 2017
Overall wall time measurements in seconds:
    time to mesh and initialize: 0
    time to run FDTD simulation: 4
    time to finalize data and save to files: 0
    FDTD solver speed: 18.9323 Mnodes/s
    total FDTD solver speed on 12 processes: 227.188 Mnodes/s

The file generation script:

clear;
closeall;

Dx_points=100;
Dy_points=Dx_points;
freq_pts=10;

# 0 is x, 90 is y
setnamed("source","polarization angle",0);
setnamed("::model","wavelength",404.3e-09);
setnamed("pillars","height",231e-9);
D_max=getnamed("::model","lattice_constant");
Dx_span=linspace(50e-09,D_max,Dx_points);
Dy_span=linspace(50e-09,D_max,Dy_points);
count=0;

for(i=1:Dx_points){
    
    for(j=1:Dy_points){
        count=count+1;
        ?count;
        
        switchtolayout;
        setnamed("pillars","Dx",Dx_span(i));
        setnamed("pillars","Dy",Dy_span(j));
        
        filename="pillarsweep" + num2str(count) + ".fsp";
        save(filename);
        
    }
}

The bash script that runs the file generation script and then runs the simulations on the files:

 #!/bin/bash          
echo Generating simulation files

rm pillarsweep*.fsp
rm pillarsweep*_p0.log

/opt/lumerical/fdtd/bin/fdtd-solutions pillar_design.fsp -nw -run pillarsweep_filegen.lsf

echo Running simulation files

for f in pillarsweep*.fsp
do
echo $f
/opt/lumerical/fdtd/mpich2/nemesis/bin/mpiexec -n 12 /opt/lumerical/fdtd/bin/fdtd-engine-mpich2nem $f
done

#4

Based on the log file generated by the simulation - it was able to complete. Can you try and copy the simulation back to your local desktop and open the simulation with FDTD CAD/Design window and check if you can see the results?