Photonics Inverse Design: The algorithm sometimes freezes when calculating gradients

[sampr]

Hi,

I am running an optimization similar to what I have described in a previous post with the same Lumerical version:

Inverse Design: Bounds are not working

Hi, I have started playing around with photonics inverse design, but have run into issues in the optimization algorithm, with optimization variables going outside their defined bounds. I am trying to run an inverse design of an MMI structure, where I try to optimize the shape of the waveguide using Automation API. I downloaded the y-branch example, and was able to run it with no problem. I then modified the file for my application. Below is what the final structure looks like. The rectangular…

I am running into an issue where the optimization works quite nicely (and I am able to achieve quite low loss structures), but occasionally lumopt freezes at what I assume is the gradient calculation. Usually it takes a minute or two to calculate the gradients, but when it freezes, I can wait for several hours, and nothing happens. See figure below for where it exactly freezes:

If I run the same optimization again, it freezes at exactly the same spot. I have been able to circumvent this problem by changing the input parameters slightly, which also changes the gradients, but this is not a solution to the problem.

I was wondering what could be causing this.

[trobertson]

Hello @sampr,

Hmm this is strange behavior. If you could share your file I would like to try and replicate it.

Is the GUI open at this point? Does it seems to freeze or can you interact with it? Does the optimizer eventually find the results or it just stays stuck as far you can tell?

Thanks,

[sampr]

Hello @trobertson

As far as I can tell, everything, including the optimizer, stops, since I stop getting any results. So far I have always close the command prompt, and restart the simulation with slightly perturbed initial parameters.

If by the GUI you mean the plots of the FOM, the gradient plot etc., I cannot interact with it since it is “not responding”. However, it is not responding even when the simulations and the optimizer are running normally.

Attached is a simulation where I tried twice and both times it got stuck during the gradient calculations on the first iteration. First time it stopped at the 46th gradient, and the second time on the 39th gradient, so not actually at the same spot.

Could it be the case that my mesh is too coarse?

MMI_opt_2D_co-opt_mod_tapered.py (20.9 KB) varFDTD_MMI_coopt.py (5.2 KB)

[sampr]

I occassionally (not every time when it gets stuck) get a memory write/read error, as well. However, this could be unrelated.

[trobertson]

Hello @sampr,

I have experienced both of these myself recently, and I am not sure exactly what is causing the problem. It seems hard to reproduce, so I am not how to log this as bug.

I am pretty sure it has to do with a lack of memory. I’ve found that reducing the number of other tasks, and changing the mesh size (bigger) to make the d_eps matrices smaller can help. Also it seems less likely to happen when you run less simulations in parallel on the same node.

The memory error message seems to only occur on one of the computer I use. It is a virtual machine, running windows. Does that sounds familiar?

Please let me know if you have further info.

Best Regards,

[sampr]

Hello @trobertson

I have so far managed to circumvent the problem by decreasing the amount of nodes on the polygon, as well as using a constant value for the permittivity of the material. This would imply that it is indeed an issue related to the lack of memory.

For the record, the machine I am running the simulation on, is not a virtual machine.