Currently, when you get the Jira Cloud add-on for Excel, however, once the panel that's pops-up on excel shows up, there's no way to keep that closed until you decide to open it again.

Instead, you have the option to close it for that session, but it keeps opening every-time the excel workbook is opened.