History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: JRA-13295
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Andreas Knecht [Atlassian]
Reporter: Bogdan Dziedzic [Atlassian]
Votes: 1
Watchers: 1
Operations

If you were logged in you would be able to see more operations.
JIRA

JIRA standalone does't run as a Windows service - Failed creating java ... jvm.dll

Created: 10/Aug/07 02:05 AM   Updated: 15/Oct/07 09:57 PM
Component/s: Installation
Affects Version/s: 3.10.1
Fix Version/s: 3.11

Time Tracking:
Not Specified

File Attachments: 1. Zip Archive installation.zip (67 kb)

Environment: clean install of windows XP or windows 2003
Issue Links:
Reference
 

Participants: Andreas Guther, Andreas Knecht [Atlassian], Anton Mazkovoi [Atlassian], Bogdan Dziedzic [Atlassian] and Daniel Schuuring
Since last comment: 44 weeks, 4 days ago
Resolution Date: 26/Aug/07 11:02 PM
Labels:


 Description  « Hide
In a client OS JIRA doesn't start up neither as a service nor as an application (the Start menu).

JIRA installed from atlassian-jira-enterprise-3.10.1.exe and the JIRA(Tomcat) service was set to be installed during the installation (log from installation attached as installation.log). User performing the installation is a member of Administrators and the service is run as the Local System account.

In Event Viewer is recoded:

From the event log
The Atlassian JIRA Enterprise Edition 3.10 service terminated with service-specific error 0 (0x0).

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

in the same time in jakarta_service*.log is recoded:

[2007-08-06 15:46:10] [info] Service JIRA060807154537 name Atlassian JIRA Enterprise Edition 3.10
[2007-08-06 15:46:10] [info] Service JIRA060807154537 installed
[2007-08-06 15:46:10] [info] Procrun finished.
[2007-08-06 15:46:11] [info] Updating service...
[2007-08-06 15:46:11] [info] Service JIRA060807154537 updated
[2007-08-06 15:46:11] [info] Update service finished.
[2007-08-06 15:46:11] [info] Procrun finished.
[2007-08-06 15:46:11] [info] Updating service...
[2007-08-06 15:46:11] [info] Service JIRA060807154537 updated
[2007-08-06 15:46:11] [info] Update service finished.
[2007-08-06 15:46:11] [info] Procrun finished.
[2007-08-06 15:46:13] [info] Running Service...
[2007-08-06 15:46:13] [info] Starting service...
[2007-08-06 15:46:13] [173 javajni.c] [error] The specified module could not be found.
[2007-08-06 15:46:13] [924 prunsrv.c] [error] Failed creating java C:\jira\jre\bin\client\jvm.dll
[2007-08-06 15:46:13] [1179 prunsrv.c] [error] ServiceStart returned 1
[2007-08-06 15:46:13] [info] Run service finished.
[2007-08-06 15:46:13] [info] Procrun finished.

It is still possible to start JIRA by calling startup_bundled_jre.bat or ...\bin\startup.bat

Problem reported by a customer and replicated on a clean Windows XP SP2 (no patches applied), MS Virtual Server 2005 R2 SP1 and JIRA 3.10.1 Ent.
Since the virtualisations product getting more and more popular, especially in hosting environments and the Microsoft solution is one of a popular one we might have more support request related to this product.

Firstly, I thought that this problem is strictly related to MS VM2005, although after some testing I have found out that any fresh install of OS, especially if there isn't %windir%\system32\MSVCR71.dll, will experience this issue.



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Anton Mazkovoi [Atlassian] - 12/Aug/07 06:41 PM
As far as I can tell, this again is more of an issue with Tomcat's service rather than our installer. We will leave this issue open and see how many problems it causes. Please use this issue to capture the amount of interest that you see on running JIRA as a service in MS Virtual Server 2005 R2 SP1.

Also, are we sure this is not a MS Virtual Server 2005 R2 SP1 bug?


Bogdan Dziedzic [Atlassian] - 20/Aug/07 01:29 AM
Please find below (I think) relevant parts from ASF Bugzilla Bug 41538 - Unable to run Tomcat as a Windows service under JDK 1.6 that seem to contain the explanation of the problem and workarounds:

The jakarta_service code uses a LOAD_WITH_ALTERED_SEARCH_PATH option when loading jvm.dll (and all its dependent DLLs).
See the following link for details on what is done when the jvm.dll is loaded by the jakarta_service code:
Dynamic-Link Library Search Order,
http://msdn2.microsoft.com/en-us/library/ms682586.aspx

The behavior depends on:

  • the OS version
  • the registry value HKLM\System\CurrentControlSet\Control\Session
    Manager\SafeDllSearchMode
  • the content of the directory specified for jvm.dll, the current directory, the system directory, the 16-bit system directory, the Windows directory, the directories that are listed in the PATH environment variable.

There are many ways it can go wrong ...
So I suggest that anyone encountering this issue, use the Filemon tool from:
http://www.microsoft.com/technet/sysinternals/FileAndDisk/Filemon.mspx
Launch it before starting the tomcat service, stopping capture after the service start has failed then save the log and have a look at the log to guess which DLL fails to load and if the search path used is correct.
I heavily suspect issues to come from DLL search path including bad versions of some DLLs (and very likely JDK1.5 DLLs while trying to load JDK1.6 DLLs).

I investigated further using Filemon ... and found that Tomcat5.exe was unable to load msvcrt71.dll.

On that machine that particular dll was not present in any of the directories on the search path. In fact it was present in JAVA_HOME\bin which is not added to the path by default. I added JAVA_HOME\bin to Path env variable and now
everything works fine (although I had to reboot the server in order to have the search path modified, merely modifying the Path env variable was not enough).

and than in the below post is a workaround that worked for me:

Tomcat does not ship msvcr71.dll as part of its installer even though Microsoft's deployment guide clearly requires this:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclib/html/_crt_c_run.2d.time_libraries.asp

I quote: "An application should use and redistribute msvcr71.dll, and it should avoid placing a copy or using an existing copy of msvcr71.dll in the system directory. Instead, the application should keep a copy of msvcr71.dll in its application directory with the program executable."

Just because the JRE happens to redistribute msvcr71.dll itself does not mean that Tomcat is not required to do the same. The point of an installer is to unpack the software and fully configure it. By asking users to manually add JAVA_HOME/bin to their PATH (where is this documented even?) you are completely missing this point.

In my humble view you have two options:

1) Follow the Microsoft deployment guide and redistribute msvcr71.dll with Tomcat. Every application I know does this.

2) Alternatively, if you insist on reusing msvcr71.dll shipped with the JRE, Tomcat's installer should either copy it into the tomcat/bin directory or modify tomcat5.exe to explicitly look for it in jre/bin.

If you experience this issue and you copy msvcr71.dll to <JIRA home path>\bin, you will be able to start the Tomcat windows service.


Daniel Schuuring - 21/Aug/07 01:28 AM
Bogdan,

From the filemon tools, I got the same information as you.

7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\Program Files\JIRA-Enterprise-3.10\bin\MSVCR71.dll NOT FOUND
Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\system32\MSVCR71.dll NOT FOUND Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\system\MSVCR71.dll NOT FOUND Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\MSVCR71.dll NOT FOUND Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\system32\MSVCR71.dll NOT FOUND Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\system32\MSVCR71.dll NOT FOUND Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\MSVCR71.dll NOT FOUND Attributes: Error
7:50:59 AM tomcat5.exe:5796 QUERY INFORMATION
C:\WINDOWS\System32\Wbem\MSVCR71.dll NOT FOUND Attributes:
Error

I simply copied the MSVCR71.dll from the Java\JDK to the JIRA\bin
folder. Everything now runs.

I also tried adding "C:\Program Files\Java\jdk1.6.0_02" to the
environmental PATH, this works as well.

So you actually have two options.

Many thanks for your help.

Daniel


Bogdan Dziedzic [Atlassian] - 21/Aug/07 08:14 PM
This is Tomcat specific issue - successfully replicated with the Tomcat's installer and than fixed with the above workaround.

Since JIRA standalone uses Tomcat as web server and Procrun to install/set the JIRA's windows service, finally the JIRA's installer should deliver nicely configured JIRA (it should work of the box - no work around needed), we should address this issue. The issue is quite easy to replicate, therefore it shouldn't be difficult to find out if the recommended workaround is the way to go and finally implement it into the installer.

The problem could be significant in terms of numbers since there are many people installing our product in a new/dedicated servers and selecting windows as their preferred OS.
BTY Recently, we have had higher number of very similar issues that appear to be related to this problem.


Anton Mazkovoi [Atlassian] - 21/Aug/07 11:39 PM
Guys,

When looking into this can you run the fix by me.

Cheers,
Anton


Anton Mazkovoi [Atlassian] - 21/Aug/07 11:39 PM
Bogdan,

Awesome work!

Cheers,
Anton


Andreas Knecht [Atlassian] - 26/Aug/07 11:02 PM
The installer now copies the msvcr71.dll file bundled in jre/bin to the main bin directory. I've tested this on windows sp2 and verified that it works correctly.

Andreas Guther - 13/Sep/07 11:35 AM
I just downloaded the JIRA-Enterprise-3.10.2 windows installer and got the same problem as described in this issue. I installed on a Windows Server 2003 Standard Edition into the default folder C:\Program Files\JIRA-Enterprise-3.10.2\.
I can see that msvcr71.dll is in the C:\Program Files\JIRA-Enterprise-3.10.2\jre\bin folder. But I cannot start either the service, nor the startup_bundled_jre.bat, or the tomcat startup.bat.

Andreas Guther - 13/Sep/07 11:58 AM
I finally got it to run. I copied the msvcr71.dll into the tomcat\bin folder since one comment was mentioning this. I am not sure if this makes a difference or not.

I definitely had a port issue which I could not see first since the catalina window opens and closes without a pausing.

My problem was, that I have another tomcat already running and the CATALINA_HOME was already set to this instance.

I would suggest that the installer alters the start files and puts in a line where necessary as follows:

set CATALINA_HOME=C:\Program Files\JIRA-Enterprise-3.10.2

or whatever the installation folder is. Currently the start batch files do nothing if CATALINA_HOME is already set. I added this line in the catalina.bat file and after that I could start without problems. I did not remove the copied dll file.


Andreas Knecht [Atlassian] - 16/Sep/07 11:20 PM
Hi Andreas,

The fix for this issue was implemented for JIRA 3.10.3 (which wont be shipped, so the fix will be available in 3.11), so the bug still exists in JIRA 3.10.2, which is why you're seeing it.

We thought about altering CATALINA_HOME, however we decided against it, since it may annoy some of our customers, who have this variable set on purpose and would like to run a different version of Tomcat.

Cheers,
Andreas