Issue Details (XML | Word | Printable)

Key: BAM-2355
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Mark Chaimungkalanont [Atlassian]
Reporter: Sam Berlin
Votes: 0
Watchers: 0
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Bamboo

Bamboo 2.0 Upgrade Task 608 Fails

Created: 10/Mar/08 04:07 PM   Updated: 01/Apr/08 07:14 PM
Component/s: Upgrading
Affects Version/s: 2.0 beta 4
Fix Version/s: 2.0 beta 6, 2.0

Time Tracking:
Not Specified

Participants: Mark Chaimungkalanont [Atlassian] and Sam Berlin
Since last comment: 42 weeks, 3 days ago
Number of comments: 5
Internal Priority: 3. Medium
Labels:


 Description  « Hide
Upgrading from 1.2.4 to 2.0b4, upgrade task 608 is failing on malformed XML.

The relevant lines from the log are:

2008-03-10 17:03:06,439 INFO [main] [UpgradeTask608BuildLogStorageUpdate] Processing file buildresults3416.xml
2008-03-10 17:03:06,439 INFO [main] [UpgradeTask608BuildLogStorageUpdate] 137 / 234 (97)
2008-03-10 17:03:06,837 ERROR [main] [UpgradeTask608BuildLogStorageUpdate] Upgrade Task failed to upgrade logs for DEF
2008-03-10 17:03:06,843 ERROR [main] [UpgradeManagerImpl] org.dom4j.DocumentException: Error on line 128407 of document : An invalid XML character (Unicode: 0x8) was found in the element content of the document. Nested exception: An invalid XML character (Unicode: 0x8) was found in the element content of the document.
org.dom4j.DocumentException: Error on line 128407 of document : An invalid XML character (Unicode: 0x8) was found in the element content of the document. Nested exception: An invalid XML character (Unicode: 0x8) was found in the element content of the document.
at org.dom4j.io.SAXReader.read(SAXReader.java:355)
at org.dom4j.io.SAXReader.read(SAXReader.java:219)
at com.atlassian.bamboo.upgrade.tasks.UpgradeTask608BuildLogStorageUpdate.doUpgrade(UpgradeTask608BuildLogStorageUpdate.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.orm.hibernate.HibernateInterceptor.invoke(HibernateInterceptor.java:117)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy33.doUpgrade(Unknown Source)
at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.runUpgradeTask(UpgradeManagerImpl.java:122)
at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.doUpgrade(UpgradeManagerImpl.java:88)
at com.atlassian.bamboo.upgrade.UpgradeLauncher.upgradeAndStartBamboo(UpgradeLauncher.java:104)
at com.atlassian.bamboo.upgrade.UpgradeLauncher.contextInitialized(UpgradeLauncher.java:37)
at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
at org.mortbay.jetty.Server.doStart(Server.java:217)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at com.atlassian.bamboo.server.Server.main(Server.java:60)
Nested exception:
org.xml.sax.SAXParseException: An invalid XML character (Unicode: 0x8) was found in the element content of the document.
at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
at org.dom4j.io.SAXReader.read(SAXReader.java:339)
at org.dom4j.io.SAXReader.read(SAXReader.java:219)
at com.atlassian.bamboo.upgrade.tasks.UpgradeTask608BuildLogStorageUpdate.doUpgrade(UpgradeTask608BuildLogStorageUpdate.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.orm.hibernate.HibernateInterceptor.invoke(HibernateInterceptor.java:117)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy33.doUpgrade(Unknown Source)
at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.runUpgradeTask(UpgradeManagerImpl.java:122)
at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.doUpgrade(UpgradeManagerImpl.java:88)
at com.atlassian.bamboo.upgrade.UpgradeLauncher.upgradeAndStartBamboo(UpgradeLauncher.java:104)
at com.atlassian.bamboo.upgrade.UpgradeLauncher.contextInitialized(UpgradeLauncher.java:37)
at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
at org.mortbay.jetty.Server.doStart(Server.java:217)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at com.atlassian.bamboo.server.Server.main(Server.java:60)
2008-03-10 17:03:06,848 INFO [main] [UpgradeManagerImpl] Completed upgrade task 608 with errors.

The relevant part of the XML is:

<BuildOutputLogEntry>
<time>1199377738674</time>
<log>2008-01-03 11:28:58,674 [main] INFO com.limewire.store.core.search.dao.SearchIndexerSolrImpl - ProductDoc: Document<stored/uncompressed<albumId:118> stored/uncompressed,indexed,tokenized<objectType:album> stored/uncompressed,indexed,omitNorms<key:album:118> stored/uncompressed,indexed,omitNorms<popularity:H@@@K> stored/uncompressed,indexed,omitNorms<rating:Ãà¨@> indexed,tkenized<provider:this-is-a-provider-product-id-1> stored/uncompressed,indexed,tokenized<combined_text:this-is-a-provider-product-id-1> stored/uncompressed<artistId:260> stored/uncompressed,indexed,omitNorms<isArtistClickable:T> stored/uncompressed<file:&gt; indexed,tokenized<territory:DE> stored/uncompressed,indexed,tokenized<combined_text:DE> stored/uncompressed,indexed,tokenized<sartist_ger:260|true|null|null> indexed,tokenized<ger_combined_text:260|true|null|null> stored/uncompressed,indexed,tokenized<origRelDate_ger:2008-01-03T11:28:58.587Z> indexed,tokenized<ger_combined_text:2008-01-03T11:28:58.587Z> stored/uncompressed,indexed,tokenized<explicitLyrics_ger:false> indexed,tokenized<ger_combined_text:false> indexed,tokenized<territory:US> stored/uncompressed,indexed,tokenized<combined_text:US> indexed,tokenized<label:Strata East Records> stored/uncompressed,indexed,tokenized<combined_text:Strata East Records> stored/uncompressed,indexed,tokenized<albumTitle:Some Title> stored/uncompressed,indexed,tokenized<combined_text:Some Title> stored/uncompressed,indexed,omitNorms<salbumTitle:Some Title> stored/uncompressed,indexed,tokenized<artist:The Artist Formerly Known As Leo> stored/uncompressed,indexed,tokenized<combined_text:The Artist Formerly Known As Leo> stored/uncompressed,indexed,omitNorms<sartist:260|true|null|The Artist Formerly Known As Leo> indexed,omitNorms<origRelDate:2008-01-03T11:28:58.568> stored/uncompressed,indexed,omitNorms<explicitLyrics:T> stored/uncompressed,indexed,tokenized<combined_text:I Have Great Talent> stored/uncompressed,indexed,tokenized<albumTrackTitles:I Have Great Talent> stored/uncompressed,indexed,tokenized<combined_text:Yanni Jr.> indexed,tokenized<territory:ES> stored/uncompressed,indexed,tokenized<combined_text:ES> stored/uncompressed,indexed,tokenized<sartist_spa:260|true|null|null> indexed,tokenized<spa_combined_text:260|true|null|null> stored/uncompressed,indexed,tokenized<origRelDate_spa:2008-01-03T11:28:58.670Z> indexed,tokenized<spa_combined_text:2008-01-03T11:28:58.670Z> stored/uncompressed,indexed,tokenized<explicitLyrics_spa:false> indexed,tokenized<spa_combined_text:false>></log>
</BuildOutputLogEntry>

It's failing on the 'H@@@^K' part . (Those are really control characters, not the characters ^, H, @, or K.) I'm not sure why the malformed data would be written to the logs in the first place... but it's likely that there was a test that wrote malformed data to System.out as part of its testing. I manually edited out the malformed data from the first buildresults###.xml that failed, but the error happened again on another... so I'm hoping there's a way to fix this without having to manually edit every file.

 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
[ Permlink | « Hide ]
Mark Chaimungkalanont [Atlassian] added a comment - 10/Mar/08 10:39 PM
Sam,

Thanks for the report.

We'll see if we can cleanup the XML before we read the XML. We did upgrade the xStream library so maybe now it's failing whereas it was more accepting before.

Do you want to post your export against this issue or a new support issue? That way we can make sure that your data will pass and upgrade correctly.

Cheers,

Mark C


[ Permlink | « Hide ]
Sam Berlin added a comment - 11/Mar/08 12:56 AM
Thanks for the info, Mark. The export (without artifacts) is ~3GB, and the build logs in question are ~40MB... so I think I'll pass on attaching the export.

It should be really easy to reproduce, though – just print out some crazy bytes (ours had 0x08, 0x00, and 0xB0 [or maybe it was 0x0B]), and then try running them through that sax parser.

FWIW we had to workaround this kind of crazy data in LimeWire, too. See: https://www.limewire.org/fisheye/browse/limecvs/core/com/limegroup/gnutella/xml/LimeXMLUtils.java?r=1.63#l452 . That was intended more replacing the whole XML string, which probably isn't what you're looking for... but I could definitely imagine building that kind of functionality into a filter InputStream or Reader or InputSource.


[ Permlink | « Hide ]
Mark Chaimungkalanont [Atlassian] added a comment - 17/Mar/08 07:04 PM
Sam,

I'm just adding the code to do a cleanup of the dodgy XML characters (we have a XMLCleaningReader we can use).

Looking through your logs, I've also noticed a concurrency issue (using SimpleDateFormat) which is now also fixed. One unresolved upgrade issue is to do with:

2008-03-10 18:14:06,619 INFO [main] [UpgradeTask608BuildLogStorageUpdate] Processing file buildresults3649.xml
2008-03-10 18:14:06,619 INFO [main] [UpgradeTask608BuildLogStorageUpdate] 517 / 982 (465)
2008-03-10 18:14:07,195 ERROR [main] [UpgradeTask608BuildLogStorageUpdate] Upgrade Task failed to upgrade logs for DEF
2008-03-10 18:14:07,269 ERROR [main] [UpgradeManagerImpl] org.dom4j.DocumentException: Error on line 130233 of document  : XML document structures must start and end within the same entity. Nested exception: XML document structures must start and end within the same entity.
org.dom4j.DocumentException: Error on line 130233 of document  : XML document structures must start and end within the same entity. Nested exception: XML document structures must start and end within the same entity.
	at org.dom4j.io.SAXReader.read(SAXReader.java:355)
	at org.dom4j.io.SAXReader.read(SAXReader.java:219)
	at com.atlassian.bamboo.upgrade.tasks.UpgradeTask608BuildLogStorageUpdate.doUpgrade(UpgradeTask608BuildLogStorageUpdate.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
	at org.springframework.orm.hibernate.HibernateInterceptor.invoke(HibernateInterceptor.java:117)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy33.doUpgrade(Unknown Source)
	at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.runUpgradeTask(UpgradeManagerImpl.java:122)
	at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.doUpgrade(UpgradeManagerImpl.java:88)
	at com.atlassian.bamboo.upgrade.UpgradeLauncher.upgradeAndStartBamboo(UpgradeLauncher.java:104)
	at com.atlassian.bamboo.upgrade.UpgradeLauncher.contextInitialized(UpgradeLauncher.java:37)
	at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
	at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
	at org.mortbay.jetty.Server.doStart(Server.java:217)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
	at com.atlassian.bamboo.server.Server.main(Server.java:60)
Nested exception: 
org.xml.sax.SAXParseException: XML document structures must start and end within the same entity.
	at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)
	at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)
	at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.endEntity(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentScannerImpl.endEntity(Unknown Source)
	at org.apache.xerces.impl.XMLEntityManager.endEntity(Unknown Source)
	at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source)
	at org.apache.xerces.impl.XMLEntityScanner.scanContent(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
	at org.dom4j.io.SAXReader.read(SAXReader.java:339)
	at org.dom4j.io.SAXReader.read(SAXReader.java:219)
	at com.atlassian.bamboo.upgrade.tasks.UpgradeTask608BuildLogStorageUpdate.doUpgrade(UpgradeTask608BuildLogStorageUpdate.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
	at org.springframework.orm.hibernate.HibernateInterceptor.invoke(HibernateInterceptor.java:117)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy33.doUpgrade(Unknown Source)
	at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.runUpgradeTask(UpgradeManagerImpl.java:122)
	at com.atlassian.bamboo.upgrade.UpgradeManagerImpl.doUpgrade(UpgradeManagerImpl.java:88)
	at com.atlassian.bamboo.upgrade.UpgradeLauncher.upgradeAndStartBamboo(UpgradeLauncher.java:104)
	at com.atlassian.bamboo.upgrade.UpgradeLauncher.contextInitialized(UpgradeLauncher.java:37)
	at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
	at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
	at org.mortbay.jetty.Server.doStart(Server.java:217)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
	at com.atlassian.bamboo.server.Server.main(Server.java:60)

Could you possible attach the file, which should be at xml-data/builds/LWS-DEF/results/buildresults3649.xml? It looks like the file has invalid XML in it but we want to check to see if this is the case.

Cheers,

Mark C


[ Permlink | « Hide ]
Sam Berlin added a comment - 17/Mar/08 07:23 PM
No worries about that particular one – I had a long running perl task to manually fix the illegal XML characters & I decided to do it another way, so aborted the task. I think it truncated the file halfway through.

[ Permlink | « Hide ]
Mark Chaimungkalanont [Atlassian] added a comment - 17/Mar/08 07:53 PM
Sam,

Thanks for that. The upgrades should work better with this release then.

Cheers,

Mark C
Powered by Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators