Issue Details (XML | Word | Printable)

Key: JRA-3383
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Jeff Turner [Atlassian]
Reporter: Jeff Turner [Atlassian]
Votes: 1
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
JIRA

Bugzilla importer: compatibility with 2.17 broken

Created: 11/Mar/04 05:38 AM   Updated: 08/Nov/06 10:01 PM
Component/s: Import / Export
Affects Version/s: 2.6 Pro
Fix Version/s: 3.0 Pro Preview

Time Tracking:
Not Specified

File Attachments: 1. Java Archive File bugzillafix2.jar (22 kB)
2. Text File bz_import_stack.txt (6 kB)

Environment: Bugzilla 2.17.5
Issue Links:
Reference
 

Participants: Colin Bendell, Ilya Pogorelsky, Jeff Turner [Atlassian], Peter Mularien, Scott Farquhar [Atlassian] and Vincent Fiano
Since last comment: 4 years, 5 weeks, 1 day ago
Resolution Date: 22/Apr/04 12:20 AM
Labels:


 Description  « Hide
BugzillaImportBean.java does a "select from product", which breaks in 2.17:

"I am currently evaluating JIRA for our development environment. During my import from our bugzilla system 2.17.5 I
get the errors below. What version of the 2.17 stream has been tested with the JIRA import service?

Error importing data from bugzilla.java.sql.SQLException: Column not found, message from server: "Unknown column 'product' in
'where clause'" at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1905) at
com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1109) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1203) at
com.mysql.jdbc.Connection.execSQL(Connection.java:2090) at
com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1496) at
com.atlassian.jira.util.BugzillaImportBean.createProjects(BugzillaImportBean.java:609) at
com.atlassian.jira.util.BugzillaImportBean.create(BugzillaImportBean.java:152) at
"



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Scott Farquhar [Atlassian] added a comment - 25/Mar/04 11:01 PM
A workaround has been found.

Bugzilla changed its table format, in the product table 'name' and 'product' are the same.

If you do something like this, then the import should work. Please backup your data first.

ALTER TABLE products ADD COLUMN product CHAR(255);

UPDATE products SET product = name;

Then you should be able to import your data.


Colin Bendell added a comment - 26/Mar/04 08:05 AM
Unfortunately that still does not completely fix the problem. Module import and version import are then also broken.

Scott Farquhar [Atlassian] added a comment - 08/Apr/04 02:54 AM
Hrm - it appears that there are still exceptions:

Error importing data from bugzilla.java.sql.SQLException:
Column not found, message from server:
"Unknown column 'program' in 'where clause'"
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1905)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1109)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1203)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2090)

We'll need to get some 2.17 data to test with.


Colin Bendell added a comment - 21/Apr/04 09:36 AM
When will 2.6.2 or 2.7 be released?

Jeff Turner [Atlassian] added a comment - 22/Apr/04 12:20 AM
Colin,

2.7 is still over a month away. However I've attached a version of the classes that should fix this bug in 2.6.1. The jar should be unpacked in the WEB-INF/classes directory (cd atlassian-jira/WEB-INF/classes ; 'jar xvf bugzillafix2.jar') to overwrite the old classes.

--Jeff


Colin Bendell added a comment - 10/Jun/04 12:40 PM
The patch you provided still falls down when importing versions.

Jeff Turner [Atlassian] added a comment - 24/Jun/04 02:31 AM
Hmm - I have tested with a smallish 2.17 database and the patch works. If anyone experiencing problems with the patched classes could mail the stacktrace, or better yet, a MySQL dump of the Bugzilla database, that would be a big help.

Ilya Pogorelsky added a comment - 24/Jun/04 08:37 AM
There still seems to be a problem in component creation step.
In the log (I turned DEBUG on - but not a whole lot of extra info is printed):
2004-06-24 09:12:56,636 INFO [atlassian.jira.util.BugzillaImportBean]

Importing Components from project(s) 'DealFX'

2004-06-24 09:12:56,636 INFO [atlassian.jira.util.BugzillaImportBean] Importing
Component: Atlantis
[WARN] IteratorTag - -Value is null! Returning an empty set.
2004-06-24 09:12:56,716 DEBUG [web.util.component.PicoJavaActionFactory] Configu
ring class using PicoJavaActionFactory com.atlassian.jira.web.action.project.ViewProjects

In the 'Bugzilla Import Progress' log window see the following:
[09:17:35]

Importing project(s) 'DealFX'
[09:17:35] Importing Project: DealFX
[09:17:35] Project: DealFX already exists. Not imported
[09:17:35] 0 projects imported from Bugzilla.
[09:17:35]

Importing Versions from project 'DealFX'

[09:17:35] Importing Version: Future
[09:17:35] Version: Future in Project: DealFX already exists. Not imported
[09:17:35] Importing Version: TBD
[09:17:35] Version: TBD in Project: DealFX already exists. Not imported
[09:17:35] Importing Version: 1.0
[09:17:35] Version: 1.0 in Project: DealFX already exists. Not imported
[09:17:35] 0 versions imported from Bugzilla.
[09:17:35]

Importing Components from project(s) 'DealFX'

[09:17:35] Importing Component: Atlantis

I am also attaching the stack trace in the import window.


Ilya Pogorelsky added a comment - 24/Jun/04 08:38 AM
nullpointer stack trace during import.

Ilya Pogorelsky added a comment - 24/Jun/04 08:38 AM
I am going against bz version 2.17.7 (obviously using patched classes)

Ilya Pogorelsky added a comment - 24/Jun/04 08:48 AM
also do you still need to have backward compatible columns fudged by hand? this is kinda bad - at least provide a sql script to perform the necessary updates.

Jeff Turner [Atlassian] added a comment - 24/Jun/04 09:01 AM
Ilya,

Thanks for the stacktrace.

In the meanwhile, the fastest solution may be to download JIRA 2.5.3 Standalone, and do the Bugzilla import with that.


Jeff Turner [Atlassian] added a comment - 24/Jun/04 09:02 AM
re. comment fudging: no, that should not be necessary with the patch.

Ilya Pogorelsky added a comment - 24/Jun/04 09:08 AM
Ok - good news - now import from 2.17.7 is working for me.

I realized that bugzilla import utility was checking the product table for existence of product column.

I created this column in the 2.17.7 schema, in my early attempts to make schema look backward compatible. but because the update bugzilla import script was finding the 'product' column, 2.16 compatible importing was attempted - which would break.

cheers.


Colin Bendell added a comment - 24/Jun/04 10:47 AM
Import generally went ok, but when I imported a product that has many thousands of bugs, I got this error (repeatedly):

2004-06-24 09:22:52,908 INFO [atlassian.jira.util.BugzillaImportBean] 0 votes im
ported from Bugzilla.
2004-06-24 09:22:52,908 INFO [atlassian.jira.util.BugzillaImportBean] Reindexing
(this may take a while)...
java.lang.IllegalArgumentException: Source GenericValue can not be null.
at com.atlassian.core.ofbiz.association.DefaultAssociationManager.getSin
kFromSource(DefaultAssociationManager.java:201)
at com.atlassian.core.ofbiz.association.DefaultAssociationManager.getSin
kFromSource(DefaultAssociationManager.java:195)
at com.atlassian.jira.scheme.AbstractSchemeManager.cacheProjectSchemes(A
bstractSchemeManager.java:406)
at com.atlassian.jira.scheme.AbstractSchemeManager.getSchemes(AbstractSc
hemeManager.java:94)
at com.atlassian.jira.notification.DefaultNotificationSchemeManager.getR
ecipients(DefaultNotificationSchemeManager.java:76)
at com.atlassian.jira.mail.IssueMailQueueItem.send(IssueMailQueueItem.ja
va:101)
at com.atlassian.mail.queue.MailQueueImpl.sendBuffer(MailQueueImpl.java:
55)
at com.atlassian.jira.service.services.mail.MailQueueService.run(MailQue
ueService.java:23)
at com.atlassian.jira.service.JiraServiceContainer.run(JiraServiceContai
ner.java:51)
at com.atlassian.jira.service.ServiceRunner.execute(ServiceRunner.java:5
0)
at org.quartz.core.JobRunShell.run(JobRunShell.java:147)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.j
ava:387)
2004-06-24 09:23:37,449 ERROR [atlassian.jira.mail.IssueMailQueueItem] Source Ge
nericValue can not be null.
java.lang.IllegalArgumentException: Source GenericValue can not be null.
at com.atlassian.core.ofbiz.association.DefaultAssociationManager.getSin
kFromSource(DefaultAssociationManager.java:201)
at com.atlassian.core.ofbiz.association.DefaultAssociationManager.getSin
kFromSource(DefaultAssociationManager.java:195)
at com.atlassian.jira.scheme.AbstractSchemeManager.cacheProjectSchemes(A
bstractSchemeManager.java:406)
at com.atlassian.jira.scheme.AbstractSchemeManager.getSchemes(AbstractSc
hemeManager.java:94)
at com.atlassian.jira.notification.DefaultNotificationSchemeManager.getR
ecipients(DefaultNotificationSchemeManager.java:76)
at com.atlassian.jira.mail.IssueMailQueueItem.send(IssueMailQueueItem.ja
va:101)
at com.atlassian.mail.queue.MailQueueImpl.sendBuffer(MailQueueImpl.java:
55)
at com.atlassian.jira.service.services.mail.MailQueueService.run(MailQue
ueService.java:23)
at com.atlassian.jira.service.JiraServiceContainer.run(JiraServiceContai
ner.java:51)
at com.atlassian.jira.service.ServiceRunner.execute(ServiceRunner.java:5
0)
at org.quartz.core.JobRunShell.run(JobRunShell.java:147)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.j
ava:387)
2004-06-24 09:23:37,449 ERROR [atlassian.mail.queue.MailQueueImpl] com.atlassian
.mail.MailException: Source GenericValue can not be null.


Jeff Turner [Atlassian] added a comment - 24/Jun/04 09:32 PM
Colin,

Those exceptions appear to be from the mail notification service running in the background, rather than from the Bugzilla importer's thread. As such, they are probably nothing to worry about. I've raised this as a separate bug though (JRA-3969)


Jeff Turner [Atlassian] added a comment - 24/Jun/04 09:41 PM
Ilya,

The stacktrace you attached indicates that the "create project" step failed, which later resulted in the "create components" step having failed. Did you run this against a clean JIRA database? Artifacts from previous failed imports could cause odd problems like this.


Jeff Turner [Atlassian] added a comment - 24/Jun/04 09:42 PM
For the record, I've successfully imported a 2700-issue test Bugzilla 2.17 database using this patch.

Peter Mularien added a comment - 03/Aug/04 08:04 AM
I have applied the patch and am still getting the stack trace Scott mentioned in his post above. Any suggestions? I noted the following message printed in the JIRA console window:

"[WARN] IteratorTag - -Value is null! Returning an empty set."

The project is imported successfully, but I assume from the stacktrace it is failing when creating versions. Probably due to bad data, but would be helpful to know what data it was.

java.sql.SQLException: Column not found, message from server: "Unknown column 'program' in 'where clause'" at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1977) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1163) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1272) at com.mysql.jdbc.Connection.execSQL(Connection.java:2236) at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1555) at com.atlassian.jira.util.BugzillaImportBean.createVersions(BugzillaImportBean.java:568) at com.atlassian.jira.util.BugzillaImportBean.create(BugzillaImportBean.java:159) at com.atlassian.jira.web.action.util.BugzillaImport.doRun(BugzillaImport.java:133) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:324) at webwork.action.ActionSupport.invokeCommand(ActionSupport.java:415) at webwork.action.ActionSupport.execute(ActionSupport.java:144) at com.atlassian.jira.action.JiraActionSupport.execute(JiraActionSupport.java:49)


Peter Mularien added a comment - 03/Aug/04 11:45 AM
Here are the SQL statements I had to run to get everything to import correctly. Note I was running Bugzilla 2.18.2, since I couldn't find a version of 2.17 to download from bugzilla.org, and had to migrate our old 2.12 system. Hopefully this helps someone. Everything imported successfully after this.

– Begin SQL - run in the Bugzilla MySQL database
alter table components add program varchar(64);
update components, products set components.program=products.name where products.id=product_id;
alter table versions add program varchar(64);
update versions, products set versions.program=products.name where products.id=product_id;
alter table components add value varchar(64);
update components set value=name;
alter table bugs add product varchar(64);
update bugs, products set bugs.product=products.name where products.id=product_id;


Jeff Turner [Atlassian] added a comment - 03/Aug/04 10:29 PM
Peter,

Thanks for the note. I've raised a separate issue (JRA-4161) to track Bugzilla 2.18 compatibility.


Vincent Fiano added a comment - 01/Sep/04 02:31 PM
I'm using the standalone, evaluation version of JIRA. I originally had column name problems before patching with this bugzillafix2.jar. Now most of my databases import, but a few throw null pointer exceptions. Since I've patched with the provided jar, the publicly available BugzillaImportBean.java available in the Bugzilla importing documentation doesn't match the class files that I'm using. Can you post the new BugzillaImportBean source, so that I can debug my importing problems?

Thanks,
Vinny

  1. Error importing data from bugzilla: java.lang.NullPointerException at com.atlassian.jira.util.BugzillaImportBean.createIssue(BugzillaImportBean.java:263) at com.atlassian.jira.util.BugzillaImportBean.createIssues(BugzillaImportBean.java:242) at com.atlassian.jira.util.BugzillaImportBean.create(BugzillaImportBean.java:163) at
    ... (and many more)

Jeff Turner [Atlassian] added a comment - 02/Sep/04 08:39 PM
Vincent,

The source code for the Bugzilla importer is online at:

http://www.atlassian.com/software/jira/docs/latest/importers/BugzillaImportBean.java