[BAM-4122] TestNG tests with DataProviders not counted correctly

Type: Bug
Resolution: Handled by Support
Priority: Low
Fix Version/s: None
Affects Version/s: 2.2.1
Component/s: Telemetry / Reporting
Labels:
- bamboo
- statistics
- test
- testng
Environment:

Hide

System Properties

System Date:
Thursday, 25 Jun 2009
System Time:
13:14:03
Up Time:
1 hour, 29 minutes, 31 seconds (since Thu Jun 25 11:44:32 EDT 2009)
Username:
root
User Timezone:
Canada/Eastern
User Locale:
English (United States)
System Encoding:
ANSI_X3.4-1968
Operating System:
Linux 2.6.24-23-server
Operating System Architecture:
i386
Available Processors:
1
Application Server Container:
jetty-6.1.5

Java / JVM Information
Java Version:
1.5.0_16
Java Vendor:
Sun Microsystems Inc.
JVM Spec. Version:
1.0
JVM Spec. Vendor:
Sun Microsystems Inc.
JVM Version:
1.5.0_16-b02
JVM Vendor:
Sun Microsystems Inc.
JVM Name:
Java HotSpot(TM) Client VM
JRE Version:
1.5.0_16-b02
JRE Name:
Java(TM) 2 Runtime Environment, Standard Edition

Network
Host Name:
mshbamboo.mtsinai.toronto.on.ca
IP Address:
142.223.188.134

Memory Statistics
Total Memory:
254 MB
Free Memory:
193 MB
Used Memory:
61 MB

Bamboo Paths
Current running directory:
/usr/local/Bamboo
Configuration Path:
/usr/local/bamboo-home/xml-data/configuration
Build Path:
/usr/local/bamboo-home/xml-data/builds
Build Working Directory:
/usr/local/bamboo-home/xml-data/build-dir
Bamboo Home:
/usr/local/bamboo-home
Bamboo logs:
/usr/local/Bamboo/atlassian-bamboo.log
Temporary directory:
/tmp

File System Statistics
Disk space free (for bamboo.home):
1 GB
Size of index:
641 KB

Instance Statistics
Number of plans:
8
Number of builds (approx):
114
Number of tests (approx):
4439
Full reindex time (approx):
9 minutes

JMS Information
Remote Broker URI:
Not enabled
Broker VM URI:
vm://bamboo

Environment Variables
BAMBOO_INSTALL=/usr/local/Bamboo
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/lib/i386/client:/usr/lib/jvm/
java-1.5.0-sun-1.5.0.16/jre/lib/i386:/usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/../lib/i386
NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat
OLDPWD=/
PATH=/sbin:/bin:/usr/sbin:/usr/bin
PREVLEVEL=N
PWD=/usr/local/Bamboo
QUIET=yes
RUNLEVEL=2
SHLVL=1
TERM=linux
UPSTART_EVENT=runlevel
UPSTART_JOB=rc2
UPSTART_JOB_ID=5
VERBOSE=no
XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt
_=/usr/bin/nohup
previous=N
runlevel=2

Enabled Plugins
Auto favourite marking - 1.0
Vendor: Atlassian Software Systems Pty Ltd
Bamboo Global Admin Sections - 1.0
Vendor: Atlassian Software Systems
Bamboo Reports - 1.0
Vendor: Atlassian Software Systems Pty Ltd
Build Labeller - ${pom.version}
Vendor: Atlassian Software Systems Pty Ltd
Build Number Stamper - 1.0
Vendor: Atlassian Software Systems Pty Ltd
Clover Coverage - 1.0
Vendor: Atlassian Software Systems Pty Ltd
Jira Integration - 1.0
Vendor: Atlassian Software Systems Pty Ltd
Nant Builder - 2.1.1
Vendor: Ross Rowe
System Builders - 1.0
Vendor: Atlassian Software Systems Pty Ltd
System Builders - 1.0
Vendor: Atlassian Software Systems Pty Ltd
System Notifications - 1.0
Vendor: Atlassian Software Systems Pty Ltd
System Repositories - 1.0
Vendor: Atlassian Software Systems Pty Ltd
Trigger Reasons - 1.0
Vendor: Atlassian Software Systems Pty Ltd
VCS Version Collector - 1.0
Vendor: Atlassian Software Systems Pty Ltd

Show
System Properties System Date: Thursday, 25 Jun 2009 System Time: 13:14:03 Up Time: 1 hour, 29 minutes, 31 seconds (since Thu Jun 25 11:44:32 EDT 2009) Username: root User Timezone: Canada/Eastern User Locale: English (United States) System Encoding: ANSI_X3.4-1968 Operating System: Linux 2.6.24-23-server Operating System Architecture: i386 Available Processors: 1 Application Server Container: jetty-6.1.5 Java / JVM Information Java Version: 1.5.0_16 Java Vendor: Sun Microsystems Inc. JVM Spec. Version: 1.0 JVM Spec. Vendor: Sun Microsystems Inc. JVM Version: 1.5.0_16-b02 JVM Vendor: Sun Microsystems Inc. JVM Name: Java HotSpot(TM) Client VM JRE Version: 1.5.0_16-b02 JRE Name: Java(TM) 2 Runtime Environment, Standard Edition Network Host Name: mshbamboo.mtsinai.toronto.on.ca IP Address: 142.223.188.134 Memory Statistics Total Memory: 254 MB Free Memory: 193 MB Used Memory: 61 MB Bamboo Paths Current running directory: /usr/local/Bamboo Configuration Path: /usr/local/bamboo-home/xml-data/configuration Build Path: /usr/local/bamboo-home/xml-data/builds Build Working Directory: /usr/local/bamboo-home/xml-data/build-dir Bamboo Home: /usr/local/bamboo-home Bamboo logs: /usr/local/Bamboo/atlassian-bamboo.log Temporary directory: /tmp File System Statistics Disk space free (for bamboo.home): 1 GB Size of index: 641 KB Instance Statistics Number of plans: 8 Number of builds (approx): 114 Number of tests (approx): 4439 Full reindex time (approx): 9 minutes JMS Information Remote Broker URI: Not enabled Broker VM URI: vm://bamboo Environment Variables BAMBOO_INSTALL=/usr/local/Bamboo LD_LIBRARY_PATH=/usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/lib/i386/client:/usr/lib/jvm/ java-1.5.0-sun-1.5.0.16/jre/lib/i386:/usr/lib/jvm/java-1.5.0-sun-1.5.0.16/jre/../lib/i386 NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat OLDPWD=/ PATH=/sbin:/bin:/usr/sbin:/usr/bin PREVLEVEL=N PWD=/usr/local/Bamboo QUIET=yes RUNLEVEL=2 SHLVL=1 TERM=linux UPSTART_EVENT=runlevel UPSTART_JOB=rc2 UPSTART_JOB_ID=5 VERBOSE=no XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt _=/usr/bin/nohup previous=N runlevel=2 Enabled Plugins Auto favourite marking - 1.0 Vendor: Atlassian Software Systems Pty Ltd Bamboo Global Admin Sections - 1.0 Vendor: Atlassian Software Systems Bamboo Reports - 1.0 Vendor: Atlassian Software Systems Pty Ltd Build Labeller - ${pom.version} Vendor: Atlassian Software Systems Pty Ltd Build Number Stamper - 1.0 Vendor: Atlassian Software Systems Pty Ltd Clover Coverage - 1.0 Vendor: Atlassian Software Systems Pty Ltd Jira Integration - 1.0 Vendor: Atlassian Software Systems Pty Ltd Nant Builder - 2.1.1 Vendor: Ross Rowe System Builders - 1.0 Vendor: Atlassian Software Systems Pty Ltd System Builders - 1.0 Vendor: Atlassian Software Systems Pty Ltd System Notifications - 1.0 Vendor: Atlassian Software Systems Pty Ltd System Repositories - 1.0 Vendor: Atlassian Software Systems Pty Ltd Trigger Reasons - 1.0 Vendor: Atlassian Software Systems Pty Ltd VCS Version Collector - 1.0 Vendor: Atlassian Software Systems Pty Ltd

Bug Fix Policy:
View Atlassian Server bug fix policy

When Bamboo builds our projects that use TestNG and have tests that use TestNG's @DataProvider to run the same test with multiple parameters, Bamboo counts them as a single test. For example, Bamboo says our current project has only 44 tests, but running the test goal in Maven shows we have 60 tests. For another project that makes extensive use of DataProviders, this discrepancy is much higher. Bamboo shows 38 tests while Maven reports 241.

Best regards,
Arthur Kalmenson

was cloned as

BAM-4188 Count ALL test results of TestNG @DataProvider feature to run the SAME TC with different parameters.

Closed

Monique Khairuliana (Inactive) made changes - 19/Aug/2019 2:03 AM

Workflow	Original: Bamboo Workflow 2016 v1 - Restricted [ 1434080 ]	New: JAC Bug Workflow v3 [ 3384048 ]
Status	Original: Resolved [ 5 ]	New: Closed [ 6 ]

Owen made changes - 07/Jul/2016 3:47 AM

Workflow

Original: Bamboo Workflow 2016 v1 [ 1407839 ]

New: Bamboo Workflow 2016 v1 - Restricted [ 1434080 ]

Marek Went (Inactive) made changes - 16/Jun/2016 10:18 AM

Workflow

Original: Bamboo Workflow 2014 v2 [ 610031 ]

New: Bamboo Workflow 2016 v1 [ 1407839 ]

James Dumay made changes - 17/Jan/2014 12:01 AM

Workflow

Original: Bamboo Workflow 2014 [ 605081 ]

New: Bamboo Workflow 2014 v2 [ 610031 ]

James Dumay made changes - 15/Jan/2014 1:35 AM

Workflow

Original: Bamboo Workflow 2010 [ 201224 ]

New: Bamboo Workflow 2014 [ 605081 ]

MarkC made changes - 11/Feb/2010 7:16 PM

Workflow	Original: reviewflow [ 177506 ]	New: Bamboo Workflow 2010 [ 201224 ]
Status	Original: Closed [ 6 ]	New: Resolved [ 5 ]

Ulrich Kuhnhardt [Atlassian] made changes - 09/Jul/2009 1:53 AM

Link

New: This issue was cloned as ~~BAM-4188~~ [ ~~BAM-4188~~ ]

Ulrich Kuhnhardt [Atlassian] made changes - 09/Jul/2009 1:44 AM

Resolution		New: Handled by Support [ 8 ]
Status	Original: Open [ 1 ]	New: Closed [ 6 ]

Arthur Arthur created issue - 25/Jun/2009 5:19 PM

Assignee:: Unassigned

Reporter:: Arthur Arthur

Affected customers:: 0 This affects my team

Watchers:: 1 Start watching this issue

Created:: 25/Jun/2009 5:19 PM

Updated:: 19/Aug/2019 2:03 AM

Resolved:: 09/Jul/2009 1:44 AM

Details

Description

Attachments

Issue Links

Forms

Activity

People

Dates