[BSERV-12777] Bitbucket Server commands timeout when trying process extremely long commit messages

Type: Bug
Resolution: Fixed
Priority: Low
Fix Version/s: 7.12.0, 7.6.5, 7.11.2
Affects Version/s: 7.0.0
Component/s: BBS DoS, Git Hosting, Repository - Commit Listing
Labels:
None

Support reference count:
1
Symptom Severity:
Severity 3 - Minor
UIS:
1
Bug Fix Policy:
View Atlassian Server bug fix policy

Issue Summary

Bitbucket Server is unable to process git output containing extremely long commit messages in a reasonable amount of time. This causes actions which use this output to timeout and fail (e.g. loading the commits page in the UI).

Steps to Reproduce

Push a commit (or commits) with extremely long commit messages (millions of characters, hundreds of thousands of lines) to a repository in Bitbucket Server.
Attempt to perform an action that causes Bitbucket Server to run a git rev-list command over those commits (e.g. viewing the commits page in the UI for that repository).

Expected Results

The git rev-list command completes quickly (~seconds, or less), and the associated action (e.g. viewing the commits page in the UI) completes successfully without any problems.

Actual Results

The git rev-list command takes an excessive amount of time to complete (multiple minutes). This will lead to for example REST requests timings out (so the UI for the commits page will show a 500 error after waiting for the commits).

The below exception is thrown in the bitbucket.log file:

c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.s.i.w.HandlerExceptionResolverImpl An error occurred while executing an external process: process timed out
com.atlassian.bitbucket.ServerException: An error occurred while executing an external process: process timed out
	at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:120)
	at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.onError(GitCommandExitHandler.java:208)
	at com.atlassian.stash.internal.scm.git.command.revlist.RevCoreExitHandler.onError(RevCoreExitHandler.java:55)
	at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onExit(DefaultCommandExitHandler.java:32)
	at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:285)
	at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.finish(NioNuProcessHandler.java:326)
	at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.onExit(NioNuProcessHandler.java:123)
	at com.zaxxer.nuprocess.internal.BasePosixProcess.onExit(BasePosixProcess.java:319)
	at com.zaxxer.nuprocess.linux.ProcessEpoll.handleExit(ProcessEpoll.java:375)
	at com.zaxxer.nuprocess.linux.ProcessEpoll.cleanupProcess(ProcessEpoll.java:334)
	at com.zaxxer.nuprocess.linux.ProcessEpoll.process(ProcessEpoll.java:272)
	at com.zaxxer.nuprocess.internal.BaseEventProcessor.run(BaseEventProcessor.java:81)
	at com.zaxxer.nuprocess.linux.ProcessEpoll.run(ProcessEpoll.java:188)
	at com.zaxxer.nuprocess.linux.LinuxProcess.run(LinuxProcess.java:114)
	at com.zaxxer.nuprocess.linux.LinProcessFactory.runProcess(LinProcessFactory.java:50)
	at com.zaxxer.nuprocess.NuProcessBuilder.run(NuProcessBuilder.java:273)
	at com.atlassian.bitbucket.internal.process.nu.NuNioProcessHelper.run(NuNioProcessHelper.java:75)
	at com.atlassian.bitbucket.internal.process.NioCommand.call(NioCommand.java:52)
	at com.atlassian.stash.internal.commit.DefaultCommitService.getCommits(DefaultCommitService.java:450)
	at com.atlassian.stash.internal.commit.DefaultCommitService.getCommits(DefaultCommitService.java:175)
	at com.atlassian.stash.internal.web.repos.RepositoryController.getCommits(RepositoryController.java:218)
	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
        ...
Caused by: com.atlassian.utils.process.ProcessTimeoutException: process timed out
	at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:269)
	... 52 common frames omitted

Workaround

Currently there is no known workaround for this behaviour.

causes: PS-74439 You do not have permission to view this issue

mentioned in: Page Failed to load

Form Name

Nate Hansberry made changes - 03/May/2021 1:01 PM

Remote Link

New: This issue links to "Page (Confluence)" [ 549333 ]

Aditya Karia made changes - 22/Mar/2021 4:01 AM

Fix Version/s	Original: 7.7.2 [ 93592 ]
Fix Version/s	Original: 7.8.2 [ 94810 ]
Fix Version/s	Original: 7.9.2 [ 94811 ]
Fix Version/s	Original: 7.10.2 [ 94837 ]

Stefan made changes - 16/Mar/2021 4:02 AM

Resolution		New: Fixed [ 1 ]
Status	Original: In Review [ 10051 ]	New: Closed [ 6 ]

Stefan made changes - 16/Mar/2021 4:02 AM

Fix Version/s		New: 7.6.5 [ 94836 ]
Fix Version/s		New: 7.10.2 [ 94837 ]
Fix Version/s		New: 7.11.2 [ 94838 ]
Fix Version/s		New: 7.7.2 [ 93592 ]
Fix Version/s		New: 7.8.2 [ 94810 ]
Fix Version/s		New: 7.9.2 [ 94811 ]
Fix Version/s		New: 7.12.0 [ 94823 ]

Stefan made changes - 10/Mar/2021 3:04 AM

Status

Original: In Progress [ 3 ]

New: In Review [ 10051 ]

SET Analytics Bot made changes - 10/Mar/2021 12:51 AM

UIS

New: 1

Bugfix Automation Bot made changes - 09/Mar/2021 3:00 PM

Support reference count

New: 1

Christian Glockner made changes - 09/Mar/2021 10:50 AM

Remote Link

New: This issue links to "PS-74439 (Atlassian Support System)" [ 534557 ]

Stefan made changes - 09/Mar/2021 10:18 AM

Description

Original: h3. Issue Summary

Bitbucket Server is unable to process {{git}} output containing extremely long commit messages in a reasonable amount of time. This causes actions which use this output to timeout and fail (e.g. loading the commits page in the UI).
h3. Steps to Reproduce
# Push a commit (or commits) with extremely long commit messages (millions of characters, hundreds of thousands of lines) to a repository in Bitbucket Server.
# Attempt to perform an action that causes Bitbucket Server to run a {{git rev-list}} command over those commits (e.g. viewing the commits page in the UI for that repository).

h3. Expected Results

The {{git rev-list}} command completes quickly (~seconds, or less), and the associated action (e.g. viewing the commits page in the UI) completes successfully without any problems.
h3. Actual Results

The {{git rev-list}} command takes an excessive amount of time to complete (multiple minutes). This will lead to for example REST requests timings out (so the UI for the commits page will show a 500 error after waiting for the commits).

The below exception is thrown in the {{bitbucket.log}} file:
{noformat}
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.s.i.w.HandlerExceptionResolverImpl An error occurred while executing an external process: process timed out
com.atlassian.bitbucket.ServerException: An error occurred while executing an external process: process timed out
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:120)
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.onError(GitCommandExitHandler.java:208)
at com.atlassian.stash.internal.scm.git.command.revlist.RevCoreExitHandler.onError(RevCoreExitHandler.java:55)
at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onExit(DefaultCommandExitHandler.java:32)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:285)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.finish(NioNuProcessHandler.java:326)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.onExit(NioNuProcessHandler.java:123)
at com.zaxxer.nuprocess.internal.BasePosixProcess.onExit(BasePosixProcess.java:319)
at com.zaxxer.nuprocess.linux.ProcessEpoll.handleExit(ProcessEpoll.java:375)
at com.zaxxer.nuprocess.linux.ProcessEpoll.cleanupProcess(ProcessEpoll.java:334)
at com.zaxxer.nuprocess.linux.ProcessEpoll.process(ProcessEpoll.java:272)
at com.zaxxer.nuprocess.internal.BaseEventProcessor.run(BaseEventProcessor.java:81)
at com.zaxxer.nuprocess.linux.ProcessEpoll.run(ProcessEpoll.java:188)
at com.zaxxer.nuprocess.linux.LinuxProcess.run(LinuxProcess.java:114)
at com.zaxxer.nuprocess.linux.LinProcessFactory.runProcess(LinProcessFactory.java:50)
at com.zaxxer.nuprocess.NuProcessBuilder.run(NuProcessBuilder.java:273)
at com.atlassian.bitbucket.internal.process.nu.NuNioProcessHelper.run(NuNioProcessHelper.java:75)
at com.atlassian.bitbucket.internal.process.NioCommand.call(NioCommand.java:52)
at com.atlassian.stash.internal.commit.DefaultCommitService.getCommits(DefaultCommitService.java:450)
at com.atlassian.stash.internal.commit.DefaultCommitService.getCommits(DefaultCommitService.java:175)
at com.atlassian.stash.internal.web.repos.RepositoryController.getCommits(RepositoryController.java:218)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
...
Caused by: com.atlassian.utils.process.ProcessTimeoutException: process timed out
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:269)
... 52 common frames omitted
{noformat}
h3. Workaround

Currently there is no known workaround for this behaviour. A workaround will be added here when available.

New: h3. Issue Summary

Bitbucket Server is unable to process {{git}} output containing extremely long commit messages in a reasonable amount of time. This causes actions which use this output to timeout and fail (e.g. loading the commits page in the UI).
h3. Steps to Reproduce
# Push a commit (or commits) with extremely long commit messages (millions of characters, hundreds of thousands of lines) to a repository in Bitbucket Server.
# Attempt to perform an action that causes Bitbucket Server to run a {{git rev-list}} command over those commits (e.g. viewing the commits page in the UI for that repository).

h3. Expected Results

The {{git rev-list}} command completes quickly (~seconds, or less), and the associated action (e.g. viewing the commits page in the UI) completes successfully without any problems.
h3. Actual Results

The {{git rev-list}} command takes an excessive amount of time to complete (multiple minutes). This will lead to for example REST requests timings out (so the UI for the commits page will show a 500 error after waiting for the commits).

The below exception is thrown in the {{bitbucket.log}} file:
{noformat}
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.b.i.p.nu.NioNuProcessHandler [/opt/git/bin/git rev-list --format=%H%x02%P%x02%aN%x02%aE%x02%at%x02%cN%x02%cE%x02%ct%n%B%n%x03END%x04 --no-min-parents --stdin --]: Dropping callback invocation (Canceled: false; Timed out: true)
c.a.s.i.w.HandlerExceptionResolverImpl An error occurred while executing an external process: process timed out
com.atlassian.bitbucket.ServerException: An error occurred while executing an external process: process timed out
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:120)
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.onError(GitCommandExitHandler.java:208)
at com.atlassian.stash.internal.scm.git.command.revlist.RevCoreExitHandler.onError(RevCoreExitHandler.java:55)
at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onExit(DefaultCommandExitHandler.java:32)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:285)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.finish(NioNuProcessHandler.java:326)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.onExit(NioNuProcessHandler.java:123)
at com.zaxxer.nuprocess.internal.BasePosixProcess.onExit(BasePosixProcess.java:319)
at com.zaxxer.nuprocess.linux.ProcessEpoll.handleExit(ProcessEpoll.java:375)
at com.zaxxer.nuprocess.linux.ProcessEpoll.cleanupProcess(ProcessEpoll.java:334)
at com.zaxxer.nuprocess.linux.ProcessEpoll.process(ProcessEpoll.java:272)
at com.zaxxer.nuprocess.internal.BaseEventProcessor.run(BaseEventProcessor.java:81)
at com.zaxxer.nuprocess.linux.ProcessEpoll.run(ProcessEpoll.java:188)
at com.zaxxer.nuprocess.linux.LinuxProcess.run(LinuxProcess.java:114)
at com.zaxxer.nuprocess.linux.LinProcessFactory.runProcess(LinProcessFactory.java:50)
at com.zaxxer.nuprocess.NuProcessBuilder.run(NuProcessBuilder.java:273)
at com.atlassian.bitbucket.internal.process.nu.NuNioProcessHelper.run(NuNioProcessHelper.java:75)
at com.atlassian.bitbucket.internal.process.NioCommand.call(NioCommand.java:52)
at com.atlassian.stash.internal.commit.DefaultCommitService.getCommits(DefaultCommitService.java:450)
at com.atlassian.stash.internal.commit.DefaultCommitService.getCommits(DefaultCommitService.java:175)
at com.atlassian.stash.internal.web.repos.RepositoryController.getCommits(RepositoryController.java:218)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24)
...
Caused by: com.atlassian.utils.process.ProcessTimeoutException: process timed out
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:269)
... 52 common frames omitted
{noformat}
h3. Workaround

Currently there is no known workaround for this behaviour.

Stefan made changes - 09/Mar/2021 10:17 AM

Status

Original: Needs Triage [ 10030 ]

New: In Progress [ 3 ]

Assignee:: Stefan

Reporter:: Stefan

Affected customers:: 1 This affects my team

Watchers:: 3 Start watching this issue

Created:: 09/Mar/2021 10:17 AM

Updated:: 03/May/2021 1:01 PM

Resolved:: 16/Mar/2021 4:02 AM

Details

Description

Issue Summary

Steps to Reproduce

Expected Results

Actual Results

Workaround

Attachments

Issue Links

Forms

Activity

People

Dates

Backbone Issue Sync