Uploaded image for project: 'Confluence Cloud'
  1. Confluence Cloud
  2. CONFCLOUD-52509

Confluence Cloud Export fails returning "com.netflix.hystrix.exception.HystrixRuntimeException: GetFileMediaApiCommand timed-out and no fallback available"

      Summary

      Confluence backup and space export fail. Below exception is thrown in the logs:

      com.netflix.hystrix.exception.HystrixRuntimeException: GetFileMediaApiCommand timed-out and no fallback available.
      	at com.netflix.hystrix.AbstractCommand$22.call(AbstractCommand.java:808)
      	at com.netflix.hystrix.AbstractCommand$22.call(AbstractCommand.java:793)
      	at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140)
      	...
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.util.concurrent.TimeoutException
      	at com.netflix.hystrix.AbstractCommand.handleTimeoutViaFallback(AbstractCommand.java:983)
      	at com.netflix.hystrix.AbstractCommand.access$500(AbstractCommand.java:59)
      	at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:598)
      	at com.netflix.hystrix.AbstractCommand$12.call(AbstractCommand.java:590)
      	at rx.internal.operators.OperatorOnErrorResumeNextViaFunction$4.onError(OperatorOnErrorResumeNextViaFunction.java:140)
      	... 18 more
      

      This incident also impacts:

      • Space imports
      • Site imports
      • Attachments upload/edition
      • Copy page(s)

      Environment

      Confluence: 1000.943.1

      Notes

      Confluence is slower than usual because of this issue: it takes long time to open a page with attachments and in the logs there is the same exception logged:

      2017-02-13 18:57:12.147210500 2017-02-13 18:57:12,141 ERROR [http-bio-1997-exec-65] [plugins.conversion.api.FileStoreConversionResult] getFile Failed to retrieve status of file fe3b47bd-a66f-4263-8d80-331f3fbf43fe from FileStore: Failure{cause=com.netflix.hystrix.exception.HystrixRuntimeException: GetFileMediaApiCommand timed-out and no fallback available., reason=TRANSPORT_ERROR, message='Hystrix command timed out', httpStatusCode=null}
      2017-02-13 18:57:12.147213500  -- url: /wiki/pages/viewpage.action | page: 55018069 | traceId: f322a57c7f6fcf10 | userName: sysadmin | referer: https://interlink.atlassian.net/wiki/display/PP/PO+0036+-+Kariera.gr+Senior+Software+Eng+ad+post | action: viewpage
      2017-02-13 18:57:12.147265500 com.netflix.hystrix.exception.HystrixRuntimeException: GetFileMediaApiCommand timed-out and no fallback available.
      

      Steps to Reproduce

      1. Try to export a Confluence backup or a space

      Expected Results

      The process ends and you get the right resources exported

      Actual Results

      The backup fails returning below exception:

      Workaround

      No workaround available at the moment.

        1. export-fails.png
          export-fails.png
          899 kB
        2. full-error.txt
          43 kB

            [CONFCLOUD-52509] Confluence Cloud Export fails returning "com.netflix.hystrix.exception.HystrixRuntimeException: GetFileMediaApiCommand timed-out and no fallback available"

            Jason Spencer added a comment - - edited

            Hello, we are still experiencing this issue and our cloud install is so slow it is almost unusable.

             

            com.netflix.hystrix.exception.HystrixRuntimeException: JerseySessionServiceValidateCommand fallback execution rejected.
            	com.netflix.hystrix.AbstractCommand.handleFallbackRejectionByEmittingError(AbstractCommand.java:1043)
            	com.netflix.hystrix.AbstractCommand.getFallbackOrThrowException(AbstractCommand.java:875)
            	com.netflix.hystrix.AbstractCommand.handleShortCircuitViaFallback(AbstractCommand.java:982)
            	com.netflix.hystrix.AbstractCommand.applyHystrixSemantics(AbstractCommand.java:557)
            	com.netflix.hystrix.AbstractCommand.access$200(AbstractCommand.java:60)
            	com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:419)
            	com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:413)
            	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46)
            	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
            	rx.Observable.unsafeSubscribe(Observable.java:10151)
            	rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
            	rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
            	rx.Observable.unsafeSubscribe(Observable.java:10151)
            	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
            	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
            	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
            	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
            	rx.Observable.unsafeSubscribe(Observable.java:10151)
            	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41)
            	rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30)
            	rx.Observable.unsafeSubscribe(Observable.java:10151)
            	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51)
            	rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35)
            	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
            	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
            	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
            	rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
            	rx.Observable.subscribe(Observable.java:10247)
            	rx.Observable.subscribe(Observable.java:10214)
            	rx.internal.operators.BlockingOperatorToFuture.toFuture(BlockingOperatorToFuture.java:51)
            	rx.observables.BlockingObservable.toFuture(BlockingObservable.java:411)
            	com.netflix.hystrix.HystrixCommand.queue(HystrixCommand.java:378)
            	com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:344)
            	io.atlassian.sessionservice.client.impl.JerseySessionServiceManager.validate(JerseySessionServiceManager.java:78)
            	io.atlassian.sessionservice.client.impl.DefaultSessionValidator.validateSession(DefaultSessionValidator.java:50)
            	io.atlassian.sessionservice.client.impl.DefaultSessionValidator.validate(DefaultSessionValidator.java:57)
            	java.util.Optional.flatMap(Optional.java:241)
            

             

            Jason Spencer added a comment - - edited Hello, we are still experiencing this issue and our cloud install is so slow it is almost unusable.   com.netflix.hystrix.exception.HystrixRuntimeException: JerseySessionServiceValidateCommand fallback execution rejected. com.netflix.hystrix.AbstractCommand.handleFallbackRejectionByEmittingError(AbstractCommand.java:1043) com.netflix.hystrix.AbstractCommand.getFallbackOrThrowException(AbstractCommand.java:875) com.netflix.hystrix.AbstractCommand.handleShortCircuitViaFallback(AbstractCommand.java:982) com.netflix.hystrix.AbstractCommand.applyHystrixSemantics(AbstractCommand.java:557) com.netflix.hystrix.AbstractCommand.access$200(AbstractCommand.java:60) com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:419) com.netflix.hystrix.AbstractCommand$4.call(AbstractCommand.java:413) rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:46) rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35) rx.Observable.unsafeSubscribe(Observable.java:10151) rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48) rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33) rx.Observable.unsafeSubscribe(Observable.java:10151) rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41) rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30) rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) rx.Observable.unsafeSubscribe(Observable.java:10151) rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:41) rx.internal.operators.OnSubscribeDoOnEach.call(OnSubscribeDoOnEach.java:30) rx.Observable.unsafeSubscribe(Observable.java:10151) rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:51) rx.internal.operators.OnSubscribeDefer.call(OnSubscribeDefer.java:35) rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48) rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30) rx.Observable.subscribe(Observable.java:10247) rx.Observable.subscribe(Observable.java:10214) rx.internal.operators.BlockingOperatorToFuture.toFuture(BlockingOperatorToFuture.java:51) rx.observables.BlockingObservable.toFuture(BlockingObservable.java:411) com.netflix.hystrix.HystrixCommand.queue(HystrixCommand.java:378) com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:344) io.atlassian.sessionservice.client.impl.JerseySessionServiceManager.validate(JerseySessionServiceManager.java:78) io.atlassian.sessionservice.client.impl.DefaultSessionValidator.validateSession(DefaultSessionValidator.java:50) io.atlassian.sessionservice.client.impl.DefaultSessionValidator.validate(DefaultSessionValidator.java:57) java.util.Optional.flatMap(Optional.java:241)  

            martin.trneny2126826849 when we mark a Cloud bug as Resolved, it indicates that it is fixed for all cloud instances. Please let us know if you're still experiencing this issue and it is showing the stack trace as indicated in the screenshot.

            jonah (Inactive) added a comment - martin.trneny2126826849 when we mark a Cloud bug as Resolved, it indicates that it is fixed for all cloud instances. Please let us know if you're still experiencing this issue and it is showing the stack trace as indicated in the screenshot.

            Atlassian, are you still rolling out this fix to the instances?? It's almost 10 days!! Could you please also then fill in Fix Version/s so we can be sure our instance has the fix?  

            Martin Trneny added a comment - Atlassian, are you still rolling out this fix to the instances?? It's almost 10 days!! Could you please also then fill in Fix Version/s so we can be sure our instance has the fix?  

            We are still unable to export spaces or more importantly backup our confluence cloud instance

            Eduardo Perez added a comment - We are still unable to export spaces or more importantly backup our confluence cloud instance

            jonah (Inactive) added a comment - - edited

            The fix for this issue is currently rolling out to all customer instances at this time. By this time tomorrow, the bug should be resolved - please let us know if you are still experiencing issues after that.

            Thank you for your patience

            Best,
            Jonah Turnquist
            Confluence Engineering team

            jonah (Inactive) added a comment - - edited The fix for this issue is currently rolling out to all customer instances at this time. By this time tomorrow, the bug should be resolved - please let us know if you are still experiencing issues after that. Thank you for your patience Best, Jonah Turnquist Confluence Engineering team

            Can we have an update or ETA ?

            Brendan Seerup added a comment - Can we have an update or ETA ?

            @Mauro I am still receiving the errow when trying to export a space.  Should this update immediately, or when can we expect a rollout?

            Thanks

            Niall Brownlee added a comment - @Mauro I am still receiving the errow when trying to export a space.  Should this update immediately, or when can we expect a rollout? Thanks

            Update:
            A fix has been released for the error "GetFileMediaApiCommand" affecting imports, space exports, attachments and page copy.

            Unfortunately we have detected a variation of the problem that still intermittently affects Confluence backups that error with:

            com.netflix.hystrix.exception.HystrixRuntimeException: CreateFileFromUploadMediaApiCommand timed-out and no fallback available.

            We have a dedicated team already working on solving this problem and will come back with more news.

            Thanks again for your patience.

            Best regards,
            Mauro

            Mauro Badii (Inactive) added a comment - Update: A fix has been released for the error "GetFileMediaApiCommand" affecting imports, space exports, attachments and page copy. Unfortunately we have detected a variation of the problem that still intermittently affects Confluence backups that error with: com.netflix.hystrix.exception.HystrixRuntimeException: CreateFileFromUploadMediaApiCommand timed-out and no fallback available. We have a dedicated team already working on solving this problem and will come back with more news. Thanks again for your patience. Best regards, Mauro

            Current Status:

            A fix has been developed and is pending release.
            We are tracking it's progress and we'll update once it's been released to Cloud sites.
            Thanks for the patience.

            Regards,
            Mauro
            Atlassian Cloud Support

            Mauro Badii (Inactive) added a comment - Current Status: A fix has been developed and is pending release. We are tracking it's progress and we'll update once it's been released to Cloud sites. Thanks for the patience. Regards, Mauro Atlassian Cloud Support

            Hello, this is also impacting use of the Confluence Cloud Copy Space add-on (https://marketplace.atlassian.com/plugins/ac-copy-space/cloud/overview). Probably because as already mentioned this add-on is trying to copy a Space. Also, dumb question, but why the reference to "netflix" in the error?

            Jordan M. Pflum added a comment - Hello, this is also impacting use of the Confluence Cloud Copy Space add-on ( https://marketplace.atlassian.com/plugins/ac-copy-space/cloud/overview ). Probably because as already mentioned this add-on is trying to copy a Space. Also, dumb question, but why the reference to "netflix" in the error?

              gvotruong Giang Vo
              dbonotto Dario B
              Affected customers:
              18 This affects my team
              Watchers:
              49 Start watching this issue

                Created:
                Updated:
                Resolved: