We couldn't load all Actvitity tabs. Refresh the page to try again.
If the problem persists, contact your Jira admin.
IMPORTANT: JAC is a Public system and anyone on the internet will be able to view the data in the created JAC tickets. Please don’t include Customer or Sensitive data in the JAC ticket.
Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-67703

JIRA Datacenter start generates large network traffic result in higher response time

      Summary

      JIRA Data Center uses a caching library called EHCache, and contains an in-memory cache of certain project, user and group data. On the very first node start (if no others are running) this cache is built from querying the database. When additional nodes are started, the cache is built from querying the existing nodes in-memory cache and propagating that data into the node that is starting up. Other caches are built only locally by filling the node's own copy of the cache directly from the database.

      Additionally when JIRA Data Center restarts a node reindex thread runs which is designed to synchronise changes between the nodes - this can also contribute to the increase in network traffic.

      Expected Behaviour

      Starting up a node should not affect other nodes performance

      Actual Behaviour

      When adding node during busy hour we can notice surge in network traffic below

       

      Which result in increased of the overall response time of all other nodes in the cluster for 3 minutes.

      node-1 node-2 node-3

      Note

      There some improvements related to the cache replication during start-up:

      Also in recent version 7.6.4 there is a fix JRASERVER-66839 which mostly contributing to this problem.

      Workaround

      Avoid adding/removing nodes during peak hour

       

        1. image.png
          image.png
          128 kB
        2. node2_slow.JPG
          node2_slow.JPG
          23 kB
        3. node3_slow.JPG
          node3_slow.JPG
          22 kB
        4. node4_slow.JPG
          node4_slow.JPG
          23 kB

            Loading...
            IMPORTANT: JAC is a Public system and anyone on the internet will be able to view the data in the created JAC tickets. Please don’t include Customer or Sensitive data in the JAC ticket.
            Uploaded image for project: 'Jira Data Center'
            1. Jira Data Center
            2. JRASERVER-67703

            JIRA Datacenter start generates large network traffic result in higher response time

                Summary

                JIRA Data Center uses a caching library called EHCache, and contains an in-memory cache of certain project, user and group data. On the very first node start (if no others are running) this cache is built from querying the database. When additional nodes are started, the cache is built from querying the existing nodes in-memory cache and propagating that data into the node that is starting up. Other caches are built only locally by filling the node's own copy of the cache directly from the database.

                Additionally when JIRA Data Center restarts a node reindex thread runs which is designed to synchronise changes between the nodes - this can also contribute to the increase in network traffic.

                Expected Behaviour

                Starting up a node should not affect other nodes performance

                Actual Behaviour

                When adding node during busy hour we can notice surge in network traffic below

                 

                Which result in increased of the overall response time of all other nodes in the cluster for 3 minutes.

                node-1 node-2 node-3

                Note

                There some improvements related to the cache replication during start-up:

                Also in recent version 7.6.4 there is a fix JRASERVER-66839 which mostly contributing to this problem.

                Workaround

                Avoid adding/removing nodes during peak hour

                 

                  1. image.png
                    image.png
                    128 kB
                  2. node2_slow.JPG
                    node2_slow.JPG
                    23 kB
                  3. node3_slow.JPG
                    node3_slow.JPG
                    22 kB
                  4. node4_slow.JPG
                    node4_slow.JPG
                    23 kB

                        Unassigned Unassigned
                        vkharisma vkharisma (Inactive)
                        Affected customers:
                        3 This affects my team
                        Watchers:
                        10 Start watching this issue

                          Created:
                          Updated:

                            Unassigned Unassigned
                            vkharisma vkharisma (Inactive)
                            Affected customers:
                            3 Vote for this issue
                            Watchers:
                            10 Start watching this issue

                              Created:
                              Updated: