[JRASERVER-69942] As an JIRA Administrator I want check Lucene Index and Issues consistency

Type: Suggestion
Resolution: Fixed
Fix Version/s: 8.4.0
Component/s: Indexing
Labels:

Feedback Policy:

We collect Jira feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

Problem Definition

In ATST 1.10.0 the index health check for JIRA was disabled. See ~~JRASERVER-67235~~

Workaround

Use REST endpoint: GET <JIRA context>/rest/api/2/index/summary
- Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

Note on fix

REST APIs:

GET - /rest/indexanalyzer/1/state?maxResults=10
- Runs the analysis, this is an expensive call, it will add additional load to DB
GET - /rest/indexanalyzer/1/statistics
- Returned if analysis where successfully done

In Jira issue can be in one of 5 different states:

Index orphan - issue present only in index, should be removed
DB orphan - issue present only in database, should be added to index
Outdated issue - if issue in index is not the same as in database
OK issue - if issues is the same in index and database
Archived issue - if issue is archived

Example:

{
    "indexOrphans": [
        {
            "issueId": 10101,
            "lastUpdated": 1528706295000
        }
    ],
    "dbOrphans": [
        {
            "issueId": 10306,
            "lastUpdated": 1529308219425
        }
    ],
    "outdatedIssueData": [
        {
            "issueId": 10102,
            "dbLastUpdated": 1528706225000,
            "indexLastUpdated": 1528706405655
        }
    ],
    "okCount": 72,
    "outdatedCount": 1,
    "indexOrphansCount": 1,
    "dbOrphansCount": 1,
    "lastAnalyzeTimestamp": 1529308274757,
    "indexHealth": 97.297295
}

Description:

indexOrphans, dbOrphans, outdatedIssueData, outdatedCount, indexOrphansCount, dbOrphansCount - see above
okCount → count of ok issues (from whole Jira)
lastAnalyzeTimestamp → timestamp when analysis where done
indexHealth → percentage of issues that or not in index or they are outdated, example: 97.297295 → 97.29% percents of issues in database are ok in index

is a regression of

JRASERVER-61501 Index consistency health check is too strict in Jira Datacenter

Closed

is cloned from

JRASERVER-65321 As an JIRA Datacenter Administrator I want check Lucene status and consistency

Closed

is related to

JRASERVER-62938 As an JIRA Administrator for JIRA Datacenter I want to know state of replication queue

Closed

JRASERVER-76343 Extend index analyzer with index fixing capability

Closed

relates to

JRASERVER-65672 As an JIRA Datacenter Administrator I want have Lucene health-check in STP

Closed

mentioned in: Page Loading...; Page Loading...; Page Loading...

(3 mentioned in)

Andriy Yakovlev [Atlassian] made changes - 06/Oct/2023 3:34 PM

Link

New: This issue is related to ~~JRASERVER-76343~~ [ ~~JRASERVER-76343~~ ]

Andriy Yakovlev [Atlassian] made changes - 10/May/2021 3:02 PM

Description

Original: h3. Problem Definition
In ATST *1.10.0* the index health check for JIRA was disabled. See ~~JRASERVER-67235~~

h3. Suggested Solution
Implement Lucene Index status and Issue consistency check

h3. Workaround
* Use REST endpoint: GET {{<JIRA context>/rest/api/2/index/summary}}
** Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

h3. Note on fix
REST APIs:
* GET - /rest/indexanalyzer/1/state?maxResults=10
** Runs the analysis
* GET - /rest/indexanalyzer/1/statistics
** Returned if analysis where successfully done

In Jira issue can be in one of 5 different states:
* Index orphan - issue present only in index, should be removed
* DB orphan - issue present only in database, should be added to index
* Outdated issue - if issue in index is not the same as in database
* OK issue - if issues is the same in index and database
* Archived issue - if issue is archived

Example:
{code:Java|borderStyle=solid}
{
    "indexOrphans": [
        {
            "issueId": 10101,
            "lastUpdated": 1528706295000
        }
    ],
    "dbOrphans": [
        {
            "issueId": 10306,
            "lastUpdated": 1529308219425
        }
    ],
    "outdatedIssueData": [
        {
            "issueId": 10102,
            "dbLastUpdated": 1528706225000,
            "indexLastUpdated": 1528706405655
        }
    ],
    "okCount": 72,
    "outdatedCount": 1,
    "indexOrphansCount": 1,
    "dbOrphansCount": 1,
    "lastAnalyzeTimestamp": 1529308274757,
    "indexHealth": 97.297295
}
{code}
Description:
* indexOrphans, dbOrphans, outdatedIssueData, outdatedCount, indexOrphansCount, dbOrphansCount - see above
* okCount → count of ok issues (from whole Jira)
* lastAnalyzeTimestamp → timestamp when analysis where done
* indexHealth → percentage of issues that or not in index or they are outdated, example: 97.297295 → 97.29% percents of issues in database are ok in index

New: h3. Problem Definition
In ATST *1.10.0* the index health check for JIRA was disabled. See ~~JRASERVER-67235~~

h3. Suggested Solution
Implement Lucene Index status and Issue consistency check

h3. Workaround
* Use REST endpoint: GET {{<JIRA context>/rest/api/2/index/summary}}
** Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

h3. Note on fix
REST APIs:
* GET - /rest/indexanalyzer/1/state?maxResults=10
** Runs the analysis, (!)(!)(!) this is an expensive call, it will add additional load to DB (!)(!)(!)
* GET - /rest/indexanalyzer/1/statistics
** Returned if analysis where successfully done

In Jira issue can be in one of 5 different states:
* Index orphan - issue present only in index, should be removed
* DB orphan - issue present only in database, should be added to index
* Outdated issue - if issue in index is not the same as in database
* OK issue - if issues is the same in index and database
* Archived issue - if issue is archived

Example:
{code:Java|borderStyle=solid}
{
    "indexOrphans": [
        {
            "issueId": 10101,
            "lastUpdated": 1528706295000
        }
    ],
    "dbOrphans": [
        {
            "issueId": 10306,
            "lastUpdated": 1529308219425
        }
    ],
    "outdatedIssueData": [
        {
            "issueId": 10102,
            "dbLastUpdated": 1528706225000,
            "indexLastUpdated": 1528706405655
        }
    ],
    "okCount": 72,
    "outdatedCount": 1,
    "indexOrphansCount": 1,
    "dbOrphansCount": 1,
    "lastAnalyzeTimestamp": 1529308274757,
    "indexHealth": 97.297295
}
{code}
Description:
* indexOrphans, dbOrphans, outdatedIssueData, outdatedCount, indexOrphansCount, dbOrphansCount - see above
* okCount → count of ok issues (from whole Jira)
* lastAnalyzeTimestamp → timestamp when analysis where done
* indexHealth → percentage of issues that or not in index or they are outdated, example: 97.297295 → 97.29% percents of issues in database are ok in index

Andriy Yakovlev [Atlassian] made changes - 10/May/2021 3:01 PM

Labels

Original: er85 pse-request

New: er85 pse-request supportability

Andriy Yakovlev [Atlassian] made changes - 10/May/2021 3:01 PM

Description

New: h3. Problem Definition
In ATST *1.10.0* the index health check for JIRA was disabled. See ~~JRASERVER-67235~~

h3. Suggested Solution
Implement Lucene Index status and Issue consistency check

h3. Workaround
* Use REST endpoint: GET {{<JIRA context>/rest/api/2/index/summary}}
** Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

h3. Note on fix
REST APIs:
* GET - /rest/indexanalyzer/1/state?maxResults=10
** Runs the analysis
* GET - /rest/indexanalyzer/1/statistics
** Returned if analysis where successfully done

In Jira issue can be in one of 5 different states:
* Index orphan - issue present only in index, should be removed
* DB orphan - issue present only in database, should be added to index
* Outdated issue - if issue in index is not the same as in database
* OK issue - if issues is the same in index and database
* Archived issue - if issue is archived

Example:
{code:Java|borderStyle=solid}
{
    "indexOrphans": [
        {
            "issueId": 10101,
            "lastUpdated": 1528706295000
        }
    ],
    "dbOrphans": [
        {
            "issueId": 10306,
            "lastUpdated": 1529308219425
        }
    ],
    "outdatedIssueData": [
        {
            "issueId": 10102,
            "dbLastUpdated": 1528706225000,
            "indexLastUpdated": 1528706405655
        }
    ],
    "okCount": 72,
    "outdatedCount": 1,
    "indexOrphansCount": 1,
    "dbOrphansCount": 1,
    "lastAnalyzeTimestamp": 1529308274757,
    "indexHealth": 97.297295
}
{code}
Description:
* indexOrphans, dbOrphans, outdatedIssueData, outdatedCount, indexOrphansCount, dbOrphansCount - see above
* okCount → count of ok issues (from whole Jira)
* lastAnalyzeTimestamp → timestamp when analysis where done
* indexHealth → percentage of issues that or not in index or they are outdated, example: 97.297295 → 97.29% percents of issues in database are ok in index

Katherine Yabut made changes - 19/Sep/2019 6:08 AM

Workflow	Original: JAC Suggestion Workflow [ 3546546 ]	New: JAC Suggestion Workflow 3 [ 3693865 ]
Status	Original: RESOLVED [ 5 ]	New: Closed [ 6 ]

Andriy Yakovlev [Atlassian] made changes - 11/Sep/2019 8:33 AM

Remote Link

New: This issue links to "Page (Confluence)" [ 449650 ]

Andriy Yakovlev [Atlassian] made changes - 11/Sep/2019 8:33 AM

Remote Link

New: This issue links to "Page (Confluence)" [ 449649 ]

Andriy Yakovlev [Atlassian] made changes - 10/Sep/2019 9:50 AM

Labels

Original: pse-request

New: er85 pse-request

Andriy Yakovlev [Atlassian] made changes - 10/Sep/2019 9:50 AM

Description

Original: h3. Problem Definition
In ATST *1.10.0* the index health check for JIRA Data Center instances was disabled. See ~~JRASERVER-67235~~

h3. Suggested Solution
Implement Lucene Index status and Issue consistency check

h3. Workaround
* Use REST endpoint: GET {{<JIRA context>/rest/api/2/index/summary}}
** Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

Andriy Yakovlev [Atlassian] made changes - 10/Sep/2019 9:48 AM

Description

Original: h3. Problem Definition
In JIRA Instance Health ver *2.1.21* the index health check for JIRA Data Center instances was disabled.

h3. Suggested Solution
Implement Lucene Index status and Issue consistency check

h3. Workaround
* Use REST endpoint: GET {{<JIRA context>/rest/api/2/index/summary}}
** Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

New: h3. Problem Definition
In ATST *1.10.0* the index health check for JIRA Data Center instances was disabled. See ~~JRASERVER-67235~~

h3. Suggested Solution
Implement Lucene Index status and Issue consistency check

h3. Workaround
* Use REST endpoint: GET {{<JIRA context>/rest/api/2/index/summary}}
** Please see docs for details: https://docs.atlassian.com/jira/REST/7.2.10/#api/2/index/summary

Assignee:: Marcin Sasinowski

Reporter:: Andriy Yakovlev [Atlassian]

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: 10/Sep/2019 9:38 AM

Updated:: 06/Oct/2023 3:34 PM

Resolved:: 10/Sep/2019 9:39 AM

Details

Description

Problem Definition

Suggested Solution

Workaround

Note on fix

Attachments

Issue Links

Forms

Activity

People

Dates