Allow the ability to exclude revisions

Original problem description

Certain customers repositories have revisions that throw errors, due to either corruption of the SCM, or some bug within the SCM's themselves. SVN often throws errors when users do funky things with symbolic links etc.

In this case, there isn't much that fisheye can do, as svn or p4 etc are the ones responsible for throwing the error.

Thus it would be great to be able to skip over these problem revisions and perhaps put a note in fisheye that these revisions were skipped (and possibly also allow admins to include a comment why these revisions were skipped).

Use cases identified in this suggestion and comments

Repository corruption

It may happen that code repository becomes corrupted and SCM is unable to read data from certain revisions.

While having on ability to exclude a revision may be a temporary workaround, the real solution is to fix a corrupted repository.

We consider this use case as out of scope of this suggestion.

Non-parseable repository data

While repository may not be corrupted, it can still contain invalid data (because SCM does not have some data validation implemented for instance) or Fisheye is unable to process such data (e.g. some corner case). An example could be: non-UTF characters in a file name or mismatch in character encodings (like FE-6909).

In order to mitigate this problem, pre-commit hooks may be set up for a repository to avoid such commits in the future.

While having an ability to exclude a revision may be a temporary workaround, the correct solution is fixing bug in Fisheye's repository indexer. Therefore please watch existing specific bug reports or raise new ones.

We consider this use case as out of scope of this suggestion.

Large repository operations which have been fully reverted

An example can be folder copy in Subversion, followed by a folder removal in next commit. Another example is a commit with some changes in file content/metadata immediately followed by a revert commit.

In such case it would be technically possible to implement a modification in Fisheye's repository scanner to skip such two commits as the repository content is identical before and after them (a diff to a previous revision should show the same data). But this requires that there are no references to excluded commits in repository history (like a branch created out of excluded commit).

This use case is in scope of this suggestion.

Large repository operations which have not been reverted

It may happen that there are some huge commits, which have not been reverted, but could be discarded because they're considered obsolete. Example: creation of a dead branch, experimental changes on some branch etc.

Defining proper exclusion patterns (if possible) may reduce severity of the problem.

It would be technically possible to implement a modification in Fisheye's repository to exclude such revisions, but only under condition that there are no references to excluded commits in later commits.

This use case is in scope of this suggestion.

Large commits with trash data

Examples are: committing Maven's /target, Node.js or Gradle caches, adding entire Desktop by accident.

In order to mitigate this problem, pre-commit hooks may be set up for a repository to avoid such commits in the future.

Another solution to mitigate the problem is to define proper exclusion patterns in repository settings.

We consider this use case as out of scope of this suggestion.

Large commits with real changes

An example could be massive code refactoring.

This cannot be skipped by Fisheye's indexer, as this data is required and there may be references to this data in further commits.

We may consider some improvements however, such as:

• process a commit in smaller chunks so that indexing thread can be released and indexing other repositories could progress (note that current indexer already works in multiple time-bound phases)
• Lucene upgrade (experience from Jira 8.0 shows that Lucene upgrade may give 50% index size reduction and 30-50% speed-up) - tracked in FE-7156

These improvements shall be tracked in separate issues.

We consider this use case as out of scope of this suggestion.

Long indexing times

Performance speed-up of certain types of operations is tracked in separate issues:

• indexing deleted files / folders / branches - FE-314, FE-5453
• indexing tags - FE-2743
• excluded paths

We consider this use case as out of scope of this suggestion.