I feel like I've banned a hundred accounts in the last week.

      It is good to see that a new spam account can only create three questions initially, but it's obvious that the Captcha on signup has now been thoroughly broken and we need to reconsider improved anti-spam-user techniques.

      I would start with
      1) A look at a better Captcha system in the short term (the fact the current one simply isn't good enough strongly suggests that better ones will be broken in the future, so it's only a quick temporary fix)

      2) Spam analysis algorithms - blacklisting of new user's postings unless they actually look coherent and vaguely relevant

      3) Absolute block on posting of links for new accounts (the main reason they do it is to get links picked up by search engines, so removing that facility removes any incentive to do this). Although I'd consider a whitelist so that links to other questions or Atlassian documentation are allowed through

          Form Name

            [CONFSERVER-47728] Spam filters have been defeated

            Timothy added a comment -

            And they have been defeated again. :|

            Timothy added a comment - And they have been defeated again. :|

            Approximately three weeks ago we have had a huge up tick in spam content being posted to answers.atlassian.com.  We've been working eliminating as much of this content as we can but it can prove to be a never ending game of wac-o-mole.  

            Several iterations of the "technical support phone number" content have been added to our anti spam measures and we are continuing to combat this abuse of our service daily.  

            Apologies for any issues this is causing.

            Kevin Collins (Inactive) added a comment - - edited Approximately three weeks ago we have had a huge up tick in spam content being posted to answers.atlassian.com.  We've been working eliminating as much of this content as we can but it can prove to be a never ending game of wac-o-mole.   Several iterations of the "technical support phone number" content have been added to our anti spam measures and we are continuing to combat this abuse of our service daily.   Apologies for any issues this is causing.

            The last few weeks i noticed a drastic increase in spam.

            Is there still something happening here on this issue? E.g. first question is moderated.

            Mike Friedrich added a comment - The last few weeks i noticed a drastic increase in spam. Is there still something happening here on this issue? E.g. first question is moderated.

            lists,

            Thanks. It means a lot to hear that my work has been having an effect. It's also always good to see that you are as active in the Answers community as you've been. I'm still waiting to see if and when anyone unseats you as the top-scoring community member. However, I'm suspecting that will not happen any time soon.

            Here's where we stand on the spam:

            (1) We had some complaints from users this week that the spam-related restrictions were preventing their legitimate access to the service. Reports were coming in to our support teams and I worked with one of our Support Engineers ( wzanchet ) to try and find a better middle ground to better serve the legitimate users while continuing to restrict the spammers. As a result of my change, an increased amount of spam made it through Friday. I apologize, that was all me.

            (2) Since the increase in spam, I spent a good part of my Saturday not only enhancing the filters but also restricting other patterns I have found. (Note: I get these emails too....)

            (3) I would make one request of you and the rest of the community. As we see new patterns emerge, please add those patterns as comments to this ticket. This will help me adapt the charlie-hates-spam script to catch them. CHS is a pretty crude and young little script and it needs to be bed a regular diet of spam samples so it grows up to becomes a spam-munching little beast we want it to be.

            Note: If you want to give credit to anyone for the improvements over the past few months, jclark@atlassian.com is the guy who deserves credit. Ever since Joe arrived in the Austin office, I have had to sit on the same team with him, hearing him remind me that "Answers needs love."

            Sam Caldwell (Inactive) added a comment - lists , Thanks. It means a lot to hear that my work has been having an effect. It's also always good to see that you are as active in the Answers community as you've been. I'm still waiting to see if and when anyone unseats you as the top-scoring community member. However, I'm suspecting that will not happen any time soon. Here's where we stand on the spam: (1) We had some complaints from users this week that the spam-related restrictions were preventing their legitimate access to the service. Reports were coming in to our support teams and I worked with one of our Support Engineers ( wzanchet ) to try and find a better middle ground to better serve the legitimate users while continuing to restrict the spammers. As a result of my change, an increased amount of spam made it through Friday. I apologize, that was all me. (2) Since the increase in spam, I spent a good part of my Saturday not only enhancing the filters but also restricting other patterns I have found. (Note: I get these emails too....) (3) I would make one request of you and the rest of the community. As we see new patterns emerge, please add those patterns as comments to this ticket. This will help me adapt the charlie-hates-spam script to catch them. CHS is a pretty crude and young little script and it needs to be bed a regular diet of spam samples so it grows up to becomes a spam-munching little beast we want it to be. Note: If you want to give credit to anyone for the improvements over the past few months, jclark@atlassian.com is the guy who deserves credit. Ever since Joe arrived in the Austin office, I have had to sit on the same team with him, hearing him remind me that "Answers needs love."

            Sam, you've done a really good job - since the work you talked about on the 9th, there's been a massive reduction in volume. Well done!

            The filters are finding and flagging things really well, we're only getting one spam per dodgy user and it really does feel like we're only banning a few accounts a day and most of them are just tidying up the "flagged as spam by administrator" ones rather than anything new.

            I have noticed a new attack pattern this week - a spammer posts a question with a title (and often body) set to some text that was the title of a recent valid question (It feels like they are choosing questions that have a couple of votes or are actively answered/commented, but I haven't done any analysis to confirm that). That trick is dodging the content filters by using phrases that the spammers know to be ok. We got loads of these on Friday.

            Now, they've started editing them to make them into spam after they've got past the filter by posting a "known good" string.

            As an example (not sure if it's still there, I'm not sure how much a ban destroys the spam): https://answers.atlassian.com/questions/30940408/how-do-i-hide-a-transition-from-a-view-screen really did have the title and body "How do I hide a transition from a view screen" when originally posted. That is a title identical to an earlier legitimate question that has a number of answers and comments. The question was later edited so that the title and body become the advert for astrological donkey-poop we have got used to.

            The volume so far is low enough that it's not a problem though.

            Nic Brough -Adaptavist- added a comment - Sam, you've done a really good job - since the work you talked about on the 9th, there's been a massive reduction in volume. Well done! The filters are finding and flagging things really well, we're only getting one spam per dodgy user and it really does feel like we're only banning a few accounts a day and most of them are just tidying up the "flagged as spam by administrator" ones rather than anything new. I have noticed a new attack pattern this week - a spammer posts a question with a title (and often body) set to some text that was the title of a recent valid question (It feels like they are choosing questions that have a couple of votes or are actively answered/commented, but I haven't done any analysis to confirm that). That trick is dodging the content filters by using phrases that the spammers know to be ok. We got loads of these on Friday. Now, they've started editing them to make them into spam after they've got past the filter by posting a "known good" string. As an example (not sure if it's still there, I'm not sure how much a ban destroys the spam): https://answers.atlassian.com/questions/30940408/how-do-i-hide-a-transition-from-a-view-screen really did have the title and body "How do I hide a transition from a view screen" when originally posted. That is a title identical to an earlier legitimate question that has a number of answers and comments. The question was later edited so that the title and body become the advert for astrological donkey-poop we have got used to. The volume so far is low enough that it's not a problem though.

            Just following up from my last post,

            (1) Over the last 24 hours we have seen a dramatic decrease in spam notifications sent to customers. While spammers are continuing to attack the AAC system, and while this attack frequency has increased, we have seen a significant reduction in email notifications sent to customers (one message per spammer before ban).

            (2) I will be continuing to work to increase the effectiveness of our anti-spam defenses.

            Sam Caldwell (Inactive) added a comment - Just following up from my last post, (1) Over the last 24 hours we have seen a dramatic decrease in spam notifications sent to customers. While spammers are continuing to attack the AAC system, and while this attack frequency has increased, we have seen a significant reduction in email notifications sent to customers (one message per spammer before ban). (2) I will be continuing to work to increase the effectiveness of our anti-spam defenses.

            Just a quick update for the Atlassian Answers community.

            (1) I have implemented something loosely called "charlie-hates-spam." This tool is a prototype for something I am hoping will have a positive effect on our spam issues.
            (2) I'm going to hold off on details about the CHS tool until I have all the features out. What I will say, for the benefit of everyone out there that I know is frustrated by the spam issue, is this: I am working hard to make our service more resilient and less spammy. This past weekend was a great step forward with the first release of CHS, and I hope to have more good news posted soon.

            Sam Caldwell (Inactive) added a comment - Just a quick update for the Atlassian Answers community. (1) I have implemented something loosely called "charlie-hates-spam." This tool is a prototype for something I am hoping will have a positive effect on our spam issues. (2) I'm going to hold off on details about the CHS tool until I have all the features out. What I will say, for the benefit of everyone out there that I know is frustrated by the spam issue, is this: I am working hard to make our service more resilient and less spammy. This past weekend was a great step forward with the first release of CHS, and I hope to have more good news posted soon.

            Thanks Sam, I was able to post my question! Glad I was able to help in some small way.

            Jordan Packer added a comment - Thanks Sam, I was able to post my question! Glad I was able to help in some small way.

            I've rolled back changes from today.
            Please test and confirm that we are working as expected one again.
            If you experience problems, please provide the content you were trying to post.

            jordan.packer, thanks for posting the content. It helped me identify what I THINK was the problem.

            Thanks also for your patience.

            Sam Caldwell (Inactive) added a comment - I've rolled back changes from today. Please test and confirm that we are working as expected one again. If you experience problems, please provide the content you were trying to post. jordan.packer , thanks for posting the content. It helped me identify what I THINK was the problem. Thanks also for your patience.

            Sorry everyone. We're trying to find the right balance with spam protection, and as everyone's pointed out our list is catching too many false positives. Thanks for commenting. I'll have a look and scale it back.

            Jeremy Largman added a comment - Sorry everyone. We're trying to find the right balance with spam protection, and as everyone's pointed out our list is catching too many false positives. Thanks for commenting. I'll have a look and scale it back.

              kcollins@atlassian.com Kevin Collins (Inactive)
              442fdb9642ee Nic Brough -Adaptavist-
              Affected customers:
              20 This affects my team
              Watchers:
              26 Start watching this issue

                Created:
                Updated:
                Resolved: