DodgyBear

[sils]

https://github.com/landscapeio/dodgy

difficulty/low type/bear proposal area/lintbears

[meetmangukiya]

@sils1297 I'd be interested in writing a bear. Maybe this one?

[Makman2]

cool why not @sils1297 :)

[meetmangukiya]

I think HappinessBear and implementation of this bear would be pretty much similar, right?

[sils]

👍 see docs.coala.io docs about writing linter bears

[meetmangukiya]

@sils1297 just in case you haven't noticed, the repo hasn't been updated in last 9 months that is as good as a year.

[jayvdb]

Ya. Unmaintained.
A language unaware regex bear would be sufficient for some of the dodgy checks, esp conflict markers.

[sils]

Nice, let's do that.

[meetmangukiya]

@sils1297 given that the repo is not maintained do we want to use it?

[sils]

As @jayvdb proposed: we can rather make an own native bear with that functionality which is usable for all languages. Essentially we just "grep" through the code for suh things right?

[meetmangukiya]

I think I'll have to get an idea of dodgy . Got to read up some stuff.

[meetmangukiya]

Can we have some words that the bear should be dodging other then the ones in dodgy, I'll add those anyway, and also I was thinking of configuring the DodgyBear with a list of words that could be maybe mentioned in the coafile of the project. Some specific words for the CI or something they use, etc. Thoughts? CC @sils1297 @underyx

[jayvdb]

As @jayvdb proposed: we can rather make an own native bear with that functionality which is usable for all languages. Essentially we just "grep" through the code for suh things right?

Allowing regex config support to KeywordBear would suffice?
It already uses regex internally, but doesnt allow configuration using regex, which is #311.

[sils]

Though I think sane defaults are key to this, having some binary options like detect_aws_keys or so is much easier than making up a regex for it

[sils]

Take a look at https://docs.gitlab.com/ee/push_rules/push_rules.html#prevent-pushing-secrets-to-the-repository and see what they do - that one is largely filename based.

[meetmangukiya]

That'd be helpful

[meetmangukiya]

I'd like to request an unassignment, sorry @sils1297 @Makman2 :( I hope the discussion would help the future assignee. Reason being I haven't been able to come up with even the little bit of code of dodgy bear, even after a week and it is getting uninteresting now :( .

[madhur-tandon]

Please assign me this, I would like to give it a shot

[swapagarwal]

@sils Maybe we can use this? https://github.com/awslabs/git-secrets

[manankalra]

Can someone guide me a little on what all understanding do I need other than getting familiar with writing a linter bear that the coala documentation demonstrates to take up this issue?

[manankalra]

@sils @Makman2
What should create_arguments() return for a global linter bear which implements a custom processing function and accepts no command-line arguments of any sort?
I guess the docs do not cover this aspect and there is only a single global linter bear implemented till now (CPPCheckBear) which does accept a --template argument.

[Makman2]

Then you return an empty iterable ;)

I guess the docs do not cover this aspect and there is only a single global linter bear implemented till now (CPPCheckBear) which does accept a --template argument.

Hm I actually think it makes kind of sense like it is now, create_arguments returns an iterable / sequence of arguments to pass to the program. If you don't want to pass any additional arguments, just return [] or tuple() ;)
And the case not passing any arguments is pretty rare^^

[manankalra]

I'm facing problems while writing a test for it and couldn't come up with a solution.
The issue is -
Initialising file_dict with the absolute paths of the files that I need to test seems to have no impact on the uut object and the bear just checks for all the files in the current directory from where pytest -k DodgyBearTest ran. This is so because dodgy accepts no command-line arguments of any sort. That is, one cannot even specify a --path for files that need to be checked. The same thing is addressed in this issue. Also, whenever dodgy runs, it just looks for .py files present in the current working directory but not in the sub-directories. It being a global bear, I couldn't use LocalBearTestHelper for validating checks.

Here's a snippet of what I've done so far for the DodgyBearTest:

import os
import unittest
from queue import Queue

from bears.python.DodgyBear import DodgyBear
from coalib.settings.Section import Section
from coalib.testing.BearTestHelper import generate_skip_decorator
from coalib.testing.LocalBearTestHelper import LocalBearTestHelper


def get_absolute_file_path(file):
    return os.path.join(os.path.dirname(__file__),
                        'dodgy_test_files', file)

@generate_skip_decorator(DodgyBear)
class DodgyBearTest(unittest.TestCase):

    def setUp(self):
        self.file_dict = {}
        self.queue = Queue()
        self.section = Section('dodgy')
        self.test_files = [get_absolute_file_path(f) for f in os.listdir(get_absolute_file_path(""))]
        for filename in self.test_files:
            with open(filename, 'r', encoding='utf-8') as content:
               self.file_dict[filename] = tuple(content.readlines())
        self.uut = DodgyBear(self.file_dict,
                             self.section,
                             self.queue)

    def get_results(self):
        return list(result.message for result in self.uut.run())

    def test(self):
        self.assertEqual(self.get_results()[0], 'Amazon Web Services secret key')

The test passes or raises an AssertionError only if the files being tested are present in the coala-bears or coala-bears/bears/python directory.

To sum up, I guess this is an issue with dodgy. Thoughts?

Other than this, I've added a commit for the implementation of DodgyBear and it seems to work fine for me.

[Makman2]

If dodgy really accepts no command line arguments, that's actually a huge flaw imo of dodgy itself.
We should really consider using dodgy at all then.

However, to make it work anyway and integrate into the coala-ecosystem, we can now deploy some OS capabilities: We could use temporary files. You create a temporary directory and symbolic mappings to the files that shall be analyzed, all ending with .py. The working directory is then this temporary directory (not sure how we can properly set it inside linter)

But really let's first consider using dodgy, and rather make them improve their CLI, because that is just bad design.

[manankalra]

So, what you mean is:

• create temporary files in the root directory i.e. coala-bears.
• run DodgyBearTest.
• delete those temp files under tearDown().

[Makman2]

This won't apply just for tests, but how the bear operates in general. But I would really suggest you sync up with the dodgy-guys and ask why they don't have any arguments to control the linting process.

[manankalra]

@Makman2 I have already written an email to one of the dodgy-guys; didn't get a reply.
Any idea why my tests are not being executed during the build? I'm getting complete coverage locally.

[Makman2]

Probably dodgy not installed in CI.

[manankalra]

Anything I need to take care of?

[Makman2]

Yes, you have to install it ;)

[manankalra]

@Makman2 I know this will be closed in favour of a better alternative but any guesses on why the tests are failing? They are all passing locally. Is writing and deleting new files in the coala-bears directory or any other directory allowed during the ci build process?
I've added dodgy in bear-requirements.txt.

[Makman2]

Is writing and deleting new files in the coala-bears directory or any other directory allowed during the ci build process?

It definitely is.

And I don't know the problem on CI. However this is something you should discuss on the PR itself because it's implementation detail ;)