Tuesday, April 8, 2014

Getting Started with pylint


Being somewhat new to Python, I figured I'd give pylint a try. In Perl we have perlcritic which serves a very similar function (although in Perl files are parsed in a DOM-like way because as some of us painfully know, the only thing that can truly parse Perl is Perl).

Setting Up

First things first, let's setup a test virtualenv so we don't clobber anything. If you want to install globally or already have a project you'd like to lint, skip this step.

jbisbee@beni:~$ mkvirtualenv test
New python executable in test/bin/python
Installing
Setuptools.............................................................................................done.
Installing
Pip....................................................................................................................................done.

Now pip install pylint

(test)jbisbee@beni:~$ pip install pylint
Downloading/unpacking pylint
...
Successfully installed pylint logilab-common astroid
Cleaning up...

So out of the box I get a warning message saying I don't have a default configuration. Before I start looking at the output, I figure I should probably figure out where the defualt rcfile is stored (~/.pylintrc) and what format its in.

jbisbee@beni$ pylint sample.py
No config file found, using default configuration
...

Generate Your ~/.pylintrc File

Lucky for us, the authors have included the handy --generate-rcfile switch that will generate one for you. The rcfile generation option also takes into account all switches you pass into pylint and will include those in your resulting file. The rcfile is printed to standard out which you can just redirect to your .pylintrc file in your home directory.

(test)jbisbee@beni:~/src/test$ pylint --generate-rcfile > ~/.pylintrc

Explore and Tweak To Your Liking

The resulting file is pretty lengthy, but I'd like to highlight some of the various options and sections that I took interest in. I took some of the descriptions directly from the documentation. disable=
Disable the message, report, category or checker with the given id(s). If you or your team ends up choosing not to enforce certain errors or warnings that pylint produces, just add the error number (or in later versions, the human readable string) to disable the messages from showing up.

output-format=colorize
Set the output format. Available formats are text, parseable, colorized, msvs (visual studio) and html. You can also give a reporter class, mypackage.mymodule.MyReporterClass. Going to default to colorize.

reports=no
Tells whether to display a full report or only the messages. No reports by default for me.

max-line-length=120
Maximum number of characters on a single line. I'm going to set this from 80 to 120. Purists out there, all I have to say is, "Haters gonna hate." :)

generated-members=objects
List of members which are set dynamically and missed by pylint inference system. Django ORM's 'objects' is only value I have presently. Feel free to suggest other common generated members.

Similarities Section
Similar lines in %s files Indicates that a set of similar lines has been detected among multiple file. This usually means that the code should be refactored to avoid this duplication. I love the idea that pylint does this and I'd like to get some feedback for folks who encountered this error and how accurate it was.

Design Section
This sections is my favorite. It points out possible design flaws in your code based on a set of rules you can tweak. I consider my OO design to be pretty darn good, but as I was quickly porting a simple all in one command line program into classes, I got flagged with 'max-attributes' error. The error states if that you have more than 7 attributes you're most likely doing something wrong. (I had 8) It pointed out 5 of those attributes belonged in a separate class of their own and made my design cleaner (which never really hurts)

IDE/Editor Support

One of the coolest things about pylint is the IDE/editor integration that can give you real time feedback as you're coding. Visit the IDE integration page on the pylint website for more information. Going attempt to setup pylint.vim soon. :)

Would Love Your Feedback

Hey guys, I'm writing these blog posts as an exercise to teach myself and hopefully have others who are in my position, stumble upon my blog posts and actually do them some good. If you have some valuable input, point out pitfalls, or just to say thank you, I would greatly appreciate it.

8 comments:

  1. The problem I have with pylint is the poor signal-to-noise ratio. It has some useful information, but it also has a ton of information that is not useful for me. The python community has largely agreed to standardize on pep8. pylint, however, establishes a lot of additional rules and conventions that are not followed by other projects.

    For example, it has a ton of warnings about naming that are not followed by any project I have seen, including the python standard library itself. Similarly, it complains if you have too many variables, too many classes, too many methods, too many functions, too many loops, or too many branches, rules which again are not followed by any project I have seen.

    Yes, you can configure it to not display those, but that requires manually disabling a couple dozen different messages. And just disabling warnings is not good, because there are useful warnings as well. As far as I have found there is no way to just disable pylint's own conventions while still showing legitimate issues.

    So perhaps it is fine if you are writing your own python code, and don't need to interact with the larger python community, and decide to follow pylint's conventions from the beginning. However, if you are trying to use pylint with an existing project, then it is much less useful.

    ReplyDelete
    Replies
    1. Pylint classifies convention issues under category ‘C’, and factoring issues (like “too many attributes”) under ‘R’. A simple `--disable=C,R` will prevent all these, while leaving intact all errors (like “no such name in scope”) and logical warnings (like “unused variable”).

      Delete
    2. Unfortunately, that is too blunt an approach since it also disables valid issues. For instance C0111, C0112, C0121, C0202, C0301, C0303, C0304, C0321, and C1001 are valid issues, not pylint-specific conventions. Similarly, R0401 is a real problem that often leads to python crashes, R0801 is a valid issue that someone would likely want to fix.

      On the other hand there are perfectly valid python conventions that are mixed in elsewhere. For instance there is absolutely nothing wrong with using * or **, but it raises a warning by pylint.

      Delete
  2. I'm using pep8 and pyflakes as my default linting tools, and just now I;m extending these with the use of pep257, and mccabe.

    To combaine them all in one simple check, I'm using pylama, it allows for easy configuration of what and when to check, ignore specified errors, groups of errors and provides a git/mercurial pre-commit hook that checks changed files.

    ReplyDelete
  3. You said that "So out of the box I get a warning message saying I don't have a default configuration. ". But the output of pylint is "No config file found, using default configuration". So I think the sentence "I don't have a default configuration." should be something like "don't have a personal configuration".

    ReplyDelete
  4. Sorry, i think you have small mistake in "output-format=colorize" - with this i can't run pylint. Should be output-format=colorized.

    And thank you for good article!

    ReplyDelete
  5. This comment has been removed by a blog administrator.

    ReplyDelete
  6. how to run whole django project in pylint

    ReplyDelete