Wednesday, April 30, 2014

Using pep257, a Python docstring style checker, with Buildout

PEP 257 documents the semantics and conventions associated with Python docstrings. According to it:

A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition. Such a docstring becomes the __doc__ special attribute of that object.

This is specially useful, for instance, when you are introspecting code and you want to know what a specific method or function does.

pep257 is a Python docstring style checker.

I already filled a feature request on plone.recipe.codeanalysis to add support for it; meanwhile, if you want to use it in your Buildout-based projects, you can add the following to your buildout configuration:

[buildout]
parts = pep257
[pep257]
recipe = zc.recipe.egg
eggs = pep257
entry-points = pep257=pep257:main
arguments = *pep257.parse_options()
view raw pep257.cfg hosted with ❤ by GitHub
After running bin/buildout you will find a bin/pep257 script.

A typical output will look like this:

(python-2.7)# hvelarde@nanovac (master * u+1) ~/collective/polls 
# bin/pep257 src/
src/collective/__init__.py:1 at module level:
        D100: Docstring missing
src/collective/polls/js_i18n_helper.py:1 at module level:
        D100: Docstring missing
src/collective/polls/js_i18n_helper.py:8 in public class `LegendOthersTranslation`:
        D101: Docstring missing
src/collective/polls/js_i18n_helper.py:14 in public method `render`:
        D102: Docstring missing
src/collective/polls/testing.py:1 at module level:
        D100: Docstring missing
src/collective/polls/testing.py:10 in public class `Fixture`:
        D101: Docstring missing
src/collective/polls/testing.py:14 in public method `setUpZope`:
        D102: Docstring missing
src/collective/polls/testing.py:19 in public method `setUpPloneSite`:
        D102: Docstring missing
src/collective/polls/config.py:1 at module level:
        D100: Docstring missing
src/collective/polls/polls.py:1 at module level:
        D100: Docstring missing
src/collective/polls/polls.py:21 in public class `IPolls`:
        D101: Docstring missing
src/collective/polls/polls.py:24 in public method `recent_polls`:
        D400: First line should end with '.', not 's'
src/collective/polls/polls.py:24 in public method `recent_polls`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:27 in public method `uid_for_poll`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:27 in public method `uid_for_poll`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:30 in public method `poll_by_uid`:
        D400: First line should end with '.', not 'd'
src/collective/polls/polls.py:30 in public method `poll_by_uid`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:33 in public method `voters_in_a_poll`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:33 in public method `voters_in_a_poll`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:36 in public method `voted_in_a_poll`:
        D400: First line should end with '.', not 'd'
src/collective/polls/polls.py:36 in public method `voted_in_a_poll`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:39 in public method `allowed_to_edit`:
        D401: First line should be imperative: 'i', not 'is'
src/collective/polls/polls.py:39 in public method `allowed_to_edit`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:39 in public method `allowed_to_edit`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:42 in public method `allowed_to_view`:
        D401: First line should be imperative: 'I', not 'Is'
src/collective/polls/polls.py:42 in public method `allowed_to_view`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:42 in public method `allowed_to_view`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:45 in public method `allowed_to_vote`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:45 in public method `allowed_to_vote`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:48 in public method `anonymous_vote_id`:
        D400: First line should end with '.', not 'd'
src/collective/polls/polls.py:48 in public method `anonymous_vote_id`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:52 in public class `Polls`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/polls.py:52 in public class `Polls`:
        D400: First line should end with '.', not 's'
src/collective/polls/polls.py:52 in public class `Polls`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:60 in public method `ct`:
        D102: Docstring missing
src/collective/polls/polls.py:64 in public method `mt`:
        D102: Docstring missing
src/collective/polls/polls.py:68 in public method `wt`:
        D102: Docstring missing
src/collective/polls/polls.py:72 in public method `member`:
        D102: Docstring missing
src/collective/polls/polls.py:75 in private method `_query_for_polls`:
        D400: First line should end with '.', not 's'
src/collective/polls/polls.py:75 in private method `_query_for_polls`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:81 in public method `uid_for_poll`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:81 in public method `uid_for_poll`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:85 in public method `recent_polls`:
        D400: First line should end with '.', not 's'
src/collective/polls/polls.py:85 in public method `recent_polls`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:98 in public method `poll_by_uid`:
        D400: First line should end with '.', not 'd'
src/collective/polls/polls.py:98 in public method `poll_by_uid`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:109 in public method `voted_in_a_poll`:
        D400: First line should end with '.', not 'd'
src/collective/polls/polls.py:109 in public method `voted_in_a_poll`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:128 in public method `allowed_to_edit`:
        D401: First line should be imperative: 'I', not 'Is'
src/collective/polls/polls.py:128 in public method `allowed_to_edit`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:128 in public method `allowed_to_edit`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:133 in public method `allowed_to_view`:
        D401: First line should be imperative: 'I', not 'Is'
src/collective/polls/polls.py:133 in public method `allowed_to_view`:
        D400: First line should end with '.', not 'l'
src/collective/polls/polls.py:133 in public method `allowed_to_view`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:137 in public method `allowed_to_vote`:
        D401: First line should be imperative: 'i', not 'is'
src/collective/polls/polls.py:137 in public method `allowed_to_vote`:
        D400: First line should end with '.', not '?'
src/collective/polls/polls.py:137 in public method `allowed_to_vote`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/polls.py:156 in public class `PollPortletRender`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/polls.py:156 in public class `PollPortletRender`:
        D204: Expected 1 blank line *after* class docstring, found 0
src/collective/polls/polls.py:156 in public class `PollPortletRender`:
        D400: First line should end with '.', not 'w'
src/collective/polls/polls.py:181 in public method `render_portlet`:
        D202: No blank lines allowed *after* method docstring, found 1
src/collective/polls/polls.py:251 in public method `render`:
        D400: First line should end with '.', not 'e'
src/collective/polls/__init__.py:1 at module level:
        D100: Docstring missing
src/collective/polls/subscribers.py:1 at module level:
        D100: Docstring missing
src/collective/polls/subscribers.py:17 in public function `fix_permissions`:
        D401: First line should be imperative: 'Thi', not 'This'
src/collective/polls/subscribers.py:17 in public function `fix_permissions`:
        D400: First line should end with '.', not 'f'
src/collective/polls/subscribers.py:17 in public function `fix_permissions`:
        D205: Blank line missing between one-line summary and description
src/collective/polls/subscribers.py:17 in public function `fix_permissions`:
        D208: Docstring is over-indented
src/collective/polls/subscribers.py:17 in public function `fix_permissions`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/subscribers.py:37 in public function `remove_votes`:
        D401: First line should be imperative: 'Thi', not 'This'
src/collective/polls/subscribers.py:37 in public function `remove_votes`:
        D400: First line should end with '.', not 't'
src/collective/polls/subscribers.py:37 in public function `remove_votes`:
        D205: Blank line missing between one-line summary and description
src/collective/polls/subscribers.py:37 in public function `remove_votes`:
        D208: Docstring is over-indented
src/collective/polls/subscribers.py:37 in public function `remove_votes`:
        D300: Expected """-quotes, got '''-quotes
src/collective/polls/setuphandlers.py:1 at module level:
        D100: Docstring missing
src/collective/polls/setuphandlers.py:12 in public class `HiddenProfiles`:
        D101: Docstring missing
src/collective/polls/setuphandlers.py:18 in public method `getNonInstallableProfiles`:
        D102: Docstring missing
src/collective/polls/setuphandlers.py:23 in public function `updateWorkflowDefinitions`:
        D103: Docstring missing
src/collective/polls/setuphandlers.py:29 in public function `setupVarious`:
        D103: Docstring missing
src/collective/polls/portlet/voteportlet.py:1 at module level:
        D100: Docstring missing
src/collective/polls/portlet/voteportlet.py:24 in public function `PossiblePolls`:
        D103: Docstring missing
src/collective/polls/portlet/voteportlet.py:46 in public class `IVotePortlet`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/portlet/voteportlet.py:46 in public class `IVotePortlet`:
        D400: First line should end with '.', not 't'
src/collective/polls/portlet/voteportlet.py:46 in public class `IVotePortlet`:
        D205: Blank line missing between one-line summary and description
src/collective/polls/portlet/voteportlet.py:87 in public class `Assignment`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/portlet/voteportlet.py:102 in public method `__init__`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:111 in public method `title`:
        D401: First line should be imperative: 'Thi', not 'This'
src/collective/polls/portlet/voteportlet.py:111 in public method `title`:
        D400: First line should end with '.', not 'e'
src/collective/polls/portlet/voteportlet.py:111 in public method `title`:
        D205: Blank line missing between one-line summary and description
src/collective/polls/portlet/voteportlet.py:118 in public class `Renderer`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/portlet/voteportlet.py:129 in public method `utility`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/portlet/voteportlet.py:135 in public method `portlet_manager_name`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:149 in public method `poll`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:170 in public method `poll_uid`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/portlet/voteportlet.py:176 in public method `getVotingResults`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:184 in public method `can_vote`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:193 in public method `available`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:202 in public method `is_closed`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:208 in public class `AddForm`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/portlet/voteportlet.py:208 in public class `AddForm`:
        D204: Expected 1 blank line *after* class docstring, found 0
src/collective/polls/portlet/voteportlet.py:217 in public method `create`:
        D102: Docstring missing
src/collective/polls/portlet/voteportlet.py:221 in public class `EditForm`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/portlet/voteportlet.py:221 in public class `EditForm`:
        D204: Expected 1 blank line *after* class docstring, found 0
src/collective/polls/portlet/__init__.py:1 at module level:
        D100: Docstring missing
src/collective/polls/Extensions/Install.py:1 at module level:
        D100: Docstring missing
src/collective/polls/Extensions/Install.py:7 in public function `uninstall`:
        D103: Docstring missing
src/collective/polls/Extensions/__init__.py:1 at module level:
        D100: Docstring missing
src/collective/polls/tests/__init__.py:1 at module level:
        D100: Docstring missing
src/collective/polls/content/__init__.py:1 at module level:
        D100: Docstring missing
src/collective/polls/content/poll.py:1 at module level:
        D100: Docstring missing
src/collective/polls/content/poll.py:34 in public class `InsuficientOptions`:
        D101: Docstring missing
src/collective/polls/content/poll.py:38 in public class `IPoll`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/content/poll.py:38 in public class `IPoll`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:80 in public method `validate_options`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:90 in public class `Poll`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/content/poll.py:90 in public class `Poll`:
        D400: First line should end with '.', not 'e'
src/collective/polls/content/poll.py:90 in public class `Poll`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:101 in public method `annotations`:
        D102: Docstring missing
src/collective/polls/content/poll.py:105 in public method `utility`:
        D102: Docstring missing
src/collective/polls/content/poll.py:109 in public method `getOptions`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:115 in private method `_getVotes`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:133 in public method `getResults`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:144 in private method `_validateVote`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:155 in private method `_setVoter`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:180 in public method `voters`:
        D102: Docstring missing
src/collective/polls/content/poll.py:186 in public method `total_votes`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:192 in public method `setVote`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:217 in public class `PollAddForm`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/content/poll.py:217 in public class `PollAddForm`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:223 in public method `create`:
        D102: Docstring missing
src/collective/polls/content/poll.py:235 in public class `PollEditForm`:
        D203: Expected 1 blank line *before* class docstring, found 0
src/collective/polls/content/poll.py:235 in public class `PollEditForm`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:241 in public method `updateWidgets`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:258 in public method `applyChanges`:
        D102: Docstring missing
src/collective/polls/content/poll.py:270 in public class `View`:
        D101: Docstring missing
src/collective/polls/content/poll.py:277 in public method `update`:
        D102: Docstring missing
src/collective/polls/content/poll.py:346 in public method `can_vote`:
        D102: Docstring missing
src/collective/polls/content/poll.py:357 in public method `can_edit`:
        D102: Docstring missing
src/collective/polls/content/poll.py:362 in public method `has_voted`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:371 in public method `poll_uid`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:377 in public method `getOptions`:
        D200: One-line docstring should not occupy 2 lines
src/collective/polls/content/poll.py:382 in public method `getResults`:
        D200: One-line docstring should not occupy 2 lines
Posted by at No comments:
Labels: , , , ,

Wednesday, January 8, 2014

How to get statistics about your contributions on a GitHub organization

GitHub, the web-based hosting service for software development projects, lets you check statistics on your contributions on specific projects easily.

You can check, for instance, the statistics on collective.cover just by visiting the following page:

https://github.com/collective/collective.cover/graphs/contributors

GitHub gives you a count of commits and a visual representation of them among time.

If you want to check only the number of commits by author, you could use the git-shortlog command:

(python-2.7)# hvelarde@nanovac (master u=) ~/collective/cover
# git shortlog -s | sort -nr
697 hvelarde
312 Juan Pablo Giménez
91 Héctor Velarde
78 Franco Pellegrini
76 Marcos F. Romero
71 quimera
71 Cleber J Santos
66 Gonzalo Almeida
62 Érico Andrei
61 Silvestre Huens
47 Maurits van Rees
32 Andre Nogueira
30 Kuno Woudt
20 Carlos de la Guardia
13 Tamosauskas
10 Thiago Curvelo
10 Davi Lima
9 Denis Krienbühl
8 tzicatl
8 Rodrigo Ferreira de Souza
8 JeanMichel FRANCOIS
7 Leonardo Rochael Almeida
5 Giorgio Borelli
5 André Nogueira
4 Juan A. Diaz
4 Fulvio Casali
4 Asko Soukka
3 lpmayos
3 Fabiano Weimar dos Santos
3 Espen Moe-Nilssen
3 Davi Lima de Medeiros
2 Takeshi Yamamoto
2 sven
2 Ricardo Bánffy
2 paul
2 Denis Bitouzé
2 Alex Clark
1 root
1 Rodrigo Ristow
1 Paul Roeland
1 Manabu TERADA
1 Malthe Borch
1 Leonardo J. Caballero G
1 Felipe Duardo
1 Emmanuelle Helly
1 adam139
1 Adam
view raw git-shortlog example hosted with ❤ by GitHub
The -s option gives you a summary output and the sort command shows the information sorted in reverse order (-nr).

You could improve this listing by using the .mailmap feature to add commits belonging to the same author using two different email addresses.

Image now that you want to get a list of all your contributions on a specific organization, lets say, the Plone collective.

Enter the GitHub API and github3.py, a Python wrapper for it.

Until today, the Plone collective has more that 1.1k repositories so we need to use the authenticated access to the API to avoid depletion of requests (rate limit allow us to make up to 60 requests per hour for unauthenticated requests and 5,000 for authenticated requests). In this specific example we used 1,171 requests to the API.

>>> user, password = 'hvelarde', 'password'
>>> from github3 import login
>>> g = login(user, password=password)
>>> o = g.organization('collective')
>>> g.ratelimit_remaining
3369
>>> my_stats = []
>>> for r in o.iter_repos():
... for c in r.iter_contributor_statistics():
... if user in repr(c.author):
... my_stats.append((r.name, c.total))
...
>>> g.ratelimit_remaining
2198
>>> len(my_stats)
74
>>> sum([total for repo, total in my_stats])
2975
>>> import operator
>>> my_stats = sorted(my_stats, key=operator.itemgetter(1), reverse=True)
>>> from pprint import pprint
>>> pprint(my_stats)
[(u'collective.cover', 607),
(u'collective.nitf', 456),
(u'collective.polls', 221),
(u'collective.z3cform.widgets', 175),
(u'collective.upload', 139),
(u'collective.newsflash', 97),
(u'collective.elections', 95),
(u'collective.syndication', 84),
(u'collective.weather', 78),
(u'collective.prettydate', 65),
(u'collective.disqus', 62),
(u'collective.newsticker', 59),
(u'collective.behavior.localdiazo', 58),
(u'sc.social.like', 53),
(u'sc.contentrules.groupbydate', 46),
(u'collective.facebook.portlets', 43),
(u'buildout.plonetest', 43),
(u'collective.googlenews', 40),
(u'collective.github.com', 39),
(u'collective.aviary', 33),
(u'collective.facebook.accounts', 32),
(u'collective.twitter.portlets', 31),
(u'collective.portlet.twitter', 26),
(u'Products.ImageEditor', 22),
(u'collective.portlet.calendar', 22),
(u'Products.PloneKeywordManager', 21),
(u'collective.behavior.localregistry', 19),
(u'collective.twitter.accounts', 18),
(u'collective.googleanalytics', 18),
(u'plone.app.imagecropping', 18),
(u'collective.developermanual', 14),
(u'Products.windowZ', 13),
(u'Products.ATGoogleVideo', 13),
(u'ploneawards.contenttypes', 12),
(u'Solgema.fullcalendar', 11),
(u'collective.z3cform.datetimewidget', 10),
(u'Products.EasyNewsletter', 10),
(u'collective.documentviewer', 10),
(u'collective.addthis', 9),
(u'collective.z3cform.datagridfield', 9),
(u'collective.portlet.usertrack', 9),
(u'ploneawards.buildout', 9),
(u'qi.portlet.TagClouds', 8),
(u'collective.lineage', 7),
(u'collective.portlet.pythonscript', 7),
(u'collective.themecustomizer', 7),
(u'collective.routes', 6),
(u'Products.Faq', 6),
(u'dexterity.membrane', 6),
(u'collective.recipe.cmd', 6),
(u'cioppino.twothumbs', 5),
(u'collective.facebook.wall', 5),
(u'templer.core', 5),
(u'collective.recipe.omelette', 5),
(u'example.conference', 4),
(u'collective.formwidget.relationfield', 4),
(u'templer.buildout', 4),
(u'i18ndude', 4),
(u'ploneawards.theme', 4),
(u'ploneawards.policy', 4),
(u'collective.oembed', 3),
(u'collective.twitter.search', 3),
(u'templer.plone', 3),
(u'collective.blog.star', 3),
(u'collective.easytemplate', 3),
(u'collective.timedevents', 3),
(u'collective.stats', 2),
(u'collective.recipe.plonesite', 2),
(u'collective.portlet.relateditems', 2),
(u'templer.plonebuildout', 1),
(u'templer.zope', 1),
(u'templer.silva', 1),
(u'collective.panels', 1),
(u'collective.portlet.tal', 1)]
>>>
view raw github3.py example hosted with ❤ by GitHub

Using github3.py we iterate over all of the organization repositories and get information of each one only if my user name is listed among the contributors. We get the results as a list of tuples (repo, total).

After that we sort the list in reverse order using the total commits as the key and make the sum of all commits in general.

As you can see I have contributed to 74 repositories on the Plone collective making 2,975 commits in total.

Not bad, isn't it? :-)

Posted by at No comments:
Labels: , , , , , , , ,
Subscribe to: Posts (Atom)