Bandit

AST parsing hits recursion limit

Bug #1650386 reported by David Wyde
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bandit
New
Low
Unassigned

Bug Description

The parsing of certain ASTs will hit Python's recursion limit. This happens if AST nodes are nested to a depth greater than "sys.getrecursionlimit()", which defaults to 1000 for me.

A simple example:

    python -c 'print ("+" * 1000) + "1"' > unary.py
    bandit unary.py

When this occurs, Bandit will report an exception:

    Exception RuntimeError: RuntimeError('maximum recursion depth exceeded',) in <generator object iter_fields at 0x108ab2eb0> ignored
    [manager] ERROR Exception occurred when executing tests against /Users/dwyde/test/unary.py. Run "bandit --debug /Users/dwyde/test/unary.py" to see the full traceback.

The start of the traceback with --debug:

    [manager] DEBUG Exception string: maximum recursion depth exceeded while calling a Python object
    [manager] DEBUG Exception traceback: Traceback (most recent call last):
      File "/Users/dwyde/code/bandit/bandit/core/manager.py", line 259, in _parse_file
        score = self._execute_ast_visitor(fname, data, nosec_lines)
      File "/Users/dwyde/code/bandit/bandit/core/manager.py", line 290, in _execute_ast_visitor
        score = res.process(data)
      File "/Users/dwyde/code/bandit/bandit/core/node_visitor.py", line 278, in process
        self.generic_visit(f_ast)
      File "/Users/dwyde/code/bandit/bandit/core/node_visitor.py", line 242, in generic_visit
        if self.pre_visit(item):
      File "/Users/dwyde/code/bandit/bandit/core/node_visitor.py", line 187, in pre_visit
        LOG.debug(ast.dump(node))
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 110, in dump
        return _format(node)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 94, in _format
        fields = [(a, _format(b)) for a, b in iter_fields(node)]
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ast.py", line 94, in _format
        fields = [(a, _format(b)) for a, b in iter_fields(node)]

Found when running Bandit against
https://bitbucket.org/logilab/astroid/src/1.4.0/astroid/tests/testdata/python2/data/joined_strings.py

Revision history for this message
Travis McPeak (travis-mcpeak) wrote :

This is a cool bug but I'm not sure what to do about it.

Seems like Bandit is probably doing the best it can in this case?

Changed in bandit:
importance: Undecided → Low
Revision history for this message
David Wyde (david-wyde) wrote :

I'm not sure what to do about this one. My first thought was to catch an exception, but pre-3.5 it's going to be a RuntimeError. Identifying the recursion error by exception string might be a bit goofy.

Note that the same idea also causes an error in an SQL-checking plugin (with strings and fewer repetitions):

$ python -c 'print "+".join(["\"a\""] * 600)' > repeat.py
$ bandit repeat.py
[main] INFO profile include tests: None
[main] INFO profile exclude tests: None
[main] INFO cli include tests: None
[main] INFO cli exclude tests: None
[main] INFO running on Python 2.7.10
[node_visitor] INFO Unable to find qualified name for module: repeat.py
[tester] ERROR Bandit internal error running: hardcoded_sql_expressions on file repeat.py at line 1: maximum recursion depth exceeded in cmpTraceback (most recent call last):
  File "/Users/dwyde/bandit/bandit/core/tester.py", line 64, in run_tests
    result = test(context)
  File "/Users/dwyde/bandit/bandit/plugins/injection_sql.py", line 92, in hardcoded_sql_expressions
    val = _evaluate_ast(context.node)
  File "/Users/dwyde/bandit/bandit/plugins/injection_sql.py", line 81, in _evaluate_ast
    out = utils.concat_string(node, node.parent)
  File "/Users/dwyde/bandit/bandit/core/utils.py", line 270, in concat_string
    _get(node, bits, stop)
  File "/Users/dwyde/bandit/bandit/core/utils.py", line 259, in _get
    if isinstance(node.left, ast.BinOp)
  File "/Users/dwyde/bandit/bandit/core/utils.py", line 259, in _get
    if isinstance(node.left, ast.BinOp)
  File "/Users/dwyde/bandit/bandit/core/utils.py", line 259, in _get
    if isinstance(node.left, ast.BinOp)
  File "/Users/dwyde/bandit/bandit/core/utils.py", line 259, in _get
    if isinstance(node.left, ast.BinOp)
...
  File "/Users/dwyde/bandit/bandit/core/utils.py", line 256, in _get
    if node != stop:
RuntimeError: maximum recursion depth exceeded in cmp

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.