[TOKEEP ?] autojsdoc extension
This commit is contained in:
Victor Feyens
parent
e218ae4991
commit
fcec9a9d7f
87
_extensions/autojsdoc/README.rst
Normal file
87
_extensions/autojsdoc/README.rst
Normal file
@ -0,0 +1,87 @@
|
||||
:orphan:
|
||||
|
||||
======================================
|
||||
JSDoc parser/Sphinx extension for Odoo
|
||||
======================================
|
||||
|
||||
Why?
|
||||
====
|
||||
|
||||
Spent about a week trying to coerce "standard" javascript tools (jsdoc_ with
|
||||
the hope of using sphinx-js_ for integration or `documentation.js`_) and
|
||||
failed to ever get a sensible result: failed to get any result with the
|
||||
current state of the documentation, significant changes/additions/fixes to
|
||||
docstrings brought this up to "garbage output" level.
|
||||
|
||||
Bug reports and mailing list posts didn't show any path to improvement on the
|
||||
ES5 codebase (if we ever go whole-hog on ES6 modules and classes things could
|
||||
be different, in fact most of JSDoc's current effort seem focused on
|
||||
ES6/ES2015 features) but both experience and looking at the mailing lists
|
||||
told me that spending more time would be wasted.
|
||||
|
||||
Even more so as I was writing visitors/rewriters to generate documentation
|
||||
from our existing structure, which broadly speaking is relatively strict, and
|
||||
thus
|
||||
|
||||
What?
|
||||
=====
|
||||
|
||||
If it were possible to generate JSDoc annotations from our relatively
|
||||
well-defined code structures, it was obviously possible to extract documentary
|
||||
information directly from it, hence this Odoo-specific package/extension
|
||||
trying to do exactly that.
|
||||
|
||||
This package should eventually provide:
|
||||
|
||||
* a command-line interface which can be invoked via ``-m autojsdoc`` (assuming
|
||||
your ``PYTHONPATH`` can find it) which should allow dumping the parsed AST
|
||||
in a convenient-ish form, possibly doing searches through the AST, a
|
||||
dependency graph extractor/analysis and a text dumper for the
|
||||
documentation.
|
||||
|
||||
* a sphinx extension (``autojsdoc.sphinx``) which can be used to integrate the
|
||||
parsed JSDoc information into the Sphinx doc.
|
||||
|
||||
How?
|
||||
====
|
||||
|
||||
Sphinx-aside, the package relies on 3 libraries:
|
||||
|
||||
* pyjsparser_, an Esprima-compliant ES5.1 parser (with bits of ES6 support),
|
||||
sadly it does not support comments in its current form so I had to fork it.
|
||||
Fed a javascript source file, pyjsparser_ simply generates a bunch of nested
|
||||
dicts representing an Esprima ast, ast-types_ does a reasonably good job of
|
||||
describing it once you understand that "bases" are basically just structural
|
||||
mixins.
|
||||
|
||||
Because the original does not, this package provides a ``visitor`` module
|
||||
for pyjsparser_ ASTs.
|
||||
|
||||
* pyjsdoc_, a one-file "port" of jsdoc, can actually do much of the JS parsing
|
||||
(using string munging) but its core semantics don't fit our needs so I'm
|
||||
only using it to parse the actual JSDoc content, and the ``jsdoc`` module
|
||||
contains some replacement classes, extensions & monkey patches for things
|
||||
`pyjsdoc`_ itself does not support, at the time of this writing:
|
||||
|
||||
- a bug in FunctionDoc.return_val
|
||||
- a type on FunctionDoc so it's compatible with ParamDoc
|
||||
- a more reliable comments-parsing function
|
||||
- a replacement ModuleDoc as the original does not materialise AMD modules
|
||||
- a ClassDoc extension to support mixins
|
||||
- two additional CommentDoc extensions for "namespaces" objects (bag of
|
||||
attributes without any more information) and mixin objects
|
||||
|
||||
* pytest_ to configure and run the test suite, which you can run by invoking
|
||||
``pytest doc/_extensions`` from the project top-level, the tests represent
|
||||
both "happy path" things we want to parse and various code patterns which
|
||||
tripped the happy path because e.g. they were matched and should not have,
|
||||
they were not matched and should have, or they were more complex than the
|
||||
happy path had expected
|
||||
|
||||
.. _ast-types: _https://github.com/benjamn/ast-types/blob/master/def/core.js
|
||||
.. _documentation.js: http://documentation.js.org
|
||||
.. _jsdoc: http://usejsdoc.org
|
||||
.. _pyjsdoc: https://github.com/nostrademons/pyjsdoc
|
||||
.. _pyjsparser: https://github.com/PiotrDabkowski/pyjsparser
|
||||
.. _pytest: https://pytest.org/
|
||||
.. _sphinx-js: https://sphinx-js-howto.readthedocs.io
|
||||
1
_extensions/autojsdoc/__init__.py
Normal file
1
_extensions/autojsdoc/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
154
_extensions/autojsdoc/__main__.py
Normal file
154
_extensions/autojsdoc/__main__.py
Normal file
@ -0,0 +1,154 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from __future__ import print_function
|
||||
import cgitb
|
||||
import fnmatch
|
||||
import io
|
||||
import logging
|
||||
|
||||
import click
|
||||
|
||||
import pyjsparser
|
||||
import sys
|
||||
|
||||
from .parser.parser import ModuleMatcher
|
||||
from .parser.visitor import Visitor, SKIP
|
||||
from .parser import jsdoc
|
||||
|
||||
class Printer(Visitor):
|
||||
def __init__(self, level=0):
|
||||
super(Printer, self).__init__()
|
||||
self.level = level
|
||||
|
||||
def _print(self, text):
|
||||
print(' ' * self.level, text)
|
||||
|
||||
def enter_generic(self, node):
|
||||
self._print(node['type'])
|
||||
self.level += 1
|
||||
|
||||
def exit_generic(self, node):
|
||||
self.level -= 1
|
||||
|
||||
def enter_Identifier(self, node):
|
||||
self._print(node['name'])
|
||||
return SKIP
|
||||
|
||||
def enter_Literal(self, node):
|
||||
self._print(node['value'])
|
||||
return SKIP
|
||||
|
||||
def enter_BinaryExpression(self, node):
|
||||
self._print(node['operator'])
|
||||
self.level += 1
|
||||
|
||||
def visit_files(files, visitor, ctx):
|
||||
for name in files:
|
||||
with io.open(name) as f:
|
||||
ctx.logger.info("%s", name)
|
||||
try:
|
||||
yield visitor().visit(pyjsparser.parse(f.read()))
|
||||
except Exception as e:
|
||||
if ctx.logger.isEnabledFor(logging.DEBUG):
|
||||
ctx.logger.exception("while visiting %s", name)
|
||||
else:
|
||||
ctx.logger.error("%s while visiting %s", e, name)
|
||||
|
||||
# bunch of modules various bits depend on which are not statically defined
|
||||
# (or are outside the scope of the system)
|
||||
ABSTRACT_MODULES = [
|
||||
jsdoc.ModuleDoc({
|
||||
'module': 'web.web_client',
|
||||
'dependency': {'web.AbstractWebClient'},
|
||||
'exports': jsdoc.NSDoc({
|
||||
'name': 'web_client',
|
||||
'doc': 'instance of AbstractWebClient',
|
||||
}),
|
||||
}),
|
||||
jsdoc.ModuleDoc({
|
||||
'module': 'web.Tour',
|
||||
'dependency': {'web_tour.TourManager'},
|
||||
'exports': jsdoc.NSDoc({
|
||||
'name': 'Tour',
|
||||
'doc': 'maybe tourmanager instance?',
|
||||
}),
|
||||
}),
|
||||
# OH FOR FUCK'S SAKE
|
||||
jsdoc.ModuleDoc({
|
||||
'module': 'summernote/summernote',
|
||||
'exports': jsdoc.NSDoc({'doc': "totally real summernote"}),
|
||||
})
|
||||
]
|
||||
|
||||
@click.group(context_settings={'help_option_names': ['-h', '--help']})
|
||||
@click.option('-v', '--verbose', count=True)
|
||||
@click.option('-q', '--quiet', count=True)
|
||||
@click.pass_context
|
||||
def autojsdoc(ctx, verbose, quiet):
|
||||
logging.basicConfig(
|
||||
level=logging.INFO + (quiet - verbose) * 10,
|
||||
format="[%(levelname)s %(created)f] %(message)s",
|
||||
)
|
||||
ctx.logger = logging.getLogger('autojsdoc')
|
||||
ctx.visitor = None
|
||||
ctx.files = []
|
||||
ctx.kw = {}
|
||||
|
||||
@autojsdoc.command()
|
||||
@click.argument('files', type=click.Path(exists=True), nargs=-1)
|
||||
@click.pass_context
|
||||
def ast(ctx, files):
|
||||
""" Prints a structure tree of the provided files
|
||||
"""
|
||||
if not files:
|
||||
print(ctx.get_help())
|
||||
visit_files(files, lambda: Printer(level=1), ctx.parent)
|
||||
|
||||
@autojsdoc.command()
|
||||
@click.option('-m', '--module', multiple=True, help="Only shows dependencies matching any of the patterns")
|
||||
@click.argument('files', type=click.Path(exists=True), nargs=-1)
|
||||
@click.pass_context
|
||||
def dependencies(ctx, module, files):
|
||||
""" Prints a dot file of all modules to stdout
|
||||
"""
|
||||
if not files:
|
||||
print(ctx.get_help())
|
||||
byname = {
|
||||
mod.name: mod.dependencies
|
||||
for mod in ABSTRACT_MODULES
|
||||
}
|
||||
for modules in visit_files(files, ModuleMatcher, ctx.parent):
|
||||
for mod in modules:
|
||||
byname[mod.name] = mod.dependencies
|
||||
|
||||
print('digraph dependencies {')
|
||||
|
||||
todo = set()
|
||||
# if module filters, roots are only matching modules
|
||||
if module:
|
||||
for f in module:
|
||||
todo.update(fnmatch.filter(byname.keys(), f))
|
||||
|
||||
for m in todo:
|
||||
# set a different box for selected roots
|
||||
print(' "%s" [color=orangered]' % m)
|
||||
else:
|
||||
# otherwise check all modules
|
||||
todo.update(byname)
|
||||
|
||||
done = set()
|
||||
while todo:
|
||||
node = todo.pop()
|
||||
if node in done:
|
||||
continue
|
||||
|
||||
done.add(node)
|
||||
deps = byname[node]
|
||||
todo.update(deps - done)
|
||||
for dep in deps:
|
||||
print(' "%s" -> "%s";' % (node, dep))
|
||||
print('}')
|
||||
|
||||
try:
|
||||
autojsdoc.main(prog_name='autojsdoc')
|
||||
except Exception:
|
||||
print(cgitb.text(sys.exc_info()))
|
||||
13
_extensions/autojsdoc/ext/__init__.py
Normal file
13
_extensions/autojsdoc/ext/__init__.py
Normal file
@ -0,0 +1,13 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from .directives import automodule_bound, autodirective_bound
|
||||
from .extractor import _get_roots
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_config_value('js_roots', _get_roots, 'env')
|
||||
modules = {}
|
||||
app.add_directive_to_domain('js', 'automodule', automodule_bound(app, modules)
|
||||
)
|
||||
autodirective = autodirective_bound(app, modules)
|
||||
for n in ['autonamespace', 'automixin', 'autoclass', 'autofunction']:
|
||||
app.add_directive_to_domain('js', n, autodirective)
|
||||
719
_extensions/autojsdoc/ext/directives.py
Normal file
719
_extensions/autojsdoc/ext/directives.py
Normal file
@ -0,0 +1,719 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import abc
|
||||
import collections
|
||||
import contextlib
|
||||
import fnmatch
|
||||
import io
|
||||
import re
|
||||
|
||||
from docutils import nodes
|
||||
from docutils.parsers.rst import Directive
|
||||
from docutils.statemachine import StringList
|
||||
from sphinx import addnodes
|
||||
from sphinx.ext.autodoc import members_set_option, bool_option, ALL
|
||||
|
||||
from autojsdoc.ext.extractor import read_js
|
||||
from ..parser import jsdoc, types
|
||||
|
||||
class DocumenterError(Exception): pass
|
||||
|
||||
@contextlib.contextmanager
|
||||
def addto(parent, newnode):
|
||||
assert isinstance(newnode, nodes.Node), \
|
||||
"Expected newnode to be a Node, got %s" % (type(newnode))
|
||||
yield newnode
|
||||
parent.append(newnode)
|
||||
|
||||
|
||||
def documenter_for(directive, doc):
|
||||
if isinstance(doc, jsdoc.FunctionDoc):
|
||||
return FunctionDocumenter(directive, doc)
|
||||
if isinstance(doc, jsdoc.ClassDoc):
|
||||
return ClassDocumenter(directive, doc)
|
||||
if isinstance(doc, jsdoc.MixinDoc):
|
||||
return MixinDocumenter(directive, doc)
|
||||
if isinstance(doc, (jsdoc.PropertyDoc, jsdoc.LiteralDoc)):
|
||||
return PropertyDocumenter(directive, doc)
|
||||
if isinstance(doc, jsdoc.InstanceDoc):
|
||||
return InstanceDocumenter(directive, doc)
|
||||
if isinstance(doc, jsdoc.Unknown):
|
||||
return UnknownDocumenter(directive, doc)
|
||||
if isinstance(doc, jsdoc.NSDoc):
|
||||
return NSDocumenter(directive, doc)
|
||||
|
||||
raise TypeError("No documenter for %s" % type(doc))
|
||||
|
||||
def to_list(doc, source=None):
|
||||
return StringList([
|
||||
line.rstrip('\n')
|
||||
for line in io.StringIO(doc)
|
||||
], source=source)
|
||||
|
||||
DIRECTIVE_OPTIONS = {
|
||||
'members': members_set_option,
|
||||
'undoc-members': bool_option,
|
||||
'private-members': bool_option,
|
||||
'undoc-matches': bool_option,
|
||||
}
|
||||
def automodule_bound(app, modules):
|
||||
class AutoModuleDirective(Directive):
|
||||
required_arguments = 1
|
||||
has_content = True
|
||||
option_spec = DIRECTIVE_OPTIONS
|
||||
|
||||
# self.state.nested_parse(string, offset, node) => parse context for sub-content (body which can contain RST data)
|
||||
# => needed for doc (converted?) and for actual directive body
|
||||
def run(self):
|
||||
self.env = self.state.document.settings.env
|
||||
modname = self.arguments[0].strip()
|
||||
|
||||
# TODO: cache/memoize modules & symbols?
|
||||
if not modules:
|
||||
read_js(app, modules)
|
||||
|
||||
mods = [
|
||||
(name, mod)
|
||||
for name, mod in modules.items()
|
||||
if fnmatch.fnmatch(name, modname)
|
||||
]
|
||||
ret = []
|
||||
for name, mod in mods:
|
||||
if mod.is_private:
|
||||
continue
|
||||
if name != modname and not (mod.doc or mod.exports):
|
||||
# this module has no documentation, no exports and was
|
||||
# not specifically requested through automodule -> skip
|
||||
# unless requested
|
||||
if not self.options.get('undoc-matches'):
|
||||
continue
|
||||
modsource = mod['sourcefile']
|
||||
if modsource:
|
||||
self.env.note_dependency(modsource)
|
||||
# not sure what that's used for as normal xrefs are resolved using the id directly
|
||||
target = nodes.target('', '', ids=['module-' + name], ismod=True)
|
||||
self.state.document.note_explicit_target(target)
|
||||
|
||||
documenter = ModuleDocumenter(self, mod)
|
||||
|
||||
ret.append(target)
|
||||
ret.extend(documenter.generate())
|
||||
return ret
|
||||
|
||||
return AutoModuleDirective
|
||||
|
||||
def autodirective_bound(app, modules):
|
||||
documenters = {
|
||||
'js:autoclass': ClassDocumenter,
|
||||
'js:autonamespace': NSDocumenter,
|
||||
'js:autofunction': FunctionDocumenter,
|
||||
'js:automixin': MixinDocumenter,
|
||||
}
|
||||
class AutoDirective(Directive):
|
||||
required_arguments = 1
|
||||
has_content = True
|
||||
option_spec = DIRECTIVE_OPTIONS
|
||||
|
||||
def run(self):
|
||||
self.env = self.state.document.settings.env
|
||||
|
||||
objname = self.arguments[0].strip()
|
||||
if not modules:
|
||||
read_js(app, modules)
|
||||
|
||||
# build complete path to object
|
||||
path = self.env.temp_data.get('autojs:prefix', []) + objname.split('.')
|
||||
# look for module/object split
|
||||
for i in range(1, len(path)):
|
||||
modname, objpath = '.'.join(path[:-i]), path[-i:]
|
||||
module = modules.get(modname)
|
||||
if module:
|
||||
break
|
||||
else:
|
||||
raise Exception("Found no valid module in " + '.'.join(path))
|
||||
|
||||
item = module
|
||||
# deref' namespaces until we reach the object we're looking for
|
||||
for k in objpath:
|
||||
item = item.get_property(k)
|
||||
|
||||
docclass = documenters[self.name]
|
||||
return docclass(self, item).generate()
|
||||
|
||||
return AutoDirective
|
||||
|
||||
class Documenter(object):
|
||||
objtype = None
|
||||
def __init__(self, directive, doc):
|
||||
self.directive = directive
|
||||
self.env = directive.env
|
||||
self.item = doc
|
||||
|
||||
@property
|
||||
def modname(self):
|
||||
return self.env.temp_data.get('autojs:module', '')
|
||||
@property
|
||||
def classname(self):
|
||||
return self.env.temp_data.get('autojs:class', '')
|
||||
def generate(self, all_members=False):
|
||||
"""
|
||||
:rtype: List[nodes.Node]
|
||||
"""
|
||||
try:
|
||||
return self._generate(all_members=all_members)
|
||||
except Exception as e:
|
||||
raise DocumenterError("Failed to document %s" % self.item) from e
|
||||
def _generate(self, all_members=False):
|
||||
objname = self.item.name
|
||||
prefixed = (self.item['sourcemodule'].name + '.' + objname) if self.item['sourcemodule'] else None
|
||||
objtype = self.objtype
|
||||
assert objtype, '%s has no objtype' % type(self)
|
||||
root = addnodes.desc(domain='js', desctype=objtype, objtype=objtype)
|
||||
with addto(root, addnodes.desc_signature(
|
||||
module=self.modname or '',
|
||||
fullname=objname,
|
||||
)) as s:
|
||||
s['class'] = self.classname
|
||||
|
||||
s['ids'] = []
|
||||
if objname:
|
||||
s['ids'].append(objname)
|
||||
if prefixed:
|
||||
s['ids'].append(prefixed)
|
||||
|
||||
if objtype:
|
||||
s += addnodes.desc_annotation(
|
||||
objtype, objtype,
|
||||
nodes.Text(' '),
|
||||
)
|
||||
|
||||
env = self.env
|
||||
if objname:
|
||||
env.domaindata['js']['objects'][objname] = (env.docname, objtype)
|
||||
if prefixed:
|
||||
env.domaindata['js']['objects'][prefixed] = (env.docname, objtype)
|
||||
|
||||
# TODO: linkcode_resolve
|
||||
s += self.make_signature()
|
||||
with addto(root, addnodes.desc_content()) as c:
|
||||
# must be here otherwise nested_parse(self.content) will not have
|
||||
# the prefix set
|
||||
self.env.temp_data.setdefault('autojs:prefix', []).append(self.item.name)
|
||||
c += self.make_content(all_members=all_members)
|
||||
self.env.temp_data['autojs:prefix'].pop()
|
||||
return [root]
|
||||
|
||||
def make_signature(self):
|
||||
"""
|
||||
:rtype: List[nodes.Node]
|
||||
"""
|
||||
return [addnodes.desc_name(self.item.name, self.item.name)]
|
||||
|
||||
@abc.abstractmethod
|
||||
def make_content(self, all_members):
|
||||
"""
|
||||
:rtype: List[nodes.Node]
|
||||
"""
|
||||
|
||||
def document_subtypes(self, subtypes):
|
||||
docs = []
|
||||
with with_mapping_value(self.directive.options, 'undoc-members', True):
|
||||
for cls in subtypes:
|
||||
docs += ClassDocumenter(self.directive, cls).generate(all_members=True)
|
||||
return docs
|
||||
|
||||
|
||||
class NSDocumenter(Documenter):
|
||||
objtype = 'namespace'
|
||||
def make_content(self, all_members):
|
||||
doc = self.item
|
||||
ret = nodes.section()
|
||||
|
||||
if doc.doc:
|
||||
self.directive.state.nested_parse(to_list(doc.doc), 0, ret)
|
||||
|
||||
self.directive.state.nested_parse(self.directive.content, 0, ret)
|
||||
|
||||
ret += self.document_properties(all_members)
|
||||
return ret.children
|
||||
|
||||
def should_document(self, member, name, all_members):
|
||||
"""
|
||||
:type member: jsdoc.CommentDoc
|
||||
:type name: str
|
||||
:type all_members: bool
|
||||
:rtype: bool
|
||||
"""
|
||||
options = self.directive.options
|
||||
|
||||
members = options.get('members') or []
|
||||
if not (all_members or members is ALL):
|
||||
# if a member is requested by name, it's always documented
|
||||
return name in members
|
||||
|
||||
# ctor params are merged into the class doc
|
||||
if member.is_constructor:
|
||||
return False
|
||||
|
||||
# only document "private" members if option is set
|
||||
if self.is_private(member, name) and not options.get('private-members'):
|
||||
return False
|
||||
|
||||
# TODO: what if member doesn't have a description but has non-desc tags set?
|
||||
# TODO: add @public to force documenting symbol? => useful for implicit typedef
|
||||
return bool(member.doc or options.get('undoc-members'))
|
||||
|
||||
def is_private(self, member, name):
|
||||
return member.is_private
|
||||
|
||||
def document_properties(self, all_members):
|
||||
ret = nodes.section()
|
||||
# TODO: :member-order: [alphabetical | groupwise | bysource]
|
||||
for (n, p) in self.item.properties:
|
||||
if not self.should_document(p, n, all_members):
|
||||
continue
|
||||
# FIXME: maybe should use property name as name inside?
|
||||
ret += documenter_for(self.directive, p).generate(all_members=True)
|
||||
return ret.children
|
||||
|
||||
_NONE = object()
|
||||
@contextlib.contextmanager
|
||||
def with_mapping_value(mapping, key, value, restore_to=_NONE):
|
||||
""" Sets ``key`` to ``value`` for the duration of the context.
|
||||
|
||||
If ``restore_to`` is not provided, restores ``key``'s old value
|
||||
afterwards, removes it entirely if there was no value for ``key`` in the
|
||||
mapping.
|
||||
|
||||
.. warning:: for defaultdict & similar mappings, may restore the default
|
||||
value (depends how the collections' .get behaves)
|
||||
"""
|
||||
if restore_to is _NONE:
|
||||
restore_to = mapping.get(key, _NONE)
|
||||
mapping[key] = value
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
if restore_to is _NONE:
|
||||
del mapping[key]
|
||||
else:
|
||||
mapping[key] = restore_to
|
||||
class ModuleDocumenter(NSDocumenter):
|
||||
objtype = 'module'
|
||||
def document_properties(self, all_members):
|
||||
with with_mapping_value(self.env.temp_data, 'autojs:module', self.item.name, ''):
|
||||
return super(ModuleDocumenter, self).document_properties(all_members)
|
||||
|
||||
def make_content(self, all_members):
|
||||
doc = self.item
|
||||
content = addnodes.desc_content()
|
||||
|
||||
if doc.exports or doc.dependencies:
|
||||
with addto(content, nodes.field_list()) as fields:
|
||||
if doc.exports:
|
||||
with addto(fields, nodes.field()) as field:
|
||||
field += nodes.field_name('Exports', 'Exports')
|
||||
with addto(field, nodes.field_body()) as body:
|
||||
ref = doc['exports'] # warning: not the same as doc.exports
|
||||
label = ref or '<anonymous>'
|
||||
link = addnodes.pending_xref(
|
||||
ref, nodes.paragraph(ref, label),
|
||||
refdomain='js',
|
||||
reftype='any',
|
||||
reftarget=ref,
|
||||
)
|
||||
link['js:module'] = doc.name
|
||||
body += link
|
||||
|
||||
if doc.dependencies:
|
||||
with addto(fields, nodes.field()) as field:
|
||||
self.make_dependencies(field, doc)
|
||||
|
||||
if doc.doc:
|
||||
# FIXME: source offset
|
||||
self.directive.state.nested_parse(to_list(doc.doc, source=doc['sourcefile']), 0, content)
|
||||
|
||||
self.directive.state.nested_parse(self.directive.content, 0, content)
|
||||
|
||||
content += self.document_properties(all_members)
|
||||
|
||||
return content
|
||||
|
||||
def make_dependencies(self, field, doc):
|
||||
field += nodes.field_name("Depends On", "Depends On")
|
||||
with addto(field, nodes.field_body()) as body:
|
||||
with addto(body, nodes.bullet_list()) as deps:
|
||||
for dep in sorted(doc.dependencies):
|
||||
ref = addnodes.pending_xref(
|
||||
dep, nodes.paragraph(dep, dep),
|
||||
refdomain='js',
|
||||
reftype='module',
|
||||
reftarget=dep,
|
||||
)
|
||||
deps += nodes.list_item(dep, ref)
|
||||
|
||||
def should_document(self, member, name, all_members):
|
||||
# member can be Nothing?
|
||||
if not member:
|
||||
return False
|
||||
modname = getattr(member['sourcemodule'], 'name', None)
|
||||
doc = self.item
|
||||
|
||||
# always document exported symbol (regardless undoc, private, ...)
|
||||
# otherwise things become... somewhat odd
|
||||
if name == doc['exports']:
|
||||
return True
|
||||
|
||||
# if doc['exports'] the module is exporting a "named" item which
|
||||
# does not need to be documented twice, if not doc['exports'] it's
|
||||
# exporting an anonymous item (e.g. object literal) which needs to
|
||||
# be documented on its own
|
||||
if name == '<exports>' and not doc['exports']:
|
||||
return True
|
||||
|
||||
# TODO: :imported-members:
|
||||
# FIXME: *directly* re-exported "foreign symbols"?
|
||||
return (not modname or modname == doc.name) \
|
||||
and super(ModuleDocumenter, self).should_document(member, name, all_members)
|
||||
|
||||
|
||||
class ClassDocumenter(NSDocumenter):
|
||||
objtype = 'class'
|
||||
|
||||
def document_properties(self, all_members):
|
||||
with with_mapping_value(self.env.temp_data, 'autojs:class', self.item.name, ''):
|
||||
return super(ClassDocumenter, self).document_properties(all_members)
|
||||
|
||||
def make_signature(self):
|
||||
sig = super(ClassDocumenter, self).make_signature()
|
||||
sig.append(self.make_parameters())
|
||||
return sig
|
||||
|
||||
def make_parameters(self):
|
||||
params = addnodes.desc_parameterlist('', '')
|
||||
ctor = self.item.constructor
|
||||
if ctor:
|
||||
params += make_desc_parameters(ctor.params)
|
||||
|
||||
return params
|
||||
|
||||
def make_content(self, all_members):
|
||||
doc = self.item
|
||||
ret = nodes.section()
|
||||
|
||||
ctor = self.item.constructor
|
||||
params = subtypes = []
|
||||
if ctor:
|
||||
check_parameters(self, ctor)
|
||||
params, subtypes = extract_subtypes(doc.name, ctor)
|
||||
|
||||
fields = nodes.field_list()
|
||||
fields += self.make_super()
|
||||
fields += self.make_mixins()
|
||||
fields += self.make_params(params)
|
||||
if fields.children:
|
||||
ret += fields
|
||||
|
||||
if doc.doc:
|
||||
self.directive.state.nested_parse(to_list(doc.doc), 0, ret)
|
||||
|
||||
self.directive.state.nested_parse(self.directive.content, 0, ret)
|
||||
|
||||
ret += self.document_properties(all_members)
|
||||
|
||||
ret += self.document_subtypes(subtypes)
|
||||
|
||||
return ret.children
|
||||
|
||||
def is_private(self, member, name):
|
||||
return name.startswith('_') or super(ClassDocumenter, self).is_private(member, name)
|
||||
|
||||
def make_super(self):
|
||||
doc = self.item
|
||||
if not doc.superclass:
|
||||
return []
|
||||
|
||||
sup_link = addnodes.pending_xref(
|
||||
doc.superclass.name, nodes.paragraph(doc.superclass.name, doc.superclass.name),
|
||||
refdomain='js', reftype='class', reftarget=doc.superclass.name,
|
||||
)
|
||||
sup_link['js:module'] = doc.superclass['sourcemodule'].name
|
||||
return nodes.field(
|
||||
'',
|
||||
nodes.field_name("Extends", "Extends"),
|
||||
nodes.field_body(doc.superclass.name, sup_link),
|
||||
)
|
||||
|
||||
def make_mixins(self):
|
||||
doc = self.item
|
||||
if not doc.mixins:
|
||||
return []
|
||||
|
||||
ret = nodes.field('', nodes.field_name("Mixes", "Mixes"))
|
||||
with addto(ret, nodes.field_body()) as body:
|
||||
with addto(body, nodes.bullet_list()) as mixins:
|
||||
for mixin in sorted(doc.mixins, key=lambda m: m.name):
|
||||
mixin_link = addnodes.pending_xref(
|
||||
mixin.name, nodes.paragraph(mixin.name, mixin.name),
|
||||
refdomain='js', reftype='mixin', reftarget=mixin.name
|
||||
)
|
||||
mixin_link['js:module'] = mixin['sourcemodule'].name
|
||||
mixins += nodes.list_item('', mixin_link)
|
||||
return ret
|
||||
|
||||
def make_params(self, params):
|
||||
if not params:
|
||||
return []
|
||||
|
||||
ret = nodes.field('', nodes.field_name('Parameters', 'Parameters'))
|
||||
with addto(ret, nodes.field_body()) as body,\
|
||||
addto(body, nodes.bullet_list()) as holder:
|
||||
holder += make_parameters(params, mod=self.modname)
|
||||
return ret
|
||||
|
||||
class InstanceDocumenter(Documenter):
|
||||
objtype = 'object'
|
||||
def make_signature(self):
|
||||
cls = self.item.cls
|
||||
ret = super(InstanceDocumenter, self).make_signature()
|
||||
if cls:
|
||||
super_ref = addnodes.pending_xref(
|
||||
cls.name, nodes.Text(cls.name, cls.name),
|
||||
refdomain='js', reftype='class', reftarget=cls.name
|
||||
)
|
||||
super_ref['js:module'] = cls['sourcemodule'].name
|
||||
ret.append(addnodes.desc_annotation(' instance of ', ' instance of '))
|
||||
ret.append(addnodes.desc_type(cls.name, '', super_ref))
|
||||
if not ret:
|
||||
return [addnodes.desc_name('???', '???')]
|
||||
return ret
|
||||
|
||||
def make_content(self, all_members):
|
||||
ret = nodes.section()
|
||||
|
||||
if self.item.doc:
|
||||
self.directive.state.nested_parse(to_list(self.item.doc), 0, ret)
|
||||
|
||||
self.directive.state.nested_parse(self.directive.content, 0, ret)
|
||||
|
||||
return ret.children
|
||||
|
||||
class FunctionDocumenter(Documenter):
|
||||
@property
|
||||
def objtype(self):
|
||||
return 'method' if self.classname else 'function'
|
||||
def make_signature(self):
|
||||
ret = super(FunctionDocumenter, self).make_signature()
|
||||
with addto(ret, addnodes.desc_parameterlist()) as params:
|
||||
params += make_desc_parameters(self.item.params)
|
||||
retval = self.item.return_val
|
||||
if retval.type or retval.doc:
|
||||
ret.append(addnodes.desc_returns(retval.type or '*', retval.type or '*'))
|
||||
return ret
|
||||
|
||||
def make_content(self, all_members):
|
||||
ret = nodes.section()
|
||||
doc = self.item
|
||||
|
||||
if doc.doc:
|
||||
self.directive.state.nested_parse(to_list(doc.doc), 0, ret)
|
||||
|
||||
self.directive.state.nested_parse(self.directive.content, 0, ret)
|
||||
|
||||
check_parameters(self, doc)
|
||||
|
||||
params, subtypes = extract_subtypes(self.item.name, self.item)
|
||||
rdoc = doc.return_val.doc
|
||||
rtype = doc.return_val.type
|
||||
if params or rtype or rdoc:
|
||||
with addto(ret, nodes.field_list()) as fields:
|
||||
if params:
|
||||
with addto(fields, nodes.field()) as field:
|
||||
field += nodes.field_name('Parameters', 'Parameters')
|
||||
with addto(field, nodes.field_body()) as body,\
|
||||
addto(body, nodes.bullet_list()) as holder:
|
||||
holder.extend(make_parameters(params, mod=doc['sourcemodule'].name))
|
||||
if rdoc:
|
||||
with addto(fields, nodes.field()) as field:
|
||||
field += nodes.field_name("Returns", "Returns")
|
||||
with addto(field, nodes.field_body()) as body,\
|
||||
addto(body, nodes.paragraph()) as p:
|
||||
p += nodes.inline(rdoc, rdoc)
|
||||
if rtype:
|
||||
with addto(fields, nodes.field()) as field:
|
||||
field += nodes.field_name("Return Type", "Return Type")
|
||||
with addto(field, nodes.field_body()) as body, \
|
||||
addto(body, nodes.paragraph()) as p:
|
||||
p += make_types(rtype, mod=doc['sourcemodule'].name)
|
||||
|
||||
ret += self.document_subtypes(subtypes)
|
||||
|
||||
return ret.children
|
||||
|
||||
def pascal_case_ify(name):
|
||||
"""
|
||||
Uppercase first letter of ``name``, or any letter following an ``_``. In
|
||||
the latter case, also strips out the ``_``.
|
||||
|
||||
=> key_for becomes KeyFor
|
||||
=> options becomes Options
|
||||
"""
|
||||
return re.sub(r'(^|_)\w', lambda m: m.group(0)[-1].upper(), name)
|
||||
def extract_subtypes(parent_name, doc):
|
||||
""" Extracts composite parameters (a.b) into sub-types for the parent
|
||||
parameter, swaps the parent's type from whatever it is to the extracted
|
||||
one, and returns the extracted type for inclusion into the parent.
|
||||
|
||||
:arg parent_name: name of the containing symbol (function, class), will
|
||||
be used to compose subtype names
|
||||
:type parent_name: str
|
||||
:type doc: FunctionDoc
|
||||
:rtype: (List[ParamDoc], List[ClassDoc])
|
||||
"""
|
||||
# map of {param_name: [ParamDoc]} (from complete doc)
|
||||
subparams = collections.defaultdict(list)
|
||||
for p in map(jsdoc.ParamDoc, doc.get_as_list('param')):
|
||||
pair = p.name.split('.', 1)
|
||||
if len(pair) == 2:
|
||||
k, p.name = pair # remove prefix from param name
|
||||
subparams[k].append(p)
|
||||
|
||||
# keep original params order as that's the order of formal parameters in
|
||||
# the function signature
|
||||
params = collections.OrderedDict((p.name, p) for p in doc.params)
|
||||
subtypes = []
|
||||
# now we can use the subparams map to extract "compound" parameter types
|
||||
# and swap the new type for the original param's type
|
||||
for param_name, subs in subparams.items():
|
||||
typename = '%s%s' % (
|
||||
pascal_case_ify(parent_name),
|
||||
pascal_case_ify(param_name),
|
||||
)
|
||||
param = params[param_name]
|
||||
param.type = typename
|
||||
subtypes.append(jsdoc.ClassDoc({
|
||||
'name': typename,
|
||||
'doc': param.doc,
|
||||
'_members': [
|
||||
# TODO: add default value
|
||||
(sub.name, jsdoc.PropertyDoc(dict(sub.to_dict(), sourcemodule=doc['sourcemodule'])))
|
||||
for sub in subs
|
||||
],
|
||||
'sourcemodule': doc['sourcemodule'],
|
||||
}))
|
||||
return params.values(), subtypes
|
||||
|
||||
def check_parameters(documenter, doc):
|
||||
"""
|
||||
Check that all documented parameters match a formal parameter for the
|
||||
function. Documented params which don't match the actual function may be
|
||||
typos.
|
||||
"""
|
||||
guessed = set(doc['guessed_params'] or [])
|
||||
if not guessed:
|
||||
return
|
||||
|
||||
documented = {
|
||||
# param name can be of the form [foo.bar.baz=default]\ndescription
|
||||
jsdoc.ParamDoc(text).name.split('.')[0]
|
||||
for text in doc.get_as_list('param')
|
||||
}
|
||||
odd = documented - guessed
|
||||
if not odd:
|
||||
return
|
||||
|
||||
app = documenter.directive.env.app
|
||||
app.warn("Found documented params %s not in formal parameter list "
|
||||
"of function %s in module %s (%s)" % (
|
||||
', '.join(odd),
|
||||
doc.name,
|
||||
documenter.modname,
|
||||
doc['sourcemodule']['sourcefile'],
|
||||
))
|
||||
|
||||
def make_desc_parameters(params):
|
||||
for p in params:
|
||||
if '.' in p.name:
|
||||
continue
|
||||
|
||||
node = addnodes.desc_parameter(p.name, p.name)
|
||||
if p.optional:
|
||||
node = addnodes.desc_optional('', '', node)
|
||||
yield node
|
||||
|
||||
def make_parameters(params, mod=None):
|
||||
for param in params:
|
||||
p = nodes.paragraph('', '', nodes.strong(param.name, param.name))
|
||||
if param.default is not None:
|
||||
p += nodes.Text('=', '=')
|
||||
p += nodes.emphasis(param.default, param.default)
|
||||
if param.type:
|
||||
p += nodes.Text(' (')
|
||||
p += make_types(param.type, mod=mod)
|
||||
p += nodes.Text(')')
|
||||
if param.doc:
|
||||
p += [
|
||||
nodes.Text(' -- '),
|
||||
nodes.inline(param.doc, param.doc)
|
||||
]
|
||||
yield p
|
||||
|
||||
|
||||
def _format_value(v):
|
||||
if v == '|':
|
||||
return nodes.emphasis(' or ', ' or ')
|
||||
if v == ',':
|
||||
return nodes.Text(', ', ', ')
|
||||
return nodes.Text(v, v)
|
||||
def make_types(typespec, mod=None):
|
||||
# TODO: in closure notation {type=} => optional, do we care?
|
||||
def format_type(t):
|
||||
ref = addnodes.pending_xref(
|
||||
t, addnodes.literal_emphasis(t, t),
|
||||
refdomain='js', reftype='class', reftarget=t,
|
||||
)
|
||||
if mod:
|
||||
ref['js:module'] = mod
|
||||
return ref
|
||||
|
||||
try:
|
||||
return types.iterate(
|
||||
types.parse(typespec),
|
||||
format_type,
|
||||
_format_value
|
||||
)
|
||||
except ValueError as e:
|
||||
raise ValueError("%s in '%s'" % (e, typespec))
|
||||
|
||||
|
||||
class MixinDocumenter(NSDocumenter):
|
||||
objtype = 'mixin'
|
||||
|
||||
class PropertyDocumenter(Documenter):
|
||||
objtype = 'attribute'
|
||||
def make_signature(self):
|
||||
ret = super(PropertyDocumenter, self).make_signature()
|
||||
proptype = self.item.type
|
||||
if proptype:
|
||||
typeref = addnodes.pending_xref(
|
||||
proptype, nodes.Text(proptype, proptype),
|
||||
refdomain='js', reftype='class', reftarget=proptype
|
||||
)
|
||||
typeref['js:module'] = self.item['sourcemodule'].name
|
||||
ret.append(nodes.Text(' '))
|
||||
ret.append(typeref)
|
||||
return ret
|
||||
|
||||
def make_content(self, all_members):
|
||||
doc = self.item
|
||||
ret = nodes.section()
|
||||
|
||||
self.directive.state.nested_parse(self.directive.content, 0, ret)
|
||||
|
||||
if doc.doc:
|
||||
self.directive.state.nested_parse(to_list(doc.doc), 0, ret)
|
||||
return ret.children
|
||||
|
||||
class UnknownDocumenter(Documenter):
|
||||
objtype = 'unknown'
|
||||
def make_content(self, all_members):
|
||||
return []
|
||||
87
_extensions/autojsdoc/ext/extractor.py
Normal file
87
_extensions/autojsdoc/ext/extractor.py
Normal file
@ -0,0 +1,87 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import io
|
||||
import os
|
||||
|
||||
import pyjsdoc
|
||||
import pyjsparser
|
||||
|
||||
try:
|
||||
from sphinx.util import status_iterator
|
||||
def it(app): return status_iterator
|
||||
except ImportError:
|
||||
# 1.2: Builder.status_iterator
|
||||
# 1.3: Application.status_iterator (with alias on Builder)
|
||||
# 1.6: sphinx.util.status_iterator (with *deprecated* aliases on Application and Builder)
|
||||
# 1.7: removed Application and Builder aliases
|
||||
# => if no sphinx.util.status_iterator, fallback onto the builder one for
|
||||
# 1.2 compatibility, remove this entirely if we ever require 1.6+
|
||||
status_iterator = None
|
||||
def it(app): return app.builder.status_iterator
|
||||
|
||||
from ..parser import jsdoc, parser
|
||||
|
||||
|
||||
def _get_roots(conf):
|
||||
env_roots = os.environ.get('AUTOJSDOC_ROOTS_PATH')
|
||||
if env_roots:
|
||||
return env_roots.split(':')
|
||||
return []
|
||||
|
||||
def read_js(app, modules):
|
||||
"""
|
||||
:type app: sphinx.application.Sphinx
|
||||
:type modules: Dict[str, jsdoc.ModuleDoc]
|
||||
"""
|
||||
roots = map(os.path.normpath, app.config.js_roots or [os.path.join(app.confdir, '..')])
|
||||
files = [
|
||||
os.path.join(r, f)
|
||||
for root in roots
|
||||
for r, _, fs in os.walk(root)
|
||||
if 'static/src/js' in r
|
||||
for f in fs
|
||||
if f.endswith('.js')
|
||||
]
|
||||
|
||||
modules.update((mod.name, mod) for mod in ABSTRACT_MODULES)
|
||||
for name in it(app)(files, "Parsing javascript files...", length=len(files)):
|
||||
with io.open(name) as f:
|
||||
ast = pyjsparser.parse(f.read())
|
||||
modules.update(
|
||||
(mod.name, mod)
|
||||
for mod in parser.ModuleMatcher(name).visit(ast)
|
||||
)
|
||||
_resolve_references(modules)
|
||||
|
||||
def _resolve_references(byname):
|
||||
# must be done in topological order otherwise the dependent can't
|
||||
# resolve non-trivial references to a dependency properly
|
||||
for name in pyjsdoc.topological_sort(
|
||||
*pyjsdoc.build_dependency_graph(
|
||||
list(byname.keys()),
|
||||
byname
|
||||
)
|
||||
):
|
||||
byname[name].post_process(byname)
|
||||
|
||||
ABSTRACT_MODULES = [
|
||||
jsdoc.ModuleDoc({
|
||||
'module': u'web.web_client',
|
||||
'dependency': {u'web.AbstractWebClient'},
|
||||
'exports': jsdoc.NSDoc({
|
||||
'name': u'web_client',
|
||||
'doc': u'instance of AbstractWebClient',
|
||||
}),
|
||||
}),
|
||||
jsdoc.ModuleDoc({
|
||||
'module': u'web.Tour',
|
||||
'dependency': {u'web_tour.TourManager'},
|
||||
'exports': jsdoc.NSDoc({
|
||||
'name': u'Tour',
|
||||
'doc': u'maybe tourmanager instance?',
|
||||
}),
|
||||
}),
|
||||
jsdoc.ModuleDoc({
|
||||
'module': u'summernote/summernote',
|
||||
'exports': jsdoc.NSDoc({'doc': u"totally real summernote"}),
|
||||
})
|
||||
]
|
||||
1
_extensions/autojsdoc/parser/__init__.py
Normal file
1
_extensions/autojsdoc/parser/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
357
_extensions/autojsdoc/parser/jsdoc.py
Normal file
357
_extensions/autojsdoc/parser/jsdoc.py
Normal file
@ -0,0 +1,357 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import re
|
||||
|
||||
import collections
|
||||
|
||||
import pyjsdoc
|
||||
|
||||
def strip_stars(doc_comment):
|
||||
"""
|
||||
Version of jsdoc.strip_stars which always removes 1 space after * if
|
||||
one is available.
|
||||
"""
|
||||
return re.sub('\n\s*?\*[\t ]?', '\n', doc_comment[3:-2]).strip()
|
||||
|
||||
class ParamDoc(pyjsdoc.ParamDoc):
|
||||
"""
|
||||
Replace ParamDoc because FunctionDoc doesn't properly handle optional
|
||||
params or default values (TODO: or compounds) if guessed_params is used
|
||||
|
||||
=> augment paramdoc with "required" and "default" items to clean up name
|
||||
"""
|
||||
def __init__(self, text):
|
||||
super(ParamDoc, self).__init__(text)
|
||||
# param name and doc can be separated by - or :, strip it
|
||||
self.doc = self.doc.strip().lstrip('-:').lstrip()
|
||||
self.optional = False
|
||||
self.default = None
|
||||
# there may not be a space between the param name and the :, in which
|
||||
# case the : gets attached to the name, strip *again*
|
||||
# TODO: formal @param/@property parser to handle this crap properly once and for all
|
||||
self.name = self.name.strip().rstrip(':')
|
||||
if self.name.startswith('['):
|
||||
self.name = self.name.strip('[]')
|
||||
self.optional = True
|
||||
if '=' in self.name:
|
||||
self.name, self.default = self.name.rsplit('=', 1)
|
||||
def to_dict(self):
|
||||
d = super(ParamDoc, self).to_dict()
|
||||
d['optional'] = self.optional
|
||||
d['default'] = self.default
|
||||
return d
|
||||
pyjsdoc.ParamDoc = ParamDoc
|
||||
|
||||
class CommentDoc(pyjsdoc.CommentDoc):
|
||||
namekey = object()
|
||||
is_constructor = False
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self[self.namekey] or self['name'] or self['guessed_name']
|
||||
def set_name(self, name):
|
||||
# not great...
|
||||
if name != '<exports>':
|
||||
self.parsed['guessed_name'] = name
|
||||
|
||||
@property
|
||||
def is_private(self):
|
||||
return 'private' in self.parsed
|
||||
|
||||
def to_dict(self):
|
||||
d = super(CommentDoc, self).to_dict()
|
||||
d['name'] = self.name
|
||||
return d
|
||||
|
||||
# don't resolve already resolved docs (e.g. a literal dict being
|
||||
# include-ed in two different classes because I don't even care anymore
|
||||
def become(self, modules):
|
||||
return self
|
||||
|
||||
class PropertyDoc(CommentDoc):
|
||||
@classmethod
|
||||
def from_param(cls, s, sourcemodule=None):
|
||||
parsed = ParamDoc(s).to_dict()
|
||||
parsed['sourcemodule'] = sourcemodule
|
||||
return cls(parsed)
|
||||
|
||||
@property
|
||||
def type(self):
|
||||
return self['type'].strip('{}')
|
||||
|
||||
def to_dict(self):
|
||||
d = super(PropertyDoc, self).to_dict()
|
||||
d['type'] = self.type
|
||||
d['is_private'] = self.is_private
|
||||
return d
|
||||
|
||||
class InstanceDoc(CommentDoc):
|
||||
@property
|
||||
def cls(self):
|
||||
return self['cls']
|
||||
|
||||
def to_dict(self):
|
||||
return dict(super(InstanceDoc, self).to_dict(), cls=self.cls)
|
||||
|
||||
class LiteralDoc(CommentDoc):
|
||||
@property
|
||||
def type(self):
|
||||
if self['type']:
|
||||
return self['type']
|
||||
valtype = type(self['value'])
|
||||
if valtype is bool:
|
||||
return 'Boolean'
|
||||
elif valtype is float:
|
||||
return 'Number'
|
||||
elif valtype is type(u''):
|
||||
return 'String'
|
||||
return ''
|
||||
|
||||
@property
|
||||
def value(self):
|
||||
return self['value']
|
||||
|
||||
def to_dict(self):
|
||||
d = super(LiteralDoc, self).to_dict()
|
||||
d['type'] = self.type
|
||||
d['value'] = self.value
|
||||
return d
|
||||
|
||||
class FunctionDoc(CommentDoc):
|
||||
type = 'Function'
|
||||
namekey = 'function'
|
||||
|
||||
@property
|
||||
def is_constructor(self):
|
||||
return self.name == 'init'
|
||||
|
||||
@property
|
||||
def params(self):
|
||||
tag_texts = self.get_as_list('param')
|
||||
# turns out guessed_params is *almost* (?) always set to a list,
|
||||
# if empty list of guessed params fall back to @params
|
||||
if not self['guessed_params']:
|
||||
# only get "primary" params (no "." in name)
|
||||
return [
|
||||
p for p in map(ParamDoc, tag_texts)
|
||||
if '.' not in p.name
|
||||
]
|
||||
else:
|
||||
param_dict = {}
|
||||
for text in tag_texts:
|
||||
param = ParamDoc(text)
|
||||
param_dict[param.name] = param
|
||||
return [param_dict.get(name) or ParamDoc('{} ' + name)
|
||||
for name in self.get('guessed_params')]
|
||||
@property
|
||||
def return_val(self):
|
||||
ret = self.get('return') or self.get('returns')
|
||||
type = self.get('type')
|
||||
if '{' in ret and '}' in ret:
|
||||
if not '} ' in ret:
|
||||
# Ensure that name is empty
|
||||
ret = re.sub(r'\}\s*', '} ', ret)
|
||||
return ParamDoc(ret)
|
||||
if ret and type:
|
||||
return ParamDoc('{%s} %s' % (type, ret))
|
||||
return ParamDoc(ret)
|
||||
|
||||
def to_dict(self):
|
||||
d = super(FunctionDoc, self).to_dict()
|
||||
d['name'] = self.name
|
||||
d['params'] = [param.to_dict() for param in self.params]
|
||||
d['return_val']= self.return_val.to_dict()
|
||||
return d
|
||||
|
||||
class NSDoc(CommentDoc):
|
||||
namekey = 'namespace'
|
||||
def __init__(self, parsed_comment):
|
||||
super(NSDoc, self).__init__(parsed_comment)
|
||||
self.members = collections.OrderedDict()
|
||||
def add_member(self, name, member):
|
||||
"""
|
||||
:type name: str
|
||||
:type member: CommentDoc
|
||||
"""
|
||||
member.set_name(name)
|
||||
self.members[name] = member
|
||||
|
||||
@property
|
||||
def properties(self):
|
||||
if self.get('property'):
|
||||
return [
|
||||
(p.name, p)
|
||||
for p in (
|
||||
PropertyDoc.from_param(p, self['sourcemodule'])
|
||||
for p in self.get_as_list('property')
|
||||
)
|
||||
]
|
||||
return list(self.members.items()) or self['_members'] or []
|
||||
|
||||
def has_property(self, name):
|
||||
return self.get_property(name) is not None
|
||||
|
||||
def get_property(self, name):
|
||||
return next((p for n, p in self.properties if n == name), None)
|
||||
|
||||
def to_dict(self):
|
||||
d = super(NSDoc, self).to_dict()
|
||||
d['properties'] = [(n, p.to_dict()) for n, p in self.properties]
|
||||
return d
|
||||
|
||||
class MixinDoc(NSDoc):
|
||||
namekey = 'mixin'
|
||||
|
||||
class ModuleDoc(NSDoc):
|
||||
namekey = 'module'
|
||||
def __init__(self, parsed_comment):
|
||||
super(ModuleDoc, self).__init__(parsed_comment)
|
||||
#: callbacks to run with the modules mapping once every module is resolved
|
||||
self._post_process = []
|
||||
|
||||
def post_process(self, modules):
|
||||
for callback in self._post_process:
|
||||
callback(modules)
|
||||
|
||||
@property
|
||||
def module(self):
|
||||
return self # lol
|
||||
|
||||
@property
|
||||
def dependencies(self):
|
||||
"""
|
||||
Returns the immediate dependencies of a module (only those explicitly
|
||||
declared/used).
|
||||
"""
|
||||
return self.get('dependency', None) or set()
|
||||
|
||||
@property
|
||||
def exports(self):
|
||||
"""
|
||||
Returns the actual item exported from the AMD module, can be a
|
||||
namespace, a class, a function, an instance, ...
|
||||
"""
|
||||
return self.get_property('<exports>')
|
||||
|
||||
def to_dict(self):
|
||||
vars = super(ModuleDoc, self).to_dict()
|
||||
vars['dependencies'] = self.dependencies
|
||||
vars['exports'] = self.exports
|
||||
return vars
|
||||
|
||||
def __str__(self):
|
||||
s = super().__str__()
|
||||
if self['sourcefile']:
|
||||
s += " in file " + self['sourcefile']
|
||||
return s
|
||||
|
||||
class ClassDoc(NSDoc):
|
||||
namekey = 'class'
|
||||
@property
|
||||
def constructor(self):
|
||||
return self.get_property('init')
|
||||
|
||||
@property
|
||||
def superclass(self):
|
||||
return self['extends'] or self['base']
|
||||
|
||||
def get_property(self, method_name):
|
||||
if method_name == 'extend':
|
||||
return FunctionDoc({
|
||||
'doc': 'Create subclass for %s' % self.name,
|
||||
'guessed_function': 'extend',
|
||||
})
|
||||
# FIXME: should ideally be a proxy namespace
|
||||
if method_name == 'prototype':
|
||||
return self
|
||||
return super(ClassDoc, self).get_property(method_name)\
|
||||
or (self.superclass and self.superclass.get_property(method_name))
|
||||
|
||||
@property
|
||||
def mixins(self):
|
||||
return self.get_as_list('mixes')
|
||||
|
||||
def to_dict(self):
|
||||
d = super(ClassDoc, self).to_dict()
|
||||
d['mixins'] = self.mixins
|
||||
return d
|
||||
|
||||
DEFAULT = object()
|
||||
class UnknownNS(NSDoc):
|
||||
params = () # TODO: log warning when (somehow) trying to access / document an unknown object as ctor?
|
||||
def get_property(self, name):
|
||||
return super(UnknownNS, self).get_property(name) or \
|
||||
UnknownNS({'name': '{}.{}'.format(self.name, name)})
|
||||
|
||||
def __getitem__(self, item):
|
||||
if self._probably_not_property(item):
|
||||
return super().__getitem__(item)
|
||||
return self.get_property(item)
|
||||
|
||||
def _probably_not_property(self, item):
|
||||
return (
|
||||
not isinstance(item, str)
|
||||
or item in (self.namekey, 'name', 'params')
|
||||
or item.startswith(('_', 'guessed_'))
|
||||
or item in self.parsed
|
||||
)
|
||||
|
||||
class Unknown(CommentDoc):
|
||||
@classmethod
|
||||
def from_(cls, source):
|
||||
def builder(parsed):
|
||||
inst = cls(parsed)
|
||||
inst.parsed['source'] = source
|
||||
return inst
|
||||
return builder
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self['name'] + ' ' + self['source']
|
||||
|
||||
@property
|
||||
def type(self):
|
||||
return "Unknown"
|
||||
|
||||
def get_property(self, p):
|
||||
return Unknown(dict(self.parsed, source=self.name, name=p + '<'))
|
||||
|
||||
def parse_comments(comments, doctype=None):
|
||||
# find last comment which starts with a *
|
||||
docstring = next((
|
||||
c['value']
|
||||
for c in reversed(comments or [])
|
||||
if c['value'].startswith(u'*')
|
||||
), None) or u""
|
||||
|
||||
# \n prefix necessary otherwise parse_comment fails to take first
|
||||
# block comment parser strips delimiters, but strip_stars fails without
|
||||
# them
|
||||
extract = '\n' + strip_stars('/*' + docstring + '\n*/')
|
||||
parsed = pyjsdoc.parse_comment(extract, u'')
|
||||
|
||||
if doctype == 'FunctionExpression':
|
||||
doctype = FunctionDoc
|
||||
elif doctype == 'ObjectExpression' or doctype is None:
|
||||
doctype = guess
|
||||
|
||||
if doctype is guess:
|
||||
return doctype(parsed)
|
||||
|
||||
# in case a specific doctype is given, allow overriding it anyway
|
||||
return guess(parsed, default=doctype)
|
||||
|
||||
def guess(parsed, default=UnknownNS):
|
||||
if 'class' in parsed:
|
||||
return ClassDoc(parsed)
|
||||
if 'function' in parsed:
|
||||
return FunctionDoc(parsed)
|
||||
if 'mixin' in parsed:
|
||||
return MixinDoc(parsed)
|
||||
if 'namespace' in parsed:
|
||||
return NSDoc(parsed)
|
||||
if 'module' in parsed:
|
||||
return ModuleDoc(parsed)
|
||||
if 'type' in parsed:
|
||||
return PropertyDoc(parsed)
|
||||
|
||||
return default(parsed)
|
||||
572
_extensions/autojsdoc/parser/parser.py
Normal file
572
_extensions/autojsdoc/parser/parser.py
Normal file
@ -0,0 +1,572 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import collections
|
||||
|
||||
import pyjsdoc
|
||||
|
||||
from . import jsdoc
|
||||
from . import utils
|
||||
from .visitor import Visitor, SKIP
|
||||
|
||||
DECLARATOR_INIT_TO_REF = ('Literal', 'Identifier', 'MemberExpression')
|
||||
|
||||
|
||||
class ModuleMatcher(Visitor):
|
||||
"""Looks for structures of the form::
|
||||
|
||||
odoo.define($string, function ($name) {
|
||||
|
||||
These are *Odoo module definitions*, upon encountering one the
|
||||
matcher:
|
||||
|
||||
* creates a module entry, optionally associated with the module comment
|
||||
* spawns off a :class:`ModuleBodyMatcher` on the function's body with the
|
||||
module name and the $name as "require" function
|
||||
"""
|
||||
def __init__(self, filename):
|
||||
super(ModuleMatcher, self).__init__()
|
||||
self.filename = filename
|
||||
self.result = []
|
||||
def enter_Program(self, node):
|
||||
pass # allows visiting toplevel
|
||||
def enter_ExpressionStatement(self, node):
|
||||
# we're interested in expression statements (toplevel call)
|
||||
if utils.match(node, {'expression': {
|
||||
'callee': {
|
||||
'object': {'name': 'odoo'},
|
||||
'property': {'name': 'define'},
|
||||
},
|
||||
}}):
|
||||
[module, *_, func] = node['expression']['arguments']
|
||||
mod = jsdoc.parse_comments(node.get('comments'), jsdoc.ModuleDoc)
|
||||
# set module name
|
||||
mod.set_name(module['value'])
|
||||
mod.parsed['sourcefile'] = self.filename
|
||||
self.result.append(mod)
|
||||
|
||||
# get name of require parameter
|
||||
require = None # a module can have no dependencies
|
||||
if func['params']:
|
||||
require = func['params'][0]['name']
|
||||
mod.parsed['dependency'], post = ModuleExtractor(mod, require).visit(func['body'])
|
||||
mod._post_process.extend(post)
|
||||
# don't recurse since we've fired off a sub-visitor for the
|
||||
# bits we're interested in
|
||||
return SKIP
|
||||
def enter_generic(self, node):
|
||||
# skip all other toplevel statements
|
||||
return SKIP
|
||||
|
||||
ref = collections.namedtuple('Ref', 'object property')
|
||||
def _name(r):
|
||||
bits = []
|
||||
while isinstance(r, ref):
|
||||
bits.append(str(r.property))
|
||||
r = r.object
|
||||
return '.'.join(reversed(bits))
|
||||
def deref(item, prop=None):
|
||||
assert isinstance(item, ref)
|
||||
|
||||
while isinstance(item, ref):
|
||||
obj = item.object
|
||||
if isinstance(obj, ref):
|
||||
obj = deref(obj)
|
||||
|
||||
if isinstance(obj, (jsdoc.NSDoc, jsdoc.Unknown)):
|
||||
item = obj.get_property(item.property)
|
||||
elif isinstance(obj, dict):
|
||||
item = obj[item.property]
|
||||
elif isinstance(obj, jsdoc.PropertyDoc):
|
||||
# f'n dynamic crap
|
||||
item = jsdoc.Unknown(obj.to_dict()).get_property(item.property)
|
||||
else:
|
||||
raise ValueError("%r (%s) should be a dict or namespace" % (obj, type(obj)))
|
||||
return item
|
||||
|
||||
def m2r(me, scope):
|
||||
# create a ref to the scope in case it's a hoisted function declaration,
|
||||
# this may false-positive on other hoisting but w/e
|
||||
if me['type'] == 'Literal':
|
||||
return jsdoc.LiteralDoc({'value': me['value']})
|
||||
if me['type'] == 'Identifier':
|
||||
return ref(scope, me['name'])
|
||||
if me['type'] != 'MemberExpression':
|
||||
raise ValueError(me)
|
||||
return ref(
|
||||
m2r(me['object'], scope),
|
||||
utils._value(me['property'], strict=True)
|
||||
)
|
||||
|
||||
NOTHING = object()
|
||||
class Declaration(object):
|
||||
__slots__ = ['id', 'comments']
|
||||
def __init__(self, id=None, comments=NOTHING):
|
||||
self.id = id
|
||||
self.comments = [] if comments is NOTHING else comments
|
||||
|
||||
class ModuleContent(object):
|
||||
__slots__ = ['dependencies', 'post']
|
||||
def __init__(self, dependencies=NOTHING, post=NOTHING):
|
||||
self.dependencies = set() if dependencies is NOTHING else dependencies
|
||||
self.post = [] if post is NOTHING else post
|
||||
def __iter__(self):
|
||||
yield self.dependencies
|
||||
yield self.post
|
||||
|
||||
class Nothing(object):
|
||||
def __init__(self, name):
|
||||
self.name = name
|
||||
def __bool__(self):
|
||||
return False
|
||||
__nonzero__ = __bool__
|
||||
|
||||
class RefProxy(object):
|
||||
def __init__(self, r):
|
||||
self._ref = r
|
||||
def become(self, modules):
|
||||
s = self
|
||||
other = deref(self._ref) or Nothing(_name(self._ref))
|
||||
# ??? shouldn't all previous refs have been resolved?
|
||||
if isinstance(other, (RefProxy, ModuleProxy)):
|
||||
other = other.become(modules)
|
||||
self.__class__ = other.__class__
|
||||
self.__dict__ = other.__dict__
|
||||
return self
|
||||
def set_name(self, name):
|
||||
pass # ???
|
||||
class ModuleProxy(object):
|
||||
def __init__(self, name):
|
||||
self._name = name
|
||||
|
||||
# replace the ModuleProxy by the module's exports
|
||||
def become(self, modules):
|
||||
s = self
|
||||
m = modules[self._name].get_property('<exports>') or Nothing(self._name)
|
||||
self.__class__ = m.__class__
|
||||
self.__dict__ = m.__dict__
|
||||
return self
|
||||
|
||||
def set_name(self, name):
|
||||
pass # FIXME: ???
|
||||
|
||||
jq = jsdoc.UnknownNS({
|
||||
'name': u'jQuery',
|
||||
'doc': u'<jQuery>',
|
||||
})
|
||||
window = jsdoc.UnknownNS({
|
||||
'doc': '<window>',
|
||||
'name': 'window',
|
||||
})
|
||||
|
||||
class BaseScope(collections.defaultdict):
|
||||
""" The base scope assumes anything it's asked for is just an unknown
|
||||
(global) namespace of some sort. Can hold a bunch of predefined params but
|
||||
avoids the variables inference system blowing up when new (browser)
|
||||
globals get used in module bodies.
|
||||
"""
|
||||
def __missing__(self, key):
|
||||
it = jsdoc.UnknownNS({
|
||||
'name': key,
|
||||
'doc': u'<%s>' % key,
|
||||
})
|
||||
self[key] = it
|
||||
return it
|
||||
BASE_SCOPE = BaseScope(None, {
|
||||
'_': jsdoc.UnknownNS({'doc': u'<underscore.js>', 'name': u'_'}),
|
||||
'$': jq, 'jQuery': jq,
|
||||
'window': window,
|
||||
'document': window.get_property('document'),
|
||||
'Date': jsdoc.ClassDoc({
|
||||
'name': u'Date',
|
||||
'doc': u'',
|
||||
}),
|
||||
'Backbone': jsdoc.UnknownNS({
|
||||
'_members': [
|
||||
('Model', jsdoc.ClassDoc({
|
||||
'name': u'Model',
|
||||
'doc': u'',
|
||||
})),
|
||||
('Collection', jsdoc.ClassDoc({
|
||||
'name': u'Collection',
|
||||
'doc': u'',
|
||||
})),
|
||||
]
|
||||
}),
|
||||
'odoo': jsdoc.UnknownNS({
|
||||
'name': u'odoo',
|
||||
'doc': u"Odoo",
|
||||
'_members': [
|
||||
('name', jsdoc.PropertyDoc({'name': u'csrf_token', 'type': u'{String}'})),
|
||||
]
|
||||
}),
|
||||
'undefined': jsdoc.LiteralDoc({'name': u'undefined', 'value': None}),
|
||||
})
|
||||
|
||||
class Scope(object):
|
||||
"""
|
||||
Add hoc scope versioning/SSA such that rebinding a symbol in a module
|
||||
scope does not screw everything up e.g. "Foo = Foo.extend({})" should not
|
||||
have the final Foo extending itself...
|
||||
"""
|
||||
def __init__(self, mapping):
|
||||
self._namemap = self._empty(mapping)
|
||||
self._targets = []
|
||||
for k, v in mapping.items():
|
||||
self[k] = v
|
||||
|
||||
@staticmethod
|
||||
def _empty(mapping):
|
||||
m = mapping.copy()
|
||||
m.clear()
|
||||
return m
|
||||
|
||||
def __setitem__(self, k, v):
|
||||
self._namemap[k] = len(self._targets)
|
||||
self._targets.append(v)
|
||||
|
||||
def freeze(self):
|
||||
d = self._empty(self._namemap)
|
||||
for k, v in self._namemap.items():
|
||||
d[k] = self._targets[v]
|
||||
return d
|
||||
|
||||
class ModuleExtractor(Visitor):
|
||||
def __init__(self, module, requirefunc):
|
||||
super(ModuleExtractor, self).__init__()
|
||||
self.module = module
|
||||
self.requirefunc = requirefunc
|
||||
self.result = ModuleContent()
|
||||
self.scope = Scope(BASE_SCOPE)
|
||||
self.declaration = None
|
||||
|
||||
def enter_BlockStatement(self, node):
|
||||
Hoistifier(self).visit(node)
|
||||
def exit_BlockStatement(self, node):
|
||||
for k, v in self.scope._namemap.items():
|
||||
if k not in BASE_SCOPE:
|
||||
self.module.add_member(k, self.scope._targets[v])
|
||||
for t in TypedefMatcher(self).visit(node):
|
||||
self.module.add_member(t.name, t)
|
||||
|
||||
def enter_VariableDeclaration(self, node):
|
||||
self.declaration = Declaration(comments=node.get('comments'))
|
||||
def enter_VariableDeclarator(self, node):
|
||||
# otherwise we've already hoisted the declaration so the variable
|
||||
# already exist initialised to undefined, and ValueExtractor does not
|
||||
# handle a None input node so it returns None which makes no sense
|
||||
if node['init']:
|
||||
self.declaration.id = node['id']['name']
|
||||
self.scope[self.declaration.id] = ValueExtractor(
|
||||
self, self.declaration
|
||||
).visit(node['init'] or [])
|
||||
self.declaration.id = None
|
||||
return SKIP
|
||||
def exit_VariableDeclaration(self, node):
|
||||
self.declaration = None
|
||||
|
||||
# as the name denotes, AssignmentExpression is an *expression*, which
|
||||
# means at the module toplevel it is wrapped into an ExpressionStatement
|
||||
# which is where the comments get attached
|
||||
def enter_ExpressionStatement(self, node):
|
||||
self.declaration = Declaration(comments=node.get('comments'))
|
||||
def exit_ExpressionStatement(self, node):
|
||||
self.declaration = None
|
||||
def enter_AssignmentExpression(self, node):
|
||||
target = node['left']
|
||||
if target['type'] == 'Identifier':
|
||||
self.declaration.id = target['name']
|
||||
self.scope[self.declaration.id] = ValueExtractor(
|
||||
self, self.declaration
|
||||
).visit(node['right'])
|
||||
self.declaration.id = None
|
||||
return SKIP
|
||||
|
||||
if target['type'] != 'MemberExpression':
|
||||
raise ValueError("Unhandled assign to %s" % target['type'])
|
||||
|
||||
# only assign to straight a.b.c patterns (OK and trivial literals)
|
||||
if target['computed'] and target['property']['type'] != 'Literal':
|
||||
return SKIP
|
||||
|
||||
name = utils._value(target['property'], strict=True)
|
||||
if isinstance(name, type(u'')) and name.endswith('.extend'):
|
||||
return SKIP # ignore overwrite of .extend (WTF)
|
||||
self.declaration.id = name
|
||||
it = ValueExtractor(
|
||||
self, self.declaration
|
||||
).visit(node['right'])
|
||||
self.declaration.id = None
|
||||
assert it, "assigned a non-value from %s to %s" % (node['right'], name)
|
||||
|
||||
@self.result.post.append
|
||||
def _augment_module(modules):
|
||||
try:
|
||||
t = deref(m2r(target['object'], self.scope.freeze()))
|
||||
except ValueError:
|
||||
return # f'n extension of global libraries garbage
|
||||
if not isinstance(t, jsdoc.NSDoc):
|
||||
# function Foo(){}; Foo.prototype = bar
|
||||
# fuck that yo
|
||||
return
|
||||
# TODO: note which module added this
|
||||
m = it
|
||||
if isinstance(m, jsdoc.LiteralDoc):
|
||||
m = jsdoc.PropertyDoc(m.to_dict())
|
||||
t.add_member(name, m)
|
||||
return SKIP
|
||||
|
||||
def enter_FunctionDeclaration(self, node):
|
||||
""" Already processed by hoistitifier
|
||||
"""
|
||||
return SKIP
|
||||
|
||||
def enter_ReturnStatement(self, node):
|
||||
self.declaration = Declaration(comments=node.get('comments'))
|
||||
if node['argument']:
|
||||
export = ValueExtractor(self, self.declaration).visit(node['argument'])
|
||||
if isinstance(export, RefProxy):
|
||||
self.module.parsed['exports'] = _name(export._ref)
|
||||
self.scope['<exports>'] = export
|
||||
self.declaration = None
|
||||
return SKIP
|
||||
|
||||
def enter_CallExpression(self, node):
|
||||
if utils.match(node, {
|
||||
'callee': {
|
||||
'type': 'MemberExpression',
|
||||
'object': lambda n: (
|
||||
n['type'] in ('Identifier', 'MemberExpression')
|
||||
# _.str.include
|
||||
and not utils._name(n).startswith('_')
|
||||
),
|
||||
'property': {'name': 'include'},
|
||||
}
|
||||
}):
|
||||
target = RefProxy(m2r(node['callee']['object'], self.scope.freeze()))
|
||||
target_name = utils._name(node['callee']['object'])
|
||||
items = ClassProcessor(self).visit(node['arguments'])
|
||||
@self.result.post.append
|
||||
def resolve_extension(modules):
|
||||
t = target.become(modules)
|
||||
if not isinstance(t, jsdoc.ClassDoc):
|
||||
return # FIXME: log warning
|
||||
# raise ValueError("include() subjects should be classes, %s is %s" % (target_name, type(t)))
|
||||
# TODO: note which module added these
|
||||
for it in items:
|
||||
if isinstance(it, dict):
|
||||
for n, member in it.items():
|
||||
t.add_member(n, member)
|
||||
else:
|
||||
t.parsed.setdefault('mixes', []).append(it.become(modules))
|
||||
|
||||
return SKIP
|
||||
|
||||
def refify(self, node, also=None):
|
||||
it = m2r(node, self.scope.freeze())
|
||||
assert isinstance(it, ref), "Expected ref, got {}".format(it)
|
||||
px = RefProxy(it)
|
||||
@self.result.post.append
|
||||
def resolve(modules):
|
||||
p = px.become(modules)
|
||||
if also: also(p)
|
||||
return px
|
||||
|
||||
class ValueExtractor(Visitor):
|
||||
def __init__(self, parent, declaration=None):
|
||||
super(ValueExtractor, self).__init__()
|
||||
self.parent = parent
|
||||
self.declaration = declaration or Declaration()
|
||||
|
||||
def enter_generic(self, node):
|
||||
self.result = jsdoc.parse_comments(
|
||||
self.declaration.comments,
|
||||
jsdoc.Unknown.from_(node['type'])
|
||||
)
|
||||
self._update_result_meta()
|
||||
return SKIP
|
||||
|
||||
def _update_result_meta(self, name=None):
|
||||
self.result.parsed['sourcemodule'] = self.parent.module
|
||||
n = name or self.declaration.id
|
||||
if n:
|
||||
self.result.set_name(n)
|
||||
|
||||
def enter_Literal(self, node):
|
||||
self.result = jsdoc.parse_comments(
|
||||
self.declaration.comments, jsdoc.LiteralDoc)
|
||||
self._update_result_meta()
|
||||
self.result.parsed['value'] = node['value']
|
||||
return SKIP
|
||||
def enter_Identifier(self, node):
|
||||
self.result = self.parent.refify(node)
|
||||
return SKIP
|
||||
|
||||
def enter_MemberExpression(self, node):
|
||||
self.result = RefProxy(ref(
|
||||
ValueExtractor(self.parent).visit(node['object']),
|
||||
utils._value(node['property'], strict=True)
|
||||
))
|
||||
self.parent.result.post.append(self.result.become)
|
||||
return SKIP
|
||||
|
||||
def enter_FunctionExpression(self, node):
|
||||
name, comments = (self.declaration.id, self.declaration.comments)
|
||||
self.result = jsdoc.parse_comments(comments, jsdoc.FunctionDoc)
|
||||
self.result.parsed['name'] = node['id'] and node['id']['name']
|
||||
self._update_result_meta()
|
||||
self.result.parsed['guessed_params'] = [p['name'] for p in node['params']]
|
||||
return SKIP
|
||||
|
||||
def enter_NewExpression(self, node):
|
||||
comments = self.declaration.comments if self.declaration else node.get('comments')
|
||||
self.result = ns = jsdoc.parse_comments(comments, jsdoc.InstanceDoc)
|
||||
self._update_result_meta()
|
||||
|
||||
def _update_contents(cls):
|
||||
if not isinstance(cls, jsdoc.ClassDoc):
|
||||
return
|
||||
ns.parsed['cls'] = cls
|
||||
self.parent.refify(node['callee'], also=_update_contents)
|
||||
return SKIP
|
||||
|
||||
def enter_ObjectExpression(self, node):
|
||||
self.result = obj = jsdoc.parse_comments(self.declaration.comments)
|
||||
self._update_result_meta()
|
||||
for n, p in MemberExtractor(parent=self.parent).visit(node['properties']).items():
|
||||
obj.add_member(n, p)
|
||||
return SKIP
|
||||
|
||||
def enter_CallExpression(self, node):
|
||||
# require(a_module_name)
|
||||
if utils.match(node, {'callee': {'type': 'Identifier', 'name': self.parent.requirefunc}}):
|
||||
depname = node['arguments'][0]['value']
|
||||
# TODO: clean this up
|
||||
self.parent.result.dependencies.add(depname)
|
||||
self.result = ModuleProxy(name=depname)
|
||||
self.parent.result.post.append(self.result.become)
|
||||
|
||||
# Class.extend(..mixins, {})
|
||||
elif utils.match(node, {
|
||||
'callee': {
|
||||
'type': 'MemberExpression',
|
||||
'object': lambda n: (
|
||||
n['type'] in ('Identifier', 'MemberExpression')
|
||||
# $.extend, $.fn.extend, _.extend
|
||||
and not utils._name(n).startswith(('_', '$'))
|
||||
),
|
||||
'property': {'name': 'extend'},
|
||||
},
|
||||
}): # creates a new class, but may not actually return it
|
||||
obj = node['callee']['object']
|
||||
comments = self.declaration.comments
|
||||
self.result = cls = jsdoc.parse_comments(comments, jsdoc.ClassDoc)
|
||||
cls.parsed['extends'] = self.parent.refify(obj)
|
||||
self._update_result_meta()
|
||||
items = ClassProcessor(self.parent).visit(node['arguments'])
|
||||
|
||||
@self.parent.result.post.append
|
||||
def add_to_class(modules):
|
||||
for item in items:
|
||||
if isinstance(item, dict):
|
||||
# methods/attributes
|
||||
for n, method in item.items():
|
||||
cls.add_member(n, method)
|
||||
else:
|
||||
cls.parsed.setdefault('mixes', []).append(item)
|
||||
|
||||
# other function calls
|
||||
else:
|
||||
self.result = jsdoc.parse_comments(self.declaration.comments, jsdoc.guess)
|
||||
self._update_result_meta()
|
||||
|
||||
return SKIP
|
||||
|
||||
class ClassProcessor(Visitor):
|
||||
def __init__(self, parent):
|
||||
super(ClassProcessor, self).__init__()
|
||||
self.result = []
|
||||
self.parent = parent
|
||||
|
||||
def enter_generic(self, node):
|
||||
self.result.append(self.parent.refify(node))
|
||||
return SKIP
|
||||
def enter_ObjectExpression(self, node):
|
||||
self.result.append(MemberExtractor(parent=self.parent, for_class=True).visit(node['properties']))
|
||||
return SKIP
|
||||
|
||||
String = type(u'')
|
||||
class MemberExtractor(Visitor):
|
||||
def __init__(self, parent, for_class=False):
|
||||
super(MemberExtractor, self).__init__()
|
||||
self.result = collections.OrderedDict()
|
||||
self.parent = parent
|
||||
self.for_class = for_class
|
||||
|
||||
def enter_Property(self, node):
|
||||
name = utils._value(node['key'])
|
||||
prop = ValueExtractor(
|
||||
self.parent,
|
||||
Declaration(id=name, comments=node.get('comments'))
|
||||
).visit(node['value'])
|
||||
|
||||
if isinstance(prop, jsdoc.LiteralDoc):
|
||||
prop = jsdoc.PropertyDoc(prop.to_dict())
|
||||
|
||||
# ValueExtractor can return a Ref, maybe this should be sent as part
|
||||
# of the decl/comments?
|
||||
if name.startswith('_') and hasattr(prop, 'parsed'):
|
||||
prop.parsed['private'] = True
|
||||
self.result[name] = prop
|
||||
|
||||
return SKIP
|
||||
|
||||
class Hoistifier(Visitor):
|
||||
"""
|
||||
Processor for variable and function declarations properly hoisting them
|
||||
to the "top" of a module such that they are available with the relevant
|
||||
value afterwards.
|
||||
"""
|
||||
def __init__(self, parent):
|
||||
super(Hoistifier, self).__init__()
|
||||
self.parent = parent
|
||||
|
||||
def enter_generic(self, node):
|
||||
return SKIP
|
||||
|
||||
# nodes to straight recurse into, others are just skipped
|
||||
enter_BlockStatement = enter_VariableDeclaration = lambda self, node: None
|
||||
|
||||
def enter_VariableDeclarator(self, node):
|
||||
self.parent.scope[node['id']['name']] = BASE_SCOPE['undefined']
|
||||
|
||||
def enter_FunctionDeclaration(self, node):
|
||||
funcname = node['id']['name']
|
||||
self.parent.scope[funcname] = fn = jsdoc.parse_comments(
|
||||
node.get('comments'),
|
||||
jsdoc.FunctionDoc,
|
||||
)
|
||||
fn.parsed['sourcemodule'] = self.parent.module
|
||||
fn.parsed['name'] = funcname
|
||||
fn.parsed['guessed_params'] = [p['name'] for p in node['params']]
|
||||
return SKIP
|
||||
|
||||
class TypedefMatcher(Visitor):
|
||||
def __init__(self, parent):
|
||||
super(TypedefMatcher, self).__init__()
|
||||
self.parent = parent
|
||||
self.result = []
|
||||
|
||||
enter_BlockStatement = lambda self, node: None
|
||||
def enter_generic(self, node):
|
||||
# just traverse all top-level statements, check their comments, and
|
||||
# bail
|
||||
for comment in node.get('comments') or []:
|
||||
if '@typedef' in comment['value']:
|
||||
extract = '\n' + jsdoc.strip_stars('/*' + comment['value'] + '\n*/')
|
||||
parsed = pyjsdoc.parse_comment(extract, u'')
|
||||
p = jsdoc.ParamDoc(parsed['typedef'])
|
||||
parsed['name'] = p.name
|
||||
parsed['sourcemodule'] = self.parent.module
|
||||
# TODO: add p.type as superclass somehow? Builtin types not in scope :(
|
||||
self.result.append(jsdoc.ClassDoc(parsed))
|
||||
|
||||
return SKIP
|
||||
6
_extensions/autojsdoc/parser/tests/README.rst
Normal file
6
_extensions/autojsdoc/parser/tests/README.rst
Normal file
@ -0,0 +1,6 @@
|
||||
:orphan:
|
||||
|
||||
These files should be run via pytest_, simply install pytest and from the top
|
||||
of the Odoo project run ``pytest doc/_extensions``.
|
||||
|
||||
.. _pytest: https://pytest.org/
|
||||
73
_extensions/autojsdoc/parser/tests/support.py
Normal file
73
_extensions/autojsdoc/parser/tests/support.py
Normal file
@ -0,0 +1,73 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import operator
|
||||
|
||||
import pyjsparser
|
||||
|
||||
from autojsdoc.parser import jsdoc, parser
|
||||
|
||||
params = operator.attrgetter('name', 'type', 'doc')
|
||||
|
||||
|
||||
def parse(s, source=None):
|
||||
tree = pyjsparser.parse(s)
|
||||
mods = parser.ModuleMatcher(source).visit(tree)
|
||||
post(mods)
|
||||
return mods
|
||||
|
||||
def post(mods):
|
||||
modules = dict(BASE_MODULES)
|
||||
modules.update((m.name, m) for m in mods)
|
||||
|
||||
for mod in mods:
|
||||
mod.post_process(modules)
|
||||
|
||||
BASE_MODULES = {
|
||||
'other': jsdoc.ModuleDoc({
|
||||
'module': 'other',
|
||||
'_members': [
|
||||
('<exports>', jsdoc.LiteralDoc({'name': 'value', 'value': "ok"})),
|
||||
],
|
||||
}),
|
||||
'dep2': jsdoc.ModuleDoc({
|
||||
'module': 'dep2',
|
||||
'_members': [
|
||||
('<exports>', jsdoc.LiteralDoc({'value': 42.})),
|
||||
],
|
||||
}),
|
||||
'dep3': jsdoc.ModuleDoc({
|
||||
'module': 'dep3',
|
||||
'_members': [
|
||||
('<exports>', jsdoc.LiteralDoc({'value': 56.})),
|
||||
],
|
||||
}),
|
||||
'Class': jsdoc.ModuleDoc({
|
||||
'module': 'Class',
|
||||
'_members': [
|
||||
('<exports>', jsdoc.ClassDoc({
|
||||
'name': 'Class',
|
||||
'doc': "Base Class"
|
||||
})),
|
||||
],
|
||||
}),
|
||||
'mixins': jsdoc.ModuleDoc({
|
||||
'module': 'mixins',
|
||||
'_members': [
|
||||
('<exports>', jsdoc.NSDoc({
|
||||
'name': 'mixins',
|
||||
'_members': [
|
||||
('Bob', jsdoc.ClassDoc({'class': "Bob"})),
|
||||
]
|
||||
})),
|
||||
],
|
||||
}),
|
||||
'Mixin': jsdoc.ModuleDoc({
|
||||
'module': 'Mixin',
|
||||
'_members': [
|
||||
('<exports>', jsdoc.MixinDoc({
|
||||
'_members': [
|
||||
('a', jsdoc.FunctionDoc({'function': 'a'})),
|
||||
]
|
||||
})),
|
||||
],
|
||||
})
|
||||
}
|
||||
227
_extensions/autojsdoc/parser/tests/test_class.py
Normal file
227
_extensions/autojsdoc/parser/tests/test_class.py
Normal file
@ -0,0 +1,227 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import pytest
|
||||
|
||||
from autojsdoc.parser import jsdoc
|
||||
from support import params, parse
|
||||
|
||||
def test_classvar():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.A', function(require) {
|
||||
var Class = require('Class');
|
||||
/**
|
||||
* This is my class-kai
|
||||
*/
|
||||
var A = Class.extend({});
|
||||
return A;
|
||||
});
|
||||
""")
|
||||
cls = mod.exports
|
||||
assert type(cls) == jsdoc.ClassDoc
|
||||
|
||||
assert type(cls.superclass) == jsdoc.ClassDoc
|
||||
assert cls.superclass.name == 'Class'
|
||||
|
||||
assert cls.name == 'A'
|
||||
assert cls.constructor is None
|
||||
assert cls.properties == []
|
||||
assert cls['doc'] == 'This is my class-kai'
|
||||
|
||||
def test_classret():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.A', function(require) {
|
||||
var Class = require('Class');
|
||||
/**
|
||||
* This is my class-kai
|
||||
*/
|
||||
return Class.extend({});
|
||||
});
|
||||
""")
|
||||
cls = mod.exports
|
||||
assert type(cls) == jsdoc.ClassDoc
|
||||
|
||||
assert cls.name == ''
|
||||
assert cls.constructor is None
|
||||
assert cls.properties == []
|
||||
assert cls['doc'] == 'This is my class-kai'
|
||||
|
||||
def test_methods():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.A', function(require) {
|
||||
var Class = require('Class');
|
||||
return Class.extend({
|
||||
/**
|
||||
* @param {Widget} parent
|
||||
*/
|
||||
init: function (parent) {},
|
||||
/**
|
||||
* @returns {Widget}
|
||||
*/
|
||||
itself: function () { return this; },
|
||||
/**
|
||||
* @param {MouseEvent} e
|
||||
*/
|
||||
_onValidate: function (e) {},
|
||||
});
|
||||
});
|
||||
""")
|
||||
cls = mod.exports
|
||||
assert len(cls.properties) == 3
|
||||
assert cls.constructor
|
||||
# assume methods are in source order
|
||||
[_, init] = cls.properties[0]
|
||||
assert init == cls.constructor
|
||||
assert init.name == 'init'
|
||||
assert not init.is_private
|
||||
assert init.is_constructor
|
||||
[param] = init.params
|
||||
assert params(param) == ('parent', 'Widget', '')
|
||||
|
||||
[_, itself] = cls.properties[1]
|
||||
assert itself.name == 'itself'
|
||||
assert not itself.is_private
|
||||
assert not itself.is_constructor
|
||||
assert not itself.params
|
||||
assert params(itself.return_val) == ('', 'Widget', '')
|
||||
|
||||
[_, _on] = cls.properties[2]
|
||||
assert _on.name == '_onValidate'
|
||||
assert _on.is_private
|
||||
assert not _on.is_constructor
|
||||
[param] = _on.params
|
||||
assert params(param) == ('e', 'MouseEvent', '')
|
||||
|
||||
def test_mixin_explicit():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.A', function (require) {
|
||||
var Class = require('Class');
|
||||
var mixins = require('mixins');
|
||||
/**
|
||||
* This is my class-kai
|
||||
* @mixes mixins.Bob
|
||||
*/
|
||||
return Class.extend({});
|
||||
});
|
||||
""")
|
||||
cls = mod.exports
|
||||
# FIXME: ClassDoc may want to m2r(mixin, scope)?
|
||||
assert cls.mixins == ['mixins.Bob']
|
||||
|
||||
def test_mixin_implicit():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.A', function(require) {
|
||||
var Class = require('Class');
|
||||
var Mixin = require('Mixin');
|
||||
/**
|
||||
* This is my class-kai
|
||||
*/
|
||||
return Class.extend(Mixin, { foo: function() {} });
|
||||
});
|
||||
""")
|
||||
cls = mod.exports
|
||||
[mixin] = cls.mixins
|
||||
assert type(mixin) == jsdoc.MixinDoc
|
||||
assert params(mixin.properties[0][1]) == ('a', 'Function', '')
|
||||
assert params(mixin.get_property('a')) == ('a', 'Function', '')
|
||||
|
||||
assert params(cls.get_property('foo')) == ('foo', 'Function', '')
|
||||
|
||||
def test_instanciation():
|
||||
[A, a] = parse("""
|
||||
odoo.define('A', function (r) {
|
||||
var Class = r('Class');
|
||||
/**
|
||||
* @class A
|
||||
*/
|
||||
return Class.extend({
|
||||
foo: function () {}
|
||||
});
|
||||
});
|
||||
odoo.define('a', function (r) {
|
||||
var A = r('A');
|
||||
var a = new A;
|
||||
return a;
|
||||
});
|
||||
""")
|
||||
assert type(a.exports) == jsdoc.InstanceDoc
|
||||
assert a.exports.cls.name == A.exports.name
|
||||
|
||||
def test_non_function_properties():
|
||||
[A] = parse("""
|
||||
odoo.define('A', function (r) {
|
||||
var Class = r('Class');
|
||||
return Class.extend({
|
||||
template: 'thing',
|
||||
a_prop: [1, 2, 3],
|
||||
'other': {a: 7}
|
||||
});
|
||||
});
|
||||
""")
|
||||
t = A.exports.get_property('template')
|
||||
assert type(t) == jsdoc.PropertyDoc
|
||||
assert params(t) == ('template', 'String', '')
|
||||
assert not t.is_private
|
||||
|
||||
def test_non_extend_classes():
|
||||
[mod] = parse("""
|
||||
odoo.define('A', function () {
|
||||
/**
|
||||
* @class
|
||||
*/
|
||||
var Class = function () {}
|
||||
return Class;
|
||||
});
|
||||
""")
|
||||
assert type(mod.exports) == jsdoc.ClassDoc
|
||||
|
||||
def test_extend():
|
||||
[a, _] = parse("""
|
||||
odoo.define('A', function (require) {
|
||||
var Class = require('Class');
|
||||
return Class.extend({});
|
||||
});
|
||||
odoo.define('B', function (require) {
|
||||
var A = require('A');
|
||||
A.include({
|
||||
/** A property */
|
||||
a: 3,
|
||||
/** A method */
|
||||
b: function () {}
|
||||
});
|
||||
});
|
||||
""")
|
||||
cls = a.exports
|
||||
assert type(cls) == jsdoc.ClassDoc
|
||||
a = cls.get_property('a')
|
||||
assert type(a) == jsdoc.PropertyDoc
|
||||
assert params(a) == ('a', 'Number', 'A property')
|
||||
b = cls.get_property('b')
|
||||
assert type(b) == jsdoc.FunctionDoc
|
||||
assert params(b) == ('b', 'Function', 'A method')
|
||||
|
||||
# TODO: also support virtual members?
|
||||
# TODO: computed properties?
|
||||
@pytest.mark.skip(reason="Need to implement member/var-parsing?")
|
||||
def test_members():
|
||||
[mod] = parse("""
|
||||
odoo.define('A', function (r) {
|
||||
var Class = r('Class');
|
||||
return Class.extend({
|
||||
init: function () {
|
||||
/**
|
||||
* This is bob
|
||||
* @var {Foo}
|
||||
*/
|
||||
this.foo = 3;
|
||||
this.bar = 42;
|
||||
/**
|
||||
* @member {Baz}
|
||||
*/
|
||||
this.baz = null;
|
||||
}
|
||||
});
|
||||
});
|
||||
""")
|
||||
cls = mod.exports
|
||||
assert params(cls.members[0]) == ('foo', 'Foo', 'This is bob')
|
||||
assert params(cls.members[1]) == ('bar', '', '')
|
||||
assert params(cls.members[2]) == ('baz', 'Baz', '')
|
||||
75
_extensions/autojsdoc/parser/tests/test_crap.py
Normal file
75
_extensions/autojsdoc/parser/tests/test_crap.py
Normal file
@ -0,0 +1,75 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Test various crap patterns found in Odoo code to ensure they don't blow up
|
||||
the parser thingie
|
||||
"""
|
||||
from autojsdoc.parser import jsdoc
|
||||
from support import parse
|
||||
|
||||
def test_export_external():
|
||||
[mod] = parse("""
|
||||
odoo.define('module', function () {
|
||||
return $.Deferred().reject();
|
||||
});
|
||||
""")
|
||||
assert isinstance(mod.exports, jsdoc.CommentDoc)
|
||||
assert mod.exports.doc == ''
|
||||
|
||||
def test_extend_jq():
|
||||
parse("""
|
||||
odoo.define('a', function (r) {
|
||||
$.extend($.expr[':'], { a: function () {} });
|
||||
$.fn.extend({ a: function () {} });
|
||||
});
|
||||
""")
|
||||
|
||||
def test_extend_dynamic():
|
||||
parse("""
|
||||
odoo.define('a', function () {
|
||||
foo.bar.baz[qux + '_external'] = function () {};
|
||||
});
|
||||
""")
|
||||
|
||||
def test_extend_deep():
|
||||
parse("""
|
||||
odoo.define('a', function () {
|
||||
var eventHandler = $.summernote.eventHandler;
|
||||
var dom = $.summernote.core.dom;
|
||||
dom.thing = function () {};
|
||||
|
||||
var fn_editor_currentstyle = eventHandler.modules.editor.currentStyle;
|
||||
eventHandler.modules.editor.currentStyle = function () {}
|
||||
});
|
||||
""")
|
||||
|
||||
def test_arbitrary():
|
||||
parse("""
|
||||
odoo.define('bob', function () {
|
||||
var page = window.location.href.replace(/^.*\/\/[^\/]+/, '');
|
||||
var mailWidgets = ['mail_followers', 'mail_thread', 'mail_activity', 'kanban_activity'];
|
||||
var bob;
|
||||
var fldj = foo.getTemplate().baz;
|
||||
});
|
||||
""")
|
||||
|
||||
def test_prototype():
|
||||
[A, B] = parse("""
|
||||
odoo.define('mod1', function () {
|
||||
var exports = {};
|
||||
exports.Foo = Backbone.Model.extend({});
|
||||
exports.Bar = Backbone.Model.extend({});
|
||||
var BarCollection = Backbone.Collection.extend({
|
||||
model: exports.Bar,
|
||||
});
|
||||
exports.Baz = Backbone.Model.extend({});
|
||||
return exports;
|
||||
});
|
||||
odoo.define('mod2', function (require) {
|
||||
var models = require('mod1');
|
||||
var _super_orderline = models.Bar.prototype;
|
||||
models.Foo = models.Bar.extend({});
|
||||
var _super_order = models.Baz.prototype;
|
||||
models.Bar = models.Baz.extend({});
|
||||
});
|
||||
""")
|
||||
|
||||
173
_extensions/autojsdoc/parser/tests/test_module.py
Normal file
173
_extensions/autojsdoc/parser/tests/test_module.py
Normal file
@ -0,0 +1,173 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
from autojsdoc.parser import jsdoc
|
||||
from support import params, parse, BASE_MODULES
|
||||
|
||||
|
||||
def test_single():
|
||||
[mod] = parse("""
|
||||
/**
|
||||
* This is a super module!
|
||||
*/
|
||||
odoo.define('supermodule', function (req) {
|
||||
var other = req('other');
|
||||
});
|
||||
""")
|
||||
assert mod.name == 'supermodule'
|
||||
assert mod.dependencies == {'other'}
|
||||
assert mod.exports is None
|
||||
assert mod.doc == "This is a super module!"
|
||||
m = mod.get_property('other')
|
||||
assert m.name == 'value', "property is exported value not imported module"
|
||||
|
||||
def test_multiple():
|
||||
[mod1, mod2, mod3] = parse("""
|
||||
odoo.define('module1', function (req) {
|
||||
return 1;
|
||||
});
|
||||
odoo.define('module2', function (req) {
|
||||
return req('dep2');
|
||||
});
|
||||
odoo.define('module3', function (req) {
|
||||
var r = req('dep3');
|
||||
return r;
|
||||
});
|
||||
""")
|
||||
assert mod1.name == 'module1'
|
||||
assert mod1.dependencies == set()
|
||||
assert isinstance(mod1.exports, jsdoc.LiteralDoc)
|
||||
assert mod1.exports.value == 1.0
|
||||
assert mod1.exports['sourcemodule'] is mod1
|
||||
assert mod1.doc == ""
|
||||
|
||||
assert mod2.name == 'module2'
|
||||
assert mod2.dependencies == {'dep2'}
|
||||
assert isinstance(mod2.exports, jsdoc.LiteralDoc)
|
||||
assert mod2.exports.value == 42.0
|
||||
assert mod2.doc == ""
|
||||
|
||||
assert mod3.name == 'module3'
|
||||
assert mod3.dependencies == {'dep3'}
|
||||
assert isinstance(mod3.exports, jsdoc.LiteralDoc)
|
||||
assert mod3.exports.value == 56.0
|
||||
assert mod3.doc == ''
|
||||
|
||||
def test_func():
|
||||
[mod] = parse("""
|
||||
odoo.define('module', function (d) {
|
||||
/**
|
||||
* @param {Foo} bar this is a bar
|
||||
* @param {Baz} qux this is a qux
|
||||
*/
|
||||
return function (bar, qux) {
|
||||
return 42;
|
||||
}
|
||||
});
|
||||
""")
|
||||
exports = mod.exports
|
||||
assert type(exports) == jsdoc.FunctionDoc
|
||||
assert exports['sourcemodule'] is mod
|
||||
|
||||
assert exports.name == ''
|
||||
assert exports.is_constructor == False
|
||||
assert exports.is_private == False
|
||||
|
||||
assert params(exports.params[0]) == ('bar', 'Foo', 'this is a bar')
|
||||
assert params(exports.params[1]) == ('qux', 'Baz', 'this is a qux')
|
||||
assert params(exports.return_val) == ('', '', '')
|
||||
|
||||
def test_hoist():
|
||||
[mod] = parse("""
|
||||
odoo.define('module', function() {
|
||||
return foo;
|
||||
/**
|
||||
* @param a_thing
|
||||
*/
|
||||
function foo(a_thing) {
|
||||
return 42;
|
||||
}
|
||||
});
|
||||
""")
|
||||
actual = mod.exports
|
||||
assert type(actual) == jsdoc.FunctionDoc
|
||||
[param] = actual.params
|
||||
assert params(param) == ('a_thing', '', '')
|
||||
|
||||
def test_export_instance():
|
||||
[mod] = parse("""
|
||||
odoo.define('module', function (require) {
|
||||
var Class = require('Class');
|
||||
/**
|
||||
* Provides an instance of Class
|
||||
*/
|
||||
return new Class();
|
||||
});
|
||||
""")
|
||||
assert type(mod.exports) == jsdoc.InstanceDoc
|
||||
assert mod.exports.doc == 'Provides an instance of Class'
|
||||
assert mod.exports['sourcemodule'] is mod
|
||||
|
||||
def test_bounce():
|
||||
[m2, m1] = parse("""
|
||||
odoo.define('m2', function (require) {
|
||||
var Item = require('m1');
|
||||
return {
|
||||
Item: Item
|
||||
};
|
||||
});
|
||||
odoo.define('m1', function (require) {
|
||||
var Class = require('Class');
|
||||
var Item = Class.extend({});
|
||||
return Item;
|
||||
});
|
||||
""")
|
||||
assert type(m2.exports) == jsdoc.NSDoc
|
||||
it = m2.exports.get_property('Item')
|
||||
assert type(it) == jsdoc.ClassDoc
|
||||
assert it['sourcemodule'] is m1
|
||||
assert sorted([n for n, _ in m1.properties]) == ['<exports>', 'Class', 'Item']
|
||||
|
||||
def test_reassign():
|
||||
[m] = parse("""
|
||||
odoo.define('m', function (require) {
|
||||
var Class = require('Class');
|
||||
/** local class */
|
||||
var Class = Class.extend({});
|
||||
return Class
|
||||
});
|
||||
""")
|
||||
assert m.exports.doc == 'local class'
|
||||
# can't use equality or identity so use class comment...
|
||||
assert m.exports.superclass.doc == 'Base Class'
|
||||
|
||||
def test_attr():
|
||||
[m1, m2] = parse("""
|
||||
odoo.define('m1', function (require) {
|
||||
var Class = require('Class');
|
||||
var Item = Class.extend({});
|
||||
return {Item: Item};
|
||||
});
|
||||
odoo.define('m2', function (require) {
|
||||
var Item = require('m1').Item;
|
||||
Item.include({});
|
||||
return Item.extend({});
|
||||
});
|
||||
""")
|
||||
assert type(m2.exports) == jsdoc.ClassDoc
|
||||
# that these two are resolved separately may be an issue at one point (?)
|
||||
assert m2.exports.superclass.to_dict() == m1.exports.get_property('Item').to_dict()
|
||||
|
||||
def test_nothing_implicit():
|
||||
[m] = parse("""
|
||||
odoo.define('m', function () {
|
||||
});
|
||||
""")
|
||||
assert m.exports is None
|
||||
|
||||
def test_nothing_explicit():
|
||||
[m] = parse("""
|
||||
odoo.define('m', function () {
|
||||
return;
|
||||
});
|
||||
""")
|
||||
assert m.exports is None
|
||||
177
_extensions/autojsdoc/parser/tests/test_namespace.py
Normal file
177
_extensions/autojsdoc/parser/tests/test_namespace.py
Normal file
@ -0,0 +1,177 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from autojsdoc.parser import jsdoc
|
||||
from support import parse, params
|
||||
|
||||
|
||||
def test_empty():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.ns', function (r) {
|
||||
return {};
|
||||
});
|
||||
""")
|
||||
assert type(mod.exports) == jsdoc.NSDoc
|
||||
assert mod.exports.properties == []
|
||||
|
||||
def test_inline():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.ns', function (r) {
|
||||
return {
|
||||
/**
|
||||
* a thing
|
||||
* @type {Boolean}
|
||||
*/
|
||||
a: true
|
||||
}
|
||||
});
|
||||
""")
|
||||
assert isinstance(mod.exports, jsdoc.NSDoc)
|
||||
[(n, p)] = mod.exports.properties
|
||||
assert n == 'a'
|
||||
assert params(p) == ('a', 'Boolean', 'a thing')
|
||||
|
||||
def test_header():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.ns', function (r) {
|
||||
/**
|
||||
* @property {Boolean} a a thing
|
||||
*/
|
||||
return { a: true }
|
||||
});
|
||||
""")
|
||||
assert isinstance(mod.exports, jsdoc.NSDoc)
|
||||
[(_, p)] = mod.exports.properties
|
||||
assert params(p) == ('a', 'Boolean', 'a thing')
|
||||
|
||||
def test_header_conflict():
|
||||
""" should the header or the inline comment take precedence? """
|
||||
[mod] = parse("""
|
||||
odoo.define('a.ns', function (r) {
|
||||
/**
|
||||
* @property {Boolean} a a thing
|
||||
*/
|
||||
return {
|
||||
/** @type {String} */
|
||||
a: true
|
||||
}
|
||||
});
|
||||
""")
|
||||
assert isinstance(mod.exports, jsdoc.NSDoc)
|
||||
[(_, p)] = mod.exports.properties
|
||||
assert params(p) == ('a', 'Boolean', 'a thing')
|
||||
|
||||
def test_mixin():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.mixin', function (r) {
|
||||
/**
|
||||
* @mixin
|
||||
*/
|
||||
return {
|
||||
/**
|
||||
* @returns {Number} a number
|
||||
*/
|
||||
do_thing: function other() { return 42; }
|
||||
}
|
||||
});
|
||||
""")
|
||||
assert isinstance(mod.exports, jsdoc.MixinDoc)
|
||||
[(n, p)] = mod.exports.properties
|
||||
assert n == 'do_thing'
|
||||
assert params(p) == ('other', 'Function', '')
|
||||
assert params(p.return_val) == ('', 'Number', 'a number')
|
||||
|
||||
def test_literal():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.ns', function (r) {
|
||||
/** whop whop */
|
||||
return {
|
||||
'a': 1,
|
||||
/** wheee */
|
||||
'b': 2,
|
||||
};
|
||||
});
|
||||
""")
|
||||
assert mod.exports.doc == 'whop whop'
|
||||
[(_1, a), (_2, b)] = mod.exports.properties
|
||||
assert params(a) == ('a', 'Number', '')
|
||||
assert params(b) == ('b', 'Number', 'wheee')
|
||||
|
||||
def test_fill_ns():
|
||||
[mod] = parse("""
|
||||
odoo.define('a.ns', function (r) {
|
||||
var Class = r('Class');
|
||||
var ns = {};
|
||||
/** ok */
|
||||
ns.a = 1;
|
||||
/** @type {String} */
|
||||
ns['b'] = 2;
|
||||
/** Ike */
|
||||
ns.c = Class.extend({});
|
||||
ns.d = function () {}
|
||||
return ns;
|
||||
});
|
||||
""")
|
||||
ns = mod.exports
|
||||
assert type(ns) == jsdoc.NSDoc
|
||||
[(_a, a), (_b, b), (_c, c), (_d, d)] = ns.properties
|
||||
assert params(a) == ('a', 'Number', 'ok')
|
||||
assert params(b) == ('b', 'String', '')
|
||||
assert type(c) == jsdoc.ClassDoc
|
||||
assert type(d) == jsdoc.FunctionDoc
|
||||
|
||||
def test_extend_other():
|
||||
[o, b] = parse("""
|
||||
odoo.define('a.ns', function () {
|
||||
/** @name outer */
|
||||
return {
|
||||
/** @name inner */
|
||||
a: {}
|
||||
};
|
||||
});
|
||||
odoo.define('b', function (r) {
|
||||
var o = r('a.ns');
|
||||
var Class = r('Class');
|
||||
/** Class 1 */
|
||||
o.a.b = Class.extend({m_b: 1});
|
||||
/** Class 2 */
|
||||
o.a['c'] = Class.extend({m_c: 1});
|
||||
});
|
||||
""")
|
||||
[(_, m)] = o.exports.properties
|
||||
assert type(m) == jsdoc.NSDoc
|
||||
|
||||
b = m.get_property('b')
|
||||
assert type(b) == jsdoc.ClassDoc
|
||||
assert b.get_property('m_b')
|
||||
assert b.doc == 'Class 1'
|
||||
|
||||
c = m.get_property('c')
|
||||
assert type(c) == jsdoc.ClassDoc
|
||||
assert c.get_property('m_c')
|
||||
assert c.doc == 'Class 2'
|
||||
|
||||
def test_ns_variables():
|
||||
[mod] = parse("""
|
||||
odoo.define('A', function (r) {
|
||||
var Class = r('Class');
|
||||
var Thing = Class.extend({});
|
||||
return {
|
||||
Thing: Thing
|
||||
};
|
||||
});
|
||||
""")
|
||||
p = mod.exports.get_property('Thing')
|
||||
assert type(p) == jsdoc.ClassDoc
|
||||
|
||||
def test_diff():
|
||||
""" Have the NS key and the underlying object differ
|
||||
"""
|
||||
[mod] = parse("""
|
||||
odoo.define('mod', function (r) {
|
||||
var Class = r('Class');
|
||||
var Foo = Class.extend({});
|
||||
return { Class: Foo };
|
||||
});
|
||||
""")
|
||||
c = mod.exports.get_property('Class')
|
||||
assert type(c) == jsdoc.ClassDoc
|
||||
assert c.name == 'Foo'
|
||||
116
_extensions/autojsdoc/parser/tests/test_params.py
Normal file
116
_extensions/autojsdoc/parser/tests/test_params.py
Normal file
@ -0,0 +1,116 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from autojsdoc.parser.jsdoc import ParamDoc
|
||||
|
||||
"Lorem ipsum dolor sit amet, consectetur adipiscing elit."
|
||||
|
||||
def test_basic():
|
||||
d = ParamDoc("Lorem ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': 'Lorem',
|
||||
'type': '',
|
||||
'optional': False,
|
||||
'default': None,
|
||||
'doc': 'ipsum dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
d = ParamDoc("{Lorem} ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': 'ipsum',
|
||||
'type': 'Lorem',
|
||||
'optional': False,
|
||||
'default': None,
|
||||
'doc': 'dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
def test_optional():
|
||||
d = ParamDoc("[Lorem] ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': 'Lorem',
|
||||
'type': '',
|
||||
'optional': True,
|
||||
'default': None,
|
||||
'doc': 'ipsum dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
d = ParamDoc("[Lorem=42] ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': 'Lorem',
|
||||
'type': '',
|
||||
'optional': True,
|
||||
'default': "42",
|
||||
'doc': 'ipsum dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
d = ParamDoc("{Lorem} [ipsum] dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': 'ipsum',
|
||||
'type': 'Lorem',
|
||||
'optional': True,
|
||||
'default': None,
|
||||
'doc': 'dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
d = ParamDoc("{Lorem} [ipsum=42] dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': 'ipsum',
|
||||
'type': 'Lorem',
|
||||
'optional': True,
|
||||
'default': '42',
|
||||
'doc': 'dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
def test_returns():
|
||||
d = ParamDoc("{} ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': '',
|
||||
'type': '',
|
||||
'optional': False,
|
||||
'default': None,
|
||||
'doc': 'ipsum dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
d = ParamDoc("{Lorem} ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d == {
|
||||
'name': '',
|
||||
'type': 'Lorem',
|
||||
'optional': False,
|
||||
'default': None,
|
||||
'doc': 'ipsum dolor sit amet, consectetur adipiscing elit.',
|
||||
}
|
||||
|
||||
def test_odd():
|
||||
d = ParamDoc("{jQuery} [$target] the node where content will be prepended").to_dict()
|
||||
assert d == {
|
||||
'name': '$target',
|
||||
'type': 'jQuery',
|
||||
'optional': True,
|
||||
'default': None,
|
||||
'doc': 'the node where content will be prepended',
|
||||
}
|
||||
|
||||
d = ParamDoc("""{htmlString} [content] DOM element,
|
||||
array of elements, HTML string or jQuery object to prepend to $target""").to_dict()
|
||||
assert d == {
|
||||
'name': 'content',
|
||||
'type': 'htmlString',
|
||||
'optional': True,
|
||||
'default': None,
|
||||
'doc': "DOM element,\n array of elements, HTML string or jQuery object to prepend to $target",
|
||||
}
|
||||
|
||||
d = ParamDoc("{Boolean} [options.in_DOM] true if $target is in the DOM").to_dict()
|
||||
assert d == {
|
||||
'name': 'options.in_DOM',
|
||||
'type': 'Boolean',
|
||||
'optional': True,
|
||||
'default': None,
|
||||
'doc': 'true if $target is in the DOM',
|
||||
}
|
||||
|
||||
d = ParamDoc("Lorem\n ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d['doc'] == 'ipsum dolor sit amet, consectetur adipiscing elit.'
|
||||
|
||||
d = ParamDoc("Lorem - ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d['doc'] == 'ipsum dolor sit amet, consectetur adipiscing elit.'
|
||||
|
||||
d = ParamDoc("Lorem : ipsum dolor sit amet, consectetur adipiscing elit.").to_dict()
|
||||
assert d['doc'] == 'ipsum dolor sit amet, consectetur adipiscing elit.'
|
||||
104
_extensions/autojsdoc/parser/tests/test_typespec.py
Normal file
104
_extensions/autojsdoc/parser/tests/test_typespec.py
Normal file
@ -0,0 +1,104 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import pytest
|
||||
|
||||
from autojsdoc.parser import types
|
||||
|
||||
def test_parser():
|
||||
assert types.parse("MouseEvent|TouchEvent") == types.Alt([
|
||||
types.Type('MouseEvent', []),
|
||||
types.Type('TouchEvent', []),
|
||||
])
|
||||
|
||||
assert types.parse('Deferred<Object>') == types.Alt([
|
||||
types.Type('Deferred', [
|
||||
types.Alt([types.Type('Object', [])])
|
||||
])
|
||||
])
|
||||
|
||||
assert types.parse('Object|undefined') == types.Alt([
|
||||
types.Type('Object', []),
|
||||
types.Literal('undefined'),
|
||||
])
|
||||
assert types.parse('any[]') == types.Alt([
|
||||
types.Type('Array', [types.Type('any', [])])
|
||||
])
|
||||
assert types.parse("'xml' | 'less'") == types.Alt([
|
||||
types.Literal('xml'),
|
||||
types.Literal('less'),
|
||||
])
|
||||
assert types.parse('Foo<Bar | Bar[] | null>') == types.Alt([
|
||||
types.Type('Foo', [types.Alt([
|
||||
types.Type('Bar', []),
|
||||
types.Type('Array', [types.Type('Bar', [])]),
|
||||
types.Literal('null'),
|
||||
])])
|
||||
])
|
||||
assert types.parse('Function<Array<Object[]>>') == types.Alt([
|
||||
types.Type('Function', [types.Alt([
|
||||
types.Type('Array', [types.Alt([
|
||||
types.Type('Array', [
|
||||
types.Type('Object', [])
|
||||
])
|
||||
])])
|
||||
])])
|
||||
])
|
||||
|
||||
|
||||
def test_tokens():
|
||||
toks = list(types.tokenize('A'))
|
||||
assert toks == [(types.NAME, 'A')]
|
||||
|
||||
toks = list(types.tokenize('"foo" | "bar"'))
|
||||
assert toks == [(types.LITERAL, 'foo'), (types.OP, '|'), (types.LITERAL, 'bar')]
|
||||
|
||||
toks = list(types.tokenize('1 or 2'))
|
||||
assert toks == [(types.LITERAL, 1), (types.OP, '|'), (types.LITERAL, 2)]
|
||||
|
||||
toks = list(types.tokenize('a.b.c.d'))
|
||||
assert toks == [
|
||||
(types.NAME, 'a'), (types.OP, '.'),
|
||||
(types.NAME, 'b'), (types.OP, '.'),
|
||||
(types.NAME, 'c'), (types.OP, '.'),
|
||||
(types.NAME, 'd'),
|
||||
]
|
||||
|
||||
toks = list(types.tokenize('Function<String, Object>'))
|
||||
assert toks == [
|
||||
(types.NAME, 'Function'),
|
||||
(types.OP, '<'),
|
||||
(types.NAME, 'String'),
|
||||
(types.OP, ','),
|
||||
(types.NAME, 'Object'),
|
||||
(types.OP, '>')
|
||||
]
|
||||
|
||||
toks = list(types.tokenize('Function<Array<Object[]>>'))
|
||||
assert toks == [
|
||||
(types.NAME, 'Function'),
|
||||
(types.OP, '<'),
|
||||
(types.NAME, 'Array'),
|
||||
(types.OP, '<'),
|
||||
(types.NAME, 'Object'),
|
||||
(types.OP, '['),
|
||||
(types.OP, ']'),
|
||||
(types.OP, '>'),
|
||||
(types.OP, '>')
|
||||
]
|
||||
|
||||
def test_peekable():
|
||||
p = types.Peekable(range(5))
|
||||
|
||||
assert p.peek() == 0
|
||||
assert next(p) == 0, "consuming should yield previously peeked value"
|
||||
next(p)
|
||||
next(p)
|
||||
assert next(p) == 3
|
||||
assert p.peek() == 4
|
||||
next(p)
|
||||
assert p.peek(None) is None
|
||||
with pytest.raises(StopIteration):
|
||||
p.peek()
|
||||
with pytest.raises(StopIteration):
|
||||
next(p)
|
||||
196
_extensions/autojsdoc/parser/types.py
Normal file
196
_extensions/autojsdoc/parser/types.py
Normal file
@ -0,0 +1,196 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
import ast
|
||||
import collections
|
||||
import io
|
||||
import token
|
||||
from tokenize import generate_tokens
|
||||
|
||||
LITERAL_NAME = ('null', 'undefined', 'true', 'false')
|
||||
|
||||
__all__ = ['tokenize', 'parse', 'iterate', 'Alt', 'Type', 'Literal']
|
||||
|
||||
class _Marker(object):
|
||||
__slots__ = ['name']
|
||||
def __init__(self, name): self.name = name
|
||||
def __repr__(self): return '<%s>' % self.name
|
||||
|
||||
OP = _Marker('OP')
|
||||
NAME = _Marker('NAME')
|
||||
LITERAL = _Marker('LITERAL')
|
||||
def tokenize(typespec):
|
||||
typespec = typespec.replace('$', 'jQuery')
|
||||
for toktype, string, _, _, _ in generate_tokens(io.StringIO(typespec).readline):
|
||||
if toktype == token.NAME:
|
||||
if string == 'or': # special case "A or B"
|
||||
# TODO: deprecation warning
|
||||
yield (OP, '|')
|
||||
else:
|
||||
yield (NAME, string)
|
||||
elif toktype == token.OP:
|
||||
if string in '|<>[].,':
|
||||
yield (OP, string)
|
||||
elif string == '>>':
|
||||
yield (OP, '>')
|
||||
yield (OP, '>')
|
||||
elif string == '*': # maybe?
|
||||
yield (NAME, 'any')
|
||||
elif string in '()':
|
||||
# TODO: deprecation warning
|
||||
# seems useless unless we add support for "..."
|
||||
continue
|
||||
else:
|
||||
raise ValueError("Unknown typespec operator %r" % string)
|
||||
elif toktype in (token.STRING, token.NUMBER): # enum-ish
|
||||
yield (LITERAL, ast.literal_eval(string))
|
||||
elif toktype == token.ENDMARKER:
|
||||
return
|
||||
elif toktype == token.NEWLINE:
|
||||
pass
|
||||
else:
|
||||
raise ValueError("Unknown typespec token %s" % token.tok_name[toktype])
|
||||
|
||||
Alt = collections.namedtuple('Alt', ['params'])
|
||||
Literal = collections.namedtuple('Literal', ['value'])
|
||||
Type = collections.namedtuple('Type', ['name', 'params'])
|
||||
|
||||
def iterate(t, fortype, forliteral):
|
||||
if isinstance(t, Alt):
|
||||
ps = Peekable(t.params)
|
||||
for param in ps:
|
||||
for it in iterate(param, fortype, forliteral):
|
||||
yield it
|
||||
if ps.peek(None):
|
||||
yield forliteral('|')
|
||||
elif isinstance(t, Type):
|
||||
yield fortype(t.name)
|
||||
if not t.params:
|
||||
return
|
||||
yield forliteral('<')
|
||||
ps = Peekable(t.params)
|
||||
for param in ps:
|
||||
for it in iterate(param, fortype, forliteral):
|
||||
yield it
|
||||
if ps.peek(None):
|
||||
yield forliteral(',')
|
||||
yield forliteral('>')
|
||||
elif isinstance(t, Literal):
|
||||
yield forliteral(t.value)
|
||||
else:
|
||||
raise ValueError('Unknown item %s' % t)
|
||||
|
||||
def parse(typespec):
|
||||
tokens = Peekable(tokenize(typespec))
|
||||
return parse_alt(tokens)
|
||||
|
||||
# alt = param
|
||||
# | param '|' alt
|
||||
def parse_alt(tokens):
|
||||
alt = Alt([])
|
||||
while True:
|
||||
alt.params.append(parse_param(tokens))
|
||||
if not next_is(tokens, (OP, '|')):
|
||||
break
|
||||
return alt
|
||||
|
||||
# param = 'null' | 'undefined' | 'true' | 'false'
|
||||
# | number
|
||||
# | string
|
||||
# | typename '[]'
|
||||
# X '[' alt ']'
|
||||
# | typename '<' params '>'
|
||||
# params = alt | alt ',' params
|
||||
def parse_param(tokens):
|
||||
t, v = tokens.peek()
|
||||
if t == LITERAL or (t == NAME and v in LITERAL_NAME):
|
||||
next(tokens)
|
||||
return Literal(str(v))
|
||||
|
||||
# # [typespec] # should this be Array<T> or an n-uple allowing multiple items?
|
||||
# if tok == (OP, '['):
|
||||
# next(tokens)
|
||||
# t = Type('Array', [parse_alt(tokens)])
|
||||
# rest = next(tokens)
|
||||
# expect(rest, (OP, ']'))
|
||||
# return t
|
||||
|
||||
t = Type(parse_typename(tokens), [])
|
||||
# type[]
|
||||
peek = tokens.peek(None)
|
||||
if peek == (OP, '['):
|
||||
next(tokens) # skip peeked
|
||||
expect(next(tokens), (OP, ']'))
|
||||
return Type('Array', [t])
|
||||
# type<typespec, ...>
|
||||
if peek == (OP, '<'):
|
||||
next(tokens) # skip peeked
|
||||
while True:
|
||||
t.params.append(parse_alt(tokens))
|
||||
n = next(tokens)
|
||||
if n == (OP, ','):
|
||||
continue
|
||||
if n == (OP, '>'):
|
||||
break
|
||||
raise ValueError("Expected OP ',' or OP ',', got %s '%s'" % n)
|
||||
return t
|
||||
|
||||
# typename = name | name '.' typename
|
||||
def parse_typename(tokens):
|
||||
typename = []
|
||||
while True:
|
||||
(t, n) = next(tokens)
|
||||
if t != NAME:
|
||||
raise ValueError("Expected a name token, got %s '%s'" % (t, n))
|
||||
if n in LITERAL_NAME:
|
||||
raise ValueError("Expected a type name, got literal %s" % n)
|
||||
typename.append(n)
|
||||
if not next_is(tokens, (OP, '.')):
|
||||
break
|
||||
return '.'.join(typename)
|
||||
|
||||
def next_is(tokens, expected):
|
||||
"""
|
||||
Consumes the next token if it's `expected` otherwise does not touch the
|
||||
tokens.
|
||||
|
||||
Returns whether it consumed a token
|
||||
"""
|
||||
if tokens.peek(None) == expected:
|
||||
next(tokens)
|
||||
return True
|
||||
return False
|
||||
|
||||
def expect(actual, expected):
|
||||
""" Raises ValueError if `actual` and `expected` are different
|
||||
|
||||
:type actual: (object, str)
|
||||
:type expected: (object, str)
|
||||
"""
|
||||
if actual != expected:
|
||||
raise ValueError("Expected %s '%s', got %s '%s'" % (expected + actual))
|
||||
|
||||
|
||||
NONE = object()
|
||||
class Peekable(object):
|
||||
__slots__ = ['_head', '_it']
|
||||
def __init__(self, iterable):
|
||||
self._head = NONE
|
||||
self._it = iter(iterable)
|
||||
|
||||
def __iter__(self):
|
||||
return self
|
||||
def __next__(self):
|
||||
if self._head is not NONE:
|
||||
r, self._head = self._head, NONE
|
||||
return r
|
||||
return next(self._it)
|
||||
next = __next__
|
||||
|
||||
def peek(self, default=NONE):
|
||||
if self._head is NONE:
|
||||
try:
|
||||
self._head = next(self._it)
|
||||
except StopIteration:
|
||||
if default is not NONE:
|
||||
return default
|
||||
raise
|
||||
return self._head
|
||||
295
_extensions/autojsdoc/parser/utils.py
Normal file
295
_extensions/autojsdoc/parser/utils.py
Normal file
@ -0,0 +1,295 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
def _name(node):
|
||||
if node['type'] == 'Identifier':
|
||||
return node['name']
|
||||
if node['type'] == 'MemberExpression':
|
||||
return "%s.%s" % (_name(node['object']), _name(node['property']))
|
||||
raise ValueError("Unnamable node type %s" % node['type'])
|
||||
|
||||
def _value(node, strict=False):
|
||||
t = node['type']
|
||||
if t == 'Identifier':
|
||||
return node['name']
|
||||
elif t == 'Literal':
|
||||
return node['value']
|
||||
msg = '<%s has no value>' % _identify(node, {})
|
||||
if strict:
|
||||
raise ValueError(msg)
|
||||
return msg
|
||||
|
||||
def _identify(valnode, ns):
|
||||
if valnode is None:
|
||||
return "None"
|
||||
# already identified and re-set in the ns?
|
||||
if isinstance(valnode, Printable):
|
||||
return valnode
|
||||
|
||||
# check other non-empty returns
|
||||
t = valnode['type']
|
||||
if t == "Literal":
|
||||
if valnode.get('regex'):
|
||||
return valnode['regex']['pattern']
|
||||
return valnode['value']
|
||||
elif t == "Identifier":
|
||||
n = valnode['name']
|
||||
return ns.get(n, Global(n))
|
||||
elif t == "NewExpression":
|
||||
return Instance(_identify(valnode['callee'], ns))
|
||||
elif t == "ObjectExpression":
|
||||
return Namespace({
|
||||
_value(prop['key']): _identify(prop['value'], ns)
|
||||
for prop in valnode['properties']
|
||||
})
|
||||
elif t == "MemberExpression":
|
||||
return Deref(
|
||||
_identify(valnode['object'], ns),
|
||||
_identify(valnode['property'], ns) if valnode['computed'] else valnode['property']['name'],
|
||||
)
|
||||
elif t == "CallExpression":
|
||||
return Call(
|
||||
_identify(valnode['callee'], ns),
|
||||
[_identify(arg, ns) for arg in valnode['arguments']],
|
||||
)
|
||||
elif t == 'BinaryExpression':
|
||||
return "%s %s %s" % (
|
||||
_identify(valnode['left'], ns),
|
||||
valnode['operator'],
|
||||
_identify(valnode['right'], ns),
|
||||
)
|
||||
else:
|
||||
return t
|
||||
|
||||
class Counter(object):
|
||||
__slots__ = ['_number']
|
||||
def __init__(self):
|
||||
self._number = 0
|
||||
|
||||
def __bool__(self):
|
||||
return bool(self._number)
|
||||
__nonzero__ = __bool__
|
||||
def __int__(self):
|
||||
return self._number
|
||||
|
||||
def get(self):
|
||||
return self._number
|
||||
def increment(self):
|
||||
self._number += 1
|
||||
return self._number
|
||||
def decrement(self):
|
||||
self._number -= 1
|
||||
return self._number
|
||||
|
||||
def __enter__(self):
|
||||
self._number += 1
|
||||
return self._number
|
||||
def __exit__(self, *args):
|
||||
self._number -= 1
|
||||
|
||||
class Printable(object):
|
||||
__slots__ = []
|
||||
depth = Counter()
|
||||
def __repr__(self):
|
||||
with self.depth as i:
|
||||
if i > 2:
|
||||
return "%s(...)" % type(self).__name__
|
||||
return "%s(%s)" % (type(self).__name__, ', '.join(
|
||||
"%s=%r" % (k, getattr(self, k))
|
||||
for k in self.__slots__
|
||||
if not k.startswith('_')
|
||||
if getattr(self, k)
|
||||
))
|
||||
|
||||
|
||||
def resolve(obj, store):
|
||||
if obj is None:
|
||||
return None
|
||||
|
||||
if isinstance(obj, (type(u''), bool, int, float)):
|
||||
return obj
|
||||
|
||||
if isinstance(obj, ModuleProxy):
|
||||
return resolve(obj.get(), store)
|
||||
|
||||
try:
|
||||
if getattr(obj, 'resolved', False):
|
||||
return obj
|
||||
return obj.resolve(store)
|
||||
except AttributeError:
|
||||
raise TypeError("Unresolvable {!r}".format(obj))
|
||||
|
||||
class Resolvable(Printable):
|
||||
"""
|
||||
For types resolved in place
|
||||
"""
|
||||
__slots__ = ['resolved']
|
||||
|
||||
def __init__(self):
|
||||
super(Resolvable, self).__init__()
|
||||
self.resolved = False
|
||||
def resolve(self, store):
|
||||
self.resolved = True
|
||||
return self
|
||||
|
||||
class Namespace(Resolvable):
|
||||
__slots__ = ['attrs']
|
||||
def __init__(self, attrs):
|
||||
super(Namespace, self).__init__()
|
||||
self.attrs = attrs
|
||||
def resolve(self, store):
|
||||
r = super(Namespace, self).resolve(store)
|
||||
self.attrs = {
|
||||
k: resolve(v, store)
|
||||
for k, v in self.attrs.items()
|
||||
}
|
||||
return r
|
||||
def __getitem__(self, key):
|
||||
assert isinstance(key, type(u'')), "%r is not a namespace key" % key
|
||||
try:
|
||||
return self.attrs[key]
|
||||
except KeyError:
|
||||
return Global(self)[key]
|
||||
|
||||
class Module(Resolvable):
|
||||
__slots__ = ('name', 'comments', 'exports', 'augments', 'dependencies', '_store')
|
||||
def __init__(self, name, store, comments=()):
|
||||
super(Module, self).__init__()
|
||||
self.name = name
|
||||
self._store = store
|
||||
self.comments = tuple(comments)
|
||||
self.exports = None
|
||||
self.augments = []
|
||||
self.dependencies = set()
|
||||
store[name] = self
|
||||
def add_dependency(self, depname):
|
||||
dep = ModuleProxy(depname, self._store)
|
||||
self.dependencies.add(dep)
|
||||
return dep
|
||||
def get(self):
|
||||
return self
|
||||
def resolve(self, store=None):
|
||||
r = super(Module, self).resolve(store)
|
||||
self.exports = resolve(self.exports, self._store)
|
||||
self.augments = [resolve(a, self._store) for a in self.augments]
|
||||
self.dependencies = {
|
||||
resolve(d, self._store)
|
||||
for d in self.dependencies
|
||||
}
|
||||
return r
|
||||
def __getitem__(self, k):
|
||||
try:
|
||||
return self.exports[k]
|
||||
except KeyError:
|
||||
return Global(self)[k]
|
||||
|
||||
class ModuleProxy(object):
|
||||
def __init__(self, name, store):
|
||||
self.name = name
|
||||
self.store = store
|
||||
def get(self):
|
||||
# web.web_client is not an actual module
|
||||
return self.store.get(self.name, '<unstored %s>' % self.name)
|
||||
def __repr__(self):
|
||||
return repr(self.get())
|
||||
def __hash__(self):
|
||||
return hash(self.name)
|
||||
def __eq__(self, other):
|
||||
return self.name == other.name
|
||||
|
||||
class Class(Resolvable):
|
||||
__slots__ = ['name', 'members', 'extends', 'mixins', 'comments']
|
||||
def __init__(self, name, extends):
|
||||
super(Class, self).__init__()
|
||||
self.name = name
|
||||
self.extends = extends
|
||||
self.members = {}
|
||||
self.mixins = []
|
||||
self.comments = []
|
||||
|
||||
def resolve(self, store):
|
||||
r = super(Class, self).resolve(store)
|
||||
self.extends = resolve(self.extends, store)
|
||||
self.mixins = [resolve(m, store) for m in self.mixins]
|
||||
return r
|
||||
|
||||
class Instance(Resolvable):
|
||||
__slots__ = ['type']
|
||||
def __init__(self, type):
|
||||
super(Instance, self).__init__()
|
||||
self.type = type
|
||||
|
||||
def resolve(self, store):
|
||||
r = super(Instance, self).resolve(store)
|
||||
self.type = resolve(self.type, store)
|
||||
if isinstance(self.type, Module):
|
||||
self.type = self.type.exports
|
||||
return r
|
||||
|
||||
def __getitem__(self, key):
|
||||
return Global(self)['key']
|
||||
|
||||
class Function(Resolvable):
|
||||
__slots__ = ['comments']
|
||||
def __init__(self, comments):
|
||||
super(Function, self).__init__()
|
||||
self.comments = comments
|
||||
|
||||
class Deref(Printable):
|
||||
__slots__ = ['object', 'property']
|
||||
def __init__(self, object, property):
|
||||
self.object = object
|
||||
self.property = property
|
||||
def resolve(self, store):
|
||||
return resolve(self.object, store)[self.property]
|
||||
|
||||
class Call(Resolvable):
|
||||
__slots__ = ['callable']
|
||||
def __init__(self, callable, arguments):
|
||||
super(Call, self).__init__()
|
||||
self.callable = callable
|
||||
def resolve(self, store):
|
||||
r = super(Call, self).resolve(store)
|
||||
self.callable = resolve(self.callable, store)
|
||||
return r
|
||||
def __getitem__(self, item):
|
||||
return Global(self)[item]
|
||||
|
||||
def get(node, it, *path):
|
||||
if not path:
|
||||
return node[it]
|
||||
return get(node[it], *path)
|
||||
|
||||
|
||||
def match(node, pattern):
|
||||
"""Checks that ``pattern`` is a subset of ``node``.
|
||||
|
||||
If a sub-pattern is a callable it will be called with the current
|
||||
sub-node and its result is the match's. Other sub-patterns are
|
||||
checked by equality against the correspoding sub-node.
|
||||
|
||||
Can be used to quickly check the descendants of a node of suitable
|
||||
type.
|
||||
|
||||
TODO: support checking list content? Would be convenient to
|
||||
constraint e.g. CallExpression based on their parameters
|
||||
|
||||
"""
|
||||
if node is None and pattern is not None:
|
||||
return
|
||||
if callable(pattern):
|
||||
return pattern(node)
|
||||
if isinstance(node, list):
|
||||
return all(len(node) > i and match(node[i], v) for i, v in pattern.items())
|
||||
if isinstance(pattern, dict):
|
||||
return all(match(node.get(k), v) for k, v in pattern.items())
|
||||
return node == pattern
|
||||
|
||||
|
||||
class Global(object):
|
||||
def __init__(self, name):
|
||||
self.name = name
|
||||
def __repr__(self):
|
||||
return '<external %s>' % self.name
|
||||
def __getitem__(self, name):
|
||||
return Global('%s.%s' % (self.name, name))
|
||||
def resolve(self, store):
|
||||
return self
|
||||
126
_extensions/autojsdoc/parser/visitor.py
Normal file
126
_extensions/autojsdoc/parser/visitor.py
Normal file
@ -0,0 +1,126 @@
|
||||
from pyjsparser.pyjsparserdata import Syntax
|
||||
|
||||
_binary = lambda n: [n['left'], n['right']]
|
||||
_function = lambda n: n['params'] + n['defaults'] + [n['body']]
|
||||
# yields children in visitation order
|
||||
_children = {
|
||||
Syntax.ArrayExpression: lambda n: n['elements'],
|
||||
Syntax.ArrayPattern: lambda n: n['elements'],
|
||||
Syntax.ArrowFunctionExpression: _function,
|
||||
Syntax.AssignmentExpression: _binary,
|
||||
Syntax.AssignmentPattern: _binary,
|
||||
Syntax.BinaryExpression: _binary,
|
||||
Syntax.BlockStatement: lambda n: n['body'],
|
||||
Syntax.BreakStatement: lambda n: [],
|
||||
Syntax.CallExpression: lambda n: [n['callee']] + n['arguments'],
|
||||
Syntax.CatchClause: lambda n: [n['param'], n['body']],
|
||||
Syntax.ClassBody: lambda n: [n['body']],
|
||||
Syntax.ClassDeclaration: lambda n: [n['superClass'], n['body']],
|
||||
Syntax.ClassExpression: lambda n: [n['superClass'], n['body']],
|
||||
Syntax.ConditionalExpression: lambda n: [n['test'], n['consequent'], n['alternate']],
|
||||
Syntax.ContinueStatement: lambda n: [],
|
||||
Syntax.DebuggerStatement: lambda n: [],
|
||||
Syntax.DoWhileStatement: lambda n: [n['body'], n['test']],
|
||||
Syntax.EmptyStatement: lambda n: [],
|
||||
Syntax.ExportAllDeclaration: lambda n: [n['source']],
|
||||
Syntax.ExportDefaultDeclaration: lambda n: [n['declaration']],
|
||||
Syntax.ExportNamedDeclaration: lambda n: ([n['declaration']] if n['declaration'] else n['specifiers']) + [n['source']],
|
||||
Syntax.ExportSpecifier: lambda n: [n['local'], n['exported']],
|
||||
Syntax.ExpressionStatement: lambda n: [n['expression']],
|
||||
Syntax.ForStatement: lambda n: [n['init'], n['test'], n['update'], n['body']],
|
||||
Syntax.ForInStatement: lambda n: [n['left'], n['right'], n['body']],
|
||||
Syntax.FunctionDeclaration: _function,
|
||||
Syntax.FunctionExpression: _function,
|
||||
Syntax.Identifier: lambda n: [],
|
||||
Syntax.IfStatement: lambda n: [n['test'], n['consequent'], n['alternate']],
|
||||
Syntax.ImportDeclaration: lambda n: n['specifiers'] + [n['source']],
|
||||
Syntax.ImportDefaultSpecifier: lambda n: [n['local']],
|
||||
Syntax.ImportNamespaceSpecifier: lambda n: [n['local']],
|
||||
Syntax.ImportSpecifier: lambda n: [n['local'], n['imported']],
|
||||
Syntax.LabeledStatement: lambda n: [n['body']],
|
||||
Syntax.Literal: lambda n: [],
|
||||
Syntax.LogicalExpression: _binary,
|
||||
Syntax.MemberExpression: lambda n: [n['object'], n['property']],
|
||||
#Syntax.MethodDefinition: lambda n: [],
|
||||
Syntax.NewExpression: lambda n: [n['callee']] + n['arguments'],
|
||||
Syntax.ObjectExpression: lambda n: n['properties'],
|
||||
Syntax.ObjectPattern: lambda n: n['properties'],
|
||||
Syntax.Program: lambda n: n['body'],
|
||||
Syntax.Property: lambda n: [n['key'], n['value']],
|
||||
Syntax.RestElement: lambda n: [n['argument']],
|
||||
Syntax.ReturnStatement: lambda n: [n['argument']],
|
||||
Syntax.SequenceExpression: lambda n: n['expressions'],
|
||||
Syntax.SpreadElement: lambda n: [n['argument']],
|
||||
Syntax.Super: lambda n: [],
|
||||
Syntax.SwitchCase: lambda n: [n['test'], n['consequent']],
|
||||
Syntax.SwitchStatement: lambda n: [n['discriminant']] + n['cases'],
|
||||
Syntax.TaggedTemplateExpression: lambda n: [n['tag'], n['quasi']],
|
||||
Syntax.TemplateElement: lambda n: [],
|
||||
Syntax.TemplateLiteral: lambda n: n['quasis'] + n['expressions'],
|
||||
Syntax.ThisExpression: lambda n: [],
|
||||
Syntax.ThrowStatement: lambda n: [n['argument']],
|
||||
Syntax.TryStatement: lambda n: [n['block'], n['handler'], n['finalizer']],
|
||||
Syntax.UnaryExpression: lambda n: [n['argument']],
|
||||
Syntax.UpdateExpression: lambda n: [n['argument']],
|
||||
Syntax.VariableDeclaration: lambda n: n['declarations'],
|
||||
Syntax.VariableDeclarator: lambda n: [n['id'], n['init']],
|
||||
Syntax.WhileStatement: lambda n: [n['test'], n['body']],
|
||||
Syntax.WithStatement: lambda n: [n['object'], n['body']],
|
||||
}
|
||||
|
||||
SKIP = object()
|
||||
class Visitor(object):
|
||||
"""
|
||||
Generic visitor for the pyjsparser AST.
|
||||
|
||||
Visitation is driven by the ``visit`` method, which iterates the tree in
|
||||
depth-first pre-order.
|
||||
|
||||
For each node, calls ``enter_$NODETYPE``, visits the children then calls
|
||||
``exit_$NODETYPE``. If the enter or exit methods are not present on the
|
||||
visitor, falls back on ``enter_generic`` and ``exit_generic``.
|
||||
|
||||
Any ``enter_`` method can return ``SKIP`` to suppress both the traversal
|
||||
of the subtree *and* the call to the corresponding ``exit_`` method
|
||||
(whether generic or specific).
|
||||
|
||||
For convenience, ``visit`` will return whatever is set as the visitor's
|
||||
``result`` attribute, ``None`` by default.
|
||||
|
||||
``visit`` can be given multiple root nodes, and it can be called multiple
|
||||
times. The ``result`` attribute is cleared at each call but not between
|
||||
two roots of the same ``visit`` call.
|
||||
"""
|
||||
def __init__(self):
|
||||
super(Visitor, self).__init__()
|
||||
self.result = None
|
||||
|
||||
def enter_generic(self, node): pass
|
||||
def exit_generic(self, node): pass
|
||||
|
||||
def visit(self, nodes):
|
||||
if isinstance(nodes, dict):
|
||||
nodes = [nodes]
|
||||
# if multiple nodes are passed in, we need to reverse the order in
|
||||
# order to traverse front-to-back rather than the other way around
|
||||
nodes = list(reversed(nodes))
|
||||
|
||||
while nodes:
|
||||
node = nodes.pop()
|
||||
# should probably filter None descendants in _children...
|
||||
if node is None:
|
||||
continue
|
||||
node_type = node['type']
|
||||
if node_type == '_exit':
|
||||
node = node['node']
|
||||
getattr(self, 'exit_' + node['type'], self.exit_generic)(node)
|
||||
continue
|
||||
|
||||
if getattr(self, 'enter_' + node_type, self.enter_generic)(node) is SKIP:
|
||||
continue
|
||||
|
||||
nodes.append({'type': '_exit', 'node': node})
|
||||
nodes.extend(reversed(_children[node_type](node)))
|
||||
|
||||
return self.result
|
||||
|
||||
Loading…
Reference in New Issue
Block a user