5bcbbf5373
Summary: *Context:* https://github.com/pytorch/pytorch/issues/53406 added a lint for trailing whitespace at the ends of lines. However, in order to pass FB-internal lints, that PR also had to normalize the trailing newlines in four of the files it touched. This PR adds an OSS lint to normalize trailing newlines. The changes to the following files (made in 54847d0adb9be71be4979cead3d9d4c02160e4cd) are the only manually-written parts of this PR: - `.github/workflows/lint.yml` - `mypy-strict.ini` - `tools/README.md` - `tools/test/test_trailing_newlines.py` - `tools/trailing_newlines.py` I would have liked to make this just a shell one-liner like the other three similar lints, but nothing I could find quite fit the bill. Specifically, all the answers I tried from the following Stack Overflow questions were far too slow (at least a minute and a half to run on this entire repository): - [How to detect file ends in newline?](https://stackoverflow.com/q/38746) - [How do I find files that do not end with a newline/linefeed?](https://stackoverflow.com/q/4631068) - [How to list all files in the Git index without newline at end of file](https://stackoverflow.com/q/27624800) - [Linux - check if there is an empty line at the end of a file [duplicate]](https://stackoverflow.com/q/34943632) - [git ensure newline at end of each file](https://stackoverflow.com/q/57770972) To avoid giving false positives during the few days after this PR is merged, we should probably only merge it after https://github.com/pytorch/pytorch/issues/54967. Pull Request resolved: https://github.com/pytorch/pytorch/pull/54737 Test Plan: Running the shell script from the "Ensure correct trailing newlines" step in the `quick-checks` job of `.github/workflows/lint.yml` should print no output and exit in a fraction of a second with a status of 0. That was not the case prior to this PR, as shown by this failing GHA workflow run on an earlier draft of this PR: - https://github.com/pytorch/pytorch/runs/2197446987?check_suite_focus=true In contrast, this run (after correcting the trailing newlines in this PR) succeeded: - https://github.com/pytorch/pytorch/pull/54737/checks?check_run_id=2197553241 To unit-test `tools/trailing_newlines.py` itself (this is run as part of our "Test tools" GitHub Actions workflow): ``` python tools/test/test_trailing_newlines.py ``` Reviewed By: malfet Differential Revision: D27409736 Pulled By: samestep fbshipit-source-id: 46f565227046b39f68349bbd5633105b2d2e9b19
2.9 KiB
Doxygen Notes
Doxygen seems to behave better if the processing between C++ and Python was split up. This is why there are two different links to cover each API.
C++ API docs work out of the box with the Caffe2 source code. Python docs require “python blocks (http://www.stack.nl/~dimitri/doxygen/manual/docblocks.html#pythonblocks)” which are (often) currently missing in the Python code.
The Python script called “process.py” that resides in the /docs folder is to prepare the docs by looking for the block and if it doesn't exist prepend and customize the python blocks section with the module's path (e.g. Module caffe2.python.examples.char_rnn). It was noted that you need to delete the previous version of docs when you regenerate the docs or else things get messy, so the script deals with that as well.
The doxygen customization includes these files in the doxygen folder:
- header.html - logo links back to the main docs page
- footer.html - includes the Facebook OSS footer
- stylesheet.css - doxygen's default CSS; tweaked to fix formatting problems with the custom logo, header, and footer
- main.css - copied from the caffe2ai CSS, so this should be refreshed after the design changes (this overrides/extends stylesheet.css)
It also extracts info from markdown files found in the source tree. A legacy installation file was in the /docs folder and this was removed. These file show up in the top navigation under “Related Pages”.
The flow to create the API documents is simple now:
- Run /caffe2_root/docs/process.py
- (TODO automatically) Copy the doxygen-c and doxygen-python folders created by the script to the gh-pages branch.
Settings that were customized:
OPTIMIZE_OUTPUT_JAVA - turned on for Python config, off for C++ config USE_MDFILE_AS_MAINPAGE - use to flag a markdown file for the mainpage EXTRACT_ALL QUIET WARN_IF_UNDOCUMENTED FILE_PATTERNS DOT_MULTI_TARGETS = YES JAVADOC_AUTOBRIEF = YES QUIET = YES SOURCE_BROWSER = YES VERBATIM_HEADERS = NO SHOW_NAMESPACES = NO for C++ config
Not using this (was in old config file, but seems to be for Latex):
EXTRA_PACKAGES = amsmath
amsfonts
xr
NOTE / TODO:
useful for xcode, currently off GENERATE_DOCSET = NO
Look at search engine integration, xml output, etc EXTERNAL_SEARCH = YES
Other Notes
To achieve better output in the Python docs: http://stackoverflow.com/questions/7690220/how-to-document-python-function-parameter-types
Swap this kind of formatting into py files:
def my_method(x, y):"""!
my_method description
@type x: int
@param x: An integer
@type y: int|string
@param y: An integer or string
@rtype: string
@return: Returns a sentence with your variables in it
"""return "Hello World! %s, %s" % (x,y)
Note that the bang (!) is added after the opening comment """! - this seems to do the trick and the remaining comments will be nicely parsed by Doxygen.