16 Commits

Author SHA1 Message Date
e54cb03a51 add/move a few apis in torch.hub (#18758)
Summary:
* `torch.hub.list('pytorch/vision')` - show all available hub models in `pytorch/vision`
* `torch.hub.show('pytorch/vision', 'resnet18')` - show docstring & example for `resnet18` in `pytorch/vision`
* Moved `torch.utils.model_zoo.load_url` to `torch.hub.load_state_dict_from_url` and deprecate `torch.utils.model_zoo`
* We have too many env to control where the cache dir is, it's not very necessary. I actually want to unify `TORCH_HUB_DIR`, `TORCH_HOME` and `TORCH_MODEL_ZOO`, but haven't done it. (more suggestions are welcome!)
* Simplify `pytorch/vision` example in doc, it was used to show how how hub entrypoint can be written so had some confusing unnecessary args.

An example of hub usage is shown below
```

In [1]: import torch

In [2]: torch.hub.list('pytorch/vision', force_reload=True)
Downloading: "https://github.com/pytorch/vision/archive/master.zip" to /private/home/ailzhang/.torch/hub/master.zip
Out[2]: ['resnet18', 'resnet50']

In [3]: torch.hub.show('pytorch/vision', 'resnet18')
Using cache found in /private/home/ailzhang/.torch/hub/vision_master

    Resnet18 model
    pretrained (bool): a recommended kwargs for all entrypoints
    args & kwargs are arguments for the function

In [4]: model = torch.hub.load('pytorch/vision', 'resnet18', pretrained=True)
Using cache found in /private/home/ailzhang/.torch/hub/vision_master
```
Pull Request resolved: https://github.com/pytorch/pytorch/pull/18758

Differential Revision: D14883651

Pulled By: ailzhang

fbshipit-source-id: 6db6ab708a74121782a9154c44b0e190b23e8309
2019-04-10 23:10:39 -07:00
01977c0a89 Change fake tqdm constructor to match real tqdm (#17636)
Summary:
Currently, the fake tqdm implementation requires an input (whereas real tqdm does not).

This caused a problem in torchvision (https://github.com/pytorch/vision/pull/770), and seems likely to cause minor irritations elsewhere.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/17636

Differential Revision: D14296530

Pulled By: ezyang

fbshipit-source-id: bc077d898773c93dab34c985a7b30525a43e558a
2019-03-03 01:06:10 -08:00
a44826e659 Fix: avoid race condition on model zoo directory creation (#16578)
Summary:
The current implementation of the `torch.utils.model_zoo.load_url`
function is prone to a race condition when creating the directory in
which it saves the loaded models, since it checks whether the
directory exists and then creates it in two separate steps. The
directory can be created after the check was made but before we
attempt to create the directory, resulting in an unhandled exception.

Instead, try to create the directory directly, and do nothing if it
already exists.

Note: for Python versions ≥ 3.2, we could simply use the
`exist_ok=True` flag on `os.makedirs`, but this is unavailable in
Python 2.7.

Signed-off-by: Antoine Busque <antoine.busque@elementai.com>
Pull Request resolved: https://github.com/pytorch/pytorch/pull/16578

Differential Revision: D13886470

Pulled By: soumith

fbshipit-source-id: 88815c8a65eec96caea32d6e9a7f83802502fdb9
2019-01-30 18:35:45 -08:00
bd19dd4b90 url download bugfix for URLs served without Content-Length header (#16153)
Summary:
Some HTTP servers dont return Content-Length, account for that

Fixes: https://github.com/pytorch/pytorch/issues/16152

Differential Revision: D13858882

Pulled By: soumith

fbshipit-source-id: e4293e9368ed4c87548d22adec1ce0c25ea4bd8f
2019-01-29 01:28:47 -08:00
c47f680086 arc lint torch/utils (#13141)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/13141

This is an example diff to show what lint rules are being applied.

Reviewed By: mingzhe09088

Differential Revision: D10858478

fbshipit-source-id: cbeb013f10f755b0095478adf79366e7cf7836ff
2018-10-25 14:59:03 -07:00
9b0cece9b0 Enable the general usage of _download_url_to_file (#9090)
Summary:
A requirement for the fix on https://github.com/pytorch/examples/issues/378.
Closes https://github.com/pytorch/pytorch/pull/9090

Reviewed By: goldsborough

Differential Revision: D8712254

Pulled By: ezyang

fbshipit-source-id: b28765f24d891890e9d88757ee4ec704e38e6af7
2018-07-02 19:55:39 -07:00
f06fcc6efa Fix bug that introduced in pull #3280 (#7292)
Apparently get() is a function of requests, not a module (not sure if in
the past get() used to be a module). Therefore, the syntax in #3280 will
alway fail with ImportError, and requests lib will never be used (kind
of defeat the purpose of that pull request).
Also, if requests lib is used, should add stream=True parameter,
otherwise requests.get() will load the whole response into memory.
2018-05-04 14:14:02 -07:00
3b58b859b2 Fix typos in docs (#6389) 2018-04-07 12:41:15 -04:00
2f27c1b56b Revert "Fix ImportError with requests in model_zoo (#5896)" (#5909)
This reverts commit 21ce93e88ff8eed1c9fa230a8c5d97d188093705.
2018-03-20 17:35:03 +01:00
21ce93e88f Fix ImportError with requests in model_zoo (#5896)
Not sure if this is a backwards compatibility issue. 
```
Python 2.7.9 (default, Apr  2 2015, 15:35:35) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests.get as urlopen
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named get
>>> from requests import get as urlopen
>>> 
```
2018-03-20 09:51:34 +01:00
d8d82d14cf Add an option to suppress download progress (#4135)
* Add an option to suppress download progress

* Add a disable option to pbar to make it a no-op

* Document progress
2017-12-14 22:19:27 -05:00
429d66549e If available try to use requests instead of urllib for model_zoo.load_url (#3280)
* try to use requests instead of urllib for load_url if available

* add noqa to silence flake warnings

* remove urllib import into the except
2017-10-31 08:53:58 -04:00
ae65236490 Fix typo 2017-07-04 15:19:05 -04:00
79d4ac670c Add map_location to load_url (#1418) 2017-05-01 10:21:30 -04:00
e7c1e6a8e3 [pep8] Fix most lint automatically with autopep8
Here's the command I used to invoke autopep8 (in parallel!):

    git ls-files | grep '\.py$' | xargs -n1 -P`nproc` autopep8 -i

Several rules are ignored in setup.cfg. The goal is to let autopep8
handle everything which it can handle safely, and to disable any rules
which are tricky or controversial to address. We may want to come back
and re-enable some of these rules later, but I'm trying to make this
patch as safe as possible.

Also configures flake8 to match pep8's behavior.

Also configures TravisCI to check the whole project for lint.
2017-01-28 01:15:51 +01:00
c9ec7fad52 Add model_zoo utility torch torch.utils (#424)
This was originally part of a torchvision PR, but I think it will be
useful outside vision, such as for distributing word embeddings.
2017-01-09 13:16:58 -05:00