Files
pytorch/tools/update_disabled_tests.sh
Zachary DeVito dab5f72543 we should have a config-based way to skip flaky tests (#30978)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/30978

This particular approach queries our issue tracker for test titles that
match the following format:

```
DISABLED test_async_grad_guard_with_grad (jit.test_async.TestAsync)
```

And then skips the python test for them. There is 1 second timeout so
if the internet flakes we still run the test suite, without disabling any
tests.

This is intended as a quick fix, similar to ninja unland, to get to a green
master. Long term test disables should go into the code.

Test Plan: Imported from OSS

Pulled By: zdevito

Differential Revision: D18890532

fbshipit-source-id: fe9447e59a6d5c9ad345f7c3ff15d63b6d2a09e2
2019-12-17 11:58:43 -08:00

9 lines
486 B
Bash
Executable File

#!/bin/bash
EXTRACTED_REPO=https://$USERNAME:$API_KEY@github.com/zdevito/pytorch_disabled_tests.git
git clone $EXTRACTED_REPO
cd pytorch_disabled_tests
curl 'https://api.github.com/search/issues?q=is%3Aissue+is%3Aopen+label%3A%22topic%3A+flaky-tests%22+repo:pytorch/pytorch+in%3Atitle+DISABLED' \
| sed 's/"score": [0-9\.]*/"score": 0.0/g' > result.json
# score changes every request, so we strip it out to avoid creating a commit every time we query.
git commit -a -m 'update'
git push