-
Notifications
You must be signed in to change notification settings - Fork 10
Open
Description
The function used to cache the input files on user's request only works with https
based paths
analysis-grand-challenge/analyses/cms-open-data-ttbar/utils.py
Lines 27 to 33 in b4625ae
def cache_files(file_paths: list, cache_dir: str, remote_prefix: str): | |
for url in file_paths: | |
out_path = Path(cache_dir) / url.removeprefix(remote_prefix).lstrip('/') | |
out_path.parent.mkdir(parents=True, exist_ok=True) | |
if not out_path.exists(): | |
with tqdm(unit='B', unit_scale=True, unit_divisor=1024, miniters=1, desc=out_path.name) as t: | |
urlretrieve(url, out_path.absolute(), reporthook=_tqdm_urlretrieve_hook(t)) |
We could support also caching files in case XRootD paths are used
Metadata
Metadata
Assignees
Labels
No labels